BACKGROUND AND TECHNICAL NOTE: SKILL SHORTAGE ASSESSMENT REPORTS

This note provides background to the skill shortage assessment reports, and also includes further technical information on methodology and definitions than is provided in the individual reports.

This note:  

• describes the structure of the Job Vacancy Monitoring Programme;
• provides some details on the Survey of Employers who have Recently Advertised (SERA) Intensive on which the skill shortage assessment reports are largely based; and
• defines a number of indicators used in the reports, and briefly outlines the methodology and data sources used to calculate these indicators.

Job Vacancy Monitoring Programme: An Overview

 The skill shortage assessment reports are one of the outputs of the Department of Labour’s (the Department’s) Job Vacancy Monitoring Programme (JVMP). The JVMP comprises a number of components and processes, which are described below.  

Job Vacancy Monitor

The Job Vacancy Monitor (JVM) is a monthly analysis of job advertisements in selected editions of 25 major newspapers¹ around New Zealand and two internet IT job boards. The purpose of the job ad series is to monitor changes in occupational labour markets. It also provides the basis of the sample for the SERA.

Each month all advertisements appearing in a single edition (usually the first Saturday of the month)² of 25 major newspapers are captured electronically. A number of attributes are captured from each advertisement including: job title, location and number of vacancies. The job title for each advertised vacancy is coded to a 5-digit occupation using the New Zealand Standard Classification of Occupations 1999. Likewise, location of the vacancy is coded to standard classifications of urban areas and regions. Advertisements that are repeated across newspapers are identified and flagged as duplicates. In addition to monitoring newspaper advertisements, the JVM also monitors two information technology internet job boards (Seek and Jobstuff). The number of new vacancies posted over the previous seven days is counted each Friday.

SERA Extensive

The SERA Extensive is an annual short telephone survey of employers who have advertised professional, technical/semi-professional, or trade vacancies. The purpose of the survey is to identify occupations in which employers are having difficulty filling vacancies.

The survey sample is drawn primarily from the JVM, but is also supplemented from other sources of advertising such as professional and trade journals, and internet job boards. Employers are approached six to eight weeks (trade occupations) or eight to ten weeks (professional occupations) after advertising and asked whether they have filled their vacancy, and how many suitable applicants applied. From this information the Department is able to calculate a fill rate for each occupation. The fill rate is the proportion of vacancies that were filled with an adequately qualified and experienced person (with the right to work in New Zealand) within ten weeks of advertising, and is a key indicator of skill shortage. Occupations with fill rates lower than about 80% are typically regarded as being in shortage, while fill rates lower than 40% usually indicate that the occupation is in acute shortage. The Department also calculates the average number of suitable applicants per vacancy, which is a useful indicator of the level of under or over supply within an occupation.

Identification of Key Occupations

The next step of the JVMP process is to identify key occupations for further in-depth analysis. A number of criteria are used to select key occupations, including some evidence that the occupation is currently in shortage. Evidence of shortage may be drawn from the SERA Extensive (a low fill rate for a specific occupation), the JVM (a rapid increase in the number of advertised vacancies may indicate that a shortage is emerging), presence of the occupation on the Immigration New Zealand skill shortage lists, or anecdotal accounts of shortage. Key occupations are selected to represent a spread of economic sectors. Key occupations may also be selected on the basis that they are of considerable strategic importance to the New Zealand economy and do not necessarily have to be in shortage all the time to be included.  

SERA Intensive

The SERA Intensive is a survey of employers in the key occupations identified in the previous step. Its objective is to gain in-depth knowledge of the causes of shortages in each occupation. The survey also draws its sample from the JVM, and is supplemented, where necessary, from other sources of advertising such as professional and trade journals and internet job boards. The survey aims to interview approximately 20 employers for each occupation.

The questionnaire used is a mix of quantitative and qualitative questions. Identical quantitative questions to those used in the SERA Extensive are included to enable a fill rate and the average number of suitable applicants per vacancy to be calculated for each occupation. Additional qualitative questions cover the following issues:  

• The supply and demand forces that are contributing to shortages;
• Implication for the employer of their inability to fill vacancies, and their response to this; and
• Skill sets that employers are unable to obtain in applicants to their vacancies.

Occupational Skill Shortage Assessment Reports

Comprehensive reports on each of the key occupations from the SERA Intensive are compiled. These reports draw together information gathered from the SERA Intensive and various other data sources that provide additional insight into the supply and demand forces contributing to shortages in each of the key occupations. These sources include:  

• Enrolments and completions for courses related to the occupation;
• Migration of persons qualified in the occupation;
• Employment data from sources such as: the Household Labour Force Survey, Censuses, and occupational associations;
• Economic data related to the sector in which the occupation is concentrated.

Draft reports are distributed to government, industry and occupational/professional associations for comment. Feedback on analysis and conclusions are incorporated into the final reports.  

SERA Intensive 2005

The skill shortage assessment reports draw on the results of the 2005 SERA Intensive. The following gives a summary of the main features of the survey:

• Interviewing was conducted from May to August 2005.
• It was based on a sample of advertised vacancies drawn from major newspapers published during April and June 2005. If there were insufficient advertisements in newspapers then other sources such as internet job sites and trade journals were used. The Department attempted to achieve a sample of at least 20 employers for each occupation. This was not possible when fewer than 20 advertisements appeared over the time period.
• A total of 885 successful interviews were conducted with employers who advertised 1,480 vacancies.
• It involved telephone interviews which took from five to 40 minutes to complete. Average duration was 10 to 15 minutes.
• The questionnaire was administered using the Blaise Computer Assisted Telephone Interviewing (CATI) system.

 Limitations of the SERA

The SERA does not necessarily survey a representative sample of employers in each occupation. The sample is based on employers who advertise, and not all employers advertise. The proportion of employers who typically advertise may vary across occupations. For instance, employers in the public service seeking to fill a vacancy are usually required to advertise. So a higher proportion of vacancies are likely to be advertised in occupations in which a high proportion of persons are employed in the public service. SERA results should, therefore, be regarded as indicative rather than representative.  

Levels of self-employment within occupations may also impact on the meaningfulness of indicators calculated from the SERA data, as the SERA does not include the portion of the workforce which is self-employed. The Department’s skill shortage assessments do not rely purely on the SERA indicators and are based on the analysis of a wide range of data sources.

Indicators used in the Reports: Definitions and Methodological Issues

Table 1 provides definitions of skill shortage indicators used in the skill shortage assessment reports.

Table 1: Skill Shortage Indicators

Indicator	Definition
Fill rate	The fill rate is the proportion of vacancies included in the SERA within an o ccupation that were filled. In order for a vacancy to be regarded as being filled, the following criteria need to be met: The appointee needs to have the right to work in NZ at the time of application; and The vacancy needs to have been filled within 10 weeks of the ready date (the ready date is either the closing date for applications, or the date when the vacancy was first advertised if there is no closing date). and The appointee needs to have the qualifications and experience to do the job (in which case the vacancy is regarded as being ‘Adequately filled’); or the appointee needs to have enough qualifications and experience to be regarded as a member of the occupation being advertised (in which case the vacancy is regarded as being ‘Compromise filled’).
Average number of suitable applicants	This indicator measures the average number of suitable applicants per advertised vacancy included in SERA. In order for an applicant to be regarded as suitable he/she: needs to have the right to work in NZ at the time of application; and needs to have the qualifications and experience to do the job; or at least the qualifications and experience to be regarded as a member of the occupation.

Table 2 provides definitions and some methodological notes on labour supply indicators used in the skill shortage assessment reports.

Table 2: Labour Supply Indicators

Indicator	Definition	Methodological Notes
Training Rate	The number of persons achieving relevant qualifications expressed as a percentage of total employment in that occupation.	The number of completions of qualifications was obtained either from training providers directly, from the Tertiary Education Commission or the Ministry of Education. The number of persons employed in each occupation is sourced from relevant workforce data where this is available; otherwise it is based on estimates from the Department’s Occupational Employment Model. This model draws on Census and Household Labour Force data.
Retirement Rate	The proportion of employed people in an occupation expected to retire each year.	This indicator is calculated from Census data. It expresses the proportion of the workforce in each occupation that retires each year. The number of persons retiring each year is estimated as 20% of the population aged 60 and above.

Glossary of Terms

 The following terms are used in the skill shortage assessment reports:

Certification – involves being issued a time-restricted (e.g. annual) practising certificate that enables a professional to conduct the business of his/her profession as set out by statute or by professional/governing council regulations.

Educational achievement – occurs when a student completes his/her certificate, diploma or degree program.

Employed - people in the working-age population who work at least one hour per week.

Job Vacancy Monitor – a monthly monitor and analysis of advertised vacancies appearing in 25 daily newspapers (covering all regions except Tasman) and two internet IT job boards.

Labour force participation rate – percentage of the working-age population that is in the labour force.

Labourforce/supply – the number of people in the working-age population who are either employed, or who are unemployed but available for work and actively seeking work.

Not in the labour force - the number of people in the working-age population who are not in work, and who are not available for or actively seeking work. For example, people who are: retired, students, or at home caring for children.

Occupational fill rate – the proportion of vacancies included in the SERA survey sample which were filled with an adequately qualified and experienced person within eight to ten weeks of advertising. In cases where an employer has appointed a person who does not have adequate qualifications and skills to be regarded as a member of the occupation for which the employer advertised, the vacancy is regarded as unfilled. For example, an employer who advertised for an accountant but was unable to find an adequately qualified and skilled accountant and appointed a bookkeeper is regarded as having an unfilled vacancy.

Occupational detachment – refers to individuals who choose not to continue practising in their occupation but retain a connection to the occupation (e.g. move into a management or supervisory role, or retain professional registration), or who leave the occupation entirely (e.g. by changing occupation or withdrawing from the labour market).

Potential labour supply – includes those people who are not in the labour force, but could be encouraged to enter if the remuneration and/or working conditions were improved. For certain professions, the potential labour supply is composed of those who are registered with a professional board, but do not currently hold an annual practicing certificate.

Registration – involves becoming a member of a professional board or council. In certain instances, registration is a pre-requisite to practising in an occupation.

Wage – the payments made to workers for the use of their labour as a factor of production.

Working-age population - the number of usually resident non-institutionalised civilian population aged 15 years and over. All labour market figures relate to those aged 15 years and over.

2005 SERA Intensive Questionnaire

Following is a copy of the SERA Intensive questionnaire used in 2004:

Please note that this questionnaire is administered in an electronic format, and the version listed below is a simplified version without routing instructions, validation rules or edits.

Questionnaire

Hi this is …….from the Department of Labour. We are doing research into skill shortages. Is there someone there who can answer some questions relating to the {occupation} vacancy you had advertised in the {paper} on {date}?

When you advertised for {occupation} in the {newspaper} on the {date} how many {occupation} were you hoping to appoint?

Would you be able to tell me whether the positions you were advertising on that date have in fact been filled and when?

IF NECESSARY ASK can I just check - am I talking to an employment agency or the employer?

Did you want people short-term, that is for less than 3 months, or for longer than that?

ALL SHORT TERM – Terminate interview here

NOT ALL SHORT TERM – Proceed

How many did you only want short term?

“In all the rest of the questions please only think about the ones that were for longer than three months.”

CHECK QUESTION

So you wanted {X} people for 3 months or longer?

“In all the rest of the questions please only think about those vacancies”

Were you advertising to replace people in existing positions, or to fill a new position(s)?

Did you want them to work in {town where advertised}?

IF NECESSARY ASK where?

Thinking about the advertisement we have talked about, how many people applied for that?

“Don’t include any applicants for short term positions.”

Did anyone else try for the job, or did you approach anyone else for the job? How many did?

So {X} number of people were considered for this job?

How many of them did you think had the qualifications and experience to do the job you needed them for?

Have you interviewed for this position?

At the time they applied, did all those {number from previous question} have the right to work in NZ?

IF NECESSARY ASK how many did?

Have you appointed anyone to that/those advertised positions?

IF NECESSARY ASK how many have you appointed?

Does the number of appointments match the number of people they are looking for?

IF NOT ASK are you still working on making appointments from among the applicants for that position?

How long do you think it will be before appointments are made to [all] the position(s) you advertised for on {date}?

IF MORE APPOINTMENTS SEEM POSSIBLE WITHIN 10 WEEKS FROM ADVERTISEMENT DATE, ARRANGE A RETURN VISIT, OTHERWISE COLLECT INFORMATION ON THOSE APPOINTED.

IF ANY APPLICANT COULDN’T WORK LEGALLY IN NEW ZEALAND ASK when you interviewed the people you’ve appointed, did they all have the right to work in NZ?

IF NECESSARY ASK how many didn’t?

For the job you wanted to fill, did you need someone with particular skills, qualifications or experience? IF YES what were they?

Were there other, nice-to-have skills or qualifications you were looking for?

IF YES what were they?

CHECK QUESTION

So you/they have appointed {number appointed}?

Would you say that (that/person) those had the skills, qualifications and experience needed for the job you wanted them for? How many did?

Thinking of those who didn’t, how many of the would you say would still be classified as a occupation even though they didn't have all that you needed for that particular job?

IF LACKED THE SKILLS

What occupation or job would you say they/the others should be classified as?

Think about the {Y} that you’d still classify as {Occupation} even though they weren’t really what you wanted. When you interviewed them did they all have the right to work in NZ? How many did?

IF ALL HAD THE SKILLS

Think about the {X} who had all the skills you needed for the job. When you interviewed them did they all have the right to work in NZ?

IF EMPLOYER HAS NOT FILLED ALL THE POSITIONS BY 10 WEEKS AFTER AD DATE

I’ll read a list of things that sometimes happen when an employer can’t fill a vacancy Because you can’t fill that {if necessary use occupation name} vacancy is it likely that you might:

• lose business to competitors?
• have to put off developing a new product or service?
• have to stop producing a certain product or service?
• have increased operating costs?
• put off introducing new technology or working practices?

Despite having that unfilled vacancy will you still be able to…

• meet your quality standards?
• meet customer needs?
• Are there other things you think will probably happen because you couldn’t fill that vacancy?

Employers do different things to cope with a staff shortage. Because you can’t fill that vacancy might you:

• outsource work?
• use contractors?
• make more use of overtime?
• recruit from overseas?
• give more training to your existing workforce in order to fill the vacancies?
• change the way existing staff do their jobs?
• increase salaries to make the job more attractive?
• use technology as a substitute for labour?
•  
• Are there other things you might do because you couldn’t fill that vacancy?

FOR ALL

Apart from the {occupation} you have appointed. Have you needed to take on {occupation} in the last 12 months?

 

FOR NO APPOINTMENT

Apart from the {occupation} you are currently trying to appoint. Have you needed to take on {occupation} in the last 12 months?

Compared to the last time you tried, would you say it’s easier or harder now to get {occupation}?

Would you say that in the last 12 months there have been any particular:

• skills that are hard to get in {occupation}?
• experience level that is hard to get?
• anything else that’s hard to get?

People sometimes leave the occupation they’re in, and go and do something different. In your view, does that happen with {occupation}

• more than with other occupations?
• less than with other occupations?
• why do you think that is?
• Would you say that there is a shortage of {occupation}?

IF YES

I’d like your opinion on some factors that people have mentioned as affecting labour shortages. Would you say that:

• businesses that employ {occupation} are expanding ?
• technology is affecting the number of {occupation} a business needs - if yes, what effect has that had?
• the {occupation} are going overseas?
• there are not as many {occupation} coming into the country?
• the current quality of training for {occupation} is good enough?
• people don’t want to become {occupation}?

Do you think this is because being the {occupation} involves:

• unusual hours of work?
• tough physical work?
• work in hot, cold or wet conditions?
• a lot of loud noise?
• danger of injury?
• it’s a low-wage occupation?
• anything else that might make it less attractive than some other jobs??

 ONLY ASK OF OCCUPATIONS WITH LICENSING AND REGISTRATION

• Would you say that:
• the way licensing is done affects the shortage of {occupation}?
• the way registration is done affects the shortage of {occupation}?

 What else do you think affects the shortage of {occupation}?

ONLY ASK IF THEY HAVE MADE AN APPOINTMENT

Looking to the next 6 months, do you think that the number of {occupation} you employ will increase, decrease or stay the same?

IF THE EMPLOYER GOT THEIR IDEAL CANDIDATE ASK

What are you paying the {occupation} who had all the skills and qualifications needed for the job?

Compared to 12 months ago, is that lower or higher or has it not changed?

 

IF ANSWERED LOWER OR HIGHER TO PREVIOUS QUESTION ASK

Would you say it’s a little (higher/lower) or a lot (higher/lower)?

 

IF THE EMPLOYER DID NOT GET THEIR IDEAL CANDIDATE ASK

If you had been able to employ the exact person you wanted for the {occupation} vacancy you advertised, what would you have paid them?

 

IF LESS THAN IDEAL OR A COMPROMISE FILL ALSO ASK

What are you paying the {{new} {occupation}} you appointed?

 

	Job 1	Job 2	Job 3
per hour
per week
per year

Do you think there is anything about this job as a {occupation} that might make it harder to fill than other {occupation} jobs?

PROBE IF NECESSARY

• unusual hours of work?
• location?
• tough physical work?
• work in hot, cold or wet conditions?
• a lot of loud noise?
• danger of injury?
• it’s a low-wage occupation?
• anything else that might make it less attractive than some other jobs?

Is there anything else you would like to say about the problems of finding {occupation} these days?

Those are all the questions. Thank you very much for your time and comments they are most appreciated.

For queries regarding this report please contact info@dol.govt.nz.

End Notes

1 Coverage of the JVM was increased from 14 newspapers to 25 newspapers in January 2005. [top]

2 JVM uses the NZ Herald from the second Wednesday, and the second Friday edition of some small regional papers. The results of the JVM are published in a monthly report which is posted on the Department of Labour website  [top]

Disclaimer: The Department of Labour has made every effort to ensure that the information contained in this report is reliable, but makes no guarantee of its accuracy or completeness and does not accept any liability for any errors. The information and opinions contained in this report are not intended to be used as a basis for commercial decisions and the Department accepts no liability for any decisions made in reliance on them. The Department may change, add to, delete from, or otherwise amend the contents of this report at any time without notice. The material contained in this report is subject to Crown copyright protection unless otherwise indicated. The Crown copyright protected material may be reproduced free of charge in any format or media without requiring specific permission. This is subject to the material being reproduced accurately and not being used in a derogatory manner or in a misleading context. Where the material is being published or issued to others, the source and copyright status should be acknowledged. The permission to reproduce Crown copyright protected material does not extend to any material in this report that is identified as being the copyright of a third party. Authorisation to reproduce such material should be obtained from the copyright holders.