Andrew Leigh

ARTIFICIAL INTELLIGENCE AT WORK: CHANGING DEMAND FOR AI SKILLS IN JOB ADVERTISEMENTS*
Australian Bureau of Statistics and Reserve Bank of Australia Joint Conference on Human Capital
11 June 2024, Sydney

I acknowledge the Gadigal people of the Eora nation, the traditional owners of these lands, and pay respect to all First Nations people present.

Barely a day goes by without someone discovering a new use for artificial intelligence. Financial institutions are using AI to detect fraud, by looking for unusual transaction patterns. AI integrated with virtual reality is being used to create highly realistic training simulations for pilots, first responders and surgeons. Musicians are using AI to create new instruments and vocal processes. Educators are using AI to personalise the learning experience. Dating coaches are using AI to train people on finding their perfect match. Gardeners are using AI to choose which plants will work best together, schedule optimal watering times and devise pest control strategies. Carers are using AI to craft fictional stories that are perfectly tailored for young listeners.

AI engines have matched and exceeded humans on a range of tests. As Stanford University’s AI Index 2024 Annual Report points out, artificial intelligence has exceeded human benchmarks on tasks such as reading comprehension, image classification and visual reasoning (see Figure 1). As AI has surpassed these benchmarks, researchers have had to identify new challenges, such as competition-level mathematics, where AI has moved from 10 per cent of human-level performance in 2021 to 90 per cent on the latest estimates (Maslej et al 2024).

Public interest in AI is growing accordingly. Figure 2 shows the monthly volume of internet searches in Australia for the phrase ‘artificial intelligence’. In the months following the release of ChatGPT on 30 November 2022, searches for AI tripled, and remain high.

There is considerable interest in the effect of AI and related technologies on the labour market. Some research considers how AI might change the nature of work. For example, Dell’Acqua et al (2023) estimate the impact of AI on the productivity of management consultants by randomly assigning consultants to undertake mock tasks with or without the assistance of ChatGPT, and find large positive impacts of the technology on productivity. Other studies have discussed the likely impact of AI on work with reference to prior technological shocks (see for example Coelli and Borland 2023).

One possible way that AI may affect the market is through increased demand for AI-related skills — that is, skills that enable people to develop or work with AI models. Recent work by OECD researchers (Borgonovi, et al. 2023) examined demand for these skills by searching the full text of job advertisements for AI-related keywords. The study, which includes Australia, relies on data from Lightcast, a company specialising in labour market data that was formed from a merger between Emsi and Burning Glass Technologies.

For Australia, the Lightcast data are extensive, but are not the most comprehensive source. From 2019 to 2022, Lightcast’s database included 1.1 to 1.3 million unique Australian job advertisements (Borgonovi, et al. 2023, p.51). By contrast, we use data from SEEK, Australia’s largest online employment marketplace. Although SEEK does not disclose the number of jobs in its database, we can confirm that its coverage for those years was around twice as large as Lightcast. Another advantage of SEEK data over the Lightcast data is that the former has extensive data on the Australian labour market, including data that is not always displayed publicly, such as the advertised salary for each role.

What constitutes an AI job? We opt to follow the approach of Borgonovi et al. (2023), which involves searching the text of job ads for a range of skill-related keywords, such as ‘AI ops’ or ‘PyTorch’. Each of these skills are classed by the authors as either ‘generic’ or ‘specific’. If a job advertisement contains at least two generic or one specific AI skills, it is classified as an ‘AI job’. A full list of these skills is provided in the Appendix.

This approach to identifying AI jobs is relatively straightforward, but is also somewhat conservative — advertisements for AI jobs that do not explicitly mention AI-related skills and terms in the text of the ad will fail to be identified as AI jobs in this methodology.

Note too that our analysis does not aim to identify any roles that have been displaced or negatively affected by the emergence of AI technology. Instead, our focus is on estimating the share of new jobs in the Australian economy that require AI skills.

We begin by applying this methodology to the SEEK dataset. Figure 3 shows the share of Australian job advertisements that are ‘AI jobs’, on a monthly basis over the period from 2017 to 2024. In 2017, just 0.06 per cent of advertisements are for AI jobs. This figure peaked at 0.2 per cent in 2021, before declining slightly to 0.17 per cent in early-2024.

There are two clear takeaways from these results. First, demand for AI skills has clearly grown substantially, approximately tripling since 2017. Second, AI jobs are extremely rare. Using Lightcast data, Borgonovi et al. (2023) find that AI jobs constitute less than 1 per cent of all advertised openings for each of the 14 countries in their sample. Nonetheless, they estimate figures that are considerably higher than ours. In 2022, Lightcast data suggest that AI jobs (defined in the same way as in our study) comprised 0.84 per cent of job advertisements in the United States, 0.54 per cent in Canada, and 0.51 per cent in the United Kingdom.

Part of the difference in our results may be related to data differences. For Australia, the Lightcast figures suggest that 0.4 per cent of 2022 employment postings were for AI jobs, around twice as large as our estimate for the same year. Given that our dataset is considerably larger, we are inclined to prefer our estimate, but it is plausible that that the Lightcast dataset has some advantages that we have not considered.

One possible theory for the declining proportion of AI jobs in the latter years of our sample is that our keywords are better at capturing the nature of AI jobs in 2017 than in 2024. However, this does not appear to be the whole story. Figure 4 plots the share of job advertisements on SEEK that simply include the phrase ‘artificial intelligence’. This too rises from 2017 to 2022, before declining slightly. The share of job advertisements that mention ‘artificial intelligence’ rose six-fold from 2017 to 2022, but fell by one-third from 2022 to 2024. At the end of the period we analyse, around 1 in 1000 Australian job postings on SEEK include the phrase ‘artificial intelligence’.

Next, we turn to examine which occupations feature the largest proportion of AI jobs, using data from the 12 months to March 2024 (inclusive). Occupations are defined at the four-digit Australian and New Zealand Standard Classification of Occupations (ANZSCO) level. These results are shown in Figure 4.

We find that the occupation that tops the list is ‘Mathematical Science Professionals’, a group that includes Data Scientists. Within this occupation, 6.3 per cent of ads for Mathematical Science Professionals are AI jobs. The rest of the top ten list is dominated by other science, technology, and research roles, all of which have 1 per cent or more of their positions as AI jobs. Some may be surprised to learn that ‘Nurse educators and researchers’, ‘Music professionals’ and ‘Social professionals’ are among the top ten occupations for their share of AI jobs.

Another way of analysing the data is by industry. SEEK’s data is not collected using the Australian and New Zealand Standard Classification of Industries. Instead, employers are asked to select from one of 29 pre-defined ‘classifications’ for their advertised roles.

As Figure 5 shows, the classifications in which AI jobs are most prevalent are Science & Technology (2.2 per cent) and Information & Communication Technology (1.23 per cent). At the other end of the spectrum, not a single job ad in the Retail & Consumer Products classification in the year to March 2024 met the ‘AI job’ criteria.

Do AI jobs pay more? In the 12 months to March 2024, the average advertised salary for AI jobs was $121,275. This is 31 per cent higher than the average for non-AI jobs. However, much of this difference reflects compositional differences — AI jobs are over-represented in occupations that tend to pay well regardless of whether AI skills are mentioned in the job ad.

To control for these compositional issues, we run fixed effects regressions, controlling for occupation and state effects. We run these regressions separately for each year in the sample. In 2017, we estimate the pay premium for AI jobs was 11 per cent. Over the ensuing years, the pay gap between AI jobs and other jobs in the same state and occupational category steadily fell. In 2023 and 2024, the final years of our sample, the pay premium for AI jobs was just 4 per cent. Averaged across the entire period 2017 to 2024 (and controlling for state, time and occupation fixed effects), the pay premium for AI jobs was 6 per cent.

Conclusion

Notwithstanding the considerable public interest in generative artificial intelligence, AI jobs constitute a tiny share of all advertised positions. In 2024, AI jobs comprised only 0.17 per cent of job postings, meaning that just 1 in 588 advertised roles were for AI jobs. From 2022 to 2024, AI jobs declined as a share of job postings. Similarly, a simple search for the phrase ‘artificial intelligence’ in posted jobs shows that the share of such positions also dropped from 2022 to 2024. In 2024, only about 1 in 1000 advertised roles contained the phase ‘artificial intelligence’.

Across occupations, AI jobs are most prevalent in science, technology and research roles. The largest share of AI jobs are among Mathematical Science Professionals, where 6.3 per cent of vacant positions are AI jobs. Across industries, AI jobs are most prevalent in science and technology. AI jobs are least prevalent (indeed, non-existent) in the retail and consumer products industry. In future work, we intend to compare trends in total employment in occupations with a high share of AI jobs with trends in employment in occupations with a low share of AI jobs.

Finally, we find that AI jobs pay higher wages than non-AI jobs. This remains true even holding constant time, geography and occupation. Without controls, AI jobs pay 31 per cent more than non-AI jobs. With controls, AI jobs pay 6 per cent more than non-AI jobs. There is some evidence that the wage premium for AI jobs declined over the period 2017 to 2024.

* This analysis was carried out in collaboration with Matt Cowgill, formerly chief economist at SEEK Australia. My thanks to Matt for his careful datacrunching and SEEK for facilitating this research. No funding was provided for this research, and any errors are my responsibility alone.

References

Acemoglu D., Autor D., Hazell, J. and Restrepo, P. (2022), ‘Artificial Intelligence and Jobs: Evidence from Online Vacancies’, Journal of Labor Economics, 40(S1).

Borgonovi, F., Calvino, F., Criscuolo, C., Samek, L., Seitz, H., Nania, J., Nitschke, J and O’Kane, L. (2023), ‘Emerging trends in AI skill demand across 14 OECD countries’, OECD Artificial Intelligence Papers, No. 2, OECD Publishing, Paris.

Coelli, Michael Bernard and Borland, Jeff, (2023) ‘The Australian Labour Market and IT-enabled Technological Change’, Melbourne Institute Working Paper No. 01/23, Melbourne Institute, Melbourne.

Dell’Acqua F, McFowland E, Mollick ER, Lifshitz‑Assaf H, Kellogg K, Rajendran S, Krayer L, Candelon F and Lakhani KR (2023) ‘Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality’, Harvard Business School Technology and Operations Management Unit Working Paper 24–013, Harvard Business School, Boston, MA.

Felten E, Raj M, Seamans R (2021) Occupational, industry, and geographic exposure to artificial intelligence: A novel dataset and its potential uses. Strategic Management Journal 42(12):2195–2217.

Maslej, Nestor Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark, (2024) The AI Index 2024 Annual Report, AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA.

Appendix: Categorisation of AI Skills

Following Borgonovi et al (2023), we classify a job posting as an ‘AI job’ if it contains at least two generic or one specific AI skill, from the list below.