Machine Learning: Recruiting for AI Skills

As AI becomes embedded within all enterprise processes, business leaders are once again lamenting the lack of AI skills available when they need to recruit. A new report from Silico reveals that almost all UK businesses (94%) are comfortable with technology taking charge of decision-making and autonomous planning with little or human involvement, indicating a departure from ‘gut instincts’ and ‘human intuition’ in the boardroom.

To ensure that the AI systems in place are reliable, essential AI skills will be needed. The question is, where are these skills going to come from? According to research from Gigged, a driven talent platform, 72% of UK businesses surveyed are currently taking part in digital transformation projects; however, 30% claim there is too much work to do and not enough people to do this work successfully 57% say that the tech talent shortage has increased compared to last year.

A massive 90% of respondents said they are experiencing a tech skills shortage to some extent. These skills challenges are present across a broad sweep of technology IT roles; however, the most significant gaps are to be found in software development (37%), a problem experienced by almost half of large (501+ employee) companies (48%), cybersecurity (37%) and digital marketing (36%).

Things are getting even more challenging, with over half (57%) of respondents saying that the tech talent shortage has increased compared to last year, rising to 74% of those companies already experiencing it to a large extent. The reasons for such an extensive talent shortage include not being able to find qualified candidates (34%), and in second place, 32% say it’s a lack of budget.

Matt Alder, Talent Acquisition Futurist, The Recruiting Future Podcast, said, “The Tech skills crisis is driving many employers to think differently about talent in their business. With advances in AI and talent marketplaces, organisations can now gain real-time insights into the availability and skills of potential candidates, both within and beyond their existing workforce. However, true competitive advantage will go to the employers who treat all workers equitably regardless of their employment status. This includes providing fair compensation, opportunities for development, and a supportive work environment that fosters engagement and productivity.”

“Skills-based hiring is a progressive approach to recruitment that prioritises a candidate’s abilities, competencies, and potential,” James McLaughlin, UK VP of WithYouWithMe, told Silicon UK. “Instead of relying on paper-based credentials, skills-first hiring uses data-driven assessments, aptitude tests, and evaluations to identify a candidate’s relevant capabilities and suitability for a specific role.

McLaughlin continued: “New AI-based roles will continue to open up as the technology evolves and business applications skyrocket. Only by assessing a candidate’s true capabilities, rather than relying solely on what’s committed to paper, will businesses source the right talent for their needs and future-proof their workforce.”

Inside knowledge

Silicon UK spoke with Liz Scott MBE, the director of the Turing Innovation Catalyst in Manchester. Silicon UK began by asking What are the essential qualities and skills to look for when recruiting AI talent?

“I think it’s important to recognise that “AI talent” isn’t one big bucket – numerous disciplines are involved in creating AI-powered products and services and a broad spectrum of skill levels. People talk about “AI talent” as if it’s just one thing, but in reality, it includes everything from data scientists to machine learning engineers, research scientists to big data analysts, deep-learning engineers to robotics scientists – and many more.

“It’s also crucial to recognise that alongside any ‘core’ AI talent being recruited into a business, there’s also a real need for an awareness and appreciation of AI and its implications across an entire organisation.

“So, for example – how do pricing structures and contract terms change when a consultancy or agency that previously charged for hours worked evolves their model and generates content or analyses in a fraction of the time using AI? What computing power is needed to process millions, if not billions, of individual data points that can now be individually analysed and organised by a sophisticated algorithm in ways that couldn’t be done before?

“There are a lot of knock-on effects of using AI to power growth, and this leads to additional skills and new talent being needed in areas that are not traditionally thought of as ‘AI talent.’”

How has the demand for AI talent evolved in recent years, and what trends do you foresee in the near future?

“We’ve seen a big explosion in demand for AI talent in recent years. But it isn’t quite as new a trend as it seems. AI has been present in our tech products and services for years now – let’s remember that it powers really simple things like predictive text – despite the rise in generative AI tools like ChatGPT and Midjourney making it seem like this is a new phenomenon.

“If we look back over the last few years, we’ve seen demand for tech roles in data grow by over 1000% from 2019 [1]. That’s a staggering increase in the demand for those skills, and it doesn’t show any signs of slowing down.

“Looking to the future, I think we’ll continue to see new roles and evolved skillsets emerging – 2023 was the first time I came across ‘prompt engineer’ as a formal job title (someone who carefully constructs text into prompts that can be interpreted and understood by generative AI tools, in case you were wondering).

“It’s really exciting to see the development of AI as a technological field driving new opportunities for skills development. The concept of better equipping people to work alongside AI or have AI as an assistant as they do their work will really grow in prominence.

“We may not all be destined to become full-time prompt engineers, but I think there will be a version of that skillset that becomes widely adopted, as more and more of us use AI tools in our everyday work.”

How important is AI professionals to have a solid academic background or formal degrees in AI-related fields?

“For me, the answer to this question really depends on what you’re looking to achieve. Just as there isn’t one generic AI skillset, I don’t think there’s one right answer in terms of academic credentials, formal degrees, or other types of learning.

“I would advise AI professionals to be really clear about what interests them and the types of roles they would like to work in, and reverse engineer from there. And if in doubt, speak to the companies you’d love to work for. Understand first-hand the type of skill they need and the mindset they’re looking for. Some roles will have a much clearer line between academia and research, and some will require more ‘real-world’ learning. Lots of roles will require both, but at different times!

“One thing to bear in mind is that due to how quickly the field is developing, it’s important that AI professionals are approaching their careers and skillsets with a lifelong learning mindset. There might be a qualification – formally academic or otherwise – to get access to the first couple of roles in a career, but we need to get much better as a nation in encouraging people to dip back into learning throughout their careers to top up expertise or develop skills in new technologies.

What are the benefits and drawbacks of hiring AI talent with a research background versus those with more industry-focused experience?

“I think it comes down to being really clear about what an organisation needs. Some foundational AI topics are best understood in depth in some of our leading universities, but there are also a plethora of great ‘alternative’ skills providers who are helping people pivot into and further develop AI skills in more ‘real-world’ environments.

“In both cases, I’d encourage organisations hiring AI talent to be inquisitive about the broader skills AI professionals are being supported with. For example, do applicants possess other helpful business skills that complement their technological and AI abilities, or is that something that the hiring organisation is going to need to take care of?

“I would also be mindful of the type of team incoming talent is landing into. What are the conditions for success for the type of professional you’ve hired? If they’re research focused, for example, what will they need to ‘dock into’ the rest of your business and how can you ensure they’ve got that from day one?”

How can organisations create an inclusive recruitment process that encourages candidates from diverse backgrounds to apply for AI roles?

“First of all, be careful about using AI for screening. There are some brilliant AI-powered recruitment, screening and assessment tools out there, but be careful about the data they’ve been trained on, and make sure you’re happy that you’ve explored – as best you can – whether any bias is built in.

“I would also think hard about the requirements for the role. Does it need a formal degree? Does it need the number of years of experience you’ve stated? How could you make the work arrangements suitable for those with caring responsibilities, disabilities, or neurodivergent conditions?

“A few other best practice practicalities for general inclusive hiring apply here. Have you stated the salary in the job listing? What routes to market are you using to promote the role? Could you tap into DEI-focused communities that might reach candidates well outside of your immediate network?

“I could go on all day, but I won’t, and I’ll just say – make sure you’ve included DEI in your job description and hiring process design. Hiring inclusively doesn’t happen by accident.”

What strategies can organisations use to retain and develop AI talent once they’ve been hired?

“All good practices around people, culture and engagement apply here. Ensure you’re welcoming your hard-found AI talent into a culture they can be proud of. Although their roles might be deeply technical, it’s worth remembering that AI professionals working outside of academia have deliberately chosen to be part of a mission that involves the practical application of their knowledge. Helping them to feel connected to what they’re enabling and allowing them to develop further and sometimes even promote their technological advancements.

“I’m hopeful that, having just come out of an incredibly hot market for broader tech talent, a lot of the companies who are now ramping up their in-house AI expertise will have already developed a lot of great practices to retain and develop their teams. If not, now is the time to invest in that, as we’ve got high and growing demand for all different types of AI talent, and the companies that win will be the ones who can go beyond just enticing talented practitioners in and can nurture them over the medium to long term.”

Skills and talent

There is little doubt that AI is profoundly impacting all businesses. Clearly, generative AI is a critical driver. Indeed, Gartner polled more than 2,500 executive leaders; 45% reported that the publicity of ChatGPT has prompted them to increase AI investments, with 70% of executives saying that their organisation is in investigation and exploration mode with generative AI, while 19% are in pilot or production mode.

The poll found that 68% of executives believe that the benefits of generative AI outweigh the risks, compared with just 5% that feel the risks outweigh the benefits. However, executives may begin to shift their perspective as investments deepen.

“Autonomous business, the next macro phase of technological change, can mitigate the impact of inflation, talent shortages and even economic downturns,” said Frances Karamouzis, Distinguished VP Analyst at Gartner. “CEOs and CIOs that leverage generative AI to drive transformation through new products and business models will find massive opportunities for revenue growth.”

Bryan Cole, Director of Engineering at Tricentis on AI recruitment, tells Silicon UK how business should approach their AI recruitment: “There are two main types of AI talent to consider when hiring depending on your specific business objectives. An enterprise developing a model for their applications, for example, will be interested in roles directly related to artificial intelligence research and design, including critical skill sets of mathematical and statistical analysis, programming in AI platforms (notably Python), and data management.

“However, enterprises using an existing AI model via API, such as ChatGPT or Bard, have much less need for deep technical architects to build the model (though they will still be required to help manage the datasets) and much more emphasis on creative thinkers and business experts needed to produce fantastic new use cases that will allow dramatic enhancements to existing features or even the creation of entirely new industries.

The skills shortage that has become chronic in some sectors may extend to AI over the short to medium term. AI is such a hot topic that reskilling is happening apace. All enterprise leaders have always needed the right people with the right skills at the right time. AI is no different. AI is such a fluid area of development that the ‘ideal’ skillset may not exist. What should come into focus rapidly is your company’s AI strategy. Once this is clearly understood, the skills needed will also become clear. Identifying whether these skills can be hired or taught should be a priority.

[1] Tech Nation People & Skills Report 2022.