Building AI that can build AI

They are a dream of researchers but perhaps a nightmare for highly skilled computer programmers: artificially intelligent machines that can build other artificially intelligent machines.

With recent speeches in both Silicon Valley and China, Jeff Dean, one of Google's leading engineers, spotlighted a Google project called AutoML. ML is short for machine learning, referring to computer algorithms that can learn to perform particular tasks on their own by analysing data. AutoML, in turn, is a machine-learning algorithm that learns to build other machine-learning algorithms.

With it, Google may soon find a way to create AI technology that can partly take the humans out of building the AI systems that many believe are the future of the technology industry.

The project is part of a much larger effort to bring the latest and greatest AI techniques to a wider collection of companies and software developers.

The tech industry is promising everything from smartphone apps that can recognise faces to cars that can drive on their own. But by some estimates, only 10,000 people worldwide have the education, experience and talent needed to build the complex and sometimes mysterious mathematical algorithms that will drive this new breed of artificial intelligence.

The world's largest tech businesses, including Google, Facebook and Microsoft, sometimes pay millions of dollars a year to AI experts, effectively cornering the market for this hard-to-find talent. The shortage isn't going away anytime soon, just because mastering these skills takes years of work.

The industry is not willing to wait. Companies are developing all sorts of tools that will make it easier for any operation to build its own AI software, including things like image and speech recognition services and online chatbots.

"We are following the same path that computer science has followed with every new type of technology," said Joseph Sirosh, a vice president at Microsoft, which recently unveiled a tool to help coders build deep neural networks, a type of computer algorithm that is driving much of the recent progress in the AI field. "We are eliminating a lot of the heavy lifting."

This is not altruism. Researchers like Dean believe that if more people and companies are working on artificial intelligence, it will propel their own research. At the same time, companies like Google, Amazon and Microsoft see serious money in the trend that Sirosh described. All of them are selling cloud-computing services that can help other businesses and developers build AI.

"There is real demand for this," said Matt Scott, a co-founder and the chief technical officer of Malong, a startup in China that offers similar services. "And the tools are not yet satisfying all the demand."

This is most likely what Google has in mind for AutoML, as the company continues to hail the project's progress. Google's chief executive, Sundar Pichai, boasted about AutoML last month while unveiling a new Android smartphone. Eventually, the Google project will help companies build systems with artificial intelligence even if they don't have extensive expertise, Dean said.

Today, he estimated, no more than a few thousand companies have the right talent for building AI, but many more have the necessary data. "We want to go from thousands of organisations solving machine learning problems to millions," he said.

Google is investing heavily in cloud-computing services - services that help other businesses build and run software - which it expects to be one of its primary economic engines in the years to come. And after snapping up such a large portion of the world's top AI researchers, it has a means of jump-starting this engine.

Neural networks are rapidly accelerating the development of AI. Rather than building an image-recognition service or a language translation app by hand, one line of code at a time, engineers can much more quickly build an algorithm that learns tasks on its own.

By analysing the sounds in a vast collection of old technical support calls, for instance, a machine-learning algorithm can learn to recognise spoken words.

But building a neural network is not like building a website or some run-of-the-mill smartphone app. It requires significant math skills, extreme trial and error, and a fair amount of intuition. Jean-François Gagné, chief executive of an independent machine-learning lab called Element AI, refers to the process as "a new kind of computer programming."

Automating the artificial

In building a neural network, researchers run dozens or even hundreds of experiments across a vast network of machines, testing how well an algorithm can learn a task like recognising an image or translating from one language to another. Then they adjust particular parts of the algorithm over and over again, until they settle on something that works. Some call it a "dark art," just because researchers find it difficult to explain why they make particular adjustments.

But with AutoML, Google is trying to automate this process. It is building algorithms that analyse the development of other algorithms, learning which methods are successful and which are not. Eventually, they learn to build more effective machine learning. Google said AutoML could now build algorithms that, in some cases, identified objects in photos more accurately than services built solely by human experts.

Barret Zoph, one of the Google researchers behind the project, believes that the same method will eventually work well for other tasks, like speech recognition or machine translation. This is not always an easy thing to wrap your head around. But it is part of a significant trend in AI research. Experts call it "learning to learn" or "meta-learning."

Many believe such methods will significantly accelerate the progress of AI in both the online and physical worlds. At the University of California, Berkeley, researchers are building techniques that could allow robots to learn new tasks based on what they have learned in the past.

"Computers are going to invent the algorithms for us, essentially," said a Berkeley professor, Pieter Abbeel. "Algorithms invented by computers can solve many, many problems very quickly - at least that is the hope."

This is also a way of expanding the number of people and businesses that can build artificial intelligence. These methods will not replace AI researchers entirely. Experts, like those at Google, must still do much of the important design work. But the belief is that the work of a few experts can help many others build their own software.

Renato Negrinho, a researcher at Carnegie Mellon University who is exploring technology similar to AutoML, said this was not a reality today but should be in the years to come. "It is just a matter of when," he said.