If machine learning is real, then why don’t we all use artificial intelligence?

The hype surrounding an area of artificial intelligence (AI) known as machine learning is for many different reasons justified. We all know the more-public examples of success with machine learning with IBM’s Watson becoming a Jeopardy champion, Google’s autonomous cars, and more-recently, a computer (or should we say a data center of computers) beating the world Go champion.

While breathless articles focus on the promise or danger of AI and machine learning, what isn’t as widely discussed is the level of effort it takes to train a machine to the point of making it genuinely useful. Machine learning isn’t an off-the-shelf product or software that you can download and train although software with machine-learning capabilities is definitely available to try out.

Machine learning is an area that is composed of over 30 different algorithms “types” and among those, there exist many variants. For instance, the words “neural network” do not begin to describe the actual way the algorithm works; there are feed-forward and backward propagation neural networks and within backward propagation, there are many variants. Each of these algorithms can theoretically be grouped into different categories based upon the nature of the algorithm or how it can be applied. Each has its own set of strengths and weaknesses.

Solving Specific Challenges

Any application of machine learning algorithms must first come with reasonable knowledge of the efficacy of the algorithm to solve a particular problem. Once we understand the proper application, we then have to understand the nature of the input required. Some of the input requirements will be understood from knowledge of the algorithm itself or its strengths and weaknesses. For instance, if a support vector machine is selected, we often can determine that the variance of input data should be relatively narrow, the more complex the input becomes (e.g., the more classes expected, the greater the error generated). Conversely, modern neural networks do a better job of handling diversity of the input with the ability to identify features and narrow-down the relevance of each.

A lot of effort is made to understand what is the proper input that will produce the desired results. For instance, it might appear straightforward to determine the input that is needed for an invoice data extraction task. However, it is more complex given the number of possible variances of that invoice, image quality, and many other variables that may (or may not be) correlated with good results. Having a good understanding of what type of input data is required can be a significant challenge.

Adjusting and Refining the Model

Assuming we’ve satisfied the first two hurdles—the efficacy of the algorithm and the nature of the required input—a third remains: adjusting and refining the model. There are some software packages with the algorithms already adjusted to a particular input. Take, for instance, IBM Wason that has four categories of APIs for various types of content. Each one of these has been trained on a particular input stream and tuned to output answers at a given error rate. These APIs can continue to be tuned through submission of more data, but they did not just learn and adjust by themselves. They required a fairly sophisticated staff who are knowledgeable about how the algorithms are used and can adjust them. Even though machine learning algorithms can be downloaded and tried, it takes good knowledge of how they work in order to obtain the best performance.

Lastly, it is not uncommon to combine different learning algorithms in order to improve overall results. In fact, Parascript’s core technologies involve often five or more learning algorithms that are combined with a sophisticated set of what are called “voting algorithms”. The ability to understand multiple approaches and then synthesize results is not for the faint of heart and not a common skill set.

How This Applies

So since machine learning is real, why aren’t we all using AI? The answer is that if you are searching for a general purpose AI product to be used by the public or even by software developers that automates solutions for a wide variety of problems, then you are out of luck. Any vendor promising this feat should be treated with serious skepticism. Such a technology doesn’t exist, and the skills needed to implement such a capability don’t exist either. A single generic platform that can learn to reliably extract data from any document is still as far fetched as a single platform that can beat both a grandmaster of chess and a master of the Chinese game go.

However, if you want a platform that can be trained on different independent tasks within a certain domain, then the promise of automation powered by machine learning is definitely achievable. We call this applied AI and it is today’s killer app.