Researchers have embraced transfer learning to address algorithm challenges

Advanced machines never fail to leave men in awe. But only researchers who worked behind the machines know how much time, cost and data it took to become a stage stealer. Training an algorithm that employs various features in a machine is quite nerve-wracking. But tech geeks have found a solution using transfer learning. Besides, companies are also unveiling a mixture of technologies like deep learning neural networks and machine learning to come up with futuristic machines.

We are often surrounded by the myth that number-crunching gets cheaper all the time. According to Moore’s law, the number of components that can be squeezed onto a microchip of a given size can double every two years with the amount of computational power available at a given cost. This idea might suggest the opinion that the cost of training a machine is falling. But that is not true. Just because data is everywhere and is easily available doesn’t mean they are open to use and inexpensive in any way. Even when the data is open for accessibility, training an algorithm takes much more effort than any other computational process. Industry analysts anticipate that worldwide spending on artificial intelligence will reach US$100 billion in 2024, double of what it is today.

The advantage of machine learning and artificial intelligence algorithm is that they can easily understand information, act and interact with our environment in the most natural and human way possible. But the performance of the models depends highly on the calculation power allocated, and the quantity and quality of data. A study conducted by Dimensional Research unravels that around 96% of organizations run into a problem with training data quality and quantity. Besides, the study also claims that most machine learning model projects require more than 100,000 data samples to perform effectively. A machine learning system is still programmed with standard one-and-zero logic, but it can modify its behavior to meet specialized goals based on patterns it discovers in the sample data. Henceforth, machine learning algorithm needs to be trained with good data, which means data is optimized according to the issue you are dealing with. Fortunately, transfer learning can help as it takes knowledge gained from a pre-trained model that was used to solve a specific task and applies it to a different, but a similar problem within the same domain. Additionally, a mixed array of technologies like deep learning neural networks and machine learning are also making the training process less burdening.

Transfer learning addresses algorithm challenges

Transfer learning is a machine learning method where a model developed for a task is reused as the starting point for a model on a second task. The technology is seen as a popular approach in deep learning where pre-trained models are used as the starting point on computer vision and natural language processing tasks, given the vast compute and time resources required to develop neural network models on these problems and from the huge jumps in a skill that they provide on related problems.

Remarkably, with the help of transfer learning, instead of starting the learning process from scratch, you start from patterns that have been learned when solving a different problem. This way, you leverage previous learning and avoid starting from nothing. Transfer learning is usually expressed through the use of pre-trained models that were trained on a large dataset to solve a problem similar to the one that we want to solve. One of the well-known examples of transfer learning is GPT-3, the largest natural language machine learning model ever built. GPT-3 is a language prediction model where an algorithm structure is designed to take one piece of language and transform it into what it predicts is the most useful following piece of language for the user. Behind the mechanism are machine learning, deep learning and transfer learning technologies that help the model to produce humanlike predictive text.

Other than this, big tech conglomerates like Microsoft, AWS, NVIDIA, IBM, etc. have leveraged the help of transfer learning toolkits to remove the burden of building models from scratch, address the data quality and quantity challenges and expedite production machine learning.

The post UNRAVELLING TRANSFER LEARNING TO MAKE MACHINES MORE ADVANCED appeared first on Artificial Intelligence.

This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding artificial intelligence.

Today, artificial intelligence programs can recognize faces and objects in photos and videos, transcribe audio in real-time, detect cancer in x-ray scans years in advance, and compete with humans in some of the most complicated games.

Until a few years ago, all these challenges were either thought insurmountable, decades away, or were being solved with sub-optimal results. But advances in neural networks and deep learning, a branch of AI that has become very popular in the past few years, has helped computers solve these and many other complicated problems.

Unfortunately, when created from scratch, deep learning models require access to vast amounts of data and compute resources. This is a luxury that many can’t afford. Moreover, it takes a long time to train deep learning models to perform tasks, which is not suitable for use cases that have a short time budget.

Fortunately, transfer learning, the discipline of using the knowledge gained from one trained AI model to another, can help solve these problems.

The cost of training deep learning models

Deep learning is a subset of machine learning, the science of developing AI through training examples. The concepts and science behind deep learning and neural networks is as old as the term “artificial intelligence” itself. But until recent years, they had been largely dismissed by the AI community for being inefficient.

The availability of vast amounts of data and compute resources in the past few years have pushed neural networks into the limelight and made it possible to develop deep learning algorithms that can solve real world problems.

To train a deep learning model, you basically must feed a neural network with lots of annotated examples. These examples can be things such as labeled images of objects or mammograms scans of patients with their eventual outcomes. The neural network will carefully analyze and compare the images and develop mathematical models that represent the recurring patterns between images of a similar category.

There already exists several large open-source datasets such as ImageNet, a database of more than 14 million images labeled in 22,000 categories, and MNIST, a dataset of 60,000 handwritten digits. AI engineers can use these sources to train their deep learning models.

However, training deep learning models also requires access to very strong computing resources. Developers usually use clusters of CPUs, GPUs or specialized hardware such as Google’s Tensor Processors (TPUs) to train neural networks in a time-efficient way. The costs of purchasing or renting such resources can be beyond the budget of individual developers or small organizations. Also, for many problems, there aren’t enough examples to train robust AI models.

The post What is transfer learning? appeared first on Artificial Intelligence.