Deep learning is a type of machine learning that uses layers of artificial neural networks to mimic the behavior of the human brain. Each synthetic neuron is very simple, but the vast numbers used create powerful data structures.
They can be used to process unstructured input of various types, primarily images and text. The deep learning algorithms identify features of interest and, in so doing, assign weights to each of the artificial neurons involved. The weights are adjusted as more input data is processed, with the successively refined results being described as learning. The style of learning can take several forms: supervised, unsupervised and reinforcement, depending on the nature of any feedback provided during training.
Some of the most popular forms of artificial intelligence rely on deep learning, including chatbots, virtual assistants and self-driving cars.
Neural Networks
Neural networks for deep learning have several hidden layers between input and output at the extremities. This is the origin of the term "deep".
© Interaction Design Foundation, CC BY-SA 4.0
Neural networks consist of interconnected artificial neurons similar in structure to the human brain. Deep learning uses layers of these networks to discover patterns in data and make decisions based on them. Unlike earlier forms of machine learning, pattern discovery in neural networks is learned rather than programmed. This can lead to the unfortunate consequence that it is not always clear how predictions have been derived.
Deep Learning vs Machine Learning
Although deep learning is a form of machine learning, there are some important differences.
Deep learning…
Is trained using large amounts of data.
Makes use of feedback in the environment, including past mistakes.
Works with unstructured data, for example, images, text and audio.
Can discover complex, non-linear relationships in the data.
Needs large numbers of GPUs (graphics processing units) that specialize in parallel processing.
Machine learning …
Can work with smaller data sets.
Requires more human intervention to correct and improve the algorithms.
Often needs additional pre- and post-processing to handle unstructured data.
Is more simplistic in terms of the relationships it can discover in the data.
Does not rely on parallel processing so can be trained with conventional CPUs (central processing units).
History of Deep Learning
The concept of artificial neurons, which are fundamental to deep learning, was first proposed in the 1940s by McCulloch and Pitts (see the illustration below). However, deep learning itself did not get much attention until later in the 20th century, when some breakthroughs were made in…
Recurrent neural networks: A type of network that uses the output of one pass as the input to the next, making them recursive or recurrent. This approach is important in language processing since the meaning of a sentence is dependent on the sequence of words.
Convolutional neural networks: A system for feature recognition loosely based on the visual processing of the human brain.
Backpropagation: Adjusting the weights in a neural network based on the error rate of the previous pass.
In the early 21st century, advances in speech, language and image processing made deep learning possible. The availability of large datasets also contributed to the training of neural networks which underpin deep learning algorithms. More recently, advances in parallel processing, in the form of Graphical Processing Units (GPUs), significantly reduced the time taken for training neural networks on very large datasets. Originally, these would have been the same GPUs found in gaming cards, but versions specialized for neural networks have since been developed.
Diagram of a human neuron. Signals are received through the dendrites (left) and sent out through the axon (right).
CC BY-SA 4.0 ShareAlike (https://creativecommons.org/licenses/by-sa/4.0/)
A graphic representation of the McCulloch-Pitts neuron. Signals are received by g (left). f (right) makes a decision whether to provide an output. All signals are binary (either 0 or 1).
© Interaction Design Foundation, CC BY-SA 4.0
Deep learning is now receiving significant attention, with applications in many fields. It is constantly improving and is making substantial contributions to artificial intelligence generally, as described below.
Applications of Deep Learning
Chatbots
Using neural networks, chatbots can convincingly mimic human conversation. In February 2024, researchers established that some were indistinguishable from humans by passing the Turing test. (The test was devised by Alan Turing in 1950 to measure the "intelligence" of machines.)
Chatbots can learn from interactions with people or from other sources, such as the vast body of conversations found on the World Wide Web. They can understand natural language and produce meaningful responses. Modern deep-learning chatbots are very sophisticated and can process complex questions and instructions. They learn from interactions over time and improve themselves when made aware of mistakes.
A chatbot explaining how they learn.
Microsoft Copilot, Fair Use
Virtual Assistants
Amazon Echo Dot with Alexa virtual assistant.
Virtual assistants (VAs) like Alexa, Google Assistant and Siri increasingly use deep learning to perform complex tasks. These include understanding spoken language, generating responses and learning from user feedback. Some related applications, such as Google Lens, use deep learning for image processing and object recognition. We are likely to see similar features built into virtual assistants as the technology improves.
Self-Driving Cars
Self-driving cars use deep learning.
In many respects, self-driving cars act as virtual assistants (see above) but require additional capabilities to deal with the complexities of driving itself:
Computer vision: Deep learning enables self-driving cars to analyze images and videos and provide relevant information or actions based on them. For example, modern cars process road signs to establish speed limits. Self-driving cars go further, using deep learning algorithms to identify important objects such as pedestrians and other vehicles.
End-to-end learning: Self-driving cars use deep learning to enhance their understanding of the environment by taking advantage of raw sensor data, such as camera images. Otherwise, they would need to rely on pre-programmed features or extra stages of processing.
