Deep Learning

Deep learning has become one of the defining technologies of modern artificial intelligence because it allows computers to learn useful representations directly from data rather than relying entirely on rules or features designed by humans.

A traditional machine-learning system might be given carefully selected variables that engineers believe are important for solving a problem. A deep-learning system can often learn many of those useful features for itself. When analysing photographs, for example, different layers of a neural network can progressively learn to recognise edges, shapes, textures, objects and increasingly complex relationships between them.

This ability to learn hierarchical representations has made deep learning particularly effective when working with enormous quantities of unstructured information such as text, images, audio and video.

What is deep learning?

Deep learning is a subset of machine learning, which is itself a subset of artificial intelligence.

The term describes machine-learning techniques based primarily on artificial neural networks containing multiple computational layers. The word "deep" refers to the depth of the network, meaning the number of layers through which information can be processed, rather than the intelligence or sophistication of the system itself.

These layers contain interconnected computational units that transform information as it moves through the network. During training, the system adjusts large numbers of numerical parameters so that its outputs become progressively better at matching the objective it has been given.

Modern deep-learning models can contain millions, billions or even trillions of parameters. Those parameters do not contain traditional hand-written rules. Instead, they encode statistical relationships learned during training.

How does deep learning work?

A deep-learning model begins with data.

Depending on the application, that data might consist of photographs, sentences, audio recordings, sensor readings, financial information, scientific measurements or many other forms of information.

The data passes through a neural network containing multiple layers. Each layer performs mathematical transformations on the information it receives before passing the resulting representation to subsequent layers.

During training, the model produces an output and compares it against an objective. An optimisation process then calculates how the model's internal parameters should change to reduce its error.

One of the fundamental techniques used to achieve this is backpropagation, which allows information about an error to be propagated backwards through the network so that relevant parameters can be adjusted.

This process is repeated across large numbers of examples. Over time, the model develops internal representations that allow it to make predictions or generate outputs when it encounters new inputs.

Once training has been completed, the model can enter inference, where it applies what it has learned to new information.

Why is deep learning important?

Deep learning changed the development of artificial intelligence because it dramatically improved computers' ability to work with forms of information that had historically been difficult to represent using conventional software.

Images are a good example. Writing explicit rules describing every possible appearance of a dog, pedestrian, tumour or road sign is practically impossible. Deep-learning systems can instead learn useful visual patterns from examples.

A similar transformation has occurred in language. Rather than attempting to manually encode every grammatical, semantic and contextual relationship found in human communication, modern deep-learning models can learn statistical representations of language from extremely large datasets.

This approach helped enable many of the breakthroughs that now define contemporary AI, from real-time speech transcription and medical imaging to generative image systems and large language models such as those used by conversational AI assistants.

Deep learning vs machine learning

Deep learning and machine learning are closely related, but they are not interchangeable terms.

Machine learning is the broader field of developing systems that learn patterns from data. It includes approaches such as decision trees, linear models, support vector machines, clustering algorithms, ensemble methods and neural networks.

Deep learning refers specifically to approaches built around deep neural networks.

Traditional machine-learning techniques can perform extremely well on relatively structured datasets and may require much less computing power or training data. Deep learning tends to become particularly valuable when dealing with highly complex or unstructured data, and when very large datasets and substantial computational resources are available.

Every deep-learning system is therefore a machine-learning system, but not every machine-learning system uses deep learning.

Deep learning vs artificial intelligence

Artificial intelligence is the broadest of the three concepts.

AI describes technologies designed to perform tasks associated with intelligent behaviour. Machine learning is one approach to building those systems, while deep learning is a specialised area within machine learning.

The relationship can be understood as a hierarchy:

Artificial Intelligence → Machine Learning → Deep Learning

This distinction matters because the terms are frequently used as though they describe the same technology. An AI system does not necessarily use machine learning, and a machine-learning system does not necessarily use deep neural networks.

What is a neural network?

A neural network is a computational system made up of interconnected mathematical units commonly called artificial neurons or nodes.

Information enters the network through an input layer, passes through one or more intermediate or hidden layers, and eventually produces an output.

Deep learning generally uses neural networks containing many such layers.

Each connection within the network is associated with numerical parameters that influence how information is transformed. Training changes these parameters until the network becomes better at performing its intended task.

Although the terminology was originally inspired by biological neurons, artificial neural networks should not be understood as direct simulations of the human brain. They are mathematical systems whose structure is only loosely inspired by biological neural networks.

What are the main types of deep-learning architecture?

Different neural-network architectures have been developed for different kinds of information and computational problems.

Convolutional neural networks (CNNs) became particularly important in computer vision because they are highly effective at identifying spatial patterns within images.

Recurrent neural networks (RNNs) were developed to process sequential information and became widely used for language, speech and time-series applications. Variants such as Long Short-Term Memory networks, or LSTMs, were designed to handle longer-term relationships within sequences.

Transformers use an attention-based architecture that allows models to identify relationships between different parts of their input efficiently. Transformers now underpin many of the world's most advanced language, multimodal and generative AI systems.

Autoencoders learn compressed representations of data and can be used for applications including dimensionality reduction, anomaly detection and generative modelling.

These architectures can also be combined, modified and scaled considerably depending on the problem being addressed.

What is deep learning used for?

Deep learning is now used across a large range of consumer, scientific and industrial applications.

Computer-vision systems use it to identify objects, interpret medical scans, inspect manufactured products and understand satellite imagery. Speech systems use it for transcription, speaker recognition and voice interfaces. Language systems use deep learning for translation, search, summarisation, question answering and text generation.

It is also used in recommendation engines, robotics, drug discovery, autonomous vehicles, fraud detection, weather forecasting, scientific modelling and predictive systems.

Perhaps its most visible recent application has been generative AI, where deep neural networks learn statistical structures within enormous datasets and use those representations to generate new text, images, audio, video or other forms of content.

What does deep learning have to do with generative AI?

Most of today's major generative AI systems are based on deep learning.

Large language models use deep neural networks, most commonly transformer architectures, trained across enormous collections of textual and other data. Image-generation systems similarly use deep-learning techniques to learn relationships between visual information, concepts and language.

Deep learning therefore provides much of the underlying computational architecture for generative AI.

The terms should not be confused, however. Deep learning describes a family of machine-learning techniques, while generative AI describes systems whose purpose includes generating new outputs.

Deep learning can be used for both generative and non-generative tasks.

Is ChatGPT an example of deep learning?

Yes. ChatGPT is built using large language models based on deep-learning techniques.

Modern large language models generally use transformer neural networks containing many layers and very large numbers of parameters. During training, these systems learn statistical relationships within language and other forms of information.

Deep learning is therefore one of the foundational technologies that makes systems such as ChatGPT possible.

Deep learning and large language models are not synonymous, however. LLMs are one particular application of deep learning, while deep-learning techniques are used across many other areas of artificial intelligence.

How is a deep-learning model trained?

Training a deep-learning model can involve processing enormous quantities of data while repeatedly adjusting the network's parameters.

The precise training process depends on the model and task. Supervised learning uses labelled examples containing known target outputs. Self-supervised learning allows systems to derive training signals from the structure of the data itself and has become particularly important for large foundation models. Reinforcement learning can train systems through feedback associated with actions and outcomes.

Training modern frontier AI models can require extensive computing infrastructure, including large clusters of specialised processors such as GPUs and AI accelerators.

Smaller deep-learning models, however, can be trained and operated with considerably fewer resources.

What are the advantages of deep learning?

One of deep learning's greatest strengths is its ability to discover useful representations within highly complex data.

Rather than requiring developers to identify every relevant feature manually, a sufficiently capable neural network can learn representations during training. This has made deep learning exceptionally powerful for images, natural language, audio and other high-dimensional information.

Deep-learning models can also improve substantially as the amount of useful training data, computing power and model capacity increases. This ability to scale has been central to many of the advances in AI seen over the past decade.

What are the limitations of deep learning?

Deep learning can also introduce significant limitations.

Large models can require extraordinary quantities of data, energy and computing infrastructure to train. Their internal decision-making processes can be difficult to interpret because useful information is distributed across enormous numbers of interacting parameters rather than expressed through easily readable rules.

Models can also inherit biases or inaccuracies from their training data, fail when conditions differ significantly from those represented during training, or produce outputs that appear convincing despite being incorrect.

A further limitation is that conventional deep-learning models do not necessarily continue learning simply because they encounter new information after deployment. Many are trained, deployed and then remain largely static until they are updated, fine-tuned or retrained.

This distinction is becoming increasingly important as AI research explores systems capable of continual learning, where intelligence can adapt to changing environments rather than remaining fixed after initial training.

Does deep learning mean an AI understands something?

Not necessarily.

A deep-learning model can learn extremely sophisticated statistical representations and perform tasks that appear to require significant intelligence. This does not by itself establish that the system understands information in the same way a human does.

Questions surrounding reasoning, understanding, consciousness and intelligence remain active areas of scientific and philosophical debate.

For practical purposes, deep learning describes how a system learns and represents information computationally. It does not provide a definitive answer about whether that system possesses human-like understanding.

Deep learning in modern AI

Deep learning now sits underneath a remarkable proportion of the modern AI industry.

The rapid development of transformers, foundation models, large language models and multimodal systems has pushed neural networks from specialised machine-learning tools into technologies used by hundreds of millions of people.

Yet deep learning is not synonymous with artificial intelligence itself. It is one approach to creating intelligent systems, albeit an exceptionally important one.

Understanding that distinction becomes increasingly useful as AI develops beyond today's predominantly pre-trained models towards systems designed to reason, adapt and potentially continue learning after deployment.