Outline of deep learning
From HandWiki
Short description: Overview of and topical guide to deep learning
The following outline is provided as an overview of, and topical guide to, deep learning:
Deep learning is a subfield of machine learning and artificial intelligence based on artificial neural networks with multiple processing layers. It emphasizes representation learning and is widely used in areas such as computer vision, natural language processing, speech recognition, recommender systems, robotics, and generative artificial intelligence.[1][2][3]
Ways to categorize deep learning
- A field of study
- A branch of artificial intelligence
- A subfield of machine learning
- A subfield of computer science
- A form of representation learning
- A class of methods based on artificial neural networks
- An approach used in computational statistics
History
Precursors
Milestones
- LeNet
- Long short-term memory
- Deep belief network
- AlexNet
- Sequence to sequence learning
- Generative adversarial network
- Residual neural network
- Transformer
- BERT
- Generative pre-trained transformer
- Diffusion model
Related histories
- History of artificial intelligence
- History of machine learning
- Timeline of machine learning
Core concepts
Learning settings
- Supervised learning
- Unsupervised learning
- Self-supervised learning
- Semi-supervised learning
- Reinforcement learning
- Transfer learning
- Multitask learning
- Multimodal learning
- Online machine learning
- Continual learning
Common tasks
- Image classification
- Object detection
- Image segmentation
- Automatic speech recognition
- Neural machine translation
- Question answering
- Automatic summarization
- Text-to-image model
- Protein structure prediction
Architectures
Feedforward and convolutional architectures
- Feedforward neural network
- Multilayer perceptron
- Convolutional neural network
- Radial basis function network
- Residual neural network
- U-Net
Recurrent and sequence architectures
- Recurrent neural network
- Long short-term memory
- Gated recurrent unit
- Sequence to sequence learning
- Recursive neural network
Representation-learning architectures
- Autoencoder
- Denoising autoencoder
- Sparse autoencoder
- Variational autoencoder
- Restricted Boltzmann machine
- Deep belief network
Attention and transformer architectures
Generative and probabilistic architectures
- Autoregressive model
- Diffusion model
- Energy-based model
- Generative adversarial network
- Mixture of experts
Graph and memory architectures
- Graph neural network
- Graph convolutional network
- Siamese network
- Neural Turing machine
- Memory network[4]
- Echo state network
- Capsule neural network
Neural network components and techniques
- Artificial neuron
- Activation function
- Embedding
- Convolution
- Pooling layer
- Attention
- Batch normalization
- Layer normalization
- Residual connections
Training and optimization
- Backpropagation
- Gradient descent
- Stochastic gradient descent
- Adam optimization[5]
- Learning rate
- Loss function
- Regularization
- Batch normalization
- Data augmentation
- Transfer learning
- Knowledge distillation
- Ensemble learning
- Curriculum learning
Datasets and benchmarks
- CIFAR-10
- ImageNet
- MNIST database
- Common Objects in Context (COCO)[6]
- General Language Understanding Evaluation (GLUE) benchmark[7]
- LibriSpeech[8][9]
- SQuAD[10][11]
Applications
Computer vision
- Computer vision
- Facial recognition system
- Image classification
- Image segmentation
- Medical imaging
- Object detection
- Optical character recognition
Natural language processing
- Automatic summarization
- Chatbot
- Information retrieval
- Large language model
- Natural language processing
- Neural machine translation
- Question answering
- Sentiment analysis
Speech and audio
- Automatic speech recognition
- Music information retrieval
- Speaker recognition
- Speech synthesis
Science and medicine
Robotics and control
- Autonomous car
- Computer game bot
- Control theory
- Robotics
Recommendation, search, and forecasting
- Anomaly detection
- Forecasting
- Fraud detection
- Recommender system
- Search engine
Generative artificial intelligence
- Deepfake
- Generative artificial intelligence
- Large language model
- Speech synthesis
- Text-to-image model
Computer graphics and video games
- Deep Learning Anti-Aliasing (DLAA)
- Deep Learning Super Sampling (DLSS)
Hardware
- AMD Instinct
- AMD XDNA
- Application-specific integrated circuit
- Deep learning processor, Neural processing unit (NPU), or Neural Engine
- Field-programmable gate array
- General-purpose computing on graphics processing units (GPGPU)
- Graphics processing unit
- NVIDIA Deep Learning Accelerator (NVDLA)
- Tensor processing unit
- Vision processing unit
- Wafer-scale integration[12]
Supporting software platforms
Software
Open-source frameworks and libraries
Neural network software
Platforms, tools, and deployment
Algorithms for deep learning and neural networks
- Backpropagation
- Conjugate gradient method
- Generalized Hebbian algorithm
- Gradient descent
- Levenberg–Marquardt algorithm
- Perceptron
- Quasi-Newton method
- Wake-sleep algorithm[17]
Methods and related topics
Representation and metric learning
- Contrastive learning
- Embedding
- Feature learning
- Manifold learning
- Metric learning
Generative modeling
- Autoregressive model
- Diffusion model
- Generative adversarial network
- Generative model
- Variational inference
Efficient and scalable deep learning
Reliability, safety, and interpretability
- Adversarial machine learning
- AI alignment
- Algorithmic bias
- Catastrophic forgetting
- Differential privacy
- Explainable artificial intelligence
- Federated learning
- Hallucination (artificial intelligence)
Conferences and workshops
- Annual Meeting of the Association for Computational Linguistics
- Conference on Computer Vision and Pattern Recognition
- Conference on Neural Information Processing Systems
- International Conference on Computer Vision
- International Conference on Learning Representations
- International Conference on Machine Learning
Organizations
Research laboratories and institutions
- Allen Institute for AI
- Alberta Machine Intelligence Institute
- European Laboratory for Learning and Intelligent Systems
- Google DeepMind
- Meta AI
- Mila
- Microsoft Research
- Vector Institute
Companies
Publications
Books
- Deep Learning[18] – Ian Goodfellow and Yoshua Bengio
- Neural Networks and Deep Learning[19] – Michael Nielsen
- Perceptrons – Marvin Minsky and Seymour Papert
Journals
- IEEE Transactions on Neural Networks and Learning Systems
- Neural Networks
- Neural Computation
Influential persons
- Alex Graves
- Alex Krizhevsky
- Andrew Ng
- Andrej Karpathy
- Ashish Vaswani
- Christopher Bishop
- Demis Hassabis
- Fei-Fei Li
- Geoffrey Hinton
- Ian Goodfellow
- Ilya Sutskever
- John Hopfield
- Jürgen Schmidhuber
- Noam Shazeer
- Oriol Vinyals
- Paul Werbos
- Quoc V. Le
- Ruslan Salakhutdinov
- Sepp Hochreiter
- Seppo Linnainmaa
- Terry Sejnowski
- Yann LeCun
- Yoshua Bengio
See also
- Artificial intelligence
- Artificial neural network
- Generative artificial intelligence
- Glossary of artificial intelligence
- Lists of open-source artificial intelligence software
- Machine learning
- Neural network software
- Outline of artificial intelligence
- Outline of computer vision
- Outline of machine learning
- Outline of robotics
References
- ↑ LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey (2015-05-27). "Deep learning". Nature 521 (7553): 436–444. doi:10.1038/nature14539. PMID 26017442. Bibcode: 2015Natur.521..436L. https://hal.science/hal-04206682.
- ↑ Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron (2016). Deep Learning. The MIT Press. ISBN 978-0-262-03561-3. https://www.deeplearningbook.org/.
- ↑ Schmidhuber, Jürgen (January 2015). "Deep learning in neural networks: An overview". Neural Networks 61: 85–117. doi:10.1016/j.neunet.2014.09.003. PMID 25462637. Bibcode: 2015NN.....61...85S.
- ↑ Biggs, David; Nuttall, Andrew (2015). Neural Memory Networks (Report). CS229 Final Report. https://cs229.stanford.edu/proj2015/367_report.pdf. Retrieved 17 April 2026.
- ↑ Akash Ajagekar (2021). "Adam". https://optimization.cbe.cornell.edu/index.php?title=Adam.
- ↑ "COCO: Common Objects in Context". https://cocodataset.org/#home.
- ↑ "GLUE Benchmark". https://gluebenchmark.com/.
- ↑ "LibriSpeech ASR corpus". https://www.openslr.org/12.
- ↑ "LibriSpeech-Long". Google DeepMind. 2024. https://github.com/google-deepmind/librispeech-long.
- ↑ "The Stanford Question Answering Dataset". https://rajpurkar.github.io/SQuAD-explorer/.
- ↑ "Stanford Question Answering Dataset". https://www.kaggle.com/datasets/stanfordu/stanford-question-answering-dataset.
- ↑ Moore, Samuel K. (1 January 2020). "Cerebras's Giant Chip Will Smash Deep Learning's Speed Barrier". IEEE Spectrum. https://spectrum.ieee.org/cerebrass-giant-chip-will-smash-deep-learnings-speed-barrier. Retrieved 17 April 2026.
- ↑ Li, Ming; Bi, Ziqian; Wang, Tianyang; Wen, Yizhu; Niu, Qian; Song, Xinyuan; Jiang, Zekun; Liu, Junyu; Peng, Benji; Zhang, Sen; Pan, Xuanhe; Xu, Jiawei; Wang, Jinlang; Chen, Keyu; Caitlyn Heqi Yin; Feng, Pohsun; Liu, Ming (2024-10-08). "Deep Learning and Machine Learning with GPGPU and CUDA: Unlocking the Power of Parallel Computing". arXiv:2410.05686 [cs.DC].
- ↑ "Accelerated PyTorch training on Mac". Apple. https://developer.apple.com/metal/pytorch/.
- ↑ "GitHub - tsawler/go-metal: A high-performance deep learning library for Go that leverages Apple's Metal for GPU acceleration on Apple Silicon.". https://github.com/tsawler/go-metal.
- ↑ "Accelerating the Machine Learning Lifecycle with MLflow". https://github.com/mlflow/mlflow.
- ↑ Quesada, Alberto (28 October 2019). "5 algorithms to train a neural network". Artelnics. https://www.neuraldesigner.com/blog/5_algorithms_to_train_a_neural_network/.
- ↑ Janishar Ali. "MIT Deep Learning Book (beautiful and flawless PDF version)". https://github.com/janishar/mit-deep-learning-book-pdf.
- ↑ Nielsen, Michael (2015). "Neural Networks and Deep Learning". Determination Press. http://neuralnetworksanddeeplearning.com/.
 |  |
