Neural network

Definition

An artificial neural network (ANN) is a model in AI that is loosely inspired by the human brain. It consists of many simple computational units, the artificial neurons, which are interconnected. These neurons are usually organized in layers, with each unit receiving numerical values (activations), performing a calculation and passing the result on to the next neurons. By adjusting the connection weights between the neurons during a training process, the network learns to map certain inputs to desired outputs.

Structure

Typically, neurons are arranged in input layer, hidden layers and output layer (if it is a feedforward network). Each connection has a weight that indicates how strongly an input signal influences the next neuron. A neuron first calculates the weighted sum of all incoming signals and then applies an activation function (e.g. sigmoid, ReLU) to it to determine the output. During training, a learning algorithm (e.g. gradient descent with backpropagation) adjusts the weights until the outputs match the expected results as closely as possible. This is also referred to as the network learning the parameters. The more hidden layers a network has, the deeper it is – this is where the term deep learning comes into play for very extensive neural networks with many layers.

Types of neural networks

There are different types of architecture:

• Feedforward networks: Information only flows in one direction (from input to output). Example: Multi-layer perceptron for classification.

• Recurrent neural networks (RNNs): Have feedback loops so that neurons also influence previous states (useful for sequence data).

• Convolutional neural networks (CNNs): Contain convolutional layers and are specialized for spatial data such as images.

• Graph neural networks (GNNs): Work on graph structures instead of regular grid data.

• Transformer networks: Rely on self-awareness instead of classical layers (relevant for language models).

Applications

Neural networks are universal approximators – they can theoretically learn almost any mathematical function, which makes them very versatile. In practice, they are used in image recognition, speech processing, control, data generation and much more. Whenever patterns in data need to be recognized or complex non-linear relationships need to be modelled, neural networks are often the first choice. Their disadvantage is their low explainability – it is often difficult to understand why a large network makes a certain decision. Nevertheless, neural networks form the foundation of most modern AI breakthroughs.