Understanding forward propagation, backpropagation, and gradient descent through a neural network built using Python, NumPy, and mathematics.
Introduction
Making a neural network produce predictions is relatively straightforward when using modern deep-learning frameworks. Understanding how the network actually learns is a different challenge.
How does it calculate its output? How does it measure an incorrect prediction? And how do derivatives determine which weights and biases need to change?
I wanted to understand these operations without hiding them behind automatic differentiation.
So I built a fully trainable neural network from scratch using Python and NumPy, implementing the mathematical operations myself without relying on PyTorch or TensorFlow.
The network follows a 2 → 3 → 1 architecture and learns the XOR function.
Despite its small size, the implementation brings together several fundamental concepts: matrix multiplication, activation functions, loss functions, the chain rule, backpropagation, and gradient descent.
In this article, I’ll break down these concepts, explain the mathematics behind them, and connect the equations to their actual Python implementation.
AI-assisted writing and visuals.
The goal isn’t just to build a working neural network. It’s to understand how it learns.
1. Why XOR?
XOR (exclusive OR) is a logical operation that returns 1 when its two binary inputs are different and 0 when they are the same.
Despite having only four possible input combinations, XOR presents an interesting challenge.
Figure 1 — XOR truth table and input-space visualization. AI-generated illustration.
A single linear classifier cannot correctly separate the two classes using one straight decision boundary.
Consider a neuron computing:
z = w₁x₁ + w₂x₂ + b
Here, x₁ and x₂ represent the input features, w₁ and w₂ are their corresponding weights, and b is the bias.
Applying a threshold to this weighted sum produces a linear decision boundary. However, the positive XOR examples occupy opposite corners of the input space, as do the negative examples.
No single straight line can correctly separate both classes.
XOR is not linearly separable.
To solve this problem, we need a network capable of learning nonlinear relationships. A hidden layer combined with nonlinear activation functions makes this possible.
2. Network Architecture
To solve XOR, I built a small fully connected feedforward neural network with a 2 → 3 → 1 architecture.
The network consists of three layers:
- Input layer: Two input features (x₁ and x₂), representing the binary XOR inputs.
- Hidden layer: Three neurons using the Rectified Linear Unit (ReLU) activation function.
- Output layer: One neuron using sigmoid activation to produce a value between 0 and 1.
Every input connects to each hidden neuron, and all three hidden neurons connect to the output neuron.
Figure 2 — The 2 → 3 → 1 neural network architecture, showing two inputs, three hidden neurons, and one output. AI-generated illustration.
Each connection has a trainable weight, while every hidden neuron and the output neuron have their own bias.
In total, the network contains 13 trainable parameters :
- 6 hidden-layer weights
- 3 hidden-layer biases
- 3 output-layer weights
- 1 output bias
These parameters determine how the network transforms inputs into predictions. During training, their values are gradually adjusted to minimize prediction error.
The hidden layer introduces nonlinearity through ReLU, allowing the network to learn patterns that a single linear classifier cannot represent.
The output layer uses sigmoid because we’re solving a binary classification problem. Its output is interpreted as a prediction score, with 0.5 used as the classification threshold.
Despite its small architecture, this network is sufficient to learn the four XOR combinations.
3. Preparing the Dataset and Initializing Parameters
Before implementing forward propagation, we need to prepare our training data and initialize the network’s weights and biases.
Preparing the XOR Dataset
Our dataset contains all four possible XOR input combinations.
import numpy as np X = np.array([ [0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0] ]) y = np.array([ [0.0], [1.0], [1.0], [0.0] ]) Here:
- X represents the input features, with shape (4, 2).
- y represents the expected outputs, with shape (4, 1).
The four rows represent training examples, while the two columns of X represent input features.
Instead of processing each example separately, we’ll use NumPy vectorization to perform calculations across the entire dataset.
Initializing Weights and Biases
Our network contains two sets of weights and biases: one for the hidden layer and another for the output layer.
rng = np.random.default_rng(42) # Hidden layer W_h = rng.normal(0, 0.5, size=(2, 3)) b_h = np.zeros(3) # Output layer W_o = rng.normal(0, 0.5, size=(3, 1)) b_o = np.zeros(1) We initialize the weights using a normal distribution with a mean of 0 and a standard deviation of 0.5.
The random seed 42 makes our initialization reproducible, allowing us to obtain the same starting weights across runs.
The biases are initially set to zero.
Why random weights?
If all hidden neurons started with identical weights and biases, they would receive identical gradients and could continue learning the same features.
Random initialization helps break this symmetry, allowing different neurons to learn different representations.
With our dataset and parameters ready, we can begin the first forward pass and observe how the network produces its initial predictions.
4. Forward Propagation: How the Network Makes Predictions
Forward propagation is the process of passing input data through a neural network to calculate its predictions.
At each trainable layer, two operations occur:
- Calculate a weighted sum and add a bias.
- Apply an activation function.
Let’s break down how these operations work in our 2 → 3 → 1 network.
Figure 3 — Forward propagation through the hidden and output layers. AI-generated illustration.
The diagram uses single-example column-vector notation, while the NumPy implementation processes row-wise batches.
Step 1: Hidden Layer - Weighted Sum
The first operation is:
Zₕ = XWₕ + bₕ
Where:
- X: Input matrix with shape (4, 2).
- Wₕ: Hidden-layer weights with shape (2, 3).
- bₕ: Hidden-layer biases with shape (3,).
- Zₕ: Resulting weighted sums with shape (4, 3).
In NumPy:
z_h = X @ W_h + b_h The @ operator performs matrix multiplication.
The dimensions are:
(4, 2) @ (2, 3) → (4, 3)
NumPy then broadcasts the three biases across all four training examples.
Each row now contains three weighted sums, one for each hidden neuron.
Step 2: Applying ReLU
Next, we introduce nonlinearity using the Rectified Linear Unit (ReLU).
ReLU(z) = max(0, z)
ReLU replaces negative values with zero while keeping positive values unchanged.
h = np.maximum(0, z_h) For example:
ReLU([-2, 3, -1]) → [0, 3, 0]
Why is this important?
Without nonlinear activation functions, stacking multiple affine layers would still produce an affine transformation. The network would remain unable to learn XOR’s nonlinear decision boundary.
The resulting hidden activation matrix has shape (4, 3).
Step 3: Output Layer - Weighted Sum
The hidden activations now become inputs to the output neuron.
Zₒ = HWₒ + bₒ
Where:
- H: Hidden activations with shape (4, 3).
- Wₒ: Output weights with shape (3, 1).
- bₒ: Output bias with shape (1,).
- Zₒ: Output weighted sums with shape (4, 1).
In NumPy:
z_o = h @ W_o + b_o Matrix multiplication:
(4, 3) @ (3, 1) → (4, 1)
We now have one output value for each training example, before activation.
Step 4: Applying Sigmoid
Finally, we apply the sigmoid activation function.
σ(z) = 1 / (1 + e⁻ᶻ)
Sigmoid maps values into the interval between 0 and 1, making it useful for producing scores in binary classification.
a_o = 1 / (1 + np.exp(-z_o)) We use 0.5 as our classification threshold:
- Output ≥ 0.5 → Class 1
- Output below 0.5 → Class 0
At this point, our network has successfully completed a forward pass.
However, making predictions is not the same as learning.
The initial outputs depend entirely on the randomly initialized parameters. To improve those predictions, the network must first measure how far they are from the expected values.
Before discussing the loss, let’s follow one actual input through our initialized network.
4.1 A Forward Pass With Actual Numbers
Let’s examine what happens when our randomly initialized network receives the input:
X = [0, 1]
Using our initial weights (seed 42), the hidden-layer weighted sums are approximately:
Zₕ = [0.470282, −0.975518, −0.651090]
Applying ReLU:
H = [0.470282, 0, 0]
Only the first hidden neuron remains active. The other two produce zero because their weighted sums are negative.
Next, we calculate the output weighted sum:
zₒ = 0.470282 × 0.063920 ≈ 0.030061
Applying sigmoid:
ŷ = 1 / (1 + e⁻⁰·⁰³⁰⁰⁶¹) ≈ 0.507515
The correct XOR target for (0, 1) is 1.
Our untrained network produces approximately 0.5075. Although this is already above the classification threshold of 0.5, the prediction still has considerable error.
This is why classification accuracy alone doesn’t tell us how well the network has learned.
We need a loss function.
5. Measuring Error With Mean Squared Error
To measure the difference between predictions and their actual targets, we’ll use Mean Squared Error (MSE).
The formula is:
MSE = (1/N) Σ(ŷᵢ − yᵢ)²
Where:
- N represents the number of training examples.
- ŷᵢ represents the predicted output.
- yᵢ represents the actual target.
- Σ represents summation across all examples.
For our network, N = 4 because we have four XOR combinations.
Consider our previous example:
Target = 1
Prediction ≈ 0.507515
The squared error becomes:
(0.507515 − 1)² ≈ 0.242542
We repeat this calculation for all four examples and compute their average.
Figure 4 — Initial XOR predictions, squared errors, and MSE before training. Values reproduced from the actual NumPy implementation.
In NumPy:
error = a_o - y squared_error = error ** 2 loss = np.mean(squared_error) At initialization, the network’s actual loss is approximately:
MSE = 0.25023636
Our objective is to reduce this value by adjusting the weights and biases.
However, calculating the loss only tells us how wrong the predictions are. It doesn’t tell us which parameters to change or by how much.
To answer that question, we need derivatives and the chain rule.
That’s where backpropagation begins.
6. Backpropagation: Understanding the Chain Rule
So far, our network can produce predictions and calculate their errors using Mean Squared Error.
But one important question remains:
How does the network know which weights and biases to change to reduce the loss?
This is where backpropagation comes in.
Backpropagation calculates the gradients of the loss with respect to each trainable parameter using the chain rule of differentiation.
A gradient tells us how sensitive the loss is to a small change in a particular parameter.
Before calculating these gradients, we need to understand how the operations depend on one another.
Figure 5 — Gradient flow from the loss through the output and hidden layers. AI-generated illustration.
6.1 Understanding the Chain Rule
Recall the equations from forward propagation:
z = wh + b
a = σ(z)
L = (a − y)²
Here:
- h represents an activation entering the neuron.
- w represents its weight.
- b represents its bias.
- z represents the weighted sum.
- a represents the activated output.
- y represents the expected target.
- L represents the squared error for one example.
Notice how these operations depend on one another.
The weight influences the weighted sum, the weighted sum influences the activation, and the activation influences the loss.
w → z → a → L
To calculate how changing the weight affects the loss, we apply the chain rule:
∂L/∂w = (∂L/∂a) × (∂a/∂z) × (∂z/∂w)
This expression contains three derivatives:
- ∂L/∂a: How the loss changes when the activation changes.
- ∂a/∂z: How the activation changes when the weighted sum changes.
- ∂z/∂w: How the weighted sum changes when the weight changes.
Multiplying them gives us the gradient of the loss with respect to the weight.
Our actual network uses four training examples, so we’ll apply this same principle to the batch MSE.
6.2 Differentiating Mean Squared Error
Recall our loss function:
L = (1/N) Σ(aᵢ − yᵢ)²
We want to calculate its derivative with respect to each prediction, aᵢ.
For one example, the squared error is:
(aᵢ − yᵢ)²
Using the power rule and chain rule:
d/daᵢ [(aᵢ − yᵢ)²] = 2(aᵢ − yᵢ)
Since MSE averages the squared errors across N examples, we include the factor 1/N:
∂L/∂aᵢ = (2/N)(aᵢ − yᵢ)
For our complete batch, the NumPy implementation is:
dl_da_o = 2 / len(X) * (a_o - y) Here, len(X) equals 4, and the resulting gradient array has shape (4, 1).
We have now calculated the first derivative needed for backpropagation.
Next, we’ll differentiate the sigmoid activation and combine both derivatives using the chain rule.
6.3 Differentiating the Sigmoid Activation
From forward propagation, recall the sigmoid function:
a = σ(z) = 1 / (1 + e⁻ᶻ)
Here, z represents the output neuron’s weighted sum, and a represents its activated output.
To understand how changes in z affect a, we differentiate sigmoid with respect to z.
The derivative is:
∂a/∂z = e⁻ᶻ / (1 + e⁻ᶻ)²
This expression can be simplified using the original sigmoid function:
∂a/∂z = σ(z)(1 − σ(z))
Since we already calculated the sigmoid output during forward propagation, we can reuse it directly:
∂a/∂z = a(1 − a)
In NumPy:
da_od_zo = a_o * (1 - a_o) This calculates the sigmoid derivative for all four training examples simultaneously.
The resulting array has shape (4, 1).
6.4 Applying the Chain Rule
We now have two derivatives.
From Section 6.2, the MSE derivative:
∂L/∂a = (2/N)(a − y)
And from Section 6.3, the sigmoid derivative:
∂a/∂z = a(1 − a)
We want to calculate how the loss changes with respect to the output weighted sum, z.
Using the chain rule:
∂L/∂z = (∂L/∂a) × (∂a/∂z)
Substituting both expressions:
∂L/∂z = (2/N)(a − y) × a(1 − a)
For the entire batch, multiplication is performed elementwise.
In NumPy:
dl_dz_o = dl_da_o * da_od_zo The resulting gradient has shape (4, 1).
We have now propagated the loss gradient backward through the sigmoid activation.
The next step is to determine how the output weights and bias contribute to this gradient.
6.5 Calculating Output Weight and Bias Gradients
Recall our output-layer equation:
Zₒ = HWₒ + bₒ
Here:
- H: Hidden-layer activations.
- Wₒ: Output-layer weights.
- bₒ: Output-layer bias.
- Zₒ: Output pre-activation values.
For a single training example, the output weighted sum is:
zₒ = h₁w₁ + h₂w₂ + h₃w₃ + bₒ
Let’s differentiate it with respect to the first output weight, w₁.
Since the other terms do not depend on w₁:
∂zₒ/∂w₁ = h₁
Similarly:
∂zₒ/∂w₂ = h₂
∂zₒ/∂w₃ = h₃
Now apply the chain rule:
∂L/∂w₁ = (∂L/∂zₒ) × h₁
The same principle applies to the remaining output weights.
Because we’re processing four examples, we sum their contributions using matrix multiplication.
∂L/∂Wₒ = Hᵀ @ (∂L/∂Zₒ)
In NumPy:
dW_o = h.T @ dl_dz_o Let’s verify the matrix dimensions:
(3, 4) @ (4, 1) → (3, 1)
The result contains three gradients, one for each output weight.
Calculating the Output Bias Gradient
The output weighted sum also depends on its bias:
zₒ = h₁w₁ + h₂w₂ + h₃w₃ + bₒ
Differentiating with respect to bₒ:
∂zₒ/∂bₒ = 1
Applying the chain rule:
∂L/∂bₒ = (∂L/∂zₒ) × 1
For the complete batch, we sum the contributions from all four examples:
db_o = np.sum(dl_dz_o, axis=0) The result has shape (1,), since our output layer contains only one bias.
Notice that we don’t divide by N again. The averaging factor was already included when differentiating MSE.
At this point, we’ve calculated the gradients for all three output weights and the output bias.
However, our network also contains six hidden-layer weights and three hidden biases.
To calculate their gradients, we need to propagate the error backward through the hidden layer and its ReLU activation.
6.6 Backpropagating Through the Hidden Layer
So far, we’ve calculated the gradients for the output layer: three weights and one bias.
But our network also contains six hidden-layer weights and three hidden biases.
To calculate their gradients, we need to propagate the loss backward through the hidden layer.
Recall the equations from forward propagation:
Zₕ = XWₕ + bₕ
H = ReLU(Zₕ)
Zₒ = HWₒ + bₒ
The hidden activations influence the output weighted sum, which influences the prediction and ultimately the loss.
We already calculated ∂L/∂Zₒ in Section 6.4.
Now we want to determine how the loss changes with respect to the hidden activations, H.
For a single example, the output weighted sum is:
zₒ = h₁w₁ + h₂w₂ + h₃w₃ + bₒ
Differentiating with respect to each hidden activation:
∂zₒ/∂h₁ = w₁
∂zₒ/∂h₂ = w₂
∂zₒ/∂h₃ = w₃
Applying the chain rule:
∂L/∂hⱼ = (∂L/∂zₒ) × wⱼ
For the entire batch, we can calculate these gradients using matrix multiplication:
∂L/∂H = (∂L/∂Zₒ) @ Wₒᵀ
In NumPy:
dl_dh = dl_dz_o @ W_o.T Let’s verify the dimensions:
(4, 1) @ (1, 3) → (4, 3)
The resulting matrix contains the loss gradients for all three hidden activations across all four training examples.
Notice that we use Wₒᵀ , the transpose of the output weight matrix, not the output weight gradients.
6.7 Differentiating ReLU
Our hidden layer uses ReLU:
h = ReLU(z) = max(0, z)
Its derivative is:
ReLU’(z) = 1 when z > 0, otherwise 0
At exactly zero, ReLU is not differentiable. Our implementation uses a derivative value of 0.
This means gradients pass through positive hidden pre-activations, while inactive ReLU units receive zero gradient through that activation.
We can calculate the derivative using a Boolean mask:
dh_dhz = (z_h > 0).astype(float) The expression (z_h > 0) returns True for positive values and False otherwise.
Using .astype(float) converts these Boolean values into 1.0 and 0.0.
Now we apply the chain rule:
∂L/∂Zₕ = (∂L/∂H) × ReLU’(Zₕ)
In NumPy:
dl_dzh = dl_dh * dh_dhz This operation uses elementwise multiplication.
Both arrays have shape (4, 3), so the result also has shape (4, 3).
We have now propagated the loss gradient backward through the hidden activation function.
6.8 Calculating Hidden Weight Gradients
Recall the hidden-layer weighted sum:
Zₕ = XWₕ + bₕ
For a single hidden neuron, its weighted sum is:
zₕ = x₁w₁ + x₂w₂ + bₕ
Differentiating with respect to the first weight:
∂zₕ/∂w₁ = x₁
Similarly:
∂zₕ/∂w₂ = x₂
Applying the chain rule:
∂L/∂w₁ = (∂L/∂zₕ) × x₁
∂L/∂w₂ = (∂L/∂zₕ) × x₂
We apply this principle to all three hidden neurons and sum contributions across the four training examples.
The matrix equation becomes:
∂L/∂Wₕ = Xᵀ @ (∂L/∂Zₕ)
In NumPy:
dW_h = X.T @ dl_dzh Checking the dimensions:
(2, 4) @ (4, 3) → (2, 3)
The resulting gradient matrix contains six values, corresponding to our six hidden-layer weights.
6.9 Calculating Hidden Bias Gradients
Finally, we calculate the hidden bias gradients.
For a hidden neuron:
zₕ = x₁w₁ + x₂w₂ + bₕ
Differentiating with respect to its bias:
∂zₕ/∂bₕ = 1
Applying the chain rule:
∂L/∂bₕ = (∂L/∂zₕ) × 1
For all four training examples, we sum the gradients along the batch dimension:
db_h = np.sum(dl_dzh, axis=0) The result has shape (3,), giving us one gradient for each hidden neuron.
6.10 Completing Backpropagation
We’ve now calculated all the gradients required to train our network.
Gradient Shape Parameters ---------------------------------- dW_h (2, 3) 6 db_h (3,) 3 dW_o (3, 1) 3 db_o (1,) 1 ---------------------------------- Total 13 The complete hidden-layer backpropagation implementation is:
# Gradient through output weights dl_dh = dl_dz_o @ W_o.T # Gradient through ReLU dh_dhz = (z_h > 0).astype(float) dl_dzh = dl_dh * dh_dhz # Hidden weight gradients dW_h = X.T @ dl_dzh # Hidden bias gradients db_h = np.sum(dl_dzh, axis=0) All these gradients are calculated using the current weights and the values saved during forward propagation.
An important distinction: backpropagation calculates gradients; it does not update the network’s parameters.
We now know how each of our 13 parameters influences the loss.
The next step is to use these gradients to adjust the weights and biases through gradient descent.
7. Gradient Descent: Updating the Parameters
We’ve now calculated the gradients for all 13 trainable parameters through backpropagation.
But calculating gradients alone doesn’t improve our network. We need to use them to update the weights and biases.
This is where gradient descent comes in.
Gradient descent is an optimization algorithm that adjusts parameters in the direction opposite to their gradients, aiming to minimize the loss function.
7.1 Understanding the Update Rule
The general gradient descent equation is:
θ_new = θ_old − α(∂L/∂θ)
Where:
- θ represents a trainable parameter, such as a weight or bias.
- α represents the learning rate.
- ∂L/∂θ represents the gradient of the loss with respect to that parameter.
The learning rate controls the size of each update.
A very large learning rate may cause unstable training, while a very small learning rate can make convergence slow.
For our XOR network, we use:
α = 0.05
Figure 6 — Gradient descent updates weights and biases using the calculated gradients. AI-generated illustration.
In this diagram, η represents the learning rate, denoted by α in our equations. Both refer to the same parameter.
7.2 Updating Weights and Biases
We apply the same update rule to all four parameter groups.
In NumPy:
# Hidden layer updates W_h = W_h - A * dW_h b_h = b_h - A * db_h # Output layer updates W_o = W_o - A * dW_o b_o = b_o - A * db_o Here, A stores our learning rate of 0.05.
Each parameter is adjusted using its corresponding gradient.
An important detail is that all gradients must be calculated before updating any weights or biases.
This ensures that every gradient corresponds to the same forward pass and parameter state.
7.3 A Real Weight Update
Let’s examine one actual update from our network’s initial training step.
The first output-layer weight begins at approximately:
w = 0.06392020
After backpropagation, its gradient is:
∂L/∂w ≈ 0.00123833
Using the gradient descent equation:
w_new = 0.06392020 − (0.05 × 0.00123833)
Therefore:
w_new ≈ 0.06385829
The weight changes only slightly, but this is how learning progresses: through repeated updates across all trainable parameters.
After updating all 13 parameters in our first training step, the MSE decreases from approximately 0.25023636 to 0.25020869.
This is a small improvement, but it demonstrates that the calculated gradients are being used to adjust the network.
7.4 Repeating the Learning Process
One update is not enough to learn the complete XOR mapping.
We repeat the training cycle:
Forward Propagation → Loss Calculation → Backpropagation → Gradient Descent
After each update, the new weights and biases are used to calculate the next predictions.
Over many iterations, the network gradually adjusts its parameters to reduce prediction errors.
In our implementation, we repeat this process for 5,000 epochs , using all four XOR examples for each update.
This method is called full-batch gradient descent.
Now that we’ve implemented forward propagation, loss calculation, backpropagation, and gradient descent, we can combine everything into one complete training loop.
8. Bringing Everything Together: The Training Loop
We’ve implemented all four fundamental operations required to train our neural network:
- Forward propagation — Calculate predictions using the current weights and biases.
- Loss calculation — Measure prediction errors using Mean Squared Error.
- Backpropagation — Calculate gradients using the chain rule.
- Gradient descent — Update weights and biases to reduce the loss.
Now we can combine these operations into a single training loop.
8.1 Implementing the Complete Training Loop
We use a learning rate of 0.05 and train the network for 5,000 epochs.
A = 0.05 epochs = 5000 for epoch in range(epochs): # Forward propagation z_h = X @ W_h + b_h h = np.maximum(0, z_h) z_o = h @ W_o + b_o a_o = 1 / (1 + np.exp(-z_o)) # Mean Squared Error error = a_o - y squared_error = error ** 2 loss = np.mean(squared_error) # Output layer backpropagation dl_da_o = 2 / len(X) * (a_o - y) da_od_zo = a_o * (1 - a_o) dl_dz_o = dl_da_o * da_od_zo dW_o = h.T @ dl_dz_o db_o = np.sum(dl_dz_o, axis=0) # Hidden layer backpropagation dl_dh = dl_dz_o @ W_o.T dh_dhz = (z_h > 0).astype(float) dl_dzh = dl_dh * dh_dhz dW_h = X.T @ dl_dzh db_h = np.sum(dl_dzh, axis=0) # Gradient descent W_h = W_h - A * dW_h b_h = b_h - A * db_h W_o = W_o - A * dW_o b_o = b_o - A * db_o # Display training progress if epoch % 500 == 0: print(f"Epoch {epoch} | Loss: {loss:.6f}") This is the complete training loop from our NumPy implementation.
Notice that every operation we previously derived appears here. There is no automatic differentiation or built-in neural network optimizer.
8.2 Understanding an Epoch
An epoch represents one complete pass through the training dataset.
Since our dataset contains only four XOR combinations, each epoch processes all four examples together.
During each epoch, the network:
- Calculates the predictions for all four examples.
- Computes the average squared error.
- Calculates gradients for all 13 parameters.
- Updates the weights and biases once.
Because the complete dataset contributes to every gradient calculation, this approach is called full-batch gradient descent.
8.3 Tracking the Training Loss
We use the following condition to print the training progress:
if epoch % 500 == 0: print(f"Epoch {epoch} | Loss: {loss:.6f}") This prints the loss every 500 epochs, allowing us to observe how the n...
