After discussing about the fundamentals of Neural Networks and Deep Learning, we have arrived to an exciting stage where we can learn how we program a Neural Network. I have created some simple programs to get some basic idea on how to program a Neural Network and I have used a Python library called PyTorch to program it. Program 1 : Calculate total numbers of parameters in a neural network Points to remember : We need to import nn submodule from main module torch We need to inherit the Module class available in torch.nn submodule We should use self otherwise method doesn't get class objects data Do not confuse about forward(), we don't call it directly, it will be called via constructor in super class. Hence using self. p.numel() return the elements in the Model. Please see my explanation in the downloaded code Need basics of Oops concepts in Python # importing modules torch, nn(neural network): nn is a sub module in main module torch import torch import torch.nn as nn...
We are going to discuss about Normalization & Optimizers which will be used across AI, we will use them in LLM's, Agentic AI framework etc. As we discussed in our previous blogs(links for previous blogs are mentioned at the end of this blog), while calculating gradients, when the previous weights are almost similar to new weights, we will land into a problem called Vanishing Gradient . Similarly when current weight is too large than the previous weights, then we will land into Exploding Gradient issue. Using Normalization problem we are going to prevent Vanishing & Exploding Gradient problems. This blogs agenda : Normalization Batch Normalization (Useful in Neural Network) Layer Normalization (Useful in LLM's) Weights Initialization Xavier He EWMA (Exponential Weighted Moving Average) We will use it in Optimizers like Momentum, NAG, Adagrad, RMSProp, Adam Normalization Normalization in Neural Networks and De...