What is PyTorch?
PyTorch is an open-source Torch based Machine Learning library for natural language processing using Python. It is similar to NumPy but with powerful GPU support. It offers Dynamic Computational Graphs that you can modify on the go with the help of autograd. PyTorch is also faster than some other frameworks. It was developed by Facebook’s AI Research Group in 2016.
What is PyTorch? PyTorch Advantages and Disadvantages PyTorch Vs. TensorFlow Installing PyTorch PyTorch Framework Basics Simple Regression with PyTorch Image Classification Example with PyTorch
PyTorch Advantages and Disadvantages
Following are the advantages and disadvantages of PyTorch:
Advantages of PyTorch
Simple Library PyTorch code is simple. It is easy to understand, and you use the library instantly. For example, take a look at the code snippet below:
As mentioned above, you can define the network model easily, and you can understand the code quickly without much training.
Dynamic Computational Graph
Image Source: Exploring Deep Learning with PyTorch Pytorch offers Dynamic Computational Graph (DAG). Computational graphs is a way to express mathematical expressions in graph models or theories such as nodes and edges. The node will do the mathematical operation, and the edge is a Tensor that will be fed into the nodes and carries the output of the node in Tensor. DAG is a graph that holds arbitrary shape and able to do operations between different input graphs. Every iteration, a new graph is created. So, it is possible to have the same graph structure or create a new graph with a different operation, or we can call it a dynamic graph.
Better Performance
Communities and researchers, benchmark and compare frameworks to see which one is faster. A GitHub repo Benchmark on Deep Learning Frameworks and GPUs reported that PyTorch is faster than the other framework in terms of images processed per second. As you can see below, the comparison graphs with vgg16 and resnet152
Native Python
PyTorch is more python based. For example, if you want to train a model, you can use native control flow such as looping and recursions without the need to add more special variables or sessions to be able to run them. This is very helpful for the training process. Pytorch also implements Imperative Programming, and it’s definitely more flexible. So, it’s possible to print out the tensor value in the middle of a computation process.
Disadvantage of PyTorch
PyTorch requires third-party applications for Visualization. It also needs an API server for production. Next in this PyTorch tutorial, we will learn about the difference between PyTorch and TensorFlow.
PyTorch Vs. Tensorflow
Installing PyTorch
Linux
It’s straightforward to install it in Linux. You can choose to use a virtual environment or install it directly with root access. Type this command in the terminal
AWS Sagemaker
Next Step, Click on Open to launch your notebook instance.
Finally, In Jupyter, Click on New and choose conda_pytorch_p36 and you are ready to use your notebook instance with Pytorch installed. Next in this PyTorch tutorial, we will learn about PyTorch framework basics.
PyTorch Framework Basics
Let’s learn the basic concepts of PyTorch before we deep dive. PyTorch uses Tensor for every variable similar to numpy’s ndarray but with GPU computation support. Here we will explain the network model, loss function, Backprop, and Optimizer.
Network Model
The network can be constructed by subclassing the torch.nn. There are 2 main parts,
The first part is to define the parameters and layers that you will use The second part is the main task called the forward process that will take an input and predict the output.
As you can see above, you create a class of nn.Module called Model. It contains 2 Conv2d layers and a Linear layer. The first conv2d layer takes an input of 3 and the output shape of 20. The second layer will take an input of 20 and will produce an output shape of 40. The last layer is a fully connected layer in the shape of 320 and will produce an output of 10. The forward process will take an input of X and feed it to the conv1 layer and perform ReLU function, Similarly, it will also feed the conv2 layer. After that, the x will be reshaped into (-1, 320) and feed into the final FC layer. Before you send the output, you will use the softmax activation function. The backward process is automatically defined by autograd, so you only need to define the forward process.
Loss Function
The loss function is used to measure how well the prediction model is able to predict the expected results. PyTorch already has many standard loss functions in the torch.nn module. For example, you can use the Cross-Entropy Loss to solve a multi-class PyTorch classification problem. It’s easy to define the loss function and compute the losses: It’s easy to use your own loss function calculation with PyTorch.
Backprop
To perform the backpropagation, you simply call the los.backward(). The error will be computed but remember to clear the existing gradient with zero_grad()
Optimizer
The torch.optim provides common optimization algorithms. You can define an optimizer with a simple step: You need to pass the network model parameters and the learning rate so that at every iteration the parameters will be updated after the backprop process.
Simple Regression with PyTorch
Let’s learn simple regression with PyTorch examples: Step 1) Creating our network model Our network model is a simple Linear layer with an input and an output shape of 1. And the network output should be like this Step 2) Test Data Before you start the training process, you need to know our data. You make a random function to test our model. Y = x3 sin(x)+ 3x+0.8 rand(100) Here is the scatter plot of our function:
Before you start the training process, you need to convert the numpy array to Variables that supported by Torch and autograd as shown in the below PyTorch regression example. Step 3) Optimizer and Loss Next, you should define the Optimizer and the Loss Function for our training process. Step 4) Training Now let’s start our training process. With an epoch of 250, you will iterate our data to find the best value for our hyperparameters. Step 5) Result As you can see below, you successfully performed PyTorch regression with a neural network. Actually, on every iteration, the red line in the plot will update and change its position to fit the data. But in this picture, it only show you the final result as shown in the below PyTorch example:
Image Classification Example with PyTorch
One of the popular methods to learn the basics of deep learning is with the MNIST dataset. It is the “Hello World” in deep learning. The dataset contains handwritten numbers from 0 – 9 with the total of 60,000 training samples and 10,000 test samples that are already labeled with the size of 28×28 pixels.
Step 1) Preprocess the Data In the first step of this PyTorch classification example, you will load the dataset using torchvision module. Before you start the training process, you need to understand the data. Torchvision will load the dataset and transform the images with the appropriate requirement for the network such as the shape and normalizing the images. The transform function converts the images into tensor and normalizes the value. The function torchvision.transforms.MNIST, will download the dataset (if it’s not available) in the directory, set the dataset for training if necessary and do the transformation process. To visualize the dataset, you use the data_iterator to get the next batch of images and labels. You use matplot to plot these images and their appropriate label. As you can see below our images and their labels.
Step 2) Network Model Configuration Now in this PyTorch example, you will make a simple neural network for PyTorch image classification. Here, we introduce you another way to create the Network model in PyTorch. We will use nn.Sequential to make a sequence model instead of making a subclass of nn.Module. Here is the output of our network model
Network Explanation
The sequence is that the first layer is a Conv2D layer with an input shape of 1 and output shape of 10 with a kernel size of 5 Next, you have a MaxPool2D layer A ReLU activation function a Dropout layer to drop low probability values. Then a second Conv2d with the input shape of 10 from the last layer and the output shape of 20 with a kernel size of 5 Next a MaxPool2d layer ReLU activation function. After that, you will flatten the tensor before you feed it into the Linear layer Linear Layer will map our output at the second Linear layer with softmax activation function
Step 3)Train the Model Before you start the training process, it is required to set up the criterion and optimizer function. For the criterion, you will use the CrossEntropyLoss. For the Optimizer, you will use the SGD with a learning rate of 0.001 and a momentum of 0.9 as shown in the below PyTorch example. The forward process will take the input shape and pass it to the first conv2d layer. Then from there, it will be feed into the maxpool2d and finally put into the ReLU activation function. The same process will occur in the second conv2d layer. After that, the input will be reshaped into (-1,320) and feed into the fc layer to predict the output. Now, you will start the training process. You will iterate through our dataset 2 times or with an epoch of 2 and print out the current loss at every 2000 batch. At each epoch, the enumerator will get the next tuple of input and corresponding labels. Before we feed the input to our network model, we need to clear the previous gradient. This is required because after the backward process (backpropagation process), the gradient will be accumulated instead of being replaced. Then, we will calculate the losses from the predicted output from the expected output. After that, we will do a backpropagation to calculate the gradient, and finally, we will update the parameters. Here’s the output of the training process Step 4) Test the Model After you train our model, you need to test or evaluate with other sets of images. We will use an iterator for the test_loader, and it will generate a batch of images and labels that will be passed to the trained model. The predicted output will be displayed and compared with the expected output.
Summary
PyTorch is an open-source Torch based Machine Learning library for natural language processing using Python. Advantages of PyTorch: 1) Simple Library, 2) Dynamic Computational Graph, 3) Better Performance, 4) Native Python PyTorch uses Tensor for every variable similar to numpy’s ndarray but with GPU computation support. One of the popular methods to learn the basics of deep learning is with the MNIST dataset.
