How to Normalize Tensors in PyTorch?

In PyTorch, normalizing a particular tensor means scaling its elements so that it has a fixed norm, 0 mean, and 1 variance. A tensor can have a certain norm, such as the L1 norm or L2 norm. PyTorch provides a “torch.nn.functional” module that has a “normalize()” function to normalize the desired tensors.

This blog will exemplify the method to normalize tensors in PyTorch.

How to Normalize Tensors in PyTorch?

To normalize tensors in PyTorch, the “torch.nn.functional.normalize()” function is used. The syntax to normalize a tensor is given below:

This function takes an input tensor, a power value “p”, and a dimension “dim” as arguments and returns a tensor of the original tensor elements’ normalized value.

Go through the following examples for a better understanding:

Example 1: Normalize 1D Tensor in PyTorch

In the first example, we will create a 1D tensor and normalize it. Let us follow the below step-by-step procedure:

Step 1: Import PyTorch Library
First, import the “torch” library and “normalize” module to normalize tensors:

Here:

• “import torch” imports the PyTorch library.
• “from torch.nn.functional import normalize” imports the “normalize” library from the “torch.nn.functional” module for normalizing the tensors:

Step 2: Create 1D Tensor
Then, create a 1D tensor and print its elements. Here, we are creating the following “Tens” tensor from a list using the “torch.tensor()” function:

This has created a 1D tensor:

Step 3: Normalize the Tensor
Now, use the “normalize()” function to normalize a tensor and input the desired tensor, power value “p” and a desired dimension “dim” as an argument. For instance, we are normalizing the “Tens” tensor with two different “p” values:

Here, “p=1.0” is used for L1 normalization and “p=2.0” is used for L2 normalization:

Step 4: Print Normalized Tensors
Finally, print the calculated normalized tensors:

The below output displays the normalized tensors:

Example 2: Normalize 2D Tensor in PyTorch

In the second example, we will create a 2D tensor and normalize it. Let us follow the provided steps:

Step 1: Import PyTorch Library
First, import the “torch” library to normalize tensors:

Step 2: Create 2D Tensor
Then, use the “torch.tensor()” function to create a 2D tensor and print its elements. Here, we are creating the following “Tens” 2D tensor:

This has created a 2D tensor:

Step 3: Normalize the Tensor
Now, normalize a tensor using the “normalize()” function and specify the desired tensor, power value “p” and a desired dimension “dim” as an argument. For instance, we are normalizing the “Tens” tensor with the following arguments:

Step 4: Print Normalized Tensors
Finally, print the calculated normalized tensors:

It can be seen that the tensor has been normalized successfully:

Moreover, users can also normalize tensors row-wise and column-wise. To do so, follow the next provided steps.

Step 5: Normalize Tensor Column-wise
To normalize a tensor column-wise, specify the “dim=0” as seen in below code:

According to the below output, the “Tens” tensor has been normalized column-wise successfully:

Step 6: Normalize Tensor Row-wise
To normalize a tensor row-wise, specify the “dim=1” as seen in below code:

The below output displays the row-wise normalized tensor:

We have efficiently explained the method to normalize tensors in PyTorch.

Note: You can access our Google Colab Notebook at this link.

Conclusion

To normalize tensors in PyTorch, first, import the “torch” library. Then, create the desired 1D or 2D tensor and view its elements. Next, use the “normalize()” method to normalize the specific tensor. Moreover, users can also normalize the 2D tensor along each row or column using the “dim” argument. Lastly, display the normalized tensor. This blog has exemplified the method to normalize tensors in PyTorch.