pytorch

How to Find Element-wise Entropy of a Tensor in PyTorch?

Entropy is a measure of uncertainty or randomness of a particular system. The element-wise entropy of a tensor is the entropy computed for each element of the tensor individually. PyTorch provides the “torch.special.entr()” method to find the entropy of any tensor. If the tensor’s element is negative, its entropy will be negative infinity. If the tensor’s element is “0”, its entropy will also be “0”. Moreover, if the tensor element is positive, its entropy will be calculated as the product of the element’s negative value by its natural logarithm.

This article will exemplify the method to find the element-wise entropy of tensors in PyTorch.

How to Find Element-wise Entropy of Tensors in PyTorch?

To find the element-wise entropy of tensors in PyTorch, the “torch.special.entr()” method is used. Users need to pass the desired tensor to this method to find its element-wise entropy.

Go through the next provided examples for a better understanding:

Example 1: Calculate/Find Element-wise Entropy of 1D Tensor

In the first example, we will create a 1D tensor and calculate its element-wise entropy. Let’s follow the provided steps:

Step 1: Import PyTorch Library

First, import the “torch” library to calculate the element-wise entropy:

import torch

Step 2: Create 1D Tensor

Then, use the “torch.tensor()” function to create a 1D tensor and print its elements. Here, we are creating the following “Tens1” 1D tensor from a list:

Tens1 = torch.tensor([3, 0.8 , -1 , 5, 0, -9])

print(Tens1)

Step 3: Calculate Element-wise Entropy

Now, utilize the “torch.special.entr()” method to calculate the element-wise entropy of the “Tens1” tensor:

tens_Entr = torch.special.entr(Tens1)

Step 4: Display Computed Entropy

Lastly, display the computed element-wise entropy of the tensor for verification:

print(tens_Entr)

The below output shows the calculated entropy of the “Tens1” tensor:

Example 2: Calculate/Find Element-wise Entropy of 2D Tensor

In the second example, we will create a 2D tensor and calculate its element-wise entropy. Let’s follow the below step-by-step procedure:

Step 1: Import PyTorch Library

First, import the “torch” library to calculate the entropy:

import torch

Step 2: Create 2D Tensor

Then, create a desired 2D tensor and print its elements. Here, we are creating the following “Tens2” 2D tensor:

Tens2 = torch.tensor([[1, 7, -3], [4, -2, 0], [-5, 0, -8]])

print(Tens2)

This has created a 2D tensor as seen below:

Step 3: Calculate Element-wise Entropy

Now, calculate the element-wise entropy of the “Tens2” tensor using the “torch.special.entr()” method:

tens2_Entr = torch.special.entr(Tens2)

Step 4: Display Computed Entropy

Finally, display the computed element-wise entropy of the tensor:

print(tens2_Entr)

The entropy of the “Tens2” tensor has been calculated successfully:

We have efficiently explained the method of calculating the element-wise entropy of tensors in PyTorch.