Entropy is a measure of uncertainty or randomness of a particular system. The element-wise entropy of a tensor is the entropy computed for each element of the tensor individually. PyTorch provides the “torch.special.entr()” method to find the entropy of any tensor. If the tensor’s element is negative, its entropy will be negative infinity. If the tensor’s element is “0”, its entropy will also be “0”. Moreover, if the tensor element is positive, its entropy will be calculated as the product of the element’s negative value by its natural logarithm.
This article will exemplify the method to find the element-wise entropy of tensors in PyTorch.
How to Find Element-wise Entropy of Tensors in PyTorch?
To find the element-wise entropy of tensors in PyTorch, the “torch.special.entr()” method is used. Users need to pass the desired tensor to this method to find its element-wise entropy.
Go through the next provided examples for a better understanding:
Example 1: Calculate/Find Element-wise Entropy of 1D Tensor
In the first example, we will create a 1D tensor and calculate its element-wise entropy. Let’s follow the provided steps:
Step 1: Import PyTorch Library
First, import the “torch” library to calculate the element-wise entropy:
Step 2: Create 1D Tensor
Then, use the “torch.tensor()” function to create a 1D tensor and print its elements. Here, we are creating the following “Tens1” 1D tensor from a list:
print(Tens1)
Step 3: Calculate Element-wise Entropy
Now, utilize the “torch.special.entr()” method to calculate the element-wise entropy of the “Tens1” tensor:
Step 4: Display Computed Entropy
Lastly, display the computed element-wise entropy of the tensor for verification:
The below output shows the calculated entropy of the “Tens1” tensor:
Example 2: Calculate/Find Element-wise Entropy of 2D Tensor
In the second example, we will create a 2D tensor and calculate its element-wise entropy. Let’s follow the below step-by-step procedure:
Step 1: Import PyTorch Library
First, import the “torch” library to calculate the entropy:
Step 2: Create 2D Tensor
Then, create a desired 2D tensor and print its elements. Here, we are creating the following “Tens2” 2D tensor:
print(Tens2)
This has created a 2D tensor as seen below:
Step 3: Calculate Element-wise Entropy
Now, calculate the element-wise entropy of the “Tens2” tensor using the “torch.special.entr()” method:
Step 4: Display Computed Entropy
Finally, display the computed element-wise entropy of the tensor:
The entropy of the “Tens2” tensor has been calculated successfully:
We have efficiently explained the method of calculating the element-wise entropy of tensors in PyTorch.
Note: You can access our Google Colab Notebook at this link.
Conclusion
To calculate/find the element-wise entropy of tensors in PyTorch, first, import the “torch” library. Then, create the desired 1D or 2D tensor and view its elements. After that, use the “torch.special.entr()” method to compute the element-wise entropy of the input tensor. Lastly, display the computed entropy. This article has exemplified the method to calculate the element-wise entropy of tensors in PyTorch.
