Python

Python Remove NaN from List

The “nan” or “NaN” is a simple term that stands for “Not a Number” and it is used to represent that the data doesn’t exist at that specific location. In most languages, it is even considered equivalent to null. However, in many cases, a NaN can be present inside a list which can halt operations to be performed on the list. In Python, there are multiple methods that allow the user to remove this NaN from the list.

This guide will illustrate the different methods the user can use to remove NaN from a list. Additionally, the numpy library will be used to create a nan value in the list throughout this post.

Method 1: Using the Comparison Operator

The NaN can easily be detected in the list by using simple string comparison using the “==” or the “!=” operator. Once detected, the user can choose to delete them from the list, or include all other values in a new list by using the append() method. To demonstrate this, start by importing numpy and create a list with some nan values:

from numpy import nan
listVar = [12,'String',nan, 56,69, nan]

 
After that create a new list that will be used to hold the non-NaN values:

newList= []

 
After that, use a for loop to iterate through every item in the “listVar” list and compare it with “nan” after using the string type casting with the str() method. In the end, append the non-NaN values into the “newList”:

for item in listVar:
    if(str(item) != "nan"):
        newList.append(item)

 
Last simply print out the original list and the new list onto the console by using the following lines of code:

print("Original List: ", listVar)
print("New list: ",newList)

 
The complete code snippet for this method is as follows:

from numpy import nan
listVar = [12,'String',nan, 56,69, nan]

newList= []
for item in listVar:
    if(str(item) != "nan"):
        newList.append(item)

print("Original List: ", listVar)
print("New list: ",newList)

 
When this code is executed, it produces the following result on the terminal:


The output verifies that the “nan” values have been removed from the list.

Method 2: Using the isnan() Method

The isnan() method is present in the “math” package as well as the “numpy” package and it is used to detect whether or not a value is “nan” or not by returning a boolean result. However, this method only works when there are only numeric values in the list along with nan.

To demonstrate this method, use the same approach as used in method one with a few little changes. To demonstrate this, use the following code snippet:

import math
from numpy import nan
listVar = [12,nan,16,69, nan,4,nan,20]

newList= []
for item in listVar:
    if(math.isnan(item) != True):
        newList.append(item)

print("Original List: ", listVar)
print("New list: ",newList)

 
When this code is executed, it produces the following results on the console:


As you can see in the output image above, the “nan” has been removed entirely from the list. In case, you want to use the numpy version of the isnan() method, then simply use the following code:

import numpy
from numpy import nan
listVar = [12,nan,16,69, nan,4,nan,20]

newList= []
for item in listVar:
    if(numpy.isnan(item) != True):
        newList.append(item)
print("Use Numpy Package")
print("Original List: ", listVar)
print("New list: ",newList)

 
When this code is executed, it will produce the following result on the terminal:


From the output, it is observable that the NaN values have been removed from the list using the numpy isnan() method.

Method 3: Using the isnull() Method From Pandas

The pandas library offers the method “isnull()” that is used to detect NaN and Null values. Which is exactly what the user requires to remove NaN from a list. Similarly, to the isnan() method in the above section, this method also returns the result in the form of a boolean value.

To demonstrate its usage, take the following code snippet:

import pandas
from numpy import nan
listVar = [12,nan,16,69, nan,4,nan,20]

newList= []
for item in listVar:
    if(pandas.isnull(item) != True):
        newList.append(item)
print("Use Pandas Package")
print("Original List: ", listVar)
print("New list: ",newList)

 
When this code is executed, it will produce the following outcome on the terminal:


That verifies that the “nan” values have been removed from the list using the isnull() method from pandas library.

Note: To install pandas, simply use the command “pip install pandas”.

Conclusion

Removing the NaN values from a list is rather an easy task. To do this, the user can utilize the comparison operator with the string type casting method str(). Additionally, the user can make use of the isnan() method that is present in the math and the numpy library or even the isnull() method inside the pandas library.

About the author

Abdul Mannan

I am curious about technology and writing and exploring it is my passion. I am interested in learning new skills and improving my knowledge and I hold a bachelor's degree in computer science.