Python Pandas

How to Fix ValueError – The truth value of a Series is ambiguous in Pandas

In this guide, we will address the ValueError – The truth value of a Series is ambiguous in Pandas. Pandas offers two primary data structures: Series, which stores 1-D data, and DataFrame, which stores data in a 2-D format.

This error occurs in two different scenarios: when you try to use the Series as a boolean value and when you filter the values on the pandas DataFrame using logical operators such as ‘and’ or ‘or.’ First, we will reproduce both errors one after another and fix them by providing examples.

Reproducing the Error for Series

Create a pandas Series with five integers and try to check if it holds at least one value using an if statement.

import pandas

# Create Series - Budgets with 5 values
Budgets = pandas.Series([2000,4000,5000,2400,1200])
if (Budgets):
print("Empty")

Output

You can see that a ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all() is raised.

Solution: Use empty, any(), all()

We can see that the error itself suggests using the empty /bool() /item() /any() /all() functions. To check whether the Series is empty or not, we can directly use the empty() function and check non-zero (values other than False or 0) values, we can use any() or all() functions.

  1. The pandas.Series.empty property returns True if the Series is empty; otherwise, it returns False.
  2. The pandas.Series.any() function returns True if any of the values in the Series are non-zero.
  3. The pandas.Series.all() function returns True if all values in the Series are non-zero.

We can specify all of them inside the if statement.

Example 1: With Empty Property

Let’s use the empty property to check if the Series ‘Budgets’ is empty or not.

import pandas

# Create Series - Budgets with 5 values
Budgets = pandas.Series([2000,4000,5000,2400,1200])

# Use the Series as a condition with empty property
if (Budgets.empty):
  print("Empty")
else:
  print("Not empty")

Output

Since the Budget holds five values, it’s not empty. The condition inside the if statement is False and the else statement is executed.

Example 2: With pandas.Series.all()

Create a Series ‘Budgets’ with three non-zero values and specify the all() function inside the if statement to check if all are non-zero values in the Budgets.

import pandas

# Create Series - Budgets with 3 values
Budgets = pandas.Series([2000,3400,9000])
if (Budgets.all()):
  print("All are non-zero values")
else:
  print("zero exists")

Output

As the Budget holds three non-zero values, all() returns True (inside the if statement).

Example 3: With pandas.Series.any()

Create Series named Budgets with five values, including zeros, and specify the any() function inside the if statement to check whether all are non-zero values in the Budgets.

import pandas

# Create Series - Budgets with 5 values
Budgets = pandas.Series([2000,3400,0,0,9000])
if (Budgets.any()):
  print("Holds atleast 1 non-zero value")
else:
  print("All are non-zeros")

Output

As the Budget holds three non-zero values, any() returns True (inside the if statement).

Reproducing the Error for DataFrame

As we discussed earlier, the same ValueError occurred when we tried to filter the records in the pandas DataFrame using logical operators like ‘and’ or ‘or.’

Scenario 1: Logical and

Create a DataFrame named day1_Data with two columns: ‘Product_Quantity’ and ‘Purchase’. Try to get the records where ‘Product_Quantity’ is greater than 20 and ‘Purchase’ is less than 4000 by specifying the logical ‘and’ operator.

import pandas

day1_Data = pandas.DataFrame({'Product_Quantity': [2,10,5,7,9,20,30,50,90,100],

'Purchase': [500,1200,700,900,1000,1500,2000,3500,4400,5000]})

print(day1_Data,"\n")

print(day1_Data[(day1_Data['Product_Quantity'] > 20) and (day1_Data['Purchase'] < 4000)])

Output

You can see that same ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all() is raised.

Scenario 2: Logical or

Create a DataFrame named day1_Data with two columns: ‘Product_Quantity’ and ‘Purchase’. Try to get the records where ‘Product_Quantity’ is greater than 20 or ‘Purchase’ is less than 4000 by specifying the logical ‘or’ operator.

import pandas

day1_Data = pandas.DataFrame({'Product_Quantity': [2,10,5,7,9,20,30,50,90,100],

'Purchase': [500,1200,700,900,1000,1500,2000,3500,4400,5000]})

print(day1_Data[(day1_Data['Product_Quantity'] > 20) or (day1_Data['Purchase'] < 4000)])

Output

You can see that same ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all() is raised.

Solution: Use Bitwise Operators

Instead of using logical operators, consider using the Bitwise operators to filter records from the pandas DataFrame. Bitwise AND (&) is used instead of logical ‘and’ operator, providing similar functionality. Bitwise OR (|) is used instead of logical ‘or’ operator, offering similar functionality

Example 1: Bitwise AND (&)

Get the records where ‘Product_Quantity’ is greater than 20 and ‘Purchase’ is less than 4000 by specifying the Bitwise AND operator.

import pandas

day1_Data = pandas.DataFrame({'Product_Quantity': [2,10,5,7,9,20,30,50,90,100],

'Purchase': [500,1200,700,900,1000,1500,2000,3500,4400,5000]})

# Use Bitwise AND (&) instead of Logical - and
print(day1_Data[(day1_Data['Product_Quantity'] > 20) & (day1_Data['Purchase'] < 4000)])

Output

There are two records with the specified conditions. Now the error is not encountered.

Example 2: Bitwise OR (|)

Get the records where ‘Product_Quantity’ is greater than 100 and ‘Purchase’ is less than 4000 by specifying the Bitwise OR operator.

import pandas

day1_Data = pandas.DataFrame({'Product_Quantity': [2,10,5,7,9,20,30,50,90,100],

'Purchase': [500,1200,700,900,1000,1500,2000,3500,4400,5000]})

# Use Bitwise OR (|) instead of Logical - or
print(day1_Data[(day1_Data['Product_Quantity'] > 100) | (day1_Data['Purchase'] < 4000)])

Output

There are eight records with the specified conditions. Now the error is not encountered.

Conclusion

A ValueError – The truth value of a Series is ambiguous in Pandas occurs when we use the Series or DataFrame with boolean values. Also, the same error occurs when we filter the records in the DataFrame by specifying the logical operators ‘and’ or ‘or’. Separately, we discussed both the scenarios and fixed the above error. We used empty property for Series and instead of logical operators with the Bitwise operators available in Python.

About the author

Gottumukkala Sravan Kumar

B tech-hon's in Information Technology; Known programming languages - Python, R , PHP MySQL; Published 500+ articles on computer science domain