Python

How to Read Pickle File Python?

There must be many queries in your mind concerning pickle files.? This article is about how to read a pickle file in python. It will cover the basic details of pickle files and the step-by-step process of correctly reading them. So, what is a pickle module or pickling?

What is a pickle module or pickling?

A pickle module or pickling is the process of serializing and deserializing python objects to a byte stream. The opposite of pickling is unpickling. Pickling is used to store python objects like classes, dictionaries, lists, etc. It is beneficial with data analysis while performing routine tasks on the data, like preprocessing. When you work with python specific data types like objects, classes, or dictionaries, pickling makes a lot of sense. It is specifically used to store the trained machine learning algorithms so that we do not have to constantly re-train them every time we want to use them. Instead, we just store the trained machine learning algorithm once, save it to a variable, and then pickle it. This way, you can save a lot of time even when you have a large amount of data, which takes a lot of time to load into memory.

Pickled data may take only 5 seconds to load, while the new data may take 5  – 10 minutes to load. Hence, it is better to pickle the dataset and then load it. This process would be far faster by 50 to 100X, in fact sometimes more than that depending on the size of the dataset. Let us see a simple illustration of how to store a dictionary into a pickle file.

Example 1:

To store a dictionary into a pickle file, first import the pickle to utilize it. After that, define your custom dictionary. The next step is to open a file and use a pickle.dump() to put or write the dictionary into the open file and close it.

import pickle

favcolor = { "red""blue", "yellow""green" }

pickle.dump( favcolor, open( "color.p", "wb" ) )

Text Description automatically generated

Example 2:

The purpose of this article is to explain how to read a pickle file. The code given below will help you read the data. The first step is to open the file with the dictionary data using the open() command, load all the data into a variable using the load() command, and save it in a new variable. Then you can print the variable to see the dictionary data.

import pickle

favcolor = pickle.load( open( “color.p, “rb”) )

Text Description automatically generated

Here is the output.

Text Description automatically generated

Example 3:

We will show you how to read the pickle file using the read_from_pickle function. The read_from_pickle function is a part of pandas 0.22. Here is an instance of how to read and write a pickle file. If you keep adding data to the file, you need to read from the file till you get what you are looking for. Here is an example of how read_from_pickle is used to read pickle files.

import os
import pickle
PFILE = 'pickle.dat'
def main():
      add_to_pickle(PFILE, 'Christmas')
      add_to_pickle(PFILE, New Year')
      add_to_pickle(PFILE, '
2022')
      add_to_pickle(PFILE, b'
Saturday')
      add_to_pickle(PFILE, 456.789)
     
      for item in read_from_pickle(PFILE):
        print(repr(item))
      os.remove(PFILE)
def add_to_pickle(p, i):
      with open(p, '
ab') as f:
        pickle.dump(i, f, pickle.HIGHEST_PROTOCOL)
def read_from_pickle(p):
      with open(p, '
rb') as f:
        try:
            while True:
               yield pickle.load(f)
        except EOFError:
          pass if name == '
main:
main()

Text Description automatically generated

The following output will be generated after executing the above code:

Text Description automatically generated with low confidence

What can you pickle and how to unpickle files?

Generally, any object can be pickled if all the attributes of that object can be pickled. However, methods, functions, and classes cannot be pickled. Moreover, it is not possible to pickle open file objects, database connections, and network connections. The first step to unpickle a file is to load it back into a python program. Use the open() command to open the file with the ‘rb’ argument as it indicated to open the file in ‘read’ mode. The ‘r’ stands for reading mode, and ‘b’ stands for ‘binary mode.’

After opening the file, assign that to a variable, then use pickle.load() with the variable and assign it to a new variable. The file data will be kept in the variable. This is how you unpickle files in python. To ensure that the file has been unpickled successfully, just print the dictionary data by using the print() command, compare it with previous dictionary data and check its type with type().

Conclusion:

This article was about reading pickle files in python. Here we have covered what the pickle module is, the process of pickling and unpickling with the help of examples. Then we explained what can be pickled or what cannot. In other words, what modules, objects, or classes can be pickled and what things cannot be pickled. The short and straightforward examples will help you learn and understand the process of reading, writing, and saving the pickle file. Read the article thoroughly and use the examples in your programs to read a pickle file.

About the author

Kalsoom Bibi

Hello, I am a freelance writer and usually write for Linux and other technology related content