Python

How to Use Shelve Module in Python

This article will cover a guide on using the “shelve” module in Python. The shelve module can be used to store Python objects as key-value pairs in an external file. These key-value pairs are serialized in byte streams, a format that Python understands. So you can save Python objects to a file and get them back again in a program by reading the file where serialized data has been stored previously. All code samples in this article are tested with Python version 3.9.7 on Ubuntu 21.10.

Shelve and Pickle

The shelve module uses Python’s “pickle” module underneath to serialize the data. The main difference between shelve and pickle modules is that shelve can be used to associate serialized objects with keys. These keys can be used as identifiers in your code and you can refer to them to get the serialized object. Pickle on the other hand doesn’t provide a native way to store serialized data in key-value pairs, without some workarounds or custom code. Thus shelve is more of a convenience module for easy serialization of Python objects and can be used to create simple databases containing serialized data.

Basic Usage and Syntax

You can open a database file using the “open” method available in the shelve module. This method can be used as a context manager so that that file is properly closed when the method call finishes. Here is a code sample:

import shelve

with shelve.open('test.db', "w") as db:
    db['apples'] = 50
    db['oranges'] = 80

The first statement imports the main shelve module into the Python program. Next, using the “with shelve.open” method call and context manager, “test.db” file is opened as “db” in write mode. The file can have any name and extension. The last two statements are used to create two new key-value pairs using Python dictionary-like syntax. These key-value pairs are then stored in the test.db file in serialized form.

After executing the above code sample, if you open the test.db file in a text editor, you should see some data similar to this:

As you can see, the data stored in the file is not in human readable form. However, if you open this file using shelve, you will be able to retrieve key-value pairs in your Python program. Here is a code sample:

import shelve

with shelve.open('test.db') as db:
    print(db['apples'])
    print(db['oranges'])

Key-value pairs serialized by shelve are identical to a dictionary type object in Python and they support all their methods. So you can refer to objects by supplying a key in square braces. In the last two statements, the value of both keys is retrieved using dictionary-like syntax. After running the above code sample, you should get the following output:

50
80

Note that Python automatically chooses an appropriate database file format for shelve, taking into consideration the operating system from which the program is being executed. However, It doesn’t affect usage of the shelve module and functionality of objects serialized using it.

Getting All Key-Value Pairs from a Database File Created Using Shelve

To get all keys and value pairs, you need to call “keys” and “values” methods, just like you would do with a dictionary type object in Python. Here is a code sample:

import shelve

with shelve.open('test.db') as db:
    items = list(db.items())
    keys = list(db.keys())
    values = list(db.values())
    print(items)
    print(keys)
    print(values)

As you can see in the above code sample, “items”, “keys” and “values” methods have been called to retrieve keys and values. These keys and values are shelve type objects, so you need to convert them to a list or any other iterable to get their actual values. After running the above code sample, you should get the following output:

[('apples', 50), ('oranges', 80)]
['apples', 'oranges']
[50, 80]

Note that if you just want to iterate over keys and values, you can do so using for loop or any other statement without converting keys and values to a Python list or any other such object.

You can Serialize Functions and Classes Using Shelve

You can serialize any Python object using shelve module, even functions and classes. Here is an example illustrating serializing of a function, retrieving it back and then calling it to do some calculations.

import shelve

def square(number):
    return number * number

with shelve.open('test.db', "w") as db:
    db['square_function'] = square

with shelve.open('test.db') as db:
    square = db['square_function']
print(square(5))

A new function called “square” has been defined. It calculates the square of a number and returns it. Next, this function is serialized using shelve and stored in the test.db database file. The serialized function is then read back into the “square” variable. Since the square variable is now an instance of the square function defined earlier, you can call it to calculate the square of a number.

After running the above code sample, you should get the following output:

25

Conclusion

Serialization is a process of storing Python objects in a database file so that they can be retrieved in a form that is exactly the same as how they were originally defined before serializing. This allows you to store complex Python objects in structured format. The shelve module uses pickle module to store and retrieve serialized Python objects. It uses key-value pair structure to handle serialized objects, making it easier to manage them.

About the author

Nitesh Kumar

I am a freelancer software developer and content writer who loves Linux, open source software and the free software community.