Python

Python Multiprocessing Pool

The deployment of two or even more central processing units throughout a processor system is known as multiprocessing. The system’s functionality could be considerably improved by multiprocessing. Python only utilizes one central processing unit because of GIL. Unlike C language or Java language, which instantly employ multiprocessing. It is a plug that only enables one process to run simultaneously. The multiprocessing package in Python, therefore, may address that issue. There are two modes in this unit which include Process and the Pool. These can help us execute a specific chunk of code concurrently.

Syntax of the pool() function:

We would explicitly construct procedures in the Process class. Although, the Pool class seems to be more useful because we can handle it dynamically. To construct a pool object, utilize the syntax:

Every parameter is optional. The ‘initargs’ is the third parameter passed to the method, the second ‘initializer’ parameter will also be provided, which is used for initialization. The ‘maxtasksperchild’ denotes the total number of operations every child process has been given. The process would be updated with a new worker procedure after completing the operations.

Let’s discuss further multiprocessing pool:

Example no 1:
Let’s have a look at the illustration below, which computes the square of the integer and sleeps the code for one sec.

import time
from multiprocessing import Pool
def square(a):
    print(f"Here the process begins:{a}")
    square = a * a
    print(f"square {a}:{square}")
    time.sleep(1)
    print(f"eHere the process ends:{a}")

if __name__ == "__main__":
    start_t = time.time()
    pool = Pool()
    pool.map(square, range(0, 9))
    pool.close()
    end_t = time.time()
    print(f"The total Time taken is {end_t-start_t} seconds")

At the beginning of the code, we have to import the time module and pool module. The pool library will be integrated from multiprocessing. We define the function square(). Next, we utilize the print() method to show the message ‘Here the process begins’. Here, we get the square of the number by multiplying the number with each other. Once again, we call the print() function to show the square of the defined number. We define the function sleep() to set the time frame of the code.

Now, the print statement prints the line which shows that the process is terminated. Let’s start with the body of the main function. We invoke the time() method of the module time. We construct a Pool class item. The method that we would like to replicate and an iterable have been provided as parameters to the map() method.

Additionally, it accepts an additional ‘chunksize’ parameter that divides the iterable into equal-sized segments and receives this one as a distinct operation. The parameter to the map() method is applied as the range. To deny new operations, utilize the pool.close() method.

Example no 2:
In this instance, we will set the data frame of the code for 2 seconds by using the sleep() method.

import time
from multiprocessing import Pool
def square(a):
    print(f"The process begins now {a}")
    square = a * a
    time.sleep(2)
    print(f"The process ends now {a}")
    return square

if __name__ == "__main__":
    pool = Pool()
    x = pool.map(square, range(0, 7))
    print(x)

The time package and pool package must be imported at the commencement of the program. Multiprocessing will be incorporated with the pool library. The method square() is defined. The message “The process begins now” is then shown by utilizing the print() function. By multiplying the values together, we may obtain the square of the number in this scenario.

To set the code’s time, we invoke the sleep() method. Further, the print command displays a line indicating that the process has ended. Within the body of the main function, we will invoke the pool() method. The function map() has been used in the next line. This function contains two parameters. The first argument shows the square of the number. And the second parameter uses the range() method.

We define the lower and higher limits within the range() function. In the end, we have been calling print() to show the result.

Example no 3:
Incredibly similar to the pool.map() function is the pool.imap(). The distinction here is that every element’s outcome is provided as immediately as it is processed, rather than having to wait for all items to be completed. Additionally, the iterable is transformed into a set by the map() function. The imap() technique, therefore, does not.

import time
from multiprocessing import Pool

def square(x1):
    print(f"sThe process has started {x1}")
    square = x1 * x1
    time.sleep(9)
    print(f" The process has ended {x1}")
    return square
if __name__ == "__main__":
    pool = Pool()
    x2 = pool.imap(square, range(0, 4))
    for j in x2:
        print(f"The result is here {j}")

First, we must include the time and pool framework. We take the pool library from the multiprocessing package. We call the square() function. This function contains the required number as its argument. To display the message indicating that the process has started, we have been calling the print() function. By multiplying the values together in this instance, we can obtain the square of the specified number. To determine the code’s time, we invoke the method sleep(). The statement that indicates that the process has ended has been printed by the print function. The square of the specified integer is then returned by the function.

Let’s begin the main function’s body now. The pool() function will be used. The next line makes use of the map() method. There are two parameters for this method. The square of the integer is provided as the first parameter. The range() function is being used as the second argument. Within the range() method, the lower and upper limitations are specified. Before ending the code, we will utilize the ‘for’ loop. We initialize the loop variable. This loop iterates until the given condition is fulfilled. After this, the print() method is applied to represent the result.

Conclusion

In this guide, we have talked about the multiprocessing pool of python. When numerous processes need to operate simultaneously or a lengthy operation would have to be sped up, multiprocessing is vital. When a process runs on a single thread, its capabilities are constrained; otherwise, this would stretch its tentacles through numerous threads. Python offers a simple interface to implement multiprocessing if the time-consuming operation can execute simultaneously and the entire system supports many processor architectures. We have executed three instances in which we set the time frame of the code.

About the author

Kalsoom Bibi

Hello, I am a freelance writer and usually write for Linux and other technology related content