Python

Python Multiprocessing Example

Python is a programming language that is beneficial in developing websites and applications. Also, involved in data visualization and analysis. It is comparatively an easy language and is often used by even non-programmers. This article refers to the use of multiprocessing in python.

Multiprocessing

Multiprocessing deals with the potential of a system that supports more than one processor at a time. All applications in multiprocessing are demolished into small patterns, and each of those patterns is self-dependent in working. In this type of process, two or more processes are used. Just let us consider that a computer is running with the use of a single processor. If we assign several processors at once, it will strew each task switch to another task after a brief time to keep all the processes running. In other words, if there are more tasks to be done, more difficult, it will be to maintain them on track of completion. Whereas allocating each task equal time is a big challenge as well. In this condition, we need to arise the concept of multiprocessing. A multiprocessing system can allow.

  • Multiprocessor: A computer that uses more than one central processor.
  • Multi-core processor: A single component that uses two or more processing units.

Multiprocessing in Python

Using the multiprocessing concept, the central processing unit can easily execute several tasks at a single time. Python uses the multiprocessing module to introduce parallel codes. To implement multiprocessing, you first need to know about the number of cores in your system. We have used a single piece of code here to know about the CPUs working at a time.

For python, you may use any other tool to accomplish the codes. We have used the SPYDER tool for this purpose that will surely execute the Python language codes.

Import multiprocessing

print("Number of CPU : ", multiprocessing.cpu_count())

Firstly, we need to import the package “pg” multiprocessing that will enable the working. For each PC, the output may vary. For me, the number of cores is 4.

Example 1:

Using a multiprocessing module, we will calculate the cube and square functions.

Let us understand the above-mentioned code. We created a process, and then we created an object to process the class.

P1 = multiprocessing.process(print(print_cube(5, )))

P2= multiprocessing.process(print(print_square(2, )))

P1, and p2 are the objects of the processes. These processes contain the functional call having a print feature that will directly call the function with arguments as parameter. Print_cube, and print_square.

We use start methods to start the process.

P1.start()

P2.start()

One of the functions used is described as.

def print_square(num):

print("Square: {}".format(num * num))

The square function will compute the square by multiplying the given number used in the function call. The function takes the number in the variable “num”.

When we execute the code, the output will display the resultant values of both functions.

Example 2:

Using data structures, you must be familiar with the use of stacks and queues in the programming language. From the multiprocessing module, we import a queue class here that is FIFO (First In First Out) data structure. There will be two processes that are working alternatively. We use two functions here. Put() function is used here to insert data in the queue. Whereas we use get() to retrieve the items from the queue. Below is the code that illustrates the working of multiprocessing in the queue. The first step is to introduce queues.

From multiprocessing import queue

After importing the module, introduce the names of cars to the array variable. We will use a for loop to insert the values inside the queue, and the put() will do it so. A “cnt” variable is used to count the number of cars used. Initialize the queue object through the statement.

Queue = Queue()

Then a loop is used to add values; this will use a “queue” keyword to use the put() function to argue as the car’s name to be inserted. The “cnt” function will be incremented after each insertion.

For car in cars:

Queue.put(car)

Cnt +=1

Similarly, for put(), a while loop is used until the whole queue is not empty.

While not queue.empty():

Print(‘item no: ‘, cnt, ‘ ‘, queue.get())

Cnt+=1

The output shows both the functions working independently. Once a function completes its work, the other process is in used.

Example 3:

This example contains two processes. In each process, “PID” is obtained through the get() function. We have used two modules. One is multiprocessing, and the other is operating system OS.

Each process is allocated a different id.

Def process():

Print (“ID of process running worker1:{}”.format(os.getpid()))

The message is concatenated with the function call to receive the process id.  In the main program, the main process id is retrieved using the same approach through the get(). Then processes are created through multiprocessing. Each process is accessed through its respective objects. i.e., for process 1, the object is P1.

P1 = multiprocessing.Process(target=process1)

Target is the function that is to be executed by the process. As the processes are created, then they need to get started. The status of each process is displayed; for this, we need to monitor the completion of the process. This will need some time to wait. We will use a function that will wait till the complete execution of the process.

P1.join()

When this join function gets the signal, it prints the statement that the execution is completed. A Boolean flag will determine either the process is alive or not; if yes, then it’s true, else it’s false.

Print(“process p1 is alive:{}”.format(p1.is_alive()))

Now execute the program and take the output. You can see that first, the main ID is displayed and then the process id of each process. After the execution is finished, the status is displayed for both the processes, that is “false”.

Example 4:

In this example, you will see the working of the multiprocessing and import time, pool, cpu_count. POOL is used to execute processes with multiple inputs and distribute this input data among the processes. A square function will calculate the square of the input value. By using the time module from the package, we will use the sleep function.

Time.sleep(3)

This function has a time expansion of 3 seconds. It means that the value of this function will be shown after this specific time. In the main program, we start the timer, and then the dummy values of each 4 cores are allocated.

With pool() as pool:

Res = pool.map(square, values)

Print(res)

Pool function is giving values and calling the function at the same time. Then we need to stop the timer. This timer function will count the seconds used for this function, known as elapsed time; it is obtained by subtracting the start value of time from the end value.

Now execute the program. The first line is displayed, and then it takes 3 seconds. After that, another result is shown.

Conclusion

“Python multiprocessing example” is a brief introduction to the multiprocessing phenomenon. Each example explained here imports different packages from the multiprocessing module. This handling of processes parallel to one another is accomplished through the number of CPUs. More the number of CPUs greater will be the chance of good multiprocessing.

About the author

Aqsa Yasin

I am a self-motivated information technology professional with a passion for writing. I am a technical writer and love to write for all Linux flavors and Windows.