Python

Scheduling tasks Using Python

Some tasks in our life are really time taking and we get bored doing those tasks repeatedly. In order to avoid those boring tasks we automate those tasks in our system. Python has many automation modules which can be used to automate our stuff. Below we will discuss the importance of automation and how to automate our stuff using python. We will also discuss some automation modules in python which help to automate our stuff. Then we will see some use cases of automation using python. At the end we will see how we can schedule our python script to run automatically at start up.

Why We Automate Our Stuff

Automation of our stuff can make our life easier. It has many advantages but some important advantages of automation are given below

  • First and the most important advantage of automation is to save time.
  • We can increase efficiency of our work (i.e. more work in less time)
  • Accuracy of the tasks scheduled can be improved.
  • By automating stuff, human interaction with system is reduced so boring stuff is automated.

Automation with Python

There are some many scripting languages like bash, perl etc. to automate manual processes but python provides feasible environment to handle our needs. It is more easy to automate stuff using Python programming language than in any other language. Python has many automation modules (i.e. subprocess, shutil, os, pyautogui, selenium etc.) that can be used in automation. We will see some use cases of how we automate our stuff.

Automation Modules in Python

Here we will discuss some automation modules in python like OS, SUBPROCESS, SHUTIL, DATETIME, SELENIUM etc. These modules are used to execute system commands and can also be used to manipulate system directories. These modules have built in functions which make it easier to perform tasks on system.

OS and SUBPROCESS modules are used for almost the same purpose. Some of the builtin functions of these modules are given below

  • chdir() \\ To Change directory
  • mkdir() \\ To Create new directory
  • rename() \\ To rename a file
  • listdir() \\ To list all files and directories
  • getcwd() \\ To know our current directory
  • system() \\ To run system commands
  • call() \\ To run system commands

Similarly SHUTIL module has many functions which allows to interact with system. Some functions of this module are given below

  • move() \\ To move a file
  • copy() \\ To copy a file
  • rmtree() \\ To remove all files in Directory and subdirectories

DATETIME module is used to find the current date and time. It is also used to find week day and many other things. We can schedule our task at any time and date using this module. In the following use cases we will use it to find only current date and time.

  • datetime.today() \\ To find current date and Time

SELENIUM Module is used to automate our website logging in process. We can automatically log into our website and update it using SELENIUM module. SELENIUM has many builtin functions which are used to perform different tasks related to browsers.

How to Execute System commands in Python

We can run system commands in python using OS and SUBPROCESS modules. Following is the syntax to run system command in python

os.system(‘command’)

OR

subprocess.call(‘command’)

For example, if we want to list all the files and directories in the current directory, we will run the following command in python script

os.system(‘ls’)

OR

subprocess.call(‘ls’)

How to Automate Stuff With Python

Python can be used to automate many kinds of daily tasks which can be time saving. Here we will see some use cases of automating our stuff. We will discuss the arranging of files in the download directory and keeping a backup of our important files.

Arranging Files in the Download Directory

When we download any type of file, by default it goes into Download directory. When we have to find any file then it may cause problem as there are many types of files mixed in Download directory. Now we will write a python script to automate the system in such a way that different types of files (e.g. mp3, png, xls etc) goes in different directories. Complete code is given below. This code will check each file type one by one then it will create corresponding directory. After this it will move file into related directories (e.g. .mp3 file will go into ‘Audio’ directory).

import os
import shutil
import datetime
while 1:

   #calculating current hour, minute and second
   today = datetime.datetime.today()
   today = str(today)
   current_hour = today[11:13]
   current_minute = today[14:16]
   current_sec = today[17:19]

   # making sure that system will arrange files at 08:00
   if current_hour == '08' and current_minute == '00' and current_sec == '00':

       # changing directory to download
       os.chdir("path_to_Download_directory")
       # saving all file names in a list
       files = os.listdir(os.getcwd())

       for filename in files:
           # ignoring directories
           if not os.path.isdir(filename):

               # selecting mp3 files
               if '.mp3' in filename:
                   # creating 'Audio' directory if not exist
                   if not os.path.exists('Audio'):
                       os.mkdir('Audio')
                   # moving file in 'Audio' directory
                   shutil.move(filename, 'Audio')

               # selecting mp4 files
               elif '.mp4' in filename:
                   # creating 'Video' directory if not exist
                   if not os.path.exists('Video'):
                       os.mkdir('Video')
                   # moving file in 'Video' directory
                   shutil.move(filename, 'Video')

               # selecting pdf files
               elif '.pdf' in filename:
                   # creating 'PDF' directory if not exist
                   if not os.path.exists('PDF'):
                       os.mkdir('PDF')
                   # moving file in PDF directory
                   shutil.move(filename, 'PDF')

               # selecting jpg and png files
               elif '.jpg' in filename or '.png' in filename:
                   # creating 'Pictures' directory if not exist
                   if not os.path.exists('Pictures'):
                       os.mkdir('Pictures')
                   # moving file in 'Pictures' directory
                   shutil.move(filename, 'Pictures')

               # selecting excel files
               elif '.xls' in filename:
                    # creating 'Excel' directory if not exists
                   if not os.path.exists('Excel'):
                       os.mkdir('Excel')
                   # moving file in 'Excel' directory
                   shutil.move(filename, 'Excel')

               # selecting '.ppt' files
               elif '.ppt' in filename:
                   # creating 'Power Point' directory if not exists
                   if not os.path.exists('Power Point'):
                       os.mkdir('Power Point')
                   # moving file in 'Power Point' directory
                   shutil.move(filename, 'Power Point')

               # selecting '.docs' files
               elif '.docx' in filename:
                   # creating 'Word File' directory if not exists
                   if not os.path.exists('Word File'):
                       os.mkdir('Word File')
                   # moving file in 'Word File' directory
                   shutil.move(filename, 'Word File')

First of all, we will save the current time in different variables then we will check if the time is exact ‘08:00’ or not. Program will check the time every day and will run the code if time is ‘08:00’, then the main code will run. In main code, First of all we change our directory to Download directory. Here we will save all filenames in a list named files. Now we will read all the files one by one and filter out only files. We will ignore directories as we are going to arrange files only.

Now we will check each file type whether it is mp3, mp4, jpg, pdf, xls, ppt and docx or not. After checking each file type we will check whether corresponding directory exists or not. For example, if file is mp3 then we will check ‘Audio’ directory exists or not. If corresponding directory does not exist, we will create the directory. After creating directory we will move our file in that directory. In this way all the files can be moved to their corresponding directories.

This code will run continuously and go on checking if the time is ‘08:00’ or not. Everyday at ‘08:00’, files will be arranged automatically. This code can be scheduled to run every time when you start your system using crontab. Crontab has been explained below.

Automatically Backup your files using system commands

You have some important files in a directory related to your project and somehow some files are deleted. Then what will you do? In this kind of situation, creating a backup of your files is important. But creating a backup of your files is really boring and time taking task. This task can be done automatically by writing a python script. Following is the code to perform this task. This code will convert every file into zip file then it will create a backup directory if does not exist. After this, zip file will be moved in backup directory.

import os
import datetime
while 1:
    # saving current time
    today = datetime.datetime.today()
    today= str(today)
    current_hour = today[11:13]
    current_minute = today[14:16]
    current_sec = today[17:19]

    # making sure code will run at exact '08:00'
    if current_hour == '08' and current_minute == '00' and current_sec == '00':
        # changing directory to documents
        os.chdir('path_to_documents_directory')
        # saving all file names in a list
        files = os.listdir(os.getcwd())
        # creating 'backup' directory if not exist
        if not os.path.exists('backup'):
            os.mkdir('backup')

        for file in files:

            # ignoring directories
            if not os.path.isdir(file):
                # defining a filename without spaces
                original_name = file
                file = file.split(" ")
                file_name = "".join(file)
                # defining zip_filename
                zip_file_name = file_name+".zip"

                # checking if file already exist in backup directory or not
                if not os.path.exists('backup/'+zip_file_name):
                    # changing file name without spaces
                    os.rename(original_name, file_name)
                    # creating zip file using system command
                    os.system("zip "+zip_file_name+" "+file_name)
                    #moving zip file in backup directory using system command
                    os.system("mv "+zip_file_name+" backup")
                    # changing filename to its original name
                    os.rename(file_name, original_name)

First of all we will save current time in variables and then we will check if the time is ‘08:00’ or not. If time is exact ‘08:00’, then the main script will run. In the main code, first of all, we go to the directory in which important files are present. Then we save all the file names in a list. Now we will create a directory ‘backup’ in which we will save all the zip files. If this directory already exists then we will ignore this. Now we will read each file and check if these are files or not. Directories will be ignored and files will be considered.

In order to create a zip file of a file using system command, first of all we will rename this file without spaces as a filename with spaces is considered a different file at every space, when run into a system command using python script. The name of the file is saved in a variable ‘original_filename’ and the file is renamed without spaces. Now we will define zip file name same as the file’s name with extension ‘.zip’.

After this we will use system command ‘zip’ to zip this file and then we will move this zip file in ‘backup’ directory using system command ‘mv’. Then we will again rename the file with its original name with spaces in it. This code will be scheduled on system to run every time automatically when you start your system using crontab.

Automate running a script using Crontab

We have written the automation scripts above. Now we want to run these scripts automatically whenever our system reboots. To do so, we add our script in crontab. In order to add task in crontab do the following steps

First of all, type the following command to edit crontab file

ubuntu@ubuntu:~$ crontab -e

-e flag means open crontab in editing mode. After opening crontab file now we will add our task in this file using following command at the  end of file

@reboot python /path/to/python/script

This will run python script automatically every time when you start up your system.

Conclusion

In this article, the importance of automation in practical life has been explained. By reading this article, you get an idea that how you can automate your simple tasks that you do every day. You should definitely search for more python libraries and functions for better and easy automation.

About the author

Usama Azad

Usama Azad

A security enthusiast who loves Terminal and Open Source. My area of expertise is Python, Linux (Debian), Bash, Penetration testing, and Firewalls. I’m born and raised in Wazirabad, Pakistan and currently doing Undergraduation from National University of Science and Technology (NUST). On Twitter i go by @UsamaAzad14