Subprocess in Python

Do you know that you can execute the external processes in Python? We will learn about this feature of Python using the module names ‘subprocess’. So let us start with an introduction to the subprocess in Python.

What is Subprocess in Python?

Subprocess is the task of executing or running other programs in Python by creating a new process. We can use subprocess when running a code from Github or running a file storing code in any other programming language like C, C++, etc.

We can also run those programs that we can run on the command line. Also, this module provides various commands to obtain the input/output/error pipes.

Subprocess in Python

We can do the above tasks in Python using the subprocess module in Python. We get this module by default when we install it. This is for the version 2.x and 3.x. So, we can import this module directly by writing the below code.

import subprocess

The methods in this module can be used to perform multiple tasks with the other programs. This module can be used as an alternative to the following functions or modules in Python:
1. commands*
2. os.system
3. os.spawn and other related functions
4. os.popen and other related functions
5. popen2*

As said before, we can obtain inputs, outputs, etc. We have different commands and these include:
1. call()
2. run()
3. check_call()
4. check_output()
5. Popen()
6. communicate()

We will discuss each of these commands in the next few sections.

Call() function in Subprocess Python

This function can be used to run an external command without disturbing it, wait till the execution is completed, and then return the output. Its syntax is

subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)

In this function, the argument

1. arg is the command that needs to be executed. We can pass multiple commands separated by a semicolon (;)

2. stdin refers to the value of the standard input stream passed as (os.pipe())

3. stdout is the value of the standard output stream

4. stderr is used to handle errors, if any, that occurred from the standard error stream

5. shell is the boolean value. If it is true then the program executes in a new shell.

This function returns the output of the execution. If there is no output, it returns the code that is executed successfully. It might also give a CalledProcessError exception.

Let us see an example.

Example of call() function:

import subprocess

subprocess.call(['ls', '-1'], shell=True) #using the call() function

Output:

In this example, the ls command is the list of all the files in a directory. The -l command lists the directories in an extended format. We set the shell argument as true because we want the subprocess to consider these commands as a single command and run. By default, it runs the command directly.

We got the output as 1 here which indicates that it executed successfully.

Run() function in Subprocess Python

This function is similar to the call() function. It runs the input commands and waits till all the commands are completed. Then runs the instance of the CompletedProcess class, which represents that the process has finished. Its syntax is

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, text=None, env=None)

In this function, the argument

1. args take the commands that are to be executed

2. stdin refers to the value of the standard input stream passed as (os.pipe())

3. stdout is the value of the standard output stream

4. stderr is used to handle errors, if any, that occurred from the standard error stream

5. When capture_output is true, stdout and stderr will be captured

6. timeout decides when the child process should be killed and waited for.

7. check argument checks if the process exits with a non-zero exit code. If this argument is true and a non-zero exit code exists, then a CalledProcessError exception will be raised.

8. If encoding or errors are specified, or text is true, then the file objects for stdin, stdout, and stderr will be opened in text mode using the given encoding and errors. By default, they will be opened in the io.TextIOWrapper format.

9. env should either be None or a mapping that defines the environment variables for the new process. These variables will be used instead of the default ones from the current process’ environment.

Example of run() function:

import subprocess

subprocess.run(['ls','-l'],shell=True) #using the run() method

Output:

Subprocess function check_call() in Python

This function runs the command(s) with the given arguments and waits for it to complete. Then it takes the return value of the code. If it is zero, it returns. Or else it raises CalledProcessError.

Its syntax is

subprocess.check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False)

In this, the parameters are:

1. arg is the command that needs to be executed. We can pass multiple commands separated by a semicolon (;)

2. stdin refers to the value of the standard input stream passed as (os.pipe())

3. We give the value of the standard output stream to the stout parameter

4. stderr is used to handle errors, if any, that occurred from the standard error stream

5. shell is the boolean value. If it is true then the program executes in a new shell.

Example of check_call() function:

import subprocess

subprocess.check_call('False',shell=True) #using the check_call() method

Output:

Subprocess function check_output()

This function is similar to the check_call() function. It runs the command(s) with the given arguments, waits for it to complete, and takes the return value of the code. If it is zero, it returns the output as a byte string. Or else it raises CalledProcessError.

Its syntax is

subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)

In this function, the parameters are:

1. arg is the command that needs to be executed. We can pass multiple commands separated by a semicolon (;)

2. stdin refers to the value of the standard input stream passed as (os.pipe())

3. stderr is used to handle errors, if any, that occurred from the standard error stream

4. shell is the boolean value. If it is true then the program executes in a new shell.

5. universal_newlines is a Boolean value. If it is set to true, then the stdout and stderr files are opened in universal newline mode.

Example of check_output() function:

import subprocess

subprocess.check_output(["echo","Welcome to PythonGeeks!"],shell=True)
 #using the check_output() method

Output:

Popen() in Subprocess Python

It is a constructor that creates and handles processes. It executes the program in a new process by using the function CreateProcess() in Windows. Its syntax is

class subprocess.Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None, text=None)

The parameters in this construct are:

1. arg can be a sequence of program arguments or else a single string or a path-like object.

2. buffer 0 means unbuffered, if it is 1 then it means line buffered, if any other positive value is given then it means to use a buffer of approximately that size and if it is negative then the default value will be used.

3. The executable argument is the program that replaces the args that is to be executed

4. stdin refers to the value of the standard input stream passed as (os.pipe())

5. stdout is the value of the standard output stream

6. stderr is used to handle errors, if any, that occurred from the standard error stream

7. A callable object is set to preexec_fn. This object will be called in the child process before its execution

8. close_fds decides if the file descriptors should be closed or should follow the default flag before the execution of the child process

9. shell is the boolean value. If it is true then the program executes in a new shell.

10. cwd is set to change the current execution directory

11. env should either be None or a mapping that defines the environment variables for the new process. These variables will be used instead of the default ones from the current process’ environment.

12. universal_newlines is a Boolean value. If it is set to true, then the stdout and stderr files are opened in universal newline mode.

13. When we set restore_signals to true (the default) all signals that Python has set to SIG_IGN are restored to SIG_DFL in the child process before the execution

14. If start_new_session is true, then setsid() will be called in the child process before the execution

15. If encoding or errors are specified, or text is true, then the file objects for stdin, stdout, and stderr will be opened in text mode using the given encoding and errors. By default, they will be opened in the io.TextIOWrapper format.

Example of Popen() function:

import subprocess

process=subprocess.Popen(['echo','"Hello World!"'],stdout=subprocess.PIPE,shell=True)
out_value=process.communicate()[0]
repr(out_value)

Output:

We will see the communicate() function seen in this example in the next section.

Communicate() function in Subprocess Python

This is one of the methods in the Popen class. It interacts with the process until the end-of-file is reached, which includes sending data to stdin, reading data from stdout, and stderr. Its syntax is

Popen.communicate(input=None, timeout=None)

The parameters in this function are:

1. input is the data that is to be sent to the child process. It must either be a string or bytes.

2. timeout is like a limit after which if still the process does not end, TimoutExpired exception will be raised.

Example of Popen() function:

import subprocess

process=subprocess.Popen(["echo","Python is fun!"],stdout=subprocess.PIPE,shell=True)
process.communicate()

Output:

Conclusion

In this article, we discussed the subprocess and its implementation in Python using different commands.
Hope you understood all the concepts covered. Happy learning!