Cloud Information and Distribution

How to Run Linux Commands With Python

Python is the language of choice for shell scripting and task automation. It is popular in system administration because it can execute shell commands using only its default libraries. In this tutorial, you will learn how to run Linux shell commands with Python using the os and subprocess modules.


PCBWay Ad

Introduction

Nowadays, automation is a buzzword as hot as a processor running at 100% in a room with no AC. Home management, data extraction, the omniscient talking boxes in the kitchen, and even self-driving cars—they are all products of automation. Well, it makes sense since we’re living in the fourth industrial revolution. Heard of it? It is where humans and machines try to work as one, and it is happening now.

Using a Raspberry Pi even opens more doors in automation. With its GPIO, you can interface the credit-card sized computer into almost anything that throws or receives digital data. And since it is a fully-fledged Linux computer, you can automate your computer tasks as well. What’s more, Python makes it unimaginably easier! There are two ways to run Linux commands with Python: using the os module and using the subprocess module.

Using the OS Module

First is the os module. According to official documentation, the os module provides a portable way of using operating system dependent functionality. Ain’t that convenient? With short python codes, you can already perform standard operating system tasks without having to interact with the Desktop interface. The system method allows you to do exactly this. To use it to run a Linux command, your code should look like below.

Sample Code using system()

import os
os.system('pwd')
os.system('cd ~')
os.system('ls -la')

This 4-liner checks your current directory, change location to your home directory, and lists all the contents in detail. It’s a pretty straightforward implementation, but there’s a downside. With system(), you are not allowed to store the resulting output as a variable.

Instead, you can use the popen() method, which is still under the os module. It opens a pipe from or to the command line. A pipe connects a command’s output to another command’s input. This makes it accessible within Python. To use popen() to store as a variable, see the example code below.

Sample code using popen()

import os
stream = os.popen('ls -la')
output = stream.readlines()

If you print the stream variable, you will see its return data. This consists of the actual commands executed, the mode, and the address. Furthermore, if you want to get the whole output as one string, change readlines() to read().

Using the Subprocess Module

The second way to run Linux commands with Python is by using the newer subprocess module. This module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. It was created to replace both os.system() and os.popen() functions.

The only method that matters in the subprocess is run(). With it, you can do everything we’ve done above and more using different arguments. Use the following codes as reference:



Writing a simple command using subprocess

import subprocess subprocess.run('ls')

Using the method like this will execute the command ls in your terminal. Unlike os.system(), it doesn’t work when you add a switch and enter it fully like subprocess.run('ls -la'). This feature allows the method to take care of quoting and escaping problems hence preventing errors with formatting. To execute ls -la, you must pass the command as a list: subprocess.run(['ls','-la'). Alternatively, you can make the shell argument True to pass the whole thing as a string. Just take note that this can pose a security risk if you’re using an untrusted input source.

Writing a command with switches

import subprocess x = subprocess.run(['ls', '-la'])
import subprocess subprocess.run(['ls -la'], shell=True)

Next, to store the command output inside a variable, simply do it just like any other data. The result won’t be what you’re expecting, however. Since the main purpose of ‘run’ is to execute the shell command within python, the result won’t be like the output you see in the terminal. It will be the return data just like in os.open. You can check it using the code below.

Storing the command output to a variable

import subprocess x = subprocess.run(['ls', '-la'])
print(x)
print(x.args)
print(x.returncode)
print(x.stdout)
print(x.stderr)

This sketch dissects the return data of your command using the method’s arguments. Here are some of the frequently used ones:

  • args – returns the actual commands executed
  • returncode – returns the return code of the output; 0 means no error
  • stdout – captured stdout from the child process
  • stderr – captured stderr stream from the child process

Since we did not capture the previous code’s output, we will get ‘none’ with both stdout and stderr arguments. To enable the capture output argument, refer to the following code:

import subprocess x = subprocess.run(['ls', '-la'], capture_output=True)

If you print x, you will get the list of items in your current directory of type bytes. Convert it to a string by writing x.stdout.decode(). Alternatively, you can pass the argument text=True with the main function. The output should now look exactly the same as what you have in the terminal.

Lastly, we will run Linux commands with Python and save the output you see in the terminal into a text file—a simple task with subprocess. You just need to redirect the stdout stream to your text file using the argument stdout.

Saving the command output to a text file

import subprocess with open('list.txt', 'w') as f: subprocess.run(['ls','-la'], stdout=f)

That’s it for this tutorial! To summarize, for quick execution of a short script, consider using os.system() and os.popen() methods. But if you have a more comprehensive script to run, you might want to use the subprocess module instead.