Python Tools

One of the reasons for the popularity of the Python programming language is its wide range of tools, modules, and their applications in various fields.

In this article, first, we will learn the four common Python tools with examples. Then we will see some other libraries related to the popular fields in which Python is used for development.

Four common tools of Python

The Dis, Tabnanny, Profile, and PDB are the four majorly used modules by Python developers. We will discuss each of these in this section.

1. Dis Module

This module is used in Python to analyze the working of the program. The dis() function in this module converts the byte code into a human-readable format. We can use this function by importing the module or from the command line.

It takes the input of the code, which can be a function, class or any Python object. And then compiles it, disassembles the code, and returns the output of the code.

Example of the dis():

import dis

def max(seq):
    m=seq[0]
    for i in range(1,len(seq)):
        if(m<seq[i]):
            m=seq[i]
    return m

dis.dis(max)

Output:

We called the function dis() using the dis module as the reference. We can see the steps in the function are printed in each line in an understandable form.

2. Tabnanny Module

We know that indentation is one of the important aspects of a python program. Without proper indentation, we might either get an unexpected output or even worse, a syntax error.

This module helps in checking if all the lines of the input code are properly indented or not. The first three letters of the module name, i.e. ‘tab’, refer to the indentation.

We can use the check() function in this module to do the above tasks. This can be used either by importing the module or from the command line. For example,

Let the below code be saved in the file named ‘loop.py’.

for i in 'PythonGeeks':
if(i in 'aeiou'):
  print(i)

We will import this Python file and check if the indentation is proper or not.

Example of the check() from Tabnanny module:

import tabnanny
import os

os.chdir('F:\\c\\Documents')
tabnanny.check('loop.py')

This module has the following functions:

a. tabnanny.check(dir or file):

This function as discussed takes the input as a directory or a python file and checks if the indentation is correct or not. It returns the result of analysis in the standard output.

b. tabnanny.verbose:

This is a flag that indicates if the verbose message will be printed or not.

c. tabnanny.filename_only:

This is also a flag, but it shows whether those Python file names are printed which has any indentation or whitespace related problem.

d. tabnanny.process_tokens(tokens):

This function is used by the check() function to process the tokens that are generated by the tokenize module.

e. exception tabnanny.NannyNag

This is the exception raised by the process_tokens() function when code is not correctly indented. This function is used and handled by the check() function.

3. Profile Module

Python provides modules cProfile and Profile for the profiling purpose. Profiling is a process of finding the statistics of how often and for how much time different parts of program execution.

The cProfile is a C extension module, which is recommended and Profile is a Python module that is imitated by cProfile. We can import any of these modules for profiling purposes.

For example, let us create a Python file with code and see its statistics. Let the following code be saved as ‘code.py’.

def func(seq):
  sum=0
  for i in seq:
    sum=sum+i
  return sum

func([1,-4,6,2,3,-6,0,9])

Now let us do profiling on this code.
Example of profiling using cProfile:

import cProfile
import os

os.chdir('F:\\c\\Documents')
cProfile.run('code.py')

Output:

One can see various values in the output. Let us discuss each of these for understanding :

a. In the first line the value 3 means, 3 calls were monitored

b. In the next few lines, we have values in different columns. Here, ncalls mean number of calls made

c. And tottime is the total time spent for that function

d. And percall is equal to the value of the quotient of tottime divided by ncalls

e. Then cumtime is the cumulative time spent in this and all subfunctions

f. And the next percall is equal to the value of the quotient of cumtime divided by ncalls

g. The last column filename:lineno(function) provides the details of the function

4. PDB Module

This is a module used for interactive debugging purposes. This module helps in

a. Setting breakpoints

b. Debugging the code line by line

c. Inspecting stack frames

d. Listing the source code

e. Evaluating arbitrary code in the context of any stack frame

It has different functions for debugging in different ways. Let us see an example of using one of these functions.

Example of set_trace() function in pdb:

import pdb
  
def sq(x):
    return x**2
  
pdb.set_trace()
a = input("Enter a number : ")
square = sq(a)
print(square)

Output:

We can see that the execution stops at the line where we insert the debugging command and we get the debugging prompt(ipdb>).

Application-Specific Tools in Python

Till now we saw the basic modules required for a Python developer. In this section, we will discuss some of the Python tools being used in the most popular fields of applications.

These tools are :

1. Data Science

a. Scikit-learn

This is one of the open-source modules commonly used by Data Scientists and Machine learning engineers. It holds different algorithms like regression, classifier, clustering, etc. It also contains various metrics and preprocessing tools.

These help the developers build a complete model using this module. It also makes the coding process simpler and also reduces the length of the code. Because one can find these friendly facilities, it is preferred by most programmers.

b. Keras

Keras is also an open-source module that is used for data science projects. It is used for machine learning and deep learning models. It is mainly used for building neural networks.

This also reduces the work of a programmer by giving the facility of more functionality in fewer lines of code. It is generally used along with frameworks like TensorFlow and Theano.

c. Theano

This library is used for handling multi-dimensional arrays. It also includes the features of NumPy. This helps to do mathematical computations on these arrays in an easier way. Its other advantages include speed, efficient use of GPU, stability, transparent differentiation of symbols, etc.

d. SciPy

The name SciPy stands for Scientific Python. It is an open-source library used for scientific and mathematical computations. It includes the facilities of the other libraries like NumPy, Pandas, and IPython.

This tool helps in doing science and mathematical relating tasks like computations on numbers, creating nonlinear structures, generating visuals, etc.

2. Automation Testing Tools

a. Selenium

Selenium is the most recommended tool for automation testing on web applications. Besides being open source, it can also work on various languages including Python, C#, Java, Pearl, PHP, Ruby, and .Net.

It can work on any browser like Chrome, Firefox, Safari, Opera, and Internet Explorer. It can also be used in different operating systems like Windows, Linux, and Mac. In addition, it can be integrated with other tools like JUnit and TestNG.

b. Robot Framework

This is an open-source generic automation testing framework. It is used for acceptance testing and acceptance test-driven development. It is based on a keyword-driven approach and its test data is in tabular format.

This tool can work with Python, Jython, PyPy, and IronPython. Besides being used for web application testing, it can also be used for android and ios applications.

c. TestComplete

This is an automatic testing framework that can be used for web, android, ios and windows applications. One needs a license to use it. Like the robot framework, it is also a keyword drive.

It is source language independent and can work on scripts in languages including Python, VBScript, and C++ script. One of the facilities it provides is that the test steps, objects, actions, and data are separate and easily recognizable.

3. Web Scraping Tools

a. Beautiful Soup

This is a widely used library for web scraping, the process of extracting data from websites. It is open-source and is written in Python. This is used to parse the HTML and XML codes. It provides an easier way to modify and read the data in the parse tree using the parser, saving the work of the programmer.

b. LXML

This is a Python library used to handle HTML and XML files. It is designed for C libraries libxml2 and libxslt, which can be accessed using the ElementTree API. This API can also be used with XPath, RelaxNG, XML Schema, XSLT, and C14N. It combines the speed, XML features, and simplicity of Python.

c. Scrapy

It is also a free and open-source Python framework for web crawling. Besides being used for web scraping, it is also used to extract data using APIs. This is a tool that can be used for all web crawling activities. It facilitates the extraction of data from the websites, processing the data, and storing it in a preferred format. It also provides facilities like data mining, automatic testing, etc. This can be used in operating systems like Windows, Linux, and macOS.

Other Common tools in Python

1. SymPy

It is an open-source Python tool used for symbolic mathematical computations. It helps in doing computer algebra and also makes the code simple and extensible.

2. NumPy

NumPy stands for Numeric Python. It is mainly used to handle multi-dimensional arrays and also provides different functions for manipulations. This module is used by data analysts and data scientists to do mathematical operations.

3. DateTime

It is a module that provides classes to get information about dates and times. It can be used to get this information at the current location or any other place in any part of the world.

4. DateUtil

This module can be thought of as an extension to the DateTime module. It provides the additional functionality of doing operations of the date and time like generating a calendar of a year etc.

5. Tkinter

Tkinter is a standard library used for GUI applications. It can be used to make GUI applications, both user interface and its functionality.

6. Pandas

This is an open-source library used for data analysis. It is built on top of NumPy. It is mainly used to access data in tabular form, process it, and save it in the required format.

7. PyTorch

It is an open-source library used to create and build Machine learning and deep learning neural network models. It is used in applications like computer vision, natural language processing, etc.

8. OpenCV

This is an open-source library used for image processing purposes. This module provides a wide range of methods to modify images and acquire different required information from the images.

Conclusion

In this article, we discussed various Python tools used for different applications. Seeing all these we can conclude about the versatility of the Python language.

Hoping that reading this article helped you. Explore more about the tools that interest you. Happy learning!