In the previous tutorials we saw how to Debug Python with Visual Studio Code?, how to set PYTHONPATH?Modules , Paths -Packages, Multi-directory projects and Virtual Environment. So far whatever we have seen were basics of Python. In this tutorial let’s move to the next level. Let’s discuss how to structure you Python project? I don’t claim to be the best in Python, I am just another student who likes to share his findings with others. While seeing how to structure the project I will try to explain the reasons according to my ability.
Context
We can edit the environment variable PYTHONPATH externally or can append the paths to sys.path(). But, is it really the right way? Answer is yes and NO. For instance if you are working on trivial modules with limited scope or a standalone executable then it is okay to append the paths in sys.path(). Generally for good packages which are going to get get distributed editing sys.path() is not recommended. The reasons are
Maintainability:
If we are dealing with stand alone executable then we need not worry of sys.path() because its scope won’t exceed beyond the executable. But, these factors come into picture if we are trying to create a distributive package. By appending paths directly into sys.path() basically we make all the modules and packages available to any module in the project. This becomes a maintainability headache especially when other users try to use your code in different ways. Also, users get into trouble due to wrong references. It’s always a good practice to call a method along with its namespace which gets difficult when there are duplicate names available. Which certainly must be avoided in order to have good maintainability.
Portability:
If you want to create a library or a package which other users can download and install, then certainly it doesn’t make sense to modify PYTHONPATH real time. Packaging is the standard practice to distribute the code. It’s always better to get the root of the package in PYTHONPATH rather than all directories by default. It avoids erroneous process of finding the root of the project using os.path() etc. With projects like Setuptools to create source distributions and twine to upload the distribution, it makes more sense to do the right thing from the beginning.
How To Structure The Project?
A “Structure” is nothing but the specific arrangement of the code. The structure decides the flow of the data both inward and outward. As mentioned earlier, the best and standard way of distributing the code is to make into a package. There are different ways to create a package in Python. Setuptools is according to me the simplest packaging tool to use. I will cover how to register and upload Python package to PyPi in next tutorial. In this tutorial I will explain whatever I have understood so far about good structuring practices. Ideally the repository should be installable using pip.
In this blog, I will try to stick to the project structure only. I will post tutorials about standard coding practices and also, the code itself. This will help you creating an installable package in Python. But, to make this package available to all you will need to register and submit (upload) it on PyPi, which I will cover in some other blog later on.
Ideally speaking the codes/modules/packages are collected in a directory. Some may call it repository, but it’s the same thing. Over time it becomes a massive repository of files. Hence, if it is not structured well then it becomes a mess.
Please take a look at following directory structure.
setup.py
LICENSE
README
requirements.txt
MyLibrary/
/__init.py
/samplemodule.py
You can refer to this public Github repository created by Me (Rohit Bapat) which has same structure. It has five basic parts (of course there can be more parts in more complex projects).
setup.py
setup.py has all it needs to make a package installable and distributable. This file must be kept at the root directory where the module package lies. In the example above ‘MyLibrary’ is the module package and hence setup.py is kept in the same directory/place.
Take a look at following sample setup.py from this github repository (I think it is fairly clear. Please post any questions if any in the comments section)
from setuptools import setup, find_packages
with open('README.md') as f:
readme = f.read()
with open('LICENSE') as f:
license = f.read()
setup(
name='MyLibrary',
version='0.0.1',
description='Sample Package to Demonstrate Structure for Python Code',
long_description=readme,
author='Rohit Bapat',
author_email='rhtbapat@gmail.com',
url='https://github.com/rhtbapat/BasicStructure',
license=license,
packages=find_packages()
)
LICENSE
Believe or not, this is the most important file apart from the code itself. The full license text and copyright claims depending on the type of the license should exist in this file.
There is actually no need to put the license file in the project but, then it just makes your code vulnerable to be used/copied by anyone. You can use chooselicense.com to choose which license fits your purpose.
README
This is an extra file which will include miscellaneous information such as Information about the author, Websites, Installations instructions etc. (Clearly not a must but should be there ideally)
requirements.txt
This requirements file should contain list of any third party dependencies
MyLibrary
This is a sample python module package I assembled for this blog. Name does not matter as long as it is mentioned correctly in setup.py
This package is like any other python package. The one in the sample Github repository has __init__.py and a samplemodule.py . It can contain sub-packages and sub-sub-packages. Keeping __init__.py empty is very normal.
Installation Using setup.py
This is probably the simplest part. Because merely creating a package doesn’t make any sense if you can’t use it in standard way. In previous blogs I talked about sys.path and modifying PYTHONPATH but, creating package like this is the best, safest and standard way. To install this package use following commands.
Windows
Mac OS X and Linux
Once, the installation is complete you will find the package installed with all the modules and sub-packages in “<Python Installation Dir>\Lib\site-packages” and don’t have to worry about PYTHONPATH etc. etc.
Thank you for reading. I just shared whatever I have learnt so far. I don’t claim this to be the perfect way (because nothing like perfect as such). If you have any questions or suggestions please do use Comments section!