Python – Multi Directory Python Project

One of the most popular way of designing the structure of the code is, to contain all the required modules and packages in one place i.e. in one folder. Now, there can be multiple directories and packages within the project. Also, there can sub-directories and sub-packages in the given project. Basic challenge here is to make sure all the modules are visible to Python. Typical structure of the code can be like following..

Project/
|------/main.py
|------/setup.py
|------/Directory1/
|      |----------/module1.py
|      |----------/module2.py
|      |----------/SubDirectory1/
|                 |-------------/submodule1.py
|                 |-------------/submodule2.py  
|      |----------/Package1/
|                 |--------/__init__.py
|                 |--------/pModule1.py
|                 |--------/pModule2.py
|------/Directory2/    
|      |----------/__init__.py
|      |----------/myClass1.py
|      |----------/myClass2.py
|      |----------/Subdirectory2/
|                 |-------------/submodule3.py
|                 |-------------/submodule4.py      

Now the important part here for consideration is how to make sure that all directories and packages within the project are visible to Python? There can be two ways to approach this problem

1. Manually add all the paths of the directories into PYTHONPATH as described in tutorial1 and tutorial2.
2. Automatically add directories and subdirectories (including packages and subpackages) into PYTHONPATH

We can always add hardcoded paths into sys.path i.e. PYTHONPATH but, is it the efficient way of working? Of course not. Then how to add the directories-subdirectories structured as above into PYTHONPATH automatically?

Let’s see following piece of code.

This code basically iterates through the directories and subdirectories within os.getcwd() and adds them in sys.path i.e. PYTHONPATH. os.getcwd() is Current Working Directory. Please check this link to know more about os.walk.

If you can observe there is extra piece of code if(str(i).find(‘__pycache__‘) == –1 and str(i).find(‘.vscode‘) == –1): before appending the path of the directory into sys.path. What it basically means is that if there is __pychache__ or .vscode in name of the directories then DO NOT append the paths into sys.path because, it will be futile. __pychache__ are directories where interpreter stores cache information and .vscode is specifically for VS Code which has nothing to do with Python code as such.

This piece of code will add all the directories and subdirectories within current working directory into PYTHONPATH. User can replace os.getcwd() with any directory of choice. I hope this information will be helpful while designing a multi-directory Python project. Please do let me know your questions and comments in the comments section!