Python - ModuleNotFoundError: no module named X

Why do we encounter ModuleNotFoundError?

A few simple reasons for ModuleNotFoundError could be:

1. The module is not installed in your Python environment

2. The module name is misspelled

3. Trying to import a module using the incorrect path

The first two reasons are easily fixable. However, the third is a tricky one. Sometimes, fixing the import path by adding correct sub-modules is easy. Still, there could be more to it if your project structure is complex and you have no choice but to append a relative path to the current directory in the PYTHONPATH list based on what is your current working directory.

Example

Let's look at an example.

Suppose you have a project with some libraries out of the src module that you install separately in your dev/prod environment. You write all tests for complete test coverage in the src/tests/ folder, including libraries tests.

project-name
  - libraries
    - utils
      - constants.py
    - library_1.py
    - __init__.py
   - src
      - api
      - tests
        - unit-tests
          - test_library_1.py
          - __init__.py
        - integration-tests

In the library_1.py file, you want to use some things from the constants.py file.

from utils.constants import X

def process(events):
  # do something using X

process({})

This will undoubtedly work when you run library_1.py, /libraries/ is your default working directory when deployed in your environment.

Things will stop working when you write unit_tests for library_1.py in test_library_1.py as the working directory for the whole project is now /project-name/, and library_1.py is using the module path relative to the library module.

from libraries.library_1 import process

def test_something():
  # test something

This test file will identify the libraries.library_1 file within project-name but will give the below error when you run the test.

ModuleNotFoundError: no module named utils

But why? The library runs fine independently, so why do we encounter this error? The logical explanation is that the Python environment finds packages in the PYTHONPATH list, which should include /project-name/, and no utils module exists.

How to resolve it?

The hack is to add the check if the /library/ exists in your sys.path. If it doesn't exist, add it. This part should be done in your libary/__init__.py file because you are accessing modules/files in the library folder and the correct path should be available in the PYTHONPATH list before you import any sub-module in any file within this folder.

try:
  sys.path.index(os.get.dirname(os.get))
except:
  sys.path.append()