python: source code layout

recently i discovered an interesting issue topic about how to layout source code in a python project; the topic is centered around one question:

put source packages in a src/ dir, or directly under the project root dir?

reference project

to make things clear, our experiments use this reference project: pypa/sampleproject @ccf222d, which is its master branch at this time of writing;

run the following commands to checkout this project:

$ cd /tmp
$ git clone https://github.com/pypa/sampleproject.git
$ cd sampleproject
$ git checkout ccf222d

to make sure our code snippets in this article can be reproduced, we also create and activate a virtualenv:

$ python3 -m virtualenv env
$ . env/bin/activate

check we are now with the python tools in the virtual env:

(env) $ which python
/tmp/sampleproject/env/bin/python
(env) $ which pip
/tmp/sampleproject/env/bin/pip

source code layout problems

there are several problems related to source code layout in a python project;

namespace pollution with editable install

when you editable install a python project, you are introducing its root path into sys.path:

(env) $ pip install -e .
(env) $ python -c 'import sys; print(sys.path)'
['', ..., '/tmp/sampleproject']

the result is, you can import the package name from anywhere:

(env) $ python -c 'import sample; print(sample.__path__)'
['/tmp/sampleproject/sample']

while the one above may seem ok, the one below looks awkward:

(env) $ python -c 'import tests; print(tests.__path__)'
['/tmp/sampleproject/tests']

when you start a new python interpreter and import a module named tests, you probably dont expect it comes from a specific package; what is worse, it may import different tests modules if there are multiple editable installed projects containing tests modules;

now clean up by uninstalling this package:

(env) $ pip uninstall sampleproject

pytest picking up modules from working tree

install this project in the normal way:

(env) $ python setup.py sdist && pip install dist/sampleproject-1.2.0.tar.gz
...
Successfully installed sampleproject-1.2.0

test with pytest

install pytest:

(env) $ pip install pytest

run pytest and it should pass:

(env) $ pytest -v
...
tests/test_simple.py::test_success PASSED                                                       [100%]

====================================== 1 passed in 0.02 seconds ======================================

but do you know what pytest is testing: the installed package, or the working tree? let us find this out by augmenting tests/test_simple.py:

# tests/test_simple.py

def test_success():
    import logging, sample
    logging.warning(sample)
    assert False

run pytest again:

(env) $ pytest -v
...
test_simple.py               5 WARNING  <module 'sample' from '/tmp/sampleproject/sample/__init__.py'>
...

something is not quite right: module sample from /tmp/sampleproject/sample/__init__.py; but this is our working tree, not a site-specific path in the env! pytest is testing our working tree, not the installed package!

this is why some people suggest using a src-layout, by putting source packages inside a top-level src dir; lets try it;

modify setup.py:

from

packages=find_packages(exclude=['contrib', 'docs', 'tests']),  # Required

packages=find_packages(where='src', exclude=['contrib', 'docs', 'tests']),  # Required
package_dir={ '': 'src' },

move source code to src dir:

(env) $ mkdir src && mv sample src/
(env) $ python setup.py sdist && pip install -U dist/sampleproject-1.2.0.tar.gz
...
Successfully installed sampleproject-1.2.0

now pytest again:

(env) $ pytest -v
...
test_simple.py               5 WARNING  <module 'sample' from '/tmp/sampleproject/env/lib/python3.6/site-packages/sample/__init__.py'>
...

this time pytest is testing the installed package; this is the correct behavior;

test with tox

tox facilitates testing across different python versions;

to save some time, we use this minimal tox.ini:

[tox]
envlist = py{27,36}

[testenv]
basepython =
    py27: python2.7
    py36: python3.6
deps =
    pytest
commands =
    pytest

to test with tox, run:

(env) $ tox
...
test_simple.py               5 WARNING  <module 'sample' from '/tmp/sampleproject/.tox/py27/lib/python2.7/site-packages/sample/__init__.pyc'>
...
test_simple.py               5 WARNING  <module 'sample' from '/tmp/sampleproject/.tox/py36/lib/python3.6/site-packages/sample/__init__.py'>
...

tox is testing the installed package in both environments; this is the correct behavior;

layout test, not source

the above approach works fine, and that is why many people favor the src-layout;

but rethink: with the above approach, we are shaping source to solve a test problem; also, we have made setup.py more complicated;

a better solution is to use a different layout on tests:

put tests in sample-tests/tests, without sample-tests/__init__.py;

to understand why this solves the problem, read the pytest doc about import rules; in this case, sample-tests does not contain a __init__.py, so pytest will determine basedir to be sample-tests; but there isnt a sample module in sample-tests, so if we import sample in tests, then that sample only comes from installed packages;

there is also a side benefit: sample-tests is not a valid python identifier, so you wont be able to import sample-tests; this avoids namespace pollution; this benefit is not available if you rename sample-tests to a valid python identifier (eg: tests);

now lets try this better solution;

uninstall old package:
```
(env) $ pip uninstall sampleproject
```
clean up git repo:
```
(env) $ rm -rf src && git checkout .
```

augment tests/test_simple.py as above:

# tests/test_simple.py

def test_success():
    import logging, sample
    logging.warning(sample)
    assert False

modify tox.ini as above:

[tox]
envlist = py{27,36}

[testenv]
basepython =
    py27: python2.7
    py36: python3.6
deps =
    pytest
commands =
    pytest

re-layout tests:

(env) $ mkdir sample-tests && mv tests sample-tests/

install project:

(env) $ python setup.py sdist && pip install dist/sampleproject-1.2.0.tar.gz

run pytest:

(env) $ pytest -v
...
test_simple.py              10 WARNING  <module 'sample' from '/tmp/sampleproject/env/lib/python3.6/site-packages/sample/__init__.py'>
...

module sample comes from installed package; this is correct behavior;

run tox:

(env) $ tox
...
test_simple.py              10 WARNING  <module 'sample' from '/tmp/sampleproject/.tox/py27/lib/python2.7/site-packages/sample/__init__.pyc'>
...
test_simple.py              10 WARNING  <module 'sample' from '/tmp/sampleproject/.tox/py36/lib/python3.6/site-packages/sample/__init__.py'>
...

module sample comes from installed package in both environments; this is correct behavior;

this proves using src-layout to address pytest problem is not necessary; it is better to leave tests problem to tests;

remarks

is it better to test working tree or installed package?

short answer: i do both;

testing working tree is faster because it doesnt require packaging and installing; and it can help discover source code errors effectively; it is recommended you do it frequently during development;

however, testing working tree may not reveal problems in packaging scripts, and in general there is no guarantee that code in working tree works the same when installed; therefore the golden rule is to test installed package, not the working tree; for completeness, the package should be installed and tested in all python versions the project intends to support;
is it better to use src-layout or non-src-layout for source code?

short answer: it varies;

as we have seen in this article, you dont need src-layout to solve pytest problem; layout tests is a better approach;

however, there may be other reasons you want to organize source code into a single src dir, such as when this makes it clear which dir contains code in a large project; it is case-by-case and better left to project authors;