Including files in Python package distributions

Including files in a distributed Python source requires manually specifying them

See: "including files in source distributions with MANIFEST.in https://packaging.python.org/guides/using-manifest-in/

There's some discussion here: https://stackoverflow.com/questions/24727709/do-python-projects-need-a-manifest-in-and-what-should-be-in-it

As demonstrated by Anthony Sottile in this video you can also use the package_data argument to setuptools.setup (or distutils.core.setup, equivalently)

Anthony doesn't mention that there's also setuptools_scm which will additionally handle incrementing your version number

The basic idea of setuptools_scm is that whichever files under version control will also be the files to be put into the source distribution (or "source control").

This makes sense to me, and I also like avoiding the need to increment the version number

There's a further discussion on this blog post which can be found linked on StackOverflow questions like this

setuptools_scm appears to have entered the spotlight around 4 or 5 years ago based on what I've seen on Twitter (I may be wrong)

There are also quite heated debates on Twitter among Python core/contributor developers, and reading it I'm inclined to agree with Paul Ganssle's conclusion that "the software is fine". (PEP 517 refers to the build tools requiring internet access among other details which I haven't fully followed)

• Pretty wild example of a setup.py using setuptools_scm: https://github.com/yeatmanlab/pyAFQ/blob/master/setup.py#L62 (again via Twitter)

As can be seen in the README of setuptools_scm, there was a recent feature added which allows you to set the important parameters in pyproject.toml (a config file you often see in Python packages). As of November 2019, setuptools version 42, you can trigger version inference (automated version numbering) by just adding a section to the pyproject.toml file,

[tool.setuptools_scm]

(I don't think you even need to put anything inside it)

Additionally, I followed the instructions to add the build requirements, as when run with just the above, there was still no inclusion of the desired files (in fact it turned out I didn't have setuptools_scm installed, so it was just silently skipping that when reading pyproject.toml)

The full pyproject.toml then looks like:

[build-system]
requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]

[tool.setuptools_scm]

(Note that the "legacy" script includes setup_requires argument to setuptools.setup which is mentioned in the Twitter thread above as deprecated/bad practice)