Packaging

Your project might already be publicly available in a GitLab/GitHub/etc. repository. But how do you actually install it? How do you make it available as a system-wide boxes command, or let your code be used by something else?

This is where packaging comes in. By creating a package, you can make your work easy to distribute, install and use.

The world of Python packaging is full of different opinions. Apply caution where necessary. This chapter was written by Chris Warrick, whose opinions may not agree with others' world views.

A little reorganization

In Lesson 11 of Part 2, we've had three Python files: boxes.py, fonts.py, and hyphen.py. We can't install all three to our users' systems, since that could lead to conflicts and other unwelcome situations. In fact, boxes is not the best name either, as it's too generic. Fortunately, nobody had uploaded a boxes package to PyPI before this chapter was written.

That said, we need to make a few changes:

1$ mkdir boxes
2$ mv boxes.py boxes/__init__.py
3$ mv fonts.py hyphen.py boxes/

Then, we'll create a new __main__.py file and reorganize code slightly. We'll move the UI code to that new file. First, change the first few lines like so:

 1# Boxes, a SVG text layout engine.
 2
 3# Copyright © 2018 Roberto Alsina.
 4
 5# Permission is hereby granted, free of charge, to any
 6# person obtaining a copy of this software and associated
 7# documentation files (the "Software"), to deal in the
 8# Software without restriction, including without limitation
 9# the rights to use, copy, modify, merge, publish,
10# distribute, sublicense, and/or sell copies of the
11# Software, and to permit persons to whom the Software is
12# furnished to do so, subject to the following conditions:
13#
14# The above copyright notice and this permission notice
15# shall be included in all copies or substantial portions of
16# the Software.
17#
18# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
19# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
20# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
21# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
22# OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
23# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
24# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
25# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
26
27"""Boxes, a SVG text layout engine."""
28
29from collections import deque
30
31from boxes.fonts import adjust_widths_by_letter
32from boxes.hyphen import insert_soft_hyphens
33
34import svgwrite
35
36__version__ = "0.14"

With that, we'll use the new names of our helper files (which are now members of our package), remove the docopt import and set __version__ (which is a good thing to have). We also added a license -- we'll explain that later.

Now, let's remove the if __name__ == "__main__": block at the end of boxes/__init__.py. This will now be a function that lives in __main__.py, like so:

 1"""
 2Usage:
 3    boxes <input> <output> [--page-size=<WxH>] [--separation=<sep>]
 4    boxes --version
 5"""
 6
 7from docopt import docopt
 8import boxes
 9
10
11def main():
12    """The main routine of Boxes."""
13    arguments = docopt(__doc__, version=f"Boxes {boxes.__version__}")
14
15    if arguments["--page-size"]:
16        p_size = [int(x) for x in arguments["--page-size"].split("x")]
17    else:
18        p_size = (30, 50)
19
20    if arguments["--separation"]:
21        separation = float(arguments["--separation"])
22    else:
23        separation = 0.05
24
25    boxes.convert(
26        input=arguments["<input>"],
27        output=arguments["<output>"],
28        page_size=p_size,
29        separation=separation,
30    )
31
32
33if __name__ == "__main__":
34    main()

We did three things here:

  1. We should ship all our code in one package (directory) so we don't make a large mess.
  2. We moved boxes.py to a boxes/__init__.py file. The contents of the __init__.py file will be what's imported when you run that.
  3. We moved user-facing code to a __main__.py file. This lets us achieve separation between library code (that others can reuse and call from their code) and UI code (that requires an interactive terminal). That filename will become important soon-ish, as will the no-argument function.

We could also consider moving our code from __init__.py to yet another submodule, but let's leave it as-is. For a larger library, having less stuff in one file would help with readability and maintenance. For Boxes, more files could make maintenance harder, as more hunting would be needed to find something.

Writing a setup.py file

To let users install our software, as well as describe what it is, we need to write a setup.py file. We're going to write a pretty basic version of it -- do note that "basic" means "enough for most use-cases".

 1#!/usr/bin/env python
 2# -*- encoding: utf-8 -*-
 3import io
 4from setuptools import setup, find_packages
 5
 6
 7setup(
 8    name="boxes",
 9    version="0.14",
10    description="A SVG text layout engine",
11    keywords="boxes,book,svg,typesetting",
12    author="Roberto Alsina",
13    author_email="ralsina@netmanagers.com.ar",
14    url="https://ralsina.gitlab.io/boxes-book/",
15    license="MIT",
16    long_description=io.open(
17        "./README.rst", "r", encoding="utf-8"
18    ).read(),
19    platforms="any",
20    zip_safe=False,
21    python_requires=">=3.6",
22    # http://pypi.python.org/pypi?%3Aaction=list_classifiers
23    classifiers=[
24        "Development Status :: 3 - Alpha",
25        "Programming Language :: Python",
26        "Topic :: Text Processing",
27        "Programming Language :: Python :: 3",
28        "Programming Language :: Python :: 3.6",
29    ],
30    packages=find_packages(exclude=("tests",)),
31    include_package_data=True,
32    install_requires=["svgwrite", "pyphen", "docopt"],
33    entry_points={"console_scripts": ["boxes = boxes.__main__:main"]},
34)

Nothing special. We've automated finding packages, and added an exception for our tests folder (we don't install that, and it isn't a part of our boxes package). We're using entry_points to install our package. Had our package been Windows-compatible (it isn't, due to harfbuzz/freetype2), this would make the lives of Windows users easier by producing .exe runner files. In other cases, we still get launchers that just work.

Note that we're not listing harfbuzz and freetype2 in requirements. This is due to them not being available in PyPI. We'll unfortunately have to work around it for the time being, leading to a non-ideal user experience.

The description will come from a README.rst file which we've written while nobody was looking. A README file is the first thing your users will see -- this is where you should describe in detail what your project is and what it does. The .rst extension stands for reStructuredText, which is used for most documentation in the Python world. See quickstart, quickref.

Choosing a license

Time for the most important decision you will make as an open-source developer: choosing a license. This decision will define the rights and obligations of users, and limits your liability if anything happens.

For this project, we're going to pick the MIT license. It’s short (you can read it and understand it without a lawyer), and lets users do everything, as long as they (a) leave in the license and copyright header, (b) they do not sue you/consider you liable.

No matter what you do, you need to:

  • Pick a license
  • Be aware of what the license entails, and what issues there might be due to this choice (eg. AGPL is banned at Google and some other places)
  • Use a known license instead of writing a new one, even if with the help of a lawyer.

You should strive to make an informed decision. ChooseALicense.com can help with this. (Also see the appendix with all licenses.)

After you've made your decision, put your license in a LICENSE file at the top of your repository. Make sure to put your name and the © year there.

Authors, changelog, requirements

Three more files. The first two are for documentation purposes:

Roberto Alsina 
Boxes Changelog
===============

0.14
    * Made Boxes an installable package (``setup.py``)
    * Reorganized code

And then, we'll list out concrete dependencies in requirements.txt:

svgwrite==1.1.12
https://github.com/ldo/harfpy/archive/master.zip#egg=harfbuzz
https://gitlab.com/ldo/python_freetype/repository/master/archive.zip#egg=freetype
pyphen
docopt==0.6.2

In the next chapter, we're going to add a docs/ directory and talk about documenting your code.

Writing MANIFEST.in and making sure everything's done right

We're almost done! We're going to need one more file:

graft boxes
graft docs
graft tests
include README.rst AUTHORS LICENSE CHANGELOG.rst setup.py requirements.txt
global-exclude __pycache__ *.pyc

BTW, if you don't have a .gitignore file yet, generate one for Python and add it, because the packaging stuff generates a bunch of files you shouldn't commit to a git repository.

Now you should be ready to build packages! First, a quick verification of our directory structure:

boxes                     # (repository root)
├── boxes                 # (Python package, contains code)
│   ├── fonts.py
│   ├── hyphen.py
│   ├── __init__.py
│   └── __main__.py
├── AUTHORS
├── CHANGELOG.rst
├── LICENSE
├── MANIFEST.in
├── README.rst
├── requirements.txt
├── setup.py
└── tests
    ├── test_create_test_boxes.py
    ├── test_fill_row.py
    ├── test_is_breaking.py
    ├── test_justify_row.py
    └── test_layout.py

Just in case, let's run pytest. Make sure you’re in the top boxes directory (the repository root). 20 passed in 0.72 seconds, we didn't break anything. Great!

Building and uploading to PyPI

The moment we've all been waiting for:

$ python setup.py sdist bdist_wheel

A few new files and folders appear. The one you actually care about is dist:

1$ ls dist
2boxes-0.14-py3-none-any.whl  boxes-0.14.tar.gz

Now, onto uploading to PyPI. First, you need to create an account on pypi.org. Make sure you verify your e-mail address. Then, put the following in ~/.pypirc (add your data at the end):

1[distutils]
2index-servers =
3    pypi
4
5[pypi]
6username:username
7password:swordfish

We can now upload to PyPI. I'm going to use the -s option to sign my package using GPG. You can skip it if you don't want to do that (requires some setup).

1$ pip install twine
2$ twine upload -s dist/boxes-0.14*

And that's it! Your package should now be on PyPI. Here's boxes: https://pypi.org/project/boxes

Let's create a commit and a git tag: (you may skip -S or -s again)

1$ git add .
2$ git commit -S -am "Version 0.14"
3$ git tag -asm "Version 0.14" v0.14
4$ git push --follow-tags origin master

Finally, you may also add the changelog to GitHub's releases page. (Or GitLab's tags page.)

Now, you should be able to run pip install boxes and use the library.

1$ pip install boxes
2Successfully installed boxes-0.14
3$ boxes --help
4Usage:
5    boxes <input> <output> [--page-size=<WxH>] [--separation=<sep>]
6    boxes --version
7$ python -c 'import boxes; print(boxes.__version__)'
80.14

Creating new releases and automating the process

What if we make some changes? We need to push our code out to the world. But if you look back, to make a new release, we have some work to do:

  • Updating version numbers (__init__.py, setup.py, possibly more)
  • Updating the changelog
  • Creating distributions for PyPI and uploading them
  • The git commit; git tag; git push --follow-tags dance
  • Updating GitHub Releases
  • Blog posts, websites, docs, announcements?

But this is only the beginning. Nikola's release checklist had 39 entries before we automated it and brought it down to 19.

Also, getting where we are took us a few paragraphs. This is pretty tiring, and there were quite a few things we could have gotten wrong.

But there are ways to solve it. One example is, if you pardon the self-promotion, Python Project Template by this chapter's author. It automates all the things mentioned above, and creating a new project. Creating a new release requires only a simple ./release.

results matching ""

    No results matching ""