Saturday, June 19, 2010

Python packaging with setuptools

We are going create a python egg distribution for a simple helloworld module.

Install tools

Let install two tools we need (consider switch to debian sid repository in order to get latest version of the tools):
deby:~# apt-get -y install python-setuptools python-virtualenv
We are going to work in isolated environment:
user1@deby:~$ virtualenv devenv
New python executable in devenv/bin/python
Installing setuptools............done.
user1@deby:~$ cd devenv/

Directory structure

Suppose our directory structure looks this way:
`-- trunk/
    |-- src/
    |   `-- greatings/
    |       |--
    |       |--
    |       `-- tests/
    |           |--
    |           `--
    `-- README.txt
We are going to place all our python code in src directory.
mkdir -p trunk/src/greatings/tests


The file is left empty and makes greating a python package. Here is content of (note that we are using docunits in order to demonstrate dependencies later, main function will be an entry point of our script):
import sys

def say():
    >>> say()
    'hello world'
    return 'hello world'

def main():
    return 0

if __name__ == '__main__':
The tests will be combined into test suites so they can be easier added for testing our setup later. Here is
from greatings import helloworld
import unittest

class HelloworldTestCase(unittest.TestCase):

    def test_say(self):
        assert 'hello world' == helloworld.say()

def suite():
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    return suite

if __name__ == '__main__':
Here is tests package file:
from greatings import helloworld
import test_helloworld

def suite():
    import unittest
    import doctest
    suite = unittest.TestSuite()
    return suite

if __name__ == '__main__':

Setup files

Here is out ~/devenv/trunk/ file:
import os
from setuptools import setup, find_packages

    name = 'greatings',
    version = '0.1',

    # Package structure
    # find_packages searches through a set of directories 
    # looking for packages
    packages = find_packages('src', exclude = ['ez_setup',
        '*.tests', '*.tests.*', 'tests.*', 'tests']),
    # package_dir directive maps package names to directories.
    # package_name:package_directory
    package_dir = {'': 'src'},

    # Not all packages are capable of running in compressed form, 
    # because they may expect to be able to access either source 
    # code or data files as normal operating system files.
    zip_safe = True,

    # Entry points
    # install the executable
    entry_points = {
        'console_scripts': ['helloworld = greatings.helloworld:main']

    # Dependencies
    # Dependency expressions have a package name on the left-hand 
    # side, a version on the right-hand side, and a comparison 
    # operator between them, e.g. == exact version, >= this version
    # or higher
    install_requires = [

    # Tests
    # Tests must be wrapped in a unittest test suite by either a
    # function, a TestCase class or method, or a module or package
    # containing TestCase classes. If the named suite is a package,
    # any submodules and subpackages are recursively added to the
    # overall test suite.
    test_suite = 'greatings.tests.suite',
    # Download dependencies in the current directory
    tests_require = 'docutils >= 0.6',

    # Meta information
    author = 'Me',
    author_email = '',
    description = 'A sample hello world application',
    url = ''
And configuration (file ~/devenv/setup.cfg):
# Just silently do your job
quiet = 1

# Where we are going to look for thrirdparty dependencies
find_links = thirdparty

# No optimization for now
optimize = 0
# Force build everything?
force = True

# We are doing development build
tag_build = dev
# Do we want to have date in file name?
tag_date = 0
# Add svn revision to the file name
tag_svn_revision = 1

# We do not want to distribute binary with source code
exclude-source-files = True

# Keep only last 10 eggs, clean up older
match = .egg
keep = 10

Third party dependencies

The next thing, we would like keep thirdparty dependencies (e.g. docutils) in a separate folder so each time we build the project it doesn't download dependencies from internet instead look at our folder, so we always have a proper version there. So let create directory thirdparty at the same level as src and download there docutils.
user1@deby:~/devenv/trunk$ mkdir thirdparty
user1@deby:~/devenv/trunk$ wget -P thirdparty/\
The directory structure should look like this:
`-- trunk/
    |-- src/
    |   `-- greatings/
    |        ...
    `-- thirdparty/
        `-- docutils-0.6.tar.gz
Install docutils that we downloaded into our environment:
../bin/easy_install thirdparty/*

Test, EGG, Source

Let ensure tests are passed:
master@deby:~/devenv/trunk$ ../bin/python test
Ran 2 tests in 0.015s

Here is how to create a binary distribution in egg format (look outcome at ~/devenv/trunk/dist directory):
../bin/python bdist_egg
... and source code:
../bin/python sdist
Both source and binary distributions are in ~/devenv/trunk/dist directory.
user1@deby:~/devenv/trunk$ ls dist/
greatings-0.1dev-py2.6.egg  greatings-0.1dev.tar.gz

Version control

Before adding the project to version control (e.g. svn), ensure the following directories are ignored:
  1. build
  2. dist
  3. src/greatings.egg-info
That's it.

No comments:

Post a Comment