Requirements

pip uses a requirements file to install specified versions of a project from the Python Package Index (PyPI).

We use a pip to ensure that we can reproduce exact python environments with specific versions of specific packages.

The exact format of the requirements file can be found in pip‘s documentation.

Default Requirements File

The default requirements file is broken into sections to make reading it clearer.

Python

These packages are just general packages that make working with Python in general easier.

ipython

IPython is a replacement Python shell that adds a bunch of useful features that are lacking in the default shell. The features that I find most useful are tab completion, class/module help using object_name?, and history. You can find more out about more features in the official documentation.

nose

nose is a test discoverer and runner for Python code. It makes writing and running test simple and fast. It also has a vast array of plugins that extend it’s functionality to capture stdout/stderr and capture logging, code coverage reports, test tagging and skipping. nose has many other features and I encourage you to experiment with them to make your testing experience better and faster.

mock

mock is a mocking library for Python tests. It is simple to use and very handy when writing fast tests.

Documentation

These packages are used to document your project. They make the writing and publishing documentation fun and easy.

Sphinx

Sphinx is main documentation platform for this project. It is simple to use, with advanced features available when needed. It uses reStructuredText as the backing text format and it is able to produce output in a bunch of different formats including HTML, PDF, ePub and others.

Jinja

Jinja is a dependency of Sphinx. It is the package that Sphinx uses as a templating engine for the HTML output.

Pygments

Pygments is a dependency of Sphinx. It is the package that Sphinx uses to highlight source code.

docutils

docutils is a dependency of Sphinx. It is the package that Sphinx uses to parse the reStructuredText markup.

Django

These requirements make Django work at it’s best.

django

It’s Django, nuf said.

pytz

pytz is a dependency of Django. It is the package that Django uses to support timezone lookup and conversion when timezone support is enabled.

py-bcrypt

py-bcrypt is used as an interface to the bcrypt library to allow for bcrypt hashing for passwords. It is not a direct dependency of Django, but we use it with Django’s BCryptPasswordHasher.

Django Utility Apps

These apps make writing Django applications and site easier.

South

South enables migrations for Django models. It has become the de-facto standard for migrations in the Django world. It supports both schema and data migrations.

django-secure

django-secure is a package containing utilities and a linter to help make your Django site more secure. It provides additional settings to apply easy security wins usually through the use of specific headers. It works best with sites that use SSL, but it also provides some benefit for those who don’t.

django-debug-toolbar

django-debug-toolbar is a package that adds a lot of additional useful information to the HTML pages that Django produces while DEBUG = True. It has panels showing information about SQL queries, templates, settings, etc.

django-nose

django-nose is a simple package that provides a Django test runner that will use nose under the covers. It makes sure that nose will correctly setup and tear down the test database.

factory_boy

factory_boy is a package that simplifies the writing of Django model factories for using in tests. It can be used to create a complex set of related models which can then be tested against. It is a good alternative to fixtures.

django-model-utils

django-model-utils is a package of handy utility classes for working with Django models. It incorperates the logic of a lot of different common model uses, ie TimeFramedModel, StatusModel, and TimeStampedModel.

Django Apps

These reusable Django apps, get your Django site up and running quickly.

django_compressor

django_compressor is a package that extends Django’s handling of static assets. It will combine and minify your CSS and JS assets into files with unique names that can be cached forever on the client.

django-appconf

django-appconf is a dependency of django_compressor. It provides a unified way for reusable Django apps to handle settings.

lxml

lxml is a dependency of django_compressor. It is an extremely fast HTML and XML parser.

BeautifulSoup

BeautifulSoup is a dependency of django_compressor. It is a slow but very forgiving HTML and XML parser.

django-waffle

django-waffle is a package for using feature flipping in Django. It provides a way to turn features of your app on and off depending on a set of rules. The rules can be as simple as a switch, or more based on more complicated logic like staff status or a weighted percentage.

Celery

Celery is a distributed task queue for Python. We use it to enable out of process execution. This means that the request/response cycle isn’t slowed down by slow steps like sending email.

celery

The base celery package. Provides all the celery task functionality.

django-celery

An add-on for celery that increases it’s functionality in a Django environment. It handles loading your Django environment before running tasks and adds all the celery commands to the manage.py interface.

billiard

billiard is a dependency of celery. It is a Python multiprocessing fork with improvements and bugfixes.

amqplib

amqplib is a dependency of celery. It is a AMQP client library.

kombu

kombu is a dependency of celery. It provides a high level interface for AMQP on top of amqplib.

anyjson

anyjson is a dependency of celery. It is a library that normalizes all the various JSON processing packages for Python.

python-dateutil

python-dateutil is a dependency of celery. It is a library for working with dates and times.

redis

redis is a required to use celery with this Redis backend. It is not a requirement of the basic celery install. Redis is a very fast key-value store that is simple to setup and run.

Adding Requirements

When you need new packages to the requirements.pip file you can follow these simple steps.

  1. Run pip freeze > requirements.pip.before to record the package state as it is now.
  2. Install the new package using pip install -v package. Note the output towards the bottom that tells you which packages where installed to meet all the requirements of the package you installed.
  3. Run pip freeze > requirements.pip.after to record the new package state.
  4. Run diff requirements.pip.before requirements.pip.after to see all the new lines show which packages and versions that were installed.
  5. Manually add the new lines to the requirements.pip file, placing them in the correct section.
  6. Run rm requirements.pip.before requirements.pip.after to cleanup the temporary files that were created.
  7. Document the addition in the commit message. Remember, other developers will now need to run pip install -v -r requirements.pip to satisfy the additional package requirements.