Python documentation


Alexandre Boucaud aboucaud@ias.u-psud.fr

Overview

Introduction

This presentation covers the inner documentation of a Python project, meaning the documentation towards other developers & collaborators, not a user manual.

  • Such documentation is written via docstrings, a line or block of characters that will be interpreted as such and be delivered to the user when requested.

  • Everything in Python is an object and every object can have its own docstring.

  • The documentation can be accessed through the builtin help() function, or via a ? at right after an object name in an IPython terminal.

In [1]:
def f():
    """This is a single-line docstring"""

help(f)
Help on function f in module __main__:

f()
    This is a single-line docstring

Python docstring

A Python docstring

  • starts with """ and ends with """ on a new line.
In [ ]:
"""This is a classic Python docstring.

It uses multiple lines.
"""

The only exception is if the docstring is one-line long.

In [ ]:
"""This is a one-line docstring."""
  • uses double quotes """ instead of simple quotes '''.
  • shall be wrapped after 72 characters.
  • has a blank line between paragraphs & sections.

The docstring structure is

  1. Short description
  2. Deprecation warning (if applicable)
  3. Extended description (optional)
  4. Sections (see methods and classes)
  5. Links (optional, use ReStructured Text syntax)
In [ ]:
"""Short description which fits in a single line.

Here is the extended description which uses paragraphs.
The paragraphs may be enriched with links like [1]_. The 
following sections are written using numpy-style doc.

Section1
--------
content of section
    blocks can be used
    
Section2
--------
new content

.. _[1]:
    http://www.python.org

"""

Numpy vs. Google style doc

  • Python offer several conventions regarding the use of docstring. You can put any type of information in the docstring, with a free format.
  • However it is best to use a standardized format which would allow other software to generate the code documentation on the fly (for e.g. Sphinx)
  • The most commonly used doc styles in Python are known as Google-style and Numpy-style. They are both very easy to write, read, and parse, and look quite similar.
  • In the remainder of this presentation, Numpy-style will be used but one can find information on Google-style doc here along with some examples here.

Method documentation

Parameters

Methods arguments are documented in the Parameters section, with their type and description. For keyword arguments, add optional and default value, or provide a restricted set of values.

In [ ]:
def func(param, param2=None, param3='X'):
    """Simple function
    
    Parameters
    ----------
    param : type
        Description of `param`
    param2 : type, optional
        Description of `param2` (default is `None`)
    param3 : {'X', 'Y', 'Z'}
        Description of `param3` (default is 'X')
        
    """

Returns

The documentation of returned values is similar to the Parameters except type is required but the value name is optional.

In [ ]:
    """
    Returns
    -------
    type
        Description of the returned value
        
    """
In [ ]:
    """
    Returns
    -------
    return_param1 : type
        Description of `return_param1`
    return_param2 : type
        Description of `return_param2`
        
    """

Yields

If the method is a generator, the Yields section should be used instead of the Returns one. The same rules apply though.

In [ ]:
    """
    Yields
    ------
    type
        Description of the yielded value
        
    """

Raises

An optional section detailing which errors get raised and under what conditions

In [ ]:
    """
    Raises
    ------
    MyCustomException
        If the data is this and not that
        
    """

Notes

An optional section that provides additional information about the code, possibly including a discussion of the algorithm. This section may include mathematical equations, written in LaTeX format

In [ ]:
    """
    Notes
    -----
    Here I can write a silly equation using :math:`\lambda`
    
    .. math:: \sqrt{\lambda^2} = 1
    
    """

References

References cited in the Notes section may be listed under the References section.

In [ ]:
    """
    References
    ----------
    You can refer to scientific articles as [1]_.

    .. [1] Astropy Collaboration, "Astropy: A community Python 
    package for astronomy", Astronomy & Astrophysics, 558, 33, 2013.
    
    """

Examples

An optional section for examples, using the doctest format.

This section is meant to illustrate usage, not to provide a testing framework.

In [ ]:
    """
    Examples
    --------
    >>> a = [4, 3]
    >>> a * 2
    [4, 3, 4, 3]
    
    Comments can be included in between examples
    
    >>> import numpy as np
    >>> np.array(a) * 2
    array([8, 6])
    
    """

Type hinting

In Python 3.X, the PEP484 allows the developers to enrich every method definition with type hinting.

In [ ]:
def func_with_annotations(param1: int, param2: float) -> str:

Such functionality is currently barely beeing used by Python libraries, except for documentation purposes.

If you are using type hinting in your code, then the type information can be removed from the docstring.

In [ ]:
def func_with_annotations(param1: int, param2: float) -> str:
    """Simple function with annotations
    
    Parameters
    ----------
    param1
        Description of `param1`
    param2
        Description of `param2`
        
    Returns
    -------
    bool
        `True` if valid, `False if not.
    
    """

Class documentation

Classes

  • use the same sections as for the methods (all except Returns are applicable).
  • have two additonal sections: Attributes and Methods.
  • The constructor (__init__) should be documented, using a Parameters section.

Attributes

In the class docstring, the Attributes section may be used to describe non-method attributes of the class.

In [ ]:
    """
    Attributes
    ----------
    x : float
        The X coordinate.
    y : float
        The Y coordinate.
    
    """

Attributes that are properties and have their own docstrings can be simply listed by name:

In [ ]:
    """
    Attributes
    ----------
    real
    imag
    x : float
        The X coordinate

    """

The Parameters section is moved to the docstring of the class constructor(s).

Methods

In general, it is not necessary to list class methods. Those that are not part of the public API have names that start with an underscore.

In some cases where only a few methods are relevant and need to be highlighted, it becomes useful to have an additional Methods section.

In [ ]:
    """
    Methods
    -------
    method_alpha(arg='rgb')
        `method_alpha` short description
    method_gamma(arg2=1.0)
        `method_gamma` short description

    """

Class properties

Properties should only be documented in their getter method, even if the setter method contains notable behavior.

In [ ]:
    @property
    def size(self):
        """float: property of the class
        
        Both getter and setter method documentation is
        defined in the getter method docstring
        
        """
        return self._size

    @size.setter
    def size(self, value):
        self._size = value

Class representation

As part of the overall documentation, it is useful to provide classes with representations, that is a definition of __repr__ and optionally __str__ special methods.

Such methods are called when the class instance is printed on screen.

In [2]:
class MyClass(object):
    def __init__(self, value1, value2):
        self.value1 = value1
        self.value2 = value2

    def __repr__(self):
        return "{!s}({!r})".format(self.__class__.__name__, self.__dict__)

print(MyClass(2, 5))
MyClass({'value2': 5, 'value1': 2})

Module documentation

The module docstring

  • appears right after the boilerplate and before the imports.

  • must be written with the exact same structure and syntax as the previous examples.

  • should explain the scope of the current module.

  • can be placed in __init__.py files to document submodules.

Should follow the structure, keeping in mind most of these sections are optional :

  1. summary
  2. extended summary
  3. routine listings
  4. see also
  5. notes
  6. references
  7. examples

Module variable docstring

In [ ]:
module_level_variable = 98765
"""int: Module level variable documented inline.

The docstring may span multiple lines. The type may optionally be specified
on the first line, separated by a colon.
"""

Comments

Comments

start with the # character in Python.

In a given Python line, everything behind this character will be escaped when interpreting the code.

There are 3 main user cases for comments in Python:

  • boilerplate
  • block comment
  • inline comment

boilerplate

Should be present at the top of every .py file, before the module docstring.

In [ ]:
# -*- coding: utf-8 -*-

# Copyright (C) 2012-2020 Euclid Developers
#
# This file is part of <this code>.
#
# <this code> is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as 
# published by the Free Software Foundation, either version 3 of 
# the License, or (at your option) any later version.
#
# <this code> is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with <this code>.  If not, see <http://www.gnu.org/licenses/>

block comments

Are used to help another developer read the code

In [ ]:
# This is an example of a block comment
# it can add some explanations on the 
# current process and for instance give
# some references.

They can also be used to write flags, such as TODO's (which are recognized in some IDEs)

In [ ]:
# TODO: check that the returned type is the same as the input type

inline comments

Are used for very specific help that only applies to the current line.

In [ ]:
url_regex = (
    r'^(?:http|ftp)s?://'  # http:// or https:// or ftp:// or ftps://
    r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+'
    r'(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|'  # domain...
    r'localhost|'  # localhost...
    r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'  # ...or ip
    r'(?::\d+)?'  # optional port
    r'(?:/?|[/?]\S+)$')

They should be used with parcimony.

Tips

  • array_like type should be used for functions that take arguments which can have not only a type ndarray, or types that can be converted to an ndarray.
  • Variable, module, function, and class names should be written between single back-ticks ( `numpy` ).
  • Use *italics*, **bold** and ``monospace``to augment the readibility elsewhere.
  • Notes and Warnings if points in the docstring deserve special emphasis. Should be used sparingly.
In [ ]:
"""
.. warning:: Warning text.

.. note:: Note text.

"""