The documentation in this section is aimed at people wishing to contribute to cupage, and can be skipped if you are simply using the tool from the command line.

class cupage.utils.CupageEncoder(skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, encoding='utf-8', default=None)[source]

Constructor for JSONEncoder, with sensible defaults.

If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, long, float or None. If skipkeys is True, such items are simply skipped.

If ensure_ascii is true (the default), all non-ASCII characters in the output are escaped with uXXXX sequences, and the results are str instances consisting of ASCII characters only. If ensure_ascii is False, a result may be a unicode instance. This usually happens if the input contains unicode strings or the encoding parameter is used.

If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.

If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.

If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation. Since the default item separator is ‘, ‘, the output might include trailing whitespace when indent is specified. You can use separators=(‘,’, ‘: ‘) to avoid this.

If specified, separators should be a (item_separator, key_separator) tuple. The default is (‘, ‘, ‘: ‘). To get the most compact JSON representation you should specify (‘,’, ‘:’) to eliminate whitespace.

If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a TypeError.

If encoding is not None, then all input strings will be transformed into unicode using that encoding prior to JSON-encoding. The default is UTF-8.


Handle datetime objects when encoding as JSON.

This simply falls through to default() if obj has no isoformat method.

Parameters:obj – Object to encode

Parse checked datetimes from cupage databases.

Parameters:obj – Object to decode

Parse human readable frequency.

Parameters:delta (str) – Frequency to parse

Order package list according to version number.

Parameters:packages (list) – Packages to sort
cupage.utils.robots_test(http, url, name, user_agent='*')[source]

Check whether a given URL is blocked by robots.txt.

  • httphttplib2.Http object to use for requests
  • url (str) – URL to check
  • name – Site name being checked
  • user_agent (str) – User agent to check in robots.txt

The following three functions are defined for purely cosmetic reasons, as they make the calling points easier to read.


Format a success message with colour, if possible.

Parameters:text (str) – Text to format

Format a failure message with colour, if possible.

Parameters:text (str) – Text to format

Format a warning message with colour, if possible.

Parameters:text (str) – Text to format


Output formatting

>>> success('well done!')
u'\x1b[38;5;10mwell done!\x1b[m\x1b(B'
>>> fail('unlucky!')