Site

Note

The documentation in this section is aimed at people wishing to contribute to cupage, and can be skipped if you are simply using the tool from the command line.

cupage.SITES = {}
Site specific configuration data
cupage.USER_AGENT = 'cupage/0.8.2 (https://github.com/JNRowe/cupage/)'

User agent to use for HTTP requests

class cupage.Site(name, url, match_func='default', options=None, frequency=None, robots=True, checked=None, matches=None)[source]

Initialise a new Site object.

Parameters:
  • name (str) – Site name
  • url (str) – URL for site
  • match_func (str) – Function to use for retrieving matches
  • options (dict) – Options for match_func
  • frequency (int) – Site check frequency
  • robots (bool) – Whether to respect a host’s robots.txt
  • checked (datetime.datetime) – Last checked date
  • matches (list) – Previous matches
check(cache=None, timeout=None, force=False, no_write=False)[source]

Check site for updates.

Parameters:
  • cache (str) – httplib2.Http cache location
  • timeout (int) – Timeout value for httplib2.Http
  • force (bool) – Ignore configured check frequency
  • no_write (bool) – Do not write to cache, useful for testing
find_default_matches(content, charset)[source]

Extract matches from content.

Parameters:
  • content (str) – Content to search
  • charset (str) – Character set for content
find_github_matches(content, charset)[source]

Extract matches from GitHub content.

Parameters:
  • content (str) – Content to search
  • charset (str) – Character set for content
find_google_code_matches(content, charset)[source]

Extract matches from Google Code content.

Parameters:
  • content (str) – Content to search
  • charset (str) – Character set for content
find_hackage_matches(content, charset)[source]

Extract matches from hackage content.

Parameters:
  • content (str) – Content to search
  • charset (str) – Character set for content
find_sourceforge_matches(content, charset)[source]

Extract matches from sourceforge content.

Parameters:
  • content (str) – Content to search
  • charset (str) – Character set for content
static package_re(name, ext, verbose=False)[source]

Generate a compiled re for the package.

Parameters:
  • name (str) – File name to check for
  • ext (str) – File extension to check
  • verbose (bool) – Whether to enable re.VERBOSE
static parse(name, options, data)[source]

Parse data generated by Sites.loader.

Parameters:
  • name (str) – Site name from config file
  • options (configobj.ConfigObj) – Site options from config file
  • data (dict) – Stored data from database file
state

Return Site state for database storage.

class cupage.Sites[source]

Site bundle wrapper.

load(config_file, database=None)[source]

Read sites from a user’s config file and database.

Parameters:
  • config_file (str) – Config file to read
  • database (str) – Database file to read
save(database)[source]

Save Sites to the user’s database.

Parameters:database (str) – Database file to write

Examples

Reading stored configuration

>>> sites = Sites()
>>> sites.load('support/cupage.conf', 'support/cupage.db')
>>> sites[0].frequency
360000

Writing updates

>>> sites.save('support/cupage.db')