zabgrab.services.metadata

Metadata saved with fetched data: enough to understand and repeat the fetch.

Files written by zabgrab store it as JSON (an HDF5 dataset, or # comment lines in delimited files). update relies on its query and items.

Functions

build_metadata(query, result, *, template, ...)

Describe a fetch.

export_metadata(dataset, *, source, filters, ...)

Describe data exported from a file.
zabgrab.services.metadata.build_metadata(query, result, *, template, command, created)[source]

Describe a fetch. Never includes credentials.

Parameters:

  • query (Query) – What was fetched, and where each setting came from.

  • result (FetchResult) – The data, and items skipped or failed.

  • template (Template or None) – The template used, copied in so the fetch can be understood (and updated with --rematch) without the config.

  • command (sequence of str) – The command line.

  • created (datetime) – When the fetch was made.

Returns:

dict – The metadata, ready for JSON.

Return type:

dict[str, Any]

zabgrab.services.metadata.export_metadata(dataset, *, source, filters, command, created)[source]

Describe data exported from a file.

The last group’s query is kept, so the export can be updated like the original, with the merged items and a record of how it was exported.

Parameters:

  • dataset (Dataset) – The merged data; it must cover some time.

  • source (str) – The file it came from.

  • filters (mapping) – The selection options used.

  • command (sequence of str) – The command line.

  • created (datetime) – When the export was made.

Returns:

dict – The metadata, ready for JSON.

Return type:

dict[str, Any]