zabgrab.services.fetch

Downloading item data: chunking, concurrency, retries and progress.

zabgrab.services.fetch.Sleep

Waits for a number of seconds (replaced in tests).

Type:

type

Module Attributes

CHUNK_SECONDS

Length of each request's time window, in seconds.

RETRY_DELAYS

Delays before retrying a request that failed because of the network.

Functions

chunks(start, end, resolution)

Split a time range into windows of whole seconds, one request each.

fetch(jobs, clients, *, start, end, ...[, sleep])

Download every job's data, concurrency items at a time.

skip_reason(item, resolution)

Explain why an item can't be fetched at a resolution, if it can't.

stored_end(query)

Get the end of a fetch's range as recorded in files, in POSIX seconds.

Classes

Job(server, item, requests)

One item to download from one server.
zabgrab.services.fetch.CHUNK_SECONDS = {Resolution.HOURLY: 2592000, Resolution.RAW: 86400}

Length of each request’s time window, in seconds. Splitting a long range bounds the size of responses and lets progress bars move.

zabgrab.services.fetch.RETRY_DELAYS = (1.0, 2.0, 4.0)

Delays before retrying a request that failed because of the network.

class zabgrab.services.fetch.Job(server, item, requests)[source]

Bases: object

One item to download from one server.

Parameters:
server

The server.

Type:

Server

item

The item.

Type:

Item

requests

Labels of the requests (the template, or __custom) that matched it.

Type:

tuple of str

property label: str

Get the item’s name for progress bars, as host: name.

zabgrab.services.fetch.chunks(start, end, resolution)[source]

Split a time range into windows of whole seconds, one request each.

Parameters:

  • start (datetime) – The time range.

  • end (datetime) – The time range.

  • resolution (Resolution) – Sets the window length (see CHUNK_SECONDS).

Returns:

list of (int, int) – Inclusive (start, end) windows in POSIX seconds, covering the range.

Return type:

list[tuple[int, int]]

zabgrab.services.fetch.stored_end(query)[source]

Get the end of a fetch’s range as recorded in files, in POSIX seconds.

For hourly trends, this is the end of the last completed hour: Zabbix only writes an hour’s trend once it’s over, so a later update must ask for it again.

Parameters:

query (Query) – The fetch.

Returns:

float – The end of the range that’s complete.

Return type:

float

zabgrab.services.fetch.skip_reason(item, resolution)[source]

Explain why an item can’t be fetched at a resolution, if it can’t.

Parameters:

  • item (Item) – The item.

  • resolution (Resolution) – Raw history or hourly trends.

Returns:

str or None – The reason, as a sentence fragment, or None if it can be fetched.

Return type:

str | None

async zabgrab.services.fetch.fetch(jobs, clients, *, start, end, resolution, agg, concurrency, ui, sleep=<function sleep>)[source]

Download every job’s data, concurrency items at a time.

Items that can’t be fetched at this resolution are skipped; items whose download fails (after retrying network errors) are reported as failed, and the others still downloaded. A rejected login stops everything.

Parameters:

  • jobs (sequence of Job) – The items to download.

  • clients (mapping of Server to ZabbixClient) – Logged-in connections to the jobs’ servers.

  • start (datetime) – The time range.

  • end (datetime) – The time range.

  • resolution (Resolution) – Raw history or hourly trends.

  • agg (sequence of Agg) – The trend statistics to keep (hourly trends only).

  • concurrency (int) – How many items to download at once (each item’s windows go in order).

  • ui (UI) – Shows progress.

  • sleep (callable, optional) – Waits between retries.

Returns:

FetchResult – The series (in job order), and items skipped or failed.

Raises:

AuthenticationFailed – If the server rejects the login partway through.

Return type:

FetchResult