zabgrab.services.dataset

The merged view of a file’s fetch groups.

Groups only exist because of how appending works, so reading operations see one dataset: each item’s data joined across groups, sorted by time, with duplicate timestamps resolved in favour of the later group.

Module Attributes

COVERAGE_TOLERANCE

Gaps between fetched ranges up to this long (seconds) still count as covered, since each update starts just after the previous fetch's end.

Functions

filter_dataset(dataset, *[, items, ...])	Limit a dataset to some items and a time range.
matches(series, pattern)	Check whether a series' item ID, name, key or host: name matches a pattern.
merge(groups, ui)	Merge fetch groups into one dataset.
overlaps(groups, series, t_from, t_to)	Find the existing groups that already hold some of a new fetch's data.
repeated_points(datasets)	Count the points each dataset shares with any of the others.
select(series, items, patterns)	Select the series that any item spec or pattern matches.
select_groups(groups, names)	Pick groups by name.
spec_matches(series, spec)	Check whether an item spec selects a series' item, as the Zabbix API would.

Classes

Dataset(series, coverage, groups[, ...])	Every series in a file (or selection of its groups), merged.
Group(*args, **kwargs)	A fetch group read from a file.
GroupRange(*args, **kwargs)	A fetch group's metadata and range, without its data.
Overlap(group, start, end, items)	Part of a new fetch's range that an existing group already covers.

zabgrab.services.dataset.COVERAGE_TOLERANCE = 1.0

Gaps between fetched ranges up to this long (seconds) still count as covered, since each update starts just after the previous fetch’s end.

class zabgrab.services.dataset.Group(*args, **kwargs)

Bases: Protocol

A fetch group read from a file.

See zabgrab.adapters.formats.hdf5.FetchGroup, which this describes without the services depending on the adapter.

class zabgrab.services.dataset.Dataset(series, coverage, groups, last_metadata=<factory>)

Bases: object

Every series in a file (or selection of its groups), merged.

Parameters:

• series (list[Series])

• coverage (list[tuple[float, float]])

• groups (list[str])

• last_metadata (dict[str, Any])

series

One series per item (and resolution), joined across groups.

Type:

list of Series

coverage

The fetched time ranges, merged, in POSIX seconds.

Type:

list of (float, float)

groups

The names of the groups merged.

Type:

list of str

last_metadata

The metadata of the last group (the most recent query).

Type:

dict

zabgrab.services.dataset.merge(groups, ui)

Merge fetch groups into one dataset.

Series are matched by server, item, resolution and columns. Each is joined across groups and sorted by time; points with the same time keep the later group’s value. The item’s details come from the latest group.

Parameters:

• groups (sequence of Group) – The groups, in the order they were written.

• ui (UI) – Warns if an item’s units changed between groups.

Returns:

Dataset – The merged data.

Raises:

ZabgrabError – If there are no groups.

Return type:

Dataset

zabgrab.services.dataset.repeated_points(datasets)

Count the points each dataset shares with any of the others.

A point is shared if another dataset has the same series (item, resolution and columns) with a point at the same time. Use it with each group read on its own, to see how much of a file is held twice.

Parameters:

datasets (sequence of Dataset) – Typically one per fetch group.

Returns:

list of int – For each dataset, how many of its points another also holds.

Return type:

list[int]

class zabgrab.services.dataset.Overlap(group, start, end, items)

Bases: object

Part of a new fetch’s range that an existing group already covers.

Parameters:

• group (str)

• start (float)

• end (float)

• items (int)

group

The existing group’s name.

Type:

str

start, end

The time range covered by both, in POSIX seconds.

Type:

float

items

How many of the new fetch’s items the group also holds.

Type:

int

class zabgrab.services.dataset.GroupRange(*args, **kwargs)

Bases: Protocol

A fetch group’s metadata and range, without its data.

See zabgrab.adapters.formats.hdf5.GroupRange.

zabgrab.services.dataset.overlaps(groups, series, t_from, t_to)

Find the existing groups that already hold some of a new fetch’s data.

A group overlaps if it holds any of the same items at the same resolution, and its range and the new fetch’s overlap for longer than an instant (an update starting where the last fetch ended doesn’t count).

Parameters:

• groups (sequence of GroupRange) – The file’s existing groups.

• series (sequence of Series) – The newly fetched series.

• t_from (float) – The new fetch’s range, in POSIX seconds.

• t_to (float) – The new fetch’s range, in POSIX seconds.

Returns:

list of Overlap – The overlapping groups, in the order they were written.

Return type:

list[Overlap]

zabgrab.services.dataset.select_groups(groups, names)

Pick groups by name.

Parameters:

• groups (sequence of Group) – Every group in the file.

• names (sequence of str) – The names wanted (all groups if empty).

Returns:

list of Group – The groups, in the order named.

Raises:

ZabgrabError – If a name doesn’t match a group.

Return type:

list[Group]

zabgrab.services.dataset.filter_dataset(dataset, *, items=(), patterns=(), start=None, end=None)

Limit a dataset to some items and a time range.

Parameters:

• dataset (Dataset) – The data.

• items (sequence of ItemSpec, optional) – Item specs (--item; see spec_matches()).

• patterns (sequence of str, optional) – Glob patterns (--match; see matches()). A series is kept if any item spec or pattern matches it, or if there are none of either.

• start (datetime, optional) – The time range to keep (unbounded if not given).

• end (datetime, optional) – The time range to keep (unbounded if not given).

Returns:

Dataset – The selected data, with its coverage cut to the time range.

Return type:

Dataset

zabgrab.services.dataset.select(series, items, patterns)

Select the series that any item spec or pattern matches.

Parameters:

• series (sequence of Series) – The series.

• items (sequence of ItemSpec) – Item specs (see spec_matches()).

• patterns (sequence of str) – Glob patterns (see matches()).

Returns:

list of Series – The matching series, in order; all of them if there are no specs or patterns.

Return type:

list[Series]

zabgrab.services.dataset.spec_matches(series, spec)

Check whether an item spec selects a series’ item, as the Zabbix API would.

Patterns match like the API’s searches (item.get and host.get with searchWildcardsEnabled): the whole value, ignoring case, with * matching any text and every other character literal. The host matches its technical or visible name. A tag (name or name:value) must be among the item’s stored tags exactly, as the API’s tag filters compare them. Every field given must match.

Parameters:

• series (Series) – The series.

• spec (ItemSpec) – The item spec.

Returns:

bool – Whether it matches.

Return type:

bool

Examples

>>> from zabgrab.model.config import Resolution, ServerConfig
>>> from zabgrab.model.server import Server
>>> from zabgrab.model.zabbix import Item, ValueType
>>> item = Item(1, 1, "pc2", "Particle counter 2", "0.3 µm particles", "p[0.3]",
...             ValueType.FLOAT)
>>> one = Series(Server("lab", ServerConfig(url="https://z.example.org")), item,
...              Resolution.RAW, {"time": []})
>>> spec_matches(one, ItemSpec(name="0.3*PARTICLES", host="*counter*"))
True
>>> spec_matches(one, ItemSpec(name="0.3"))
False

zabgrab.services.dataset.matches(series, pattern)

Check whether a series’ item ID, name, key or host: name matches a pattern.

Parameters:

• series (Series) – The series.

• pattern (str) – A pattern in Python’s fnmatch syntax (*, ?, [seq] and [!seq]), which must match the whole of one of them, ignoring case.

Returns:

bool – Whether it matches.

Return type:

bool