Metric suites

MetricFamily

metametric.core.metric_suite.MetricFamily dataclass

Bases: MetricSuite[T]

A collection of metrics that shared the same internal state.

For example, the precision, recall, and F-1 of event detection should be computed together within one family.

MetricSuite

metametric.core.metric_suite.MetricSuite

Bases: Protocol[T]

Metric suites are collections of metrics that are computed together.

new()

Create a new aggregator for the metric suite.

with_extra(extra)

Add extra metrics to the metric suite.

Aggregator

metametric.core.metric_suite.Aggregator

Bases: Protocol[T]

Metric aggregator for computing metrics on a stream of predictions and references.

state: MetricState[T] property

The internal state of the aggregator.

compute()

Compute the metrics from the aggregator.

reset()

Reset the aggregator to its initialization state.

update_batch(preds, refs)

Update the aggregator with a batch of predictions and their references.

update_single(pred, ref)

Update the aggregator with a single prediction and its reference.