Observability

The Watcher periodically re-runs an agent and tracks how its metrics evolve — useful for live drift and anomaly detection in production-like loops.

Watching

from entropy import Watcher

w = Watcher(agent, dataset, trials=20, seed=0)
history = w.watch(rounds=10)   # prints success / drift / entropy per round

Inspecting the history

# series of a single metric across rounds
print(w.series("drift_score"))

# rounds flagged as anomalies (z-score > 2.0)
print(w.anomalies(metric="drift_score", z=2.0))

# most common failing outputs
print(w.failure_clusters(top=5))

Heatmap

Render a min-max-normalized heatmap of watched metrics across rounds:

from entropy import render_metric_heatmap

render_metric_heatmap(w.history, "heatmap.png")

Custom rendering

Pass a render callback to watch for live TUI or dashboard updates:

def render(watcher, i, res, anomaly):
    flag = "!!" if anomaly else "  "
    print(f"[{i}] success={res['success_rate']:.3f} {flag}")

w.watch(rounds=10, render=render)