API Reference

Everything below is importable from the top-level entropy package (e.g. from entropy import Suite). Modules are listed by area; see the Metrics Catalog for the full metric list.

Core — `entropy.core`

Symbol	Signature / Notes
`Suite(seed=0, metrics=None)`	Monte Carlo runner. `.run(agent, dataset, trials=100, extractor=None) -> dict`.
`assert_stable(results, **thresholds)`	Regression gate; thresholds like `success_rate__gt=0.9`. Raises on violation.
`AgentRun`	Dataclass: `run_id, input, output, events, cost, metadata`.
`Event` / typed events	`ActionEvent, ReasoningEvent, ToolCallEvent, ObservationEvent, MemoryReadEvent, MemoryWriteEvent, ErrorEvent, StateTransitionEvent`.
`Trace`	Dataclass: `runs, nodes, edges, metadata` + `.json() .df() .graph() .otel() .zip()`.
`Node`, `Edge`	Behavior graph primitives.
`instrument(agent)`	Wrap a callable (or framework agent) to return an `AgentRun`.
`trace()`, `event(type, name, **data)`, `current_trace()`	Manual tracing context & emitters.

Metrics — `entropy.metrics`

Symbol	Notes
`metric(name=None)`	Decorator registering a metric class.
`Metric`	Base class; implement `compute(self, batch)`.
`Batch`	Per-input aggregation: outputs, behaviors, actions, tool_calls, successes, costs, errored, recovered.
`default_metrics()`	List the canonical Suite metric names.
`layers`	Mapping of evaluation layer → metrics (see Metrics).

Datasets — `entropy.datasets`

Symbol	Notes
`Dataset(list)`	List of cases; `from_json / from_jsonl / from_yaml / from_csv / from_hf`, `.save()`.
`Case(input, expected=None, check=None, metadata=None)`	Generic case.
`GoldenCase`, `Scenario`, `BehaviorCase`, `FailureCase`, `AdversarialCase`	Specialized cases.

Simulation — `entropy.simulation`

Symbol	Notes
`ScriptedUserSimulator(script)`, `RandomUserSimulator(pool, seed=0)`, `LLMUserSimulator(client, model)`	User turn generators with `.next()` / `.reset()`.
`StatefulEnv(state=None)`, `GridWorld(w, h, goal)`	Environments with `.step(action)` / `.reset()`.
`AdversarialSimulator(seed=0, attacks=None)`	`.iterate(text)`, `.perturb(text, kind)`.

Chaos — `entropy.chaos`

Symbol	Notes
`ChaosRunner(seed=0)`	`.run(agent, dataset, faults, trials=50) -> dict`, `.summary(result)`.
`chaos_inject(agent, fault, p=1.0, seed=0)`	Wrap an agent so a fault fires with probability `p`.
`chaos_available()`	List available fault names.

Observability — `entropy.observability`

Symbol	Notes
`Watcher(agent, dataset, trials=20, seed=0)`	`.watch(rounds, interval, render)`, `.series()`, `.anomalies()`, `.failure_clusters()`.
`render_metric_heatmap(history, path, metrics=None)`	Writes a PNG heatmap.

Reports & Dashboard — `entropy.reports`, `entropy.dashboard`

Symbol	Notes
`export_report(results, fmt, path, title="EntroPy Report")`	HTML / JSON / Markdown / PDF / Jupyter.
`export(results, fmt, path, title)`	Lower-level dispatcher.
`build_dashboard(results, out_dir)`, `serve_dashboard(out_dir, port=8000)`	Static dashboard HTML.

Integrations — `entropy.integrations`

Symbol	Notes
`from_langchain / from_langgraph / from_openai / from_crewai / from_pydanticai / from_autogen / from_google_adk / from_mcp / from_custom`	Wrap a framework agent into a callable.
`adapter(name)`, `Adapter`, `register_adapter`	Define custom adapters.
`find_adapter(agent)`, `list_adapters()`	Discovery helpers.

Plugins — `entropy.plugins`

Symbol	Notes
`exporter(name=None)`	Register a report exporter `fn(results, path) -> str`.
`discover(group="entropy.plugins")`	Import entry-point plugins.