EntroPy measures what other frameworks miss: uncertainty, drift, stochastic behavior, and emergent failures — via Monte Carlo evaluation over repeated runs and entropy-based metrics. Framework-agnostic, runs locally, no vendor lock-in.
Run any agent N times and aggregate behavior across trials — not a single lucky pass.
Success, reliability, behavioral & trajectory entropy, drift, robustness, safety and more.
Track how an agent's behavior distribution shifts across trials and over time.
Inject faults (API failures, timeouts, tool crashes) and measure resilience.
Adapters for LangChain, LangGraph, OpenAI, CrewAI, PydanticAI, AutoGen, Google ADK, MCP — all lazy.
Reports, dashboards, observability and a CLI — everything runs on your machine.
pip install entropy-ai # core
pip install "entropy-ai[langchain]" # optional: a framework adapter
pip install "entropy-ai[pdf,jupyter]" # optional: PDF / Jupyter reportsfrom entropy import Suite, Dataset, Case
def my_agent(inp): ...
# return an output, an AgentRun, or dict with events
dataset = Dataset([Case(input="q1", expected="correct")])
results = Suite(seed=42).run(my_agent, dataset, trials=100)
print(results) # 50+ metrics