Datasets

A Dataset is a list of Case objects. Each case defines an input, an expected output (or a custom check), and metadata.

Building datasets

from entropy import Dataset, Case

dataset = Dataset([
    Case(input="q1", expected="A"),
    Case(input="q2", expected="B"),
])

Custom checks

When expected isn't enough, supply a check callable. It takes the agent output and returns a bool:

Case(input="sum?", check=lambda out: out.isdigit() and int(out) > 0)

Specialized case types

Case type	Purpose
`Case`	Generic input/expected/check.
`GoldenCase`	Known-good pair the agent must reproduce exactly.
`Scenario`	Stateful scenario; `setup` is applied to the environment first.
`BehaviorCase`	Checks behavior (not just output) via `behavior_check`.
`FailureCase`	Expects the agent to fail gracefully (success = handled).
`AdversarialCase`	Provokes unsafe behavior; carries an `attack` tag.

Loading from files

Dataset provides class-method loaders for several formats:

Dataset.from_json("cases.json")      # {"cases": [{"input": ..., "expected": ...}]}
Dataset.from_jsonl("cases.jsonl")
Dataset.from_yaml("cases.yaml")      # requires the pyyaml dependency
Dataset.from_csv("cases.csv")         # columns: input, expected, type
Dataset.from_hf("squad", split="train")  # requires the hf extra

Case records may include a "type" field (case, golden, scenario, behavior, failure, adversarial) to select the right subclass automatically.

Saving

dataset.save("cases.json")

CLI

Inspect any dataset without code: entropy dataset --path cases.json.