A New Tool for AI Behavioral Evaluation: How to Use?

Bloom is now live as an open source framework designed to test behavioral traits in advanced AI models. The tool focuses on one key challenge in AI alignment. Behavioral evaluations take time and often age quickly. As models improve, fixed tests lose relevance. Bloom aims to solve that problem by generating new evaluation scenarios on demand.

Instead of testing models with the same prompts repeatedly, Bloom creates fresh situations every run. As a result, researchers can measure whether a behavior exists without relying on stale benchmarks.

What Makes It Different

Bloom is built for speed, scale, and flexibility. Researchers define a single behavior they want to measure. It then creates many scenarios to test how often that behavior appears and how severe it is.

Unlike older evaluation methods, Bloom does not depend on manually written conversations. Instead, it automatically generates, runs, and judges interactions. This reduces engineering effort and makes evaluations easier to update as models evolve.

Bloom also integrates with tools like Weights & Biases and exports Inspect-compatible transcripts. These features help researchers analyze results at scale.

How It Works Step by Step

Bloom follows a four-stage pipeline. First, it studies the behavior definition and examples provided by the researcher. Next, it designs diverse scenarios meant to trigger that behavior. Then, it runs these scenarios by simulating users and tools in real time. Finally, a judge model scores each conversation and produces summary metrics.

Because Bloom uses a configurable seed file, evaluations stay reproducible. At the same time, each run produces new scenarios, keeping the tests fresh and harder to game.

Benchmark Results Show Clear Separation

To demonstrate Bloom’s reliability, researchers tested it on four alignment-relevant behaviors. These include delusional sycophancy, long-horizon sabotage, self-preservation, and self-preferential bias. The evaluations covered 16 frontier AI models.

Bloom successfully separated baseline models from intentionally misaligned ones. In validation tests, its scores closely matched human judgments. Some judge models showed especially strong agreement at high and low extremes, which matters most for safety thresholds.

Why Bloom Matters for AI Alignment

Bloom complements earlier tools like Petri by offering targeted behavior measurement. While Petri explores broad behavioral profiles, Bloom zooms in on one trait at a time.

As AI models grow more capable, fast and adaptable evaluations become essential. Bloom offers a practical way to keep pace without relying on outdated tests. For researchers focused on alignment and safety, this could change how behavioral risks are measured.