Expert-built RL environments beyond software engineering

The work professionals are paid for, rebuilt as environments an agent can learn from. Each ships with a reference solution and a verifier stress-tested against reward hacking.

Our team

50,000+RL environments built
NeurIPS · ICML · Naturewhere our RL research has appeared

Domains

The agent works in the tools of the trade and delivers the files a professional would.

  • Electronics

  • Architecture

  • Motion design

  • 3D animation

  • Games

  • Biology

Custom

We draw on our network of experts to build tasks in your domain.

Built by domain experts

  1. The expert does the work

    A working professional solves the task in their own tools. That is the reference solution.

  2. We build the task

    The brief, the environment and the verifier, in Harbor format.

  3. We probe and attack the verifier

    Known good and bad work first, then attacks.

  4. QA

    Only a task that passes ships.

Formats

Tasks ship in the Harbor format, so they run in the harness you may already use.

  • Harbor format
  • Every probe, attack and model trial included

Licensing

Where a task uses a firm's real files, they are licensed, with signed consent on file.

A reward you can train against

A verifier that can be gamed teaches a model to game it. We attack ours before it ships and close what the attacks find.

blocked 0passed 0

Every task passes three layers of QA

  1. 01

    Automated probes

    The reference scores full reward on every run. A no-op scores zero. Breaking one requirement loses reward.

  2. 02

    Attacks

    Agents try to get reward without doing the work. We close every hole before the task ships.

  3. 03

    Expert review

    The expert confirms the scores on model trials match their own judgment.

Scores are computed by code from the delivered files, in an isolated container with no network.

The right difficulty for your model to learn from

Too easy teaches nothing. Too hard gives no signal. We run your model on every task and ship the ones in your target band.

Example task✓ accepted

Rig a character in Blender and animate a kick from reference video

A task we shipyour model, 50 trials
22%in target
Saturated task
90%too easy
Task out of reach
0%no signal

Environments built for your lab

Custom environments to your spec. We start with a pilot.

  1. Scope

    Domains, tools, target model

  2. Sample

    Nine tasks to judge quality

  3. Pilot

    A small batch in your domains

  4. Calibrate

    Runs against your model

  5. Deliver

    Harbor, with every trial

↺ After the pilot: scale, checked against your evals

> request --samples

Nine runnable tasks with reference solutions, verifiers and every trial.

Domains