01
Functional correctness
Software & terminal
Terminal-Bench 2.1
Terminal tasks for parsing, concurrent services and reliable systems tools.
Benchmark / task domain
Terminal-Bench 2.1 format
The work
Build reliable tools inside terminal environments.
Agents build and repair tools that process data, coordinate concurrent work and maintain service state. Tasks emphasize functional correctness and reliable execution.
Delivery can include
- 01
- Harbor-format task packages with Markdown instructions and task.toml configuration
- 02
- Docker environment definitions, code and task inputs
- 03
- Reference solutions and executable evaluation code
How we evaluate the task
Define how to evaluate the result.
02
System behavior and integration
03
Completeness of task outputs
WORK WITH ASTRAFLUX
What should your
model be able to do?
Share the capability, task types and delivery requirements. We’ll discuss the right program and representative tasks.
Contact us norman383@astraflux.spaceBenchmark names indicate reference directions. Task scope and delivery specifications are agreed for each project.