API exploration from specs
Have the model read an API's specification and then call the API itself — chaining requests, trying variations and comparing what comes back with what the spec promises.
How it works
An API specification is a contract with many clauses nobody checks one by one. The model reads it, plans sequences of calls that exercise real workflows (create, read back, update, list, delete), and runs them against a test environment, adapting as it goes: using an ID from one response in the next request, trying a field the spec marks optional with and without a value, sending the wrong type, calling endpoints in an unexpected order, and repeating a request to check idempotency. Every response is compared with the specification — status codes, fields, types, error formats — and with earlier responses. Mismatches are reported with the exact requests that produced them.
When to use it
New or changed APIs before consumers build on them; APIs with a specification but thin test coverage; checking that documentation and implementation still agree; and exploring third-party APIs before integrating with them.
Limitations
The specification is the model's main oracle, and specifications are often incomplete or wrong; a mismatch may be a documentation bug. Business rules that aren't in the spec — who may see which records, what totals should come to — need to be supplied or they go untested. Calls are real: an agent with write access will create and delete data, and with production credentials it can do real damage. Rate limits and costs apply.
Watch for
If the spec is generated from the code, every response matches it by construction and the run reports no mismatches. Check where the spec comes from, and supply business rules separately, such as that one customer can’t read another’s orders.
What you still own
The credentials and environment the agent gets, the business rules it should check, and deciding whether each mismatch is a bug in the API or the spec.
Cite this
Qlarify Labs. (2026). API exploration from specs. Retrieved from https://labs.qlarify.fi/ai-in-testing/api-exploration-from-specs