Live online workshop
Testing AI Agents: evals, regression suites and done-gates your agents cannot skip
Stop taking "done" on trust. Build an eval set, a regression suite your agent cannot fool and a CI gate it cannot skip, on your own repo, in one live session. Live online on Google Meet, 1 day, 3 hours, small group.
Ergebnisse
You will leave able to
- Name the ways an agent reports "done" without proof (a deleted or skipped test, an assertion that checks nothing, a green run that executed zero tests, a check passed by stale data) and the check that catches each.
- Put each claim at the layer that can prove it: unit test, eval set, regression suite or acceptance run.
- Write an eval set for a job your agent repeats: fixed cases, pass or fail checks, and a floor that only moves up.
- Write YAML regression tests against a running service whose assertions fail on empty, null or stale data, and lint them so a weak test cannot be merged.
- Run the suite in CI and make the pipeline verdict the done-gate, with integrity rules in the agent's instructions: never delete, skip or loosen a test to go green.
- Turn each incident into a ledger entry that a reviewer checks on the next piece of work.
Für wen
Built for
- Dev leads whose team already ships code written by coding agents: the agent says "done", the tests are green, and the bug still reaches main
- Senior developers who review agent merge requests: you cannot read every line, so you need proof you can check in a minute
- Tech leads and CTOs deciding how far to let agents run: more autonomy needs a gate that holds without you watching
Termine
Upcoming dates
- Di., 2. Feb.09:00–12:00
Zeit in Dublin · Englisch Online
12 Plätze · findet ab 4 Anmeldungen statt
Plätze verfügbar420,00 €
inkl. MwSt.
Preis
Price
420,00 €bis 13. Jan.
inkl. MwSt.
danach 495,00 €
Team mit 5 Personen: 1.980,00 €
Ablauf
Agenda
Modul 1
0:00 to 0:15: How agents fake green
Modul 2
0:15 to 0:50: Evals for agent work
Modul 3
0:50 to 1:35: A regression suite the agent cannot fool
Modul 4
1:45 to 2:25: The done-gate
Modul 5
2:25 to 2:50: Break it on purpose
Modul 6
2:50 to 3:00: Q&A and close
Trainer
Your trainer
Ihr Trainer
David Marsa
Founder of Neomanex. Builds agent-run software companies: ConvOps, Gnosari and the Neomanex directory. Teaches the workflows he runs his own company on.
Kurse
Related workshops
Read before you come
Häufige Fragen
Questions
What is the Testing AI Agents workshop?
Testing AI Agents is a live online Neomanex workshop on proving that work done by coding agents is actually done. In one 3-hour session on Google Meet, each attendee builds an eval set, a YAML regression suite and a CI gate on their own repo, then hands their agent a change and watches the gate decide. Neomanex ships its own products through the same gates.
Who should take Testing AI Agents?
Neomanex wrote Testing AI Agents for dev leads, senior developers and tech leads whose teams already merge code written by coding agents. You should have reviewed merge requests an agent opened. Teams still aligning how each developer's agent works should start with Claude Code for Teams; teams ready to split work across several agents should look at Build Your AI Agent Team.
How is this different from an AI evals course?
Most evals courses teach how to score a model's answers: prompts, judges and datasets for an AI product. Neomanex's Testing AI Agents covers agents that change code and run work, so the tests check what the agent changed in a running system, inside the CI pipeline that decides the merge. It is one live session taught by a company that ships this way, not a multi-week course.
Do I have to use regrun?
No. Neomanex built regrun and uses it for every product's regression suites, so the trainer demonstrates with it, and it installs free from PyPI. The habits the session teaches carry to any runner: assertions that fail on empty data, a sweep of your own fixtures, a test count you read, and a verdict from the pipeline. Your existing test framework works for the eval part.
What do I need ready before the session?
Arrive with a coding agent (Claude Code recommended), git, a terminal, Python 3.11 or later and a repo with a service, API or CLI you can run locally. You also need permission to edit that repo's CI, on GitLab or GitHub. If none of that can leave your company, Neomanex gives you a sample repo with a small API and a pipeline, so every exercise still runs.
What time does Testing AI Agents run?
Each run happens in one of two slots. The Europe slot starts at 17:00 and ends at 20:00 Malaysia time: in Dublin and London that is 09:00 to 12:00 in winter and 10:00 to 13:00 in summer. The APAC slot runs 10:00 to 13:00 in Malaysia and Singapore. Dates on the runs table appear in your own time zone.
What if I cannot attend after booking?
Testing AI Agents seats are not refunded. A colleague can take your place at no charge: send Neomanex their name and email before the session begins, and the seat, the recording and the certificate transfer to them. For anything this policy does not cover, write to Neomanex and the request is looked at on its own.
Will I get the recording?
Yes. The session is recorded, and Neomanex emails the recording to everyone booked on that run within 24 hours. You see the trainer's screen, the live suite and pipeline, and the group's questions. The recording backs up the live session and is not available to buy separately.
Is there a certificate?
Attendees who join the Testing AI Agents session live receive a Neomanex certificate of completion 24 hours afterwards, naming the workshop, the date and the trainer. It records attendance at a Neomanex workshop. It is not an accredited qualification, and Neomanex does not claim it counts toward any training or funding scheme.
Neomanex is not affiliated with Anthropic. Claude and Claude Code are trademarks of Anthropic.