What you will be able to do
- List every agent, runner, schedule, family, environment and key your company runs, one row each.
- Find agents nobody registered by tracing each key to the runners that use it.
- Name the off switch on every agent, runner, schedule, family and key, and what it stops.
- Pause a schedule on purpose and confirm no run fired after it.
- Retire an agent without deleting its history or breaking a workflow that names it.
Key takeaways
- An AI agent inventory is a register that lists every AI agent a company runs, with its owner, the systems it reaches, and the switch that stops it.
- Keep the register in the records the agents run from, so the switch on each row is the one that stops it.
- Find unregistered agents from the keys and the schedules, not from memory.
- Test an off switch by checking that nothing ran after it, and treat turning back on as part of the switch.
An AI agent inventory is a register that lists every AI agent a company runs, with its owner, the systems it reaches, and the switch that stops it. In our register each row is the record the agent runs from, so the switch on the row is the one that stops it. An inventory you cannot switch off from is a list of things you hope are stopped.
By the end you have a register of every AI agent your company runs, the keys each one reaches, and an off switch on every row except environments, used and checked. For a small company it takes an afternoon. What agent sprawl is covers the why; this page is how to prevent agent sprawl in practice.
The examples use Kettlewick, a fictional online coffee roaster selling beans by subscription, and Ines, who runs its operations. Names and times are invented; field names, statuses, rules and thresholds are the real ones.
What you need before you start
You need every agent to live in a system that keeps it as a record with an on/off control. We use ConvOps.
| You need | Why |
|---|---|
| Where your agents run today | Cloud runners, a laptop, scheduled jobs, chat tools. An agent you cannot place, you cannot stop |
| One place where each is a record with an on/off state | A switch you have to go looking for is not a switch |
| One owner per family of jobs | Owners and review cadence are covered in our AI governance operating model |
We keep ours in ConvOps(opens in new tab), the system that runs the whole company on scheduled agents.
Step 1: List the six kinds of row
Keep the register in the records the agents run from. A second copy in a spreadsheet drifts the day someone adds an agent, and the switch is never in the spreadsheet.
| Kind | What it records | Its off switch |
|---|---|---|
| Agent | A named definition with saved revisions; runs get the active one | Disable |
| Runner (ConvOps calls it an executor) | Where agent runs execute, and which keys it uses | Disable the runner, stop its open runs and pause the schedules that name it |
| Schedule | A rule such as FREQ=DAILY;BYHOUR=18;BYMINUTE=0, its runner, its last runs | Pause |
| Family (a program in ConvOps) | A group of schedules with one purpose and one owner | Pause, which disarms every member |
| Environment | The tools, repositories and secrets a run receives | None: it holds secrets as references |
| Key | A model provider credential and the runners that use it | Revoke it at the provider |
The register row (a copyable AI agent registry template)
ConvOps holds most of these fields as records. Two are yours: ConvOps records an owner on a family, not on an agent or a runner, so you fill in the owner, and you write the last switch test.
name: review-replier
kind: agent
owner: Ines
purpose: drafts replies to customer reviews
runner: cloud-runner-main
schedule/trigger: Review replies, daily 18:00
keys it reaches: Model key (subscription), SHOP_API_KEY (reference)
off switch: disable (refused while Reply to reviews names it)
last switch test: <date, what you switched, what you saw after>
A list someone keeps by hand tells you an agent exists. It is not the thing that stops it.
Step 2: Find the agents nobody registered
Unregistered agents are found from the keys and the clock, not by asking around.
Sweep from the keys inward: each key's record in ConvOps lists the runners that use it, with the credential's state and never its value. Match every runner against the register. A runner with no row is an agent nobody registered.
At Kettlewick, "Model key (subscription)" lists cloud-runner-main and laptop-runner. Only the first has a row, so laptop-runner gets one before anything else happens.
Then sweep from the clock outward:
- Every schedule, paused ones included. The Paused tab keeps each row with its rule.
- "Missed its last fire". A repeating schedule gets this flag after one late fire.
- "No run for too long". The flag turns to this after 1.5 times the schedule's longest normal gap: 36 hours for a daily job. 1.5 is the default, settable from 1 to 10.
- A family whose verdict reads
silent. Itssilent_countsays how many members went quiet.
Staff chatbots on personal accounts are a different problem: shadow AI in the enterprise.
Step 3: Give every agent an off switch (an AI agent kill switch you can find)
Every row except an environment gets exactly one switch you can name, and ConvOps refuses the one switch that would break a workflow still using the agent.
| Switch | What it stops | What it keeps |
|---|---|---|
| Pause a schedule | The next fires, and its open runs, cancelled with the reason "schedule stopped" | The rule, the last runs, the runner for the restart |
| Pause a family | Disarms every member; you choose whether live runs stop or finish | Which members were armed. Resume re-arms only those, with one catch-up run each by default |
| Disable the runner, stop its open runs and pause the schedules that name it | Manual dispatch to the runner is refused, its open runs are cancelled, and the schedules that name it stop firing | The runner's record and connections |
| Disable an agent | It leaves the agent set of every later run | Every saved revision |
| Revoke a key | Every runner listed on the key's record | The record of which runners used it |
Assigning a schedule to a paused family warns by default, and it does not fire until the family resumes. Secrets have no switch because the register never holds them. ConvOps keeps each secret as a reference with a label and a state such as set, and never accepts or returns the value.
The switch ConvOps refuses
ConvOps refuses to disable an agent while a step of an enabled workflow names it. The refusal is HTTP 409, code agent_in_use, writes nothing, and names every workflow and step that holds the agent. You choose: clear those steps' agent, or disable those workflows first.
At Kettlewick, disabling review-replier is refused because the enabled workflow "Reply to reviews" names it in step draft-reply. Switching it off would break the job mid-workflow, so the system makes you decide first. When only disabled workflows name the agent, the disable succeeds and reports them. An isolated runner keeps each run apart from your machine, as in how we stop Claude Code deleting files.
Step 4: Test the off switch, one run traced
A switch is tested by watching what happens after it, not by flipping it. The proof is a fire time that passes with nothing on the row.
Ines tests the switch on "Review replies" (daily 18:00, family "Customer care", runner cloud-runner-main) while a reply template is rewritten:
| # | Time | What happens | What she reads |
|---|---|---|---|
| 1 | Yesterday 18:00 | Scheduled run | completed, kept as the last run |
| 2 | Today 12:00 | Manual run kw-run-208 | Still open, todo |
| 3 | Today 17:40 | Ines pauses the schedule | In the same change kw-run-208 is cancelled: "schedule stopped" |
| 4 | Today 18:00 | The fire time passes | No new run on the row or in "Customer care" |
| 5 | Next morning | She checks again | Paused tab, rule and last run intact, cloud-runner-main remembered |
| 6 | Verdict | She writes the result on the register row | The verdict line below |
| 7 | Restart | She re-enables, naming cloud-runner-main | The next run shows that runner |
The verdict line is a note you write on the register row:
off switch tested: paused 17:40, no run at 18:00, open run cancelled, restart runner cloud-runner-main
A pass is no run after the fire time, the open run cancelled, and a restart that lands on the runner it names. If the remembered runner no longer exists, ConvOps refuses the restart until you name another runner or clear it. Test a runner the same way: pause the schedules that name it, disable it, stop its open runs, and confirm nothing new starts. Watching agents while they run is AI agent observability.
Step 5: Retire, do not delete
Never delete an agent. Disable it, keep its history, and treat turning back on as part of the switch.
| Row | How it retires |
|---|---|
| Agent | Disabled; ConvOps has no delete action for agents, so every revision stays |
| Family | Retired, which is final and disarms every member |
| Runner | Delete is refused while a live task names it, and the refusal lists those tasks |
Turning back on is part of the switch: the two rules it needs are in Common mistakes below.
Common mistakes
We used our off switches wrong three times. Each time the stop looked done, the work came back, and it left a rule.
Pausing the schedule but not its runs
We paused a schedule and left its open runs open, and the recovery job, which re-sends runs that have gone quiet, sent them again. At Kettlewick: "Old promo emails" is paused, its open run kw-run-117 stays todo, and the recovery job sends it on its next pass.
The rule: a stop is two writes in one change. The schedule goes on hold and every open run is cancelled with "schedule stopped"; finished runs are never rewritten. A plain pause in ConvOps does both, and the same rule is why the runner switch always stops its open runs too.
Letting a runner come back into stale runs
We brought a runner back after an outage without clearing its queue, so the recovery job re-sent every stale run at once, and the flood starved the live run. At Kettlewick that is cloud-runner-cheap returning to a queue nobody needed any more.
The rule: cancel stale runs before the runner comes back.
Restarting a schedule without its runner
We restarted a schedule meant for a new runner and did not check its next run, which fired on the old runner. At Kettlewick: "Monthly price research" is restarted for a new runner, and nobody checks which runner its next run used.
The rule: name the runner when you restart, and check the next run used it. ConvOps remembers the runner across a pause and refuses a restart whose remembered runner no longer exists. This is how a failure becomes a rule across everything we run.
Steps
List the six kinds of row
Create a free ConvOps account at my.convops.app/register, then record every agent, runner, schedule, family, environment and key as a row, kept in the records the agents run from.
Find the agents nobody registered
Trace every key to the runners that use it and read every schedule, paused or flagged, to find agents with no row.
Name the off switch on each row
Name one switch per row (pause, disable or revoke; an environment has none, it holds secrets as references) and know which switch the system refuses while a workflow still uses the agent.
Test the off switch
Pause a schedule before a known fire time, confirm no run follows and its open runs are cancelled, and write the result on the row.
Retire, do not delete
Disable agents instead of deleting them, cancel stale runs before a runner returns, and name the runner when you restart a schedule.
Frequently asked questions
What is an AI agent inventory?
An AI agent inventory is a register that lists every AI agent a company runs, with its owner, the systems it reaches, and the switch that stops it. In ConvOps each row is the record the agent runs from: the agent, its runner, its schedule, its family and the keys it uses, each with its own off switch: pause or disable in ConvOps, and revoke at the provider for a key. The owner and the last switch test are the two fields the company adds itself.
Who needs an AI agent inventory?
Founders and operators of small companies whose AI agents run on schedules, runners and model keys need one. ConvOps keeps those agents as records in one place, so any agent can be found, traced to the keys it spends, and stopped from the same row. A company needs it before the first agent nobody remembers starts spending a key.
What should an AI agent inventory record?
Each row records the name, kind, owner, purpose, runner, schedule or trigger, the keys it reaches, its off switch and the last switch test. ConvOps holds most of these as records: the agent definition, the runner, the schedule rule and the key with the runners that use it. The owner of an agent and the result of the last switch test are written on the row by the company.
How do I find AI agents nobody registered?
Create a free ConvOps account at my.convops.app/register, then list every key and read the runners each key's record names. Any runner without a row is an agent nobody registered. Next, read every schedule, paused ones included, and every family whose verdict is silent. ConvOps flags a schedule "Missed its last fire" after one late fire and "No run for too long" after 1.5 times its longest normal gap.
Is there a kill switch for AI agents?
Yes, one per row rather than one for everything. In ConvOps you pause a schedule, pause a family, disable a runner, then stop its open runs and pause the schedules that name it, or disable an agent, and you revoke a key at its provider. ConvOps refuses to disable an agent while a step of an enabled workflow names it, and the refusal names that workflow and step.
How do I test that an AI agent's off switch works?
Create a free ConvOps account at my.convops.app/register, then pause a schedule shortly before a known fire time. Confirm that no run appears after the fire time and that open runs were cancelled with the reason "schedule stopped". ConvOps keeps the rule and the last run on the paused row. Write the result on the register row, then restart naming the runner and check the next run used it.
How do I retire an AI agent without breaking a workflow?
Create a free ConvOps account at my.convops.app/register and disable the agent instead of deleting it: ConvOps has no delete action for agents and keeps every revision. ConvOps refuses the disable while a step of an enabled workflow names the agent, and the refusal lists those steps. Clear the step's agent or disable the workflow first, then disable the agent.