Skip to content

Promotion reviews

A workflow moves up the autonomy ladder one level at a time, and only after a promotion review. This page covers what the review checks, who attends, what is recorded and what can come out of it.

Every move up takes a review, during the Embedded Build and under Operate & Automate alike. A review is held as soon as a workflow is ready, remotely if need be. It does not wait for a calendar date, and a ready workflow does not wait for the quarterly on-site week. A review that falls near that week is held in person. Ready means two things: the workflow has done enough real work at its current level to be measured against its threshold, and the workflow owner wants to take the next step. Our forward-deployed engineers can suggest a review. They cannot push through a promotion the workflow owner does not want.

Nothing moves up between reviews. Moving down never needs one. Operate & Automate covers how reviews fit into the work after the build.

1. Measured accuracy on real work, against a threshold set in advance

Section titled “1. Measured accuracy on real work, against a threshold set in advance”

The threshold is set for the workflow, with the workflow owner, and written down before the level starts. Nobody adjusts it after seeing the results.

What counts as correct is written down too, in the workflow owner’s terms. For a sales order that means the right customer and ship-to address and, on every line, the right part number, quantity, unit of measure, price and date.

The measurement uses real work: everything the workflow handled in the period, not a test set picked for the occasion. What gets measured depends on the level the workflow is leaving.

Moving from What is measured
Level 0 (Recorded) to Level 1 (Assisted) The agent is tested on the work recorded at Level 0. It drafts from the same documents your people worked from, and its drafts are compared with what they entered. Nobody at the desk sees a draft until the review is passed.
Level 1 to Level 2 (Prepared) How often the agent’s draft matched what the person keyed, field by field.
Level 2 to Level 3 (Supervised) How often a person approved the prepared transaction without changing it, and what they changed when they did.
Level 3 to Level 4 (Autonomous) What spot-checks found in the work the agent completed on its own, and which errors surfaced later, such as a credit or a corrected order.

The test before Level 1 sends recorded documents to an AI model, so it runs only after the provider is agreed with you. See AI providers and your data.

Our engineers break the results down by customer or supplier and by type of document, so a good overall figure cannot hide a group the agent handles badly. The breakdown also shapes the limits for the next level: the agent acts on its own only for the groups it handles well.

These docs state no accuracy figures. A threshold only means something for one workflow at one company.

Accuracy does not settle it alone. The review asks three questions about the next level: what is the worst realistic mistake, how soon would someone notice and what would it take to put right.

A wrong ship-to address caught at approval costs a minute. The same mistake on an order that has shipped costs freight, a return and some of the customer’s patience. Where a mistake is expensive, the answer is tighter limits or staying at the current level. A workflow can pass on accuracy and still be held back here.

The review checks both by tracing real cases, not by reading a description of how things should work.

  • Logged. For each kind of action the agent would take at the next level, the audit trail holds the inputs, the decision and the result.
  • Reversible. For each kind of action there is a written way to put it right. It has been tried, and it names who does it. Cancel the order before it is released to the warehouse. Void the invoice before it is paid. Send a correction after a wrong email.

An action with no way back, such as releasing a payment, is not given to an agent to take on its own.

The workflow owner decides. If the first three checks pass and the workflow owner is not comfortable, the workflow stays where it is. Our engineers bring the evidence. They do not make the call.

Who Why
The workflow owner Decides and signs off
One of our engineers who built and runs the workflow Brings the measurements, cases from the audit trail and the proposed limits
Someone who does the work every day Knows what is behind the figures: which corrections were trivial and which were close calls
Your IT manager or ERP administrator Attends when the next level gives the agent new access to the ERP
Your controller Attends when the next level lets the agent commit money, as in purchasing or AP

Each review leaves a written record. It is part of the data you own. It holds:

  • the workflow, the date and who attended
  • the level the workflow is moving from and to
  • the measurements, set against the threshold that was written down in advance
  • the cost of a wrong action at the new level, and how each kind of action is reversed
  • the limits for the new level: amounts, customer and supplier lists, time windows
  • how much completed work will be sampled, if the new level is Level 3 or Level 4
  • the threshold the workflow must meet before the next review
  • what would send the workflow back down
  • the decision and the workflow owner’s sign-off.
  1. Promote. The new level starts with the limits in the record.
  2. Not yet. The review names what has to change, such as a rule, a data fix or more evidence for one group of customers. The workflow stays at its level and is measured again.
  3. Stay for good. The workflow owner decides the current level is the right one for the long term. Not every workflow should reach Level 4. Some stay at Level 2 or 3 for good, and that is a fine result.

A promotion is not permanent. The record says what would send the workflow back down, and the workflow owner can act on it at any time, without a review. Moving back up takes a new one.