Writing "check again in March" isn't enough. Put it in someone's calendar, and also check again straight away if something big changes.
Review dates slip because nobody owns them and nothing reminds anyone. I set two kinds: a calendar date, and a list of events that trigger a review early, like a model update or a spike in complaints. Each one has an owner and an automatic reminder.
Pair each register row with a cadence (set by priority) and event triggers (model or prompt change, policy change, volume shift, error-rate breach, incident). Automate reminders with assignment and due dates, record whether the review happened, and report review completion as a control metric.
Why review dates get missed
A review date in a spreadsheet column is a hope. Nobody gets a reminder, the owner has moved on or forgotten, and the date passes without anyone noticing. When someone finally looks, half the dates are in the past.
There's a second problem. Calendar reviews happen on a fixed schedule, but AI risks change on events. A model update, a new data source or a change of policy can change a risk overnight. A review in six months is too late for that.
Two kinds of review date
I give each register row both.
The calendar date depends on priority from Likelihood, impact and reversibility on one sheet.
| Priority score | Calendar review |
|---|---|
| 40 and above | Monthly |
| 15 to 39 | Quarterly |
| Below 15 | Every six months |
The triggers are events that bring the review forward, whatever the calendar says.
| Trigger | Example |
|---|---|
| The model, prompt or agent changes | Microsoft updates the model behind an agent; someone edits its instructions |
| The data changes | A new SharePoint site is added as a knowledge source |
| The policy changes | The rule the agent applies is rewritten |
| Volume moves sharply | Use doubles in a month |
| The error rate moves | The weekly sample finds more errors than the agreed limit |
| Something goes wrong | A complaint or incident involving the decision |
Making the reminder impossible to ignore
I'd put each calendar review in a tool that assigns it to a named person with a due date, such as a task in Microsoft Planner or an item in a SharePoint list with an owner column. A scheduled Power Automate flow can then check each week for overdue reviews and message the owner in Teams, copying their manager after a set number of days. The patterns for that are in Nudge patterns with Power Automate, Teams and Autopilot.
Triggers are harder to automate, because many of them happen outside the register. The most reliable approach I can see is to name, for each trigger, who would know first, and make telling the risk owner part of their change process. For example, whoever edits an agent's instructions logs the change in the register.
A worked example
This example is illustrative. The customer service register has five rows. After scoring, the call summary row (31.5) and escalation row (28) are quarterly, and the others are six-monthly.
In week six, the team adds a new knowledge source to the summary agent: archived case notes from an old system. That's a data trigger. The agent owner logs it, the risk owner gets an automatic message, and a review happens that week instead of in week thirteen. The review finds that old notes use different codes, so summaries misread some cases. The team adds a line to the agent's instructions and five test cases to its test set.
Without the trigger, the problem would have run for seven weeks before the quarterly review.
Record that the review happened
The register should show the date each review actually took place and what it decided, even if the decision was "no change". Then review completion itself becomes a number: what percentage of reviews happened on time this quarter? If that number drops, the register is decaying, whatever the risk scores say.
What I'm still checking
I'm unsure how to set the "error rate moves" trigger without enough data to know the normal range. For a new agent, I'd set a deliberately cautious limit for the first two months and widen it once the weekly sample shows what normal looks like.
Sources
This article describes my own method. It mentions Microsoft Planner, SharePoint lists, Power Automate and Teams as places to hold and send reminders, which are standard Microsoft 365 tools; specific flow actions are covered and sourced in the Nudges collection. All figures are illustrative.