When a computer sends someone a message, it has chosen to interrupt them. If it does that too often, people stop listening. So choosing when to send it matters.
Each nudge makes a choice: whether today's numbers are worth interrupting someone. That choice has costs both ways. Too many nudges and people ignore them; too few and problems slip through. I set each nudge's limit like any other decision threshold, and review whether it's earning its place.
An alert is a binary classifier with a threshold; false alarms cost attention and erode trust, misses cost the risk. Set the trigger using the same cost-ratio method as decision thresholds, track response and action rates, and retire or retune alerts with low action rates.
The decision inside every alert
When a flow checks a number and decides whether to message someone, it's making a yes-or-no call. Send or don't send. Like any decision, it can be wrong in two ways. It can send when nothing needed doing, which wastes someone's attention. Or it can stay quiet when something did, and the problem grows.
That framing helps, because it means the tools on this site for setting thresholds also apply to nudges.
The cost of a false alarm
The obvious cost of a false alarm is a few minutes of someone's time. The less obvious cost is trust. If most alerts from a source turn out to be nothing, people learn to ignore them, including the one that matters. So the real cost of a false alarm is higher than the minutes it takes to read.
Setting the trigger
In Setting a threshold you can defend, the threshold comes from the cost of a wrong yes compared with a wrong no. For a nudge, a wrong yes is a false alarm and a wrong no is a missed problem.
A worked example
These figures are illustrative. A weekly nudge checks an agent's sampled error count and alerts the risk owner if it's above a limit.
| Value | |
|---|---|
| Cost of a false alarm (time, plus some loss of trust) | About £30 |
| Cost of missing a real rise in errors for a week | About £300 |
| Threshold: false alarm ÷ (false alarm + miss) | 30 ÷ 330 ≈ 0.09 |
So it's worth sending the nudge whenever the chance that errors have really risen is above about 9%. That's a low bar, which says this nudge should lean towards firing. If the cost of a miss were only £60, the threshold would be 30 ÷ 90 ≈ 0.33, and the nudge should be much quieter.
Turning a probability into a count limit needs some knowledge of the normal error rate. With a sample of 20 and a normal rate of around 5%, one error a week is expected. Two might be chance. Four is very unlikely to be chance. Where exactly to draw the line depends on the costs above, and I'd start cautious and adjust once there are a few months of data.
Track what happens after the nudge
For each nudge I'd record three things: did it fire, did anyone respond, and did the response lead to an action other than "accept"? Over a quarter, that gives an action rate.
| Action rate | What I'd do |
|---|---|
| High: most nudges lead to action | Keep it; it's earning its place |
| Middling | Check whether the limit is set well |
| Low: almost every response is "accept" | Raise the limit, merge it with another nudge, or retire it |
A nudge with a low action rate does quiet harm, because it teaches someone to ignore that channel.
Who decides the nudge's limit
The risk owner should own the limit, since they bear the interruption and the consequences of a miss. Changing it is a small decision worth recording, with the reason, in the decision record format.
Where I got stuck
The cost of lost trust is real and hard to put a figure on. I've folded it into the false alarm cost as a rough uplift, which is a judgement call. A better approach would be to measure how response times change as false alarms rise, but that needs more data than a new nudge has.
Sources
This article applies the threshold method already on this site and uses no external facts or figures. All costs and rates in the example are illustrative.