ZSE-04 · Iterate · Scaling AI in Enterprises

04

Iterate —
Feedback Loops

Build-measure-learn for AI adoption. What Copilot analytics and Viva Insights actually tell you. The retrospective cadence. Mapping your real jagged frontier through systematic experiment.

"The first version of anything is a hypothesis. Treat AI deployment as an experiment, not a programme."
— Craig Stanley
Four Zones Reference

Zone 1: Employee Only · Zone 2: Copilot Assisted · Zone 3: Cowork · Zone 4: Automated. Full framework in Issue 01.

01 / Build-Measure-Learn

Applying Lean Thinking to AI Adoption

The lean startup principle — build a minimum viable product, measure what happens, learn from the result, and repeat — is the most appropriate mental model for enterprise AI deployment. Not because AI adoption is a startup, but because it shares the startup's essential condition: genuine uncertainty about what will work. The jagged frontier that Ethan Mollick named in the BCG research is, at its core, the same problem Eric Ries described when he said that the only way to find product-market fit is to be in the market. You cannot predict which AI use cases will stick in your organisation from outside it. You have to run the experiment.

This means treating each deployment wave as a structured hypothesis. Before Wave 1 goes live, write down your hypotheses: we expect that Copilot will be most valuable for email drafting and meeting summaries in the first wave; we expect that Zone 2 tasks will show measurable time savings before Zone 3 tasks do; we expect that the Finance team will find it more useful than the Operations team because of their document-heavy workflows. Write these down. Assign a metric to each one. At the end of the wave, test your hypotheses explicitly.

The organisations that iterate fastest on AI adoption are not the ones with the most sophisticated analytics. They are the ones with the most disciplined feedback loops — the ones that have built rituals for capturing what they are learning and structures for acting on it. Microsoft Viva Engage (for qualitative learning), the Microsoft 365 admin reporting suite, and Microsoft Viva Insights (for behavioural signals) provide all the infrastructure you need. The gap is usually the discipline, not the data.

02 / What to Measure

Copilot Analytics and Viva Insights

The Microsoft 365 Admin Centre provides Copilot usage analytics that most organisations do not fully utilise. The key reports cover active usage days (number of days each user has actively used Copilot, broken down by product), feature usage (which Copilot capabilities are being used — meeting summaries, email drafts, document generation, data analysis), and adoption trends over time. These are your leading indicators — the signals that tell you whether behaviour is changing before you can measure outcomes.

Microsoft Viva Insights adds a layer of organisational intelligence to the raw adoption data. It provides aggregate signals on meeting load, focus time, collaboration patterns, and — increasingly — AI usage integrated with wellbeing and productivity metrics. Viva Insights can tell you, at the team level, whether Copilot usage correlates with reduced meeting load or increased focus time. These are the intermediate outcome metrics that bridge the gap between "people are using the tool" and "this is making us more productive."

The distinction between seat utilisation and active usage is critical and widely misunderstood. Seat utilisation — the percentage of licensed users who have ever opened Copilot — is a procurement metric, not an adoption metric. A user who opened Copilot twice in three months contributes to your seat utilisation number but has not adopted the tool. Active usage days — the number of days on which a user has genuinely engaged with Copilot features — is the metric that predicts durable behaviour change.

Metrics that illuminate
  • Active usage days per user per month
  • Feature adoption breadth (number of distinct features used)
  • Champion vs non-champion usage comparison
  • Week-on-week retention (users who continue to use after first week)
  • Qualitative sentiment from check-ins
  • Specific task time savings (self-reported, validated)
Metrics that mislead
  • Raw seat utilisation percentage
  • Self-reported productivity gains (without validation)
  • Cost savings (too early, too hard to attribute)
  • Number of prompts entered (effort, not value)
  • Feature usage totals (without denominator — per user rates matter)

03 / The Retrospective Cadence

Monthly Team Retrospectives on AI Use

The retrospective is the most underused tool in the AI adoption toolkit. Every agile team runs sprint retrospectives; very few teams run retrospectives specifically on their AI usage. The monthly AI retrospective — run at team level, facilitated by the champion, thirty minutes maximum — is the mechanism that turns individual experimentation into collective learning.

The structure is simple and should not be over-engineered. Four questions, honest answers, documented and shared. The value is not in the questions themselves but in the discipline of asking them regularly, in the safety to say "this didn't work," and in the aggregation of team-level insights into a programme-level picture of the frontier.

Monthly AI Retrospective Template

01What did AI help with this month? Specific task, specific outcome, specific time saved or quality improved. Not "it was useful for emails" — "I used Copilot to draft the Q3 stakeholder report and cut the drafting time from four hours to 90 minutes."
02Where did it get in the way? Tasks where AI made things worse, produced unreliable output, or created more work to correct than it saved. These discoveries are as valuable as the wins — they map the boundary of the frontier.
03What prompt or workflow worked well enough to share? Specific enough that a colleague in another team could replicate it. Add it to the Viva Engage champions community.
04What are we going to try next month? One specific experiment. One measurable hypothesis. One person responsible for reporting back.

The retrospective outputs should flow to the champions network in Viva Engage. Not as formal reports — as honest posts. "Our Finance team tried Copilot for variance analysis commentary this month and here is what we found." These posts, aggregated across teams and over time, build the organisational map of the frontier that no single deployment lead could produce alone.

04 / The Jagged Edge in Practice

Mapping Your Real Frontier

Ethan Mollick's concept of the jagged frontier is not just a theoretical framing — it is a practical research question. The frontier that exists in your organisation right now is different from the one documented in the BCG study in 2023, different from the one your technology vendor will describe in their capability briefings, and different from what any external consultant can tell you. It is your frontier. The only way to map it is to deploy, experiment, and capture what you find.

The simplest infrastructure for frontier mapping is an AI win / AI fail log — a shared channel or document where team members record two things: a task where AI genuinely helped (with enough detail to be replicable) and a task where AI fell short or made things worse (with enough detail to warn colleagues away). This does not need to be elaborate. A running Teams channel post, a shared OneNote page, or a Viva Engage community thread all work. The discipline is in the habit, not the system.

When you aggregate these logs across teams and departments, patterns emerge that are specific to your organisation. You will find that AI is remarkably reliable for certain task types in your context but fails in ways that were not predicted on others. You will find that certain prompting patterns work consistently and others do not. You will find that some roles benefit significantly from Zone 2 Copilot assistance while others find the output quality insufficient for their standards. This is your frontier map — built empirically, updated continuously, and vastly more accurate than any pre-deployment analysis could have been.

The frontier also shifts over time as model capabilities improve. A task that fell outside the frontier six months ago may have moved inside it with the latest model update. This is why frontier mapping is a continuous process, not a one-time exercise. The AI win / AI fail log is not a project deliverable — it is an ongoing operational intelligence system.

"The frontier is jagged in ways that are specific to your organisation, your roles, your data. No external source can map it for you. Only your own people, experimenting systematically, can do that."
Adapted from Dell'Acqua, Mollick et al. — "Navigating the Jagged Technological Frontier", HBS WP 24-013, 2023

05 / Rapid Iteration

From Experiment to Embedded Workflow in Six Weeks

The micro-sprint model for AI workflow development takes a team from "we are trying Copilot for meeting summaries" to a refined, team-specific, documented workflow in six weeks. The framework is deliberately tight — six weeks is short enough to maintain energy, long enough to generate real learning.

Week one: define the specific task, the current process, and the hypothesis for how AI will improve it. Be specific about what "improvement" means: time saved, quality improved, consistency increased. Week two: everyone on the team tries the task with AI at least twice using whatever approach they choose. No rules yet. Week three: share what you found. What worked? What did not? Identify the approach that produced the best results. Week four: everyone uses that approach consistently. Refine the prompt or workflow based on what you learn. Week five: document the workflow — the prompt template, the review checklist, the exception cases. Week six: share with other teams via Viva Engage. Move to the next task.

The Brynjolfsson et al. NBER study of 14,000 customer support agents found that AI-assisted agents improved productivity by 14% overall, but that the gains were concentrated among newer, less experienced workers — those who benefited most from access to the expertise embedded in AI responses. The implication for micro-sprint design is to ensure that experienced workers are the ones defining the quality standard for AI output, while less experienced workers may be better positioned to adopt the workflow quickly. Design for both.

+14%
Productivity gain — AI-assisted customer support agents
Brynjolfsson, Li, Raymond — NBER WP 31161, 2023
14k
Agents studied over six months
Brynjolfsson, Li, Raymond — NBER WP 31161, 2023

06 / Key Moves

Five Actions for the Programme Lead

Key Moves — Issue 04 · Iterate

1
Write down your three deployment hypotheses before Wave 2 starts. Which teams will benefit most? Which task types will show value first? Which Copilot features will have the highest adoption? Write them down. Test them explicitly at the wave end.
2
Set up your active usage day tracking in the M365 Admin Centre this week. Export the baseline. Set a weekly calendar reminder to review it. Active usage days is the single metric that tells you most about whether behaviour has genuinely changed.
3
Create the AI win / AI fail log in Viva Engage or Teams. Announce it to champions with a specific ask: post one win and one fail per week. The first three posts should come from you — model the honesty and specificity you want from others.
4
Schedule the first monthly AI retrospective with each Wave 1 team. Thirty minutes. Use the four-question template. Publish the outputs to the champions community. Ask one other team to do the same and share what they find.
5
Identify one task for a micro-sprint in each Wave 1 business area. The task should be a Zone 2 or Zone 3 task that teams find time-consuming and where the current quality is inconsistent. Brief the relevant champion on the six-week sprint framework. Start next Monday.

Citations

[1] Dell'Acqua, F., McFowland, E., Mollick, E.R., et al. "Navigating the Jagged Technological Frontier." Harvard Business School Working Paper 24-013, 2023.

[2] Brynjolfsson, E., Li, D., and Raymond, L.R. "Generative AI at Work." NBER Working Paper 31161, 2023.

[3] Mollick, E. Co-Intelligence: Living and Working with AI. Portfolio/Penguin, 2024.

[4] Microsoft Viva Insights: microsoft.com/en-us/microsoft-viva/insights

← Issue 03: Adopt Issue 05: Scale →