This guide uses two types of learning material:
- Worked scenarios are hypothetical composites based on common AI
initiative patterns. They are prompts for testing assumptions, not reports of named organisations or observed outcomes.
- Evidence vignettes summarise a public source and draw a deliberately
narrow lesson from it. They do not prove that every organisation, task, or AI system will produce the same result.
Use the same questions to test the relevant assumptions in your own AI Value Canvas (AVC). These are focused examples, not completed ten-element Canvases.
Worked scenario: The e-commerce recommender that worked, slowly
Context and Horizon
A mid-size e-commerce business moves an AI-driven product recommender—a system that suggests products to shoppers—from Pilot toward Scale. The expected benefit is higher conversion and margin from recommendations that relevant shoppers actually use.
Canvas decisions and gaps
The scenario assumes the model produces a modest conversion improvement, but fewer eligible shoppers use the recommendations than forecast. Its modelled result is below the commercial target even though the model is technically functional.
- Customer / User Segments — identify the eligible shoppers, the
moments when recommendations are useful, and the reasons a shopper may ignore them.
- AI Opportunity Fit — compare the recommender with merchandising
rules, search improvements, and human curation before assuming a model is the best intervention.
- Value & Metrics of Success — measure adoption among eligible users,
incremental conversion and margin against a baseline, and repeat use; do not treat raw model accuracy as business value.
- Cost Drivers — include build, integration, evaluation, monitoring,
support, and continuing infrastructure costs in the economic case.
Assumption boundary and lesson
This scenario is hypothetical. Test base, downside, and upside adoption assumptions before committing to Scale. A useful downside question is: “If only half of eligible shoppers use the recommendations, does the incremental margin still justify the full economic cost?”
Reader next step
Record the adoption denominator, conversion baseline, expected margin, and run costs in your Canvas. Set a Pilot gate that requires both user adoption and incremental business value, not technical performance alone.
Worked scenario: The HR chatbot that lost trust
Context and Horizon
A human resources (HR) team launches a chatbot for questions about benefits, leave, and payroll, intending to reduce service workload. This hypothetical project is treated as ready for Scale before it has established that employees want the service or that its answers are dependable.
Canvas decisions and gaps
Employees prefer email because it supports follow-up questions and has a clear human owner. A small number of inconsistent or invented policy answers erode trust, while HR spends additional time correcting answers without eliminating the original email demand.
- Customer / User Segments — learn how employees seek help, when a
follow-up is needed, and when they expect a qualified person to own the answer.
- AI Opportunity Fit — compare the chatbot with better email routing,
searchable policy pages, and additional HR capacity.
- Data & Model Considerations — assign an owner for authoritative
policy sources, content freshness, retrieval quality, evaluation of leave and payroll questions, and escalation to HR.
- Risks & Guardrails — pilot with real users, show uncertainty rather
than inventing an answer, and route consequential questions to a qualified human before the employee acts.
- Value & Metrics of Success — measure resolved requests, correction
effort, employee confidence, and workload displacement—not only chatbot sessions.
Assumption boundary and lesson
This scenario is hypothetical. It represents both an adoption failure and a reliability failure: a chatbot interface alone does not create a useful service.
Reader next step
Before a Pilot, define the authoritative source for every high-impact answer, the escalation path, the evaluation set, and the method for measuring whether workload is reduced rather than transferred.
Evidence vignette: Air Canada's chatbot misinformation
Context and evidence boundary
In Moffatt v. Air Canada, 2024 BCCRT 149, a passenger relied on incorrect bereavement-fare information supplied by Air Canada's chatbot. The Civil Resolution Tribunal of British Columbia found Air Canada liable for negligent misrepresentation. This is a legal decision specific to its facts and jurisdiction; it is not a universal legal rule for every chatbot or organisation.
AVC editorial application
The following is AVC guidance, not an additional finding of the tribunal. For consequential chatbot answers, identify the authoritative source, test the system against it, show uncertainty rather than inventing an answer, and provide a prompt route to a qualified human and a remedy path when the system is wrong.
Reader next step
For each consequential chatbot answer in your Canvas, record the content owner, update process, evaluation method, escalation route, and who can correct or compensate an affected person.
Evidence vignette: Measure productivity before scaling
Context and evidence boundary
A 2025 randomised controlled trial by METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” (2025), an AI evaluation research organisation, studied 16 experienced open-source developers working in their own repositories. For the tools and setting studied, participants with access to early-2025 AI tools took 19% longer to complete tasks on average. This result is not a claim about all developers, tasks, or AI tools.
AVC editorial application
Perceived time savings are not evidence of productivity. Define the task, baseline or comparison group, quality standard, rework, review effort, and cost per completed task before claiming a gain.
Reader next step
Run an Experiment or Pilot in the actual workflow. Set an evidence gate that compares time, quality, rework, and total cost with a credible baseline before committing to Scale.
Apply the page to your Canvas
For the initiative in front of you, state which assumptions are hypothetical, which are observed, and which still need evidence. Use the next Horizon to test the uncertainties that would most change the decision to stop, adapt, or scale.