This free curriculum covers every domain assessed in the Oxford Certified AI Product Manager examination: deciding when to use AI, framing ML problems, data, evaluation, generative AI, user experience, responsible AI, experimentation, MLOps, build-versus-buy and stakeholder management. Read each chapter, note the key points, and then sit the 75-question examination.
AI should solve a real user problem better than simpler alternatives. Machine learning fits problems where patterns are complex, change over time or are hard to express as rules — fraud detection, recommendations, image recognition, forecasting. Problems with simple, stable rules (such as calculating tax) are better solved with conventional software.
Before investing, confirm the user problem, the value AI adds, data availability and the cost of errors. Define clear acceptance criteria: the performance level and constraints (accuracy, latency, cost, fairness) needed for the feature to be useful and launchable.
Start with the user problem, not the technology
Use ML for complex, changing patterns; use rules for simple, stable logic
Check data availability and the cost of wrong predictions
Agree acceptance criteria before development
Key takeaway: AI is a means to solve a problem — never the goal itself.
Chapter 2
Framing problems as machine-learning tasks
Supervised learning learns from labelled examples. Classification predicts categories (will this customer churn — yes or no?); regression predicts numbers (next week's order volume). Unsupervised learning finds structure without labels, such as clustering customers into segments. Reinforcement learning learns through rewards from actions.
A well-framed ML problem answers: what decision or action will the prediction support, how will success be measured for users and the business, what data exists and is it labelled, and what happens when the model is wrong? A "minimum viable model" — simple but good enough — gets real user feedback early.
Classification = categories; regression = numbers; clustering = groups without labels
Supervised learning needs labelled outcomes
Link every prediction to a decision or action
Start with a minimum viable model and iterate
Key takeaway: A precise problem definition prevents months of building the wrong model.
Chapter 3
Data strategy and quality
Model quality depends on data quality: "garbage in, garbage out". Training data must represent the real users and situations the product will serve; unrepresentative data leads to poor or unfair performance for some groups. Labelling quality improves with clear guidelines, multiple annotators and agreement checks.
Data is split into training, validation and test sets. The test set is never used during training, so it estimates performance on unseen data. Overfitting — memorising training data, including noise — produces excellent training scores but poor real-world results. Synthetic data can help when real data is scarce or sensitive, but its realism must be validated, and it should not replace evaluation on real data.
Representative data prevents poor and unfair results
Invest in labelling guidelines and quality checks
Keep a held-out test set
Watch for overfitting
Validate synthetic data carefully
Key takeaway: Your data strategy is your model strategy.
Chapter 4
Evaluating models
Accuracy is misleading on imbalanced data: a model that predicts "not fraud" for every transaction can be 99.5% accurate and useless. A confusion matrix counts true positives, false positives, true negatives and false negatives. Precision asks: of the items flagged, how many were correct? Recall asks: of all actual positives, how many were found?
The right balance depends on business costs. Medical screening with specialist review prioritises recall to avoid missed cases; a spam filter that hides important emails has a false-positive problem. Adjusting the decision threshold trades precision against recall. Offline metrics do not always predict production impact, so online experiments are still needed. Communicate performance to executives in business terms: losses prevented, time saved, risks and limitations.
Never rely on accuracy alone for imbalanced problems
Precision = correctness of positives; recall = coverage of positives
Choose thresholds based on the cost of each error type
Validate offline results with online experiments
Translate metrics into business impact
Key takeaway: The best metric is the one that reflects the real cost of being wrong.
Chapter 5
Generative AI and large language models
Large language models generate text by predicting the next token based on patterns learned in training. They can produce fluent but false statements — hallucinations. The context window limits how much text the model can consider at once.
Retrieval-augmented generation (RAG) retrieves relevant documents and provides them as context, grounding answers in company knowledge and enabling citations. Prompt engineering shapes behaviour; fine-tuning helps when a consistent style, format or specialised behaviour cannot be achieved with prompts. Evaluation requires a realistic test set with clear criteria, combining automated checks and human review, plus red-teaming to find harmful or incorrect outputs.
Security matters: prompt injection hides instructions inside content (emails, web pages) to hijack an assistant. Treat external content as untrusted, limit the assistant's permissions and require confirmation for sensitive actions.
LLMs predict tokens; they can hallucinate
RAG grounds answers in your documents and enables citations
Prompting first; fine-tune for consistent specialised behaviour
Evaluate with realistic test sets and red-teaming
Defend against prompt injection with least privilege and confirmations
Key takeaway: Generative AI is powerful but probabilistic — design for verification.
Chapter 6
Designing the AI user experience
AI outputs are probabilistic, so the experience must set clear expectations, show uncertainty where useful and keep users in control: let them edit, undo or reject outputs, and present generated content as a draft. Collect feedback — both implicit (clicks, purchases, skips) and explicit (ratings) — to improve models.
Human-in-the-loop designs route uncertain or high-impact cases to people while automating clear ones. When the assistant cannot answer reliably, it should decline gracefully or hand off to a human. Watch for automation bias — users accepting suggestions without checking — and design reviews and friction where errors are costly. Users should know when they are interacting with AI.
Set expectations and disclose AI involvement
Keep users in control: edit, undo, reject
Route low-confidence or high-impact cases to humans
Collect structured feedback
Counter automation bias in high-stakes decisions
Key takeaway: Good AI UX builds the right amount of trust — not blind trust.
Chapter 7
Responsible AI, privacy and regulation
Bias can come from unrepresentative data, biased labels or proxies for protected characteristics. Responsible teams test performance across user groups, document intended use and limitations in model cards, provide explanations for important decisions and offer a way to contest them.
Privacy by design minimises personal data, removes or pseudonymises unnecessary identifiers before training and ensures a lawful basis. The EU AI Act takes a risk-based approach: some practices, such as social scoring by public authorities leading to unjustified detrimental treatment, are prohibited; high-risk uses such as recruitment or credit scoring face strict requirements on data, documentation, transparency and human oversight; chatbots carry transparency obligations. Marketing must be accurate — never claim an AI "never makes mistakes".
Test across groups and document limitations
Explainability supports trust and accountability
Minimise and protect personal data
Know the EU AI Act risk categories
Keep marketing claims honest
Key takeaway: Responsible AI is a product requirement, not an afterthought.
Chapter 8
Experimentation and product metrics
A/B tests measure the causal impact of an AI feature against a control group. Define a primary metric that reflects user value (for example task completion time or suggestion acceptance) and guardrail metrics that must not worsen (complaints, latency, error rates). Run experiments for a planned duration with sufficient sample size rather than stopping at the first positive day.
Business metrics should reflect real value: for fraud detection, losses prevented balanced against false alarms and customer friction; for assistants, time saved and successful task completion.
Primary metric = user value; guardrails = things that must not get worse
Plan duration and sample size in advance
Avoid vanity metrics such as tokens generated
Measure business value and costs together
Key takeaway: Experiments turn opinions about AI value into evidence.
Chapter 9
MLOps: running models in production
Models degrade. Data drift changes the input distribution; concept drift changes the relationship between inputs and outcomes. MLOps practices version data, code and models; automate training and deployment pipelines; and monitor performance and data quality with alerts.
Shadow deployments run a new model alongside the current one without affecting users, allowing safe comparison. When performance drops — for example after a major change in customer behaviour — investigate drift, retrain or recalibrate with recent data and strengthen monitoring. Product managers remain responsible for AI performance after launch.
Version everything: data, code, models
Monitor quality and drift continuously
Automate retraining and deployment
Use shadow deployments to compare safely
Ownership continues after launch
Key takeaway: Launching a model is the beginning of its life, not the end of the project.
Chapter 10
Build, buy, cost and latency
Third-party foundation model APIs offer fast time to market and low upfront cost; building proprietary models makes sense when unique data and the capability are core differentiators. Reduce vendor lock-in by abstracting model calls behind an internal interface and keeping evaluation sets to compare providers.
Inference cost and latency shape the product. Route simple requests to smaller models, cache repeated answers, shorten prompts and stream responses to reduce waiting. Contracts must state how the provider uses, stores and retains your data, including whether it is used for training.
Buy for speed; build for differentiation
Abstract providers and keep evaluation sets
Manage cost with model routing and caching
Reduce latency with streaming and smaller models
Check data-usage terms in every AI contract
Key takeaway: Cost, latency and data terms are product decisions, not just engineering details.
Chapter 11
Leading AI products and stakeholders
AI product managers coordinate data scientists, ML engineers, designers, user researchers, legal, privacy and compliance teams and business leaders. Early roadmaps emphasise learning milestones that validate value, feasibility and data quality before scaling.
Communicate in business language, be explicit about limitations and safeguards, and align on acceptance criteria and success metrics before development. Honest communication — internally and externally — protects users and the company's reputation.
Coordinate technical, design, legal and business stakeholders
Roadmap learning milestones before scaling
Speak in business outcomes and be clear about limits
Align on criteria and metrics early
Key takeaway: Trust in AI products is built through clear, honest communication.
Ready? Take the free exam
Take the exam and earn your digital badge free of charge. After passing, you may order the official certificate.