Part I Foundational AI/ML Concepts 1

Chapter 1 Artificial Intelligence and Machine Learning: What Every Product Manager Needs to Know 3

AI vs. ml 4

Why This Matters to a PM 4

Key Differences Between AI and ml 5

Common Misconceptions for PMs: Myths vs. Reality 7

Your Glossary as a PM 7

Grounding the Concepts: Real-World AI in Action 10

The AI PM’s Guiding Principles 14

Chapter Summary and Key Takeaways 16

Key Takeaways 16

Onward: Peeking Under the Hood 17

Chapter 2 How Machine Learning Models Learn: A Peek Under the Hood 19

The Learning Process: Training, Validation, and Testing 20

How Models Learn: An Example with k-Nearest Neighbors (k-NN) 22

Applying k-NN (with k=1): 23

Another Example: Testing an Unknown Fruit 26

Evaluating Model Performance 27

The Confusion Matrix: A Foundation for Understanding 27

Key Classification Metrics (and Their PM Implications) 28

The Precision-Recall Trade-Off 29

Choosing the Right Metric 30

Overfitting and Underfitting: Striking the Right Balance for Real-World Performance 31

Overfitting: Memorizing Instead of Learning 31

Underfitting: Missing the Forest for the Trees 32

Visual Analogy: Fitting a Curve 32

Finding the Sweet Spot: Generalization 33

The PM’s Role 33

Human-in-the-Loop: Blending AI Power with Human Expertise 34

What Is Human-in-the-Loop? 34

Why HITL Is Essential for Product Managers (and Their Products) 35

How to Implement HITL (PM Considerations) 37

Chapter Summary and Key Takeaways 38

Key Takeaways 39

Onward: Understanding the Broader Process 39

Chapter 3 The Big Picture: AI, ML, and You 41

Understanding the Relationship Between AI, ML, and Product Goals 41

Types of Machine Learning: Understanding the Spectrum of Learning 44

Supervised Learning: Guiding the Model with Labeled Examples 46

Technical Deep Dive: How Supervised Learning Models Learn from Labeled Data 48

Critical Considerations for Product Managers 54

Unsupervised Learning: Discovering Hidden Patterns in Your Data 55

Technical Deep Dive: How Unsupervised Learning Models Discover Patterns 57

Critical Considerations for Product Managers 60

Reinforcement Learning: Learning Through Trial and Error 61

Technical Deep Dive: How Reinforcement Learning Agents Learn Optimal Policies 63

The Learning Process: Exploration, Exploitation, and Q-Learning 65

Critical Considerations for Product Managers 67

Generative AI: Powering a New Era of Language-Based Applications 67

Technical Deep Dive: How LLMs Understand and Generate Language 69

Critical Considerations for Product Managers 72

The “Gotchas”: A PM’s Guide to LLM Limitations and Risks 73

Navigating the Nuances of Generative AI: Understanding GenAI Evaluations— Ensuring Quality and Trust 75

Prompt Engineering: The Art and Science of Talking to AI 84

Types of Machine Learning: A Recap 89

Introduction to Neural Networks and Deep Learning: The Engines of Complex Pattern Recognition 92

Neural Networks: Mimicking the Brain’s Connections (But Not Really) 92

How Neural Networks Learn: Adjusting the Connections 94

Technical Deep Dive: The Mechanics of Neural Networks and Deep Learning 95

Challenges in Deep Learning 98

Chapter Summary and Key Takeaways 99

Key Takeaways 99

Onward: Mapping the Process 100

Chapter 4 The AI Lifecycle 101

Problem Definition and Business Understanding: The “Why” 102

Data Collection and Exploration: Understanding Your Ingredients 103

Data Preprocessing: Preparing the Ingredients 104

Feature Engineering: Crafting the Inputs for Success 104

Model Selection and Training: Choosing the Right Algorithm 105

Model Evaluation and Tuning: Ensuring Quality 106

Model Deployment and Monitoring: Bringing AI to Life (and Keeping It Healthy) 107

Retraining and Maintenance: Keeping Your Model Up-to-Date 108

Chapter Summary and Key Takeaways 109

Key Takeaways 109

Onward: Exploring the AI PM Roles 110

Part II AI PM Specializations 111

Chapter 5 AI-Experiences PM: Shaping User Interaction with AI 113

Key Responsibilities: Shaping the AI User Experience 114

Day-to-Day Activities 117

Required Skills and Knowledge: The AI-Experiences PM Toolkit 120

Core Product Management Craft and Practices 120

Engineering Foundations for PMs 121

Essential Leadership and Collaboration Skills 122

AI Lifecycle and Operational Awareness 123

Illustrative Example: A Day in the Life of an AI-Experiences PM 124

Challenges and Complexities 127

How the AI-Experiences PM Interacts with Other Roles 129

Chapter Summary and Key Takeaways 134

Key Takeaways 134

Onward: Architecting the AI Foundation 135

Chapter 6 AI-Builder PM: Architecting the Foundation of Intelligent Systems 137

Key Responsibilities: Building and Managing the AI Foundation 138

Day-to-Day Activities 141

Required Skills and Knowledge: The AI-Builder PM’s Technical and Strategic Toolkit 144

Core Product Management Craft and Practices 145

Engineering Foundations for PMs 146

Essential Leadership and Collaboration Skills 147

AI Lifecycle and Operational Awareness 148

Illustrative Example: A Day in the Life of an AI-Builder PM 149

Challenges and Complexities 152

How the AI-Builder PM Interacts with Other Roles 154

Chapter Summary and Key Takeaways 156

Key Takeaways 157

Onward: Supercharging the PM Workflow 158

Chapter 7 AI-Enhanced PM: Supercharging Product Management with AI 159

Key Responsibilities: Augmenting PM Workflows and Decision-Making with AI 160

Day-to-Day Activities 162

Required Skills and Knowledge: The AI-Enhanced PM’s Toolkit 165

Core Product Management Craft and Practices 165

Engineering Foundations for PMs 166

Essential Leadership and Collaboration Skills 167

AI Lifecycle and Operational Awareness 168

Illustrative Example: A Day in the Life of an AI-Enhanced PM 169

Examples of AI Tools 172

Challenges and Complexities 173

How the AI-Enhanced PM Interacts with Other Roles 175

Skill Comparison: AI-Experiences PM, AI-Builder PM, and AI-Enhanced PM 177

Chapter Summary and Key Takeaways 184

Key Takeaways 185

Onward: From Theory to Action 185

Part III Connecting the Dots Between AI/ML Knowledge and PM Craft 187

Chapter 8 Identifying and Evaluating AI Opportunities 189

Uncovering Potential Use Cases—Mining Your Product for AI Gold 189

Recognizing Data-Rich Problem Areas 190

Analyzing Existing Data Sources 192

Asking the Right Questions 193

AI/ML Capability Matching: Connecting Problems to Solutions 194

Understanding Your AI/ML Toolkit: Key Capabilities 195

Matching Capabilities to Problems: A Practical Approach 200

Feature: Search Functionality in a Document Management System 200

Feature: Customer Support Chatbot 201

Feature: Reporting Dashboard for Marketing Campaigns 201

Finding AI Opportunities in the User Journey 202

Mapping the User Journey: Charting the Course 202

Identifying Pain Points and Opportunities: The AI Detective Work 204

Applying AI/ML to Enhance Touchpoints: The Transformation 205

Feature Enhancement Through AI/ML— Transforming Existing Functionality 208

Identifying Enhancement Opportunities: Finding the Weak Spots 209

Applying AI/ML to Enhance Features: The Transformation Process 210

Feature: Standard Search Functionality 212

Feature: Data Entry Form 212

Feature: Reporting Dashboard 212

Proactive Product Management—Anticipating User Needs with AI 213

Understanding the Power of Prediction and Automation 213

Key Areas for Predictive and Automation Opportunities 214

Identifying Opportunities: A Practical Approach 216

Responsible AI Foundations—Ethical and Feasibility Considerations 217

Ethical Considerations: The “Do No Harm” Principle 217

Feasibility Considerations: Can We Actually Build This? 220

Practical Ideation Techniques for AI/ML Use Cases—Thinking Like an AI-First Product Manager 221

Ideation Techniques: Unleashing Your AI Creativity 222

“AI Feature Storming”: The Brain Dump 222

“AI Scenario Planning”: Walking in the User’s Shoes 223

“Data Opportunity Mapping”: Leveraging Your Data Assets 223

“AI Capability Alignment”: The Matching Game 224

“AI-Powered Feature Reverse Engineering”: Learning from Others 225

Cultivating an AI-First Mindset 226

Chapter Summary and Key Takeaways 226

Key Takeaways 227

Onward: Measuring the Value of Your Ideas 227

Chapter 9 ROI Calculation for AI Projects: Measuring the Impact and Demonstrating Value 229

From Model Performance to Business Impact: A PM’s Guide to AI Metrics 229

Defining AI/ML-Specific Metrics: The Foundation for Measuring ROI 230

The Importance of Baselines: Knowing Where You Started 230

Understanding the Confusion Matrix: Decoding Classification Performance 231

Key Performance Metrics for AI/ML Models: Beyond the Confusion Matrix 233

Context Matters: Selecting the Right Metrics for Your AI/ML Application 237

1. Define Your Business Goals (and Connect Them to User Needs) 237

2. Consider the Type of AI/ML Application (and Its Inherent Trade-Offs) 238

3. Evaluate the Cost of Errors: The Risk Assessment 239

4. Translate Technical Metrics into Business Impact 240

Important Considerations 240

End-to-End Example—Predicting Churn in a Subscription Service 241

1. Identify the Business Goal: Defining the “Why” 241

2. Define the AI/ML Application and Solution 242

3. Identify Data Sources and Engineer Features: The Raw Materials 243

Available Data 243

Feature Engineering 243

The Product Manager’s Role in This Stage 244

4. Select the Metrics: Defining Success 245

The Cost of Errors: Prioritizing What Matters 245

Our Chosen Metrics 246

5. Establish Baseline Metrics: Setting the Starting Point 246

6. Conduct Model Training and Evaluation: Building and Testing the AI 247

7. Conduct A/B Testing: Measuring Real-World Impact 248

8. Calculate the Results and ROI: Quantifying the Value 248

Translating Results into Business Impact 249

Monitoring for Long-Term Success 249

9. Monitor and Maintain the Model for Long-Term Success 250

A/B Testing for AI and ML Projects: Validating Impact and Optimizing Performance 251

What Is A/B Testing (in a Nutshell)? 251

Why Is A/B Testing Especially Important for AI/ML? 252

How to Conduct A/B Testing for AI and ML: A Step-by-Step Guide 253

Key Considerations for AI/ML A/B Testing 258

Chapter Summary and Key Takeaways 259

Key Takeaways 259

Onward: From the Lab to a Live Product 260

Chapter 10 Building and Deploying AI Solutions: From Lab to Live 261

MLOps: The Key to Reliable and Scalable AI 261

Key Components of MLOps—The AI Production Line 264

CI/CD, IaC, and Collaboration: The Foundational Pillars of MLOps 271

Glossary of Key MLOps Terms 272

MLOps End-to-End Example: Churn Prediction in a Subscription Service (Product Manager’s Perspective) 274

Chapter Summary and Key Takeaways 278

Key Takeaways 278

Onward: Building with Integrity 279

Chapter 11 Responsible AI and Ethical Considerations: Building AI with Integrity 281

Understanding AI Bias and Fairness: The Foundation of Responsible AI 281

Identifying Potential Biases: Where Bias Can Creep In 282

Mitigating Potential Biases: A Proactive Approach 285

Protected Classes and AI Fairness— Designing for Inclusion 287

What are Protected Classes? 287

Why Focus on Protected Classes? (The Legal and Ethical Imperative) 287

How Protected Classes Relate to AI Bias: The Mechanisms of Discrimination 288

Mitigating Bias Related to Protected Classes: Actionable Steps for PMs 289

AI Ethics and Legal Compliance—From Principles to Practice 291

Understanding the Ethical Landscape: Core Principles 291

Understanding the Legal Landscape: Key Regulations 292

Actionable Steps for Product Managers: Building Ethically and Legally Compliant AI 293

Engaging with the Community and External Stakeholders 297

Chapter Summary and Key Takeaways 298

Key Takeaways 298

Onward: Paving Your Path 299

Chapter 12 Conclusion: Paving Your Own Path to AI PM 301

Embrace Lifelong Learning: Stay Curious and Iterative 302

Cultivate a User-Centric AI Mindset 303

Deepen Cross-Functional Collaboration Skills 303

Build a Distinct AI Portfolio (Show, Don’t Just Tell) 304

Develop a Personal Vision for Your AI Career 305

Keep Resilience and Adaptability at the Core 305

Final Thoughts 306

Index 307