Uncertain Knowledge and Reasoning Methods

🔊 Press the button below to hear the intercepted transmission.

First Impressions

What are the first words that comes to mind when you heard the above audio?

GARV

The Qualification Problem

Why Strict Logic Fails

  • Strict Logic: IF bird THEN flies (100% True or False)
  • The Problem: What about penguins? Injured birds? Birds in casts?
  • The Challenge: Impossible to list every exception to every rule
  • The Consequence: AI cannot function on rigid rules alone
Real Example: A rules-based spam filter might say "IF has word 'free' THEN spam." But legitimate emails about free shipping get blocked. The rule fails.
IF Bird THEN Flies Penguin Injured Bird Toy Bird Exceptions Grow Infinitely
GARV

Degrees of Belief

Probability as the Solution

  • Prior Probability P(A): What the AI believes before evidence
  • Posterior Probability P(A|B): What the AI believes after observing evidence
  • The Shift: From binary (True/False) to continuous (0 to 1)
  • The Gain: AI can now handle uncertainty naturally
Example: Instead of "Bird flies: TRUE," we say "Probability this bird flies: 0.85." Much more realistic.
0 1 Probability Belief Peak Uncertainty
SUVARNA

The Full Joint Probability Distribution

The Master Lookup Table

  • Definition: A table with every possible combination of events and their probabilities
  • The Promise: Look up ANY scenario instantly
  • The Power: Contains all information about the system
  • The Problem: Practically impossible to build for real systems
Example: Imagine tracking weather for every location on Earth for every hour. One table per day would require MORE data than all computers combined could store.
Event A Event B P(A,B) T T 0.42 T F 0.18 F T 0.26 F F 0.14 Scales to 2^n entries...
SUVARNA

The Scaling Crisis

Why the JPD Breaks Down

  • The Formula: n boolean variables = 2n - 1 probabilities
  • The Reality: Just 30 variables = Over 1 BILLION rows
  • The Result: Memory exhausted. Computation impossible.
  • The Lesson: We need a smarter approach

JPD Approach

✗ Complete but unusable
✗ Exponential storage
✗ Slow computation
✗ Practical breakdown at n=20

Smarter Approach

✓ Approximate but fast
✓ Linear storage
✓ Real-time computation
✓ Scales to n=1000+

5 vars 31 10 vars 1K 20 vars 1M 30 vars 1B 💥 Exponential Growth = System Collapse
SUVARNA

The Shortcut: Conditional Independence

AI's Lifeline

  • Absolute Independence: P(A ∧ B) = P(A) · P(B) (Very rare)
  • Conditional Independence: Knowing C makes A and B independent
  • Example: Toothache and probe scratch are independent if we know there's a cavity
  • The Win: Reduces billions of calculations to hundreds
Real Example: Rain and your broken sprinkler aren't independent. BUT if you know whether your neighbor has water, they become independent. They don't affect each other anymore—they only affect water availability.
A B C A C Complex Network All Connected → Simplified Chain Key Variables Only
GARV

Bayes' Theorem

The Engine of AI Learning

P(A|B) = [P(B|A) · P(A)] / P(B)
  • P(A|B): Posterior probability (belief after evidence)
  • P(B|A): Likelihood (how likely is the evidence given A?)
  • P(A): Prior probability (belief before evidence)
  • P(B): Evidence (total probability of observing B)
Why This Matters: Bayes' Theorem is the mathematical foundation of learning. Every time an AI system sees new data, it updates beliefs using this formula. It's how self-driving cars, medical diagnosis systems, and spam filters improve.
GARV

How Bayes Actually Works

From Math to Meaning

Posterior = [Likelihood × Prior] / Evidence
  • Step 1: Start with what you believed (Prior)
  • Step 2: Observe new evidence
  • Step 3: Update your belief mathematically
  • Step 4: New belief becomes the prior for next observation
Real Application: Medical diagnosis. Prior: "1% of people have disease X." Evidence: "Patient tests positive." Likelihood: "Test is 95% accurate." Posterior: Update to maybe 16% based on ONE positive test.
Belief Evolution Prior + Evidence Posterior Initial Uncertainty After 1st Observation After Many Observations
KETKI

The Malware Paradox

Why Human Intuition Fails

🚨 SCENARIO: Your file was flagged by our security scanner.

• 1% of all files are malware
• Our scanner is 90% accurate
• False-positive rate is 10%

What is the probability your file is ACTUALLY malware?
  • Intuitive Guess: Most humans say 90%
  • Your Gut: "The scanner is 90% accurate, so 90%!"
  • The Trap: You're ignoring the base rate (1% is rare!)
  • Next: Watch Bayes demolish your intuition...
Why We Fail: Our brains are wired to focus on the test accuracy (90%) and forget that the disease/malware is rare (1%). We commit what psychologists call "base rate neglect."

The Trust Factor

Would you trust an AI with a 70% certainty score?

Think about the consequences of being wrong...

KETKI

The Bayes Calculation

Why Your Intuition Was Wrong

P(Malware|Flagged) = [0.90 × 0.01] / 0.108 = 8.3%
  • The Correct Answer: Only 8.3% (not 90%!)
  • Why: Even though the test is accurate, malware is rare (1%)
  • The Insight: Most flags are false positives
  • The Lesson: AI uses Bayes. Your gut uses cognitive bias.
Real-World Impact: This exact mistake is why medical tests can be scary. A positive test for a rare disease might only mean 5% chance you have it. Doctors know this. Most patients don't.
Out of 100,000 Files 1,000 Malware 99,000 Clean Files What Gets Flagged? 900 True Positives 9,900 False Positives 900 Real ÷ 10,800 Total = 8.3% Rare events + imperfect tests = counterintuitive probabilities
KETKI

The Contradiction Detector

Human Bias vs. AI Logic

Scenario: The "Star" Candidate

You are hiring. 95% of applicants are average, 5% are "Stars."
Your AI screening tool is 90% accurate.

The tool flags a candidate as a "STAR." What do you do?

HIRE IMMEDIATELY

"The AI says they are a star, and it's 90% accurate!"

REQUEST SECOND INTERVIEW

"90% is good, but let's be careful."

The Truth

Human Instinct

"They are probably a Star (90% confident)."

Bayesian Logic

Actually, there is only a 32% chance they are a Star.

Speaker Note: Explain the Base Rate Fallacy. Because Stars are so rare (5%), even a 90% accurate test produces many more "False Stars" than real ones. Human brains ignore the 5% base rate and only see the 90% accuracy. This is why we contradict logic.
SUVARNA

Bayesian Networks

Mapping Uncertainty Visually

  • What It Is: A directed graph showing cause-and-effect relationships
  • Nodes: Random variables (Cloudy, Rain, Sprinkler, WetGrass)
  • Edges: Direct causal influence
  • CPTs: Conditional Probability Tables at each node
  • The Power: Visualize AND quantify an entire system
Real Application: Netflix uses Bayesian Networks to model: user-watches-movie → affects-recommendation → affects-future-clicks. Each connection has probabilities.
Cloudy Sprinkler Rain WetGrass

Interactive: Network Structure Builder

Drag the concepts to map the system's causal structure.

☁️ Cloudy
🌧️ Rain
🚿 Sprinkler
🌱 Wet Grass

Click two nodes to connect them

Note: This emphasizes structure (how variables influence each other) before we even touch the numbers.

SUVARNA

Inside the Network: CPTs

Conditional Probability Tables

  • CPT Attached to: Every node in the Bayesian Network
  • What It Shows: How parent nodes influence child nodes
  • Example: If Cloudy=True, how likely is Rain?
  • The Quantification: Turns abstract relationships into hard numbers
Cloudy P(Rain=true)
True 0.90 (90% chance)
False 0.10 (10% chance)
Reading the Table: "If it's cloudy, there's an 80% chance of rain. If it's clear, only 10%." This captures the real-world relationship beautifully.
KETKI

Decision Theory

Probability Meets Action

Definition: Decision Theory = Probability Theory + Utility Theory

An AI doesn't just calculate probabilities. It calculates the VALUE of each outcome and CHOOSES the best action.
  • Probability: How likely is it to rain? (80%)
  • Utility: What's the value of being dry? (+10 happiness)
  • Disutility: What's the cost of carrying an umbrella? (-2 happiness)
  • Decision: Bring umbrella? YES. Expected happiness = 6.8
Business Example: Should a company launch a product? Probability of success: 70%. Revenue if success: $10M. Loss if failure: $2M. Expected value: 0.7×10M - 0.3×2M = $6.4M. LAUNCH.
KETKI

Maximum Expected Utility

The Principle of Rational Action

MEU = max { Σ P(outcome|action) × U(outcome) }
  • What It Means: Calculate payoff for every possible action
  • How It Works: Multiply probability × utility for each outcome
  • The Choice: Pick the action with the highest expected value
  • The Guarantee: This is THE most rational decision possible
Why Humans Don't Do This: Our brains can't quickly multiply probabilities and utilities. We use shortcuts (heuristics) that often fail. AI has no such limitation.
Bring Umbrella If Rain (80%) Utility = +8 If No Rain (20%) Utility = +2 Expected: 6.8 No Umbrella If Rain (80%) Utility = -5 If No Rain (20%) Utility = +5 Expected: -1.0 RATIONAL CHOICE: Bring Umbrella
GARV

Rationality in Action

The Black Hole Heuristic Game

The Scenario: An AI navigates a 3×3 grid. One hidden cell is the "Black Hole" (score = -100). The AI doesn't know where it is, but it can reason probabilistically.
  • The Challenge: Maximize score without knowing danger zones
  • The Strategy: Sacrifice low numbers in risky areas, keep high numbers safe
  • The Math: Every move is an MEU calculation
  • The Result: The AI wins because it reasons under uncertainty systematically
Human vs AI: Humans might say "I'll just avoid the risky area." AI says "Cell (1,1) has 1/9 chance of being the hole. If I go there, expected loss is -100/9 ≈ -11. So place my low-value token there." Much smarter.

Final Takeaway

After seeing the math and the logic, how do you feel about AI decision making now?

Common Questions

Because the real world has infinite exceptions. To account for every possibility (like a bird not flying because it has a wing in a cast), our rules would grow too large to manage. Probability allows us to capture all these exceptions in a single number.

It provides a mathematical way to update "prior" beliefs based on new evidence, resulting in a more accurate "posterior" probability. This iterative process is how AI systems refine their understanding as they see more data.

It simplifies complex probability problems by showing which variables actually affect each other. By using "conditional independence," we avoid calculating billions of unnecessary combinations, making AI reasoning possible on normal computers.

AI uses Decision Theory, specifically Maximum Expected Utility (MEU). It calculates the "payoff" for every possible action multiplied by the probability of success, then chooses the path that offers the highest mathematical value.

Key Takeaways

From Uncertainty to Rational Action

01

The Logic Gap

Strict "If-Then" logic fails in the real world because exceptions are infinite. AI needs a more flexible way to reason.

02

Probability Theory

By moving from binary (True/False) to degrees of belief (0-1), AI can represent and handle uncertainty naturally.

03

Bayesian Learning

Bayes' Theorem provides the engine for updating beliefs, allowing systems to learn and adapt as they see new data.

04

System Mapping

Bayesian Networks visualize complex relationships and use conditional independence to make computation efficient.

05

Rational Decisions

Decision Theory combines probability with utility (value), enabling AI to choose the most beneficial path forward.

The Big Picture

Every modern AI system—from the recommendations on your phone to autonomous vehicles—relies on these core principles to survive in an unpredictable world. By computing probabilities and maximizing expected utility, AI turns uncertainty into its greatest strength.