Reasoning Under Uncertainty
Probability as the logic of an uncertain world.
A medical test catches 90% of cases of a disease that 1 in 100 people have, wrongly flags 9% of healthy people, and comes back positive for you. How worried should you be? Intuition is badly off here, and the fix, Bayes' rule, is the foundation of every machine that reasons when it cannot be sure.
What you’ll learn
- Explain why AI needs probability, not just logic, to act in an uncertain world.
- Update a belief with Bayes' rule, and recognize the base-rate fallacy.
- Read a Bayesian network and describe how evidence flows through it, including explaining away.
- Describe filtering: tracking a hidden state over time with a predict–update loop.
- Explain expected utility and the trade-off between exploring and exploiting.
Why logic is not enough
Logic deals in certainties: a sentence is entailed or it is not. The world rarely cooperates. "A toothache means a cavity" is false, because it might be gum disease, and patching the rule with every exception ("…or gum disease, or an abscess, or…") never ends.
Probability summarizes what you do not know. Instead of a yes-or-no rule you hold a degree of belief, , read "the probability of a cavity given a toothache". This conditional probability is the basic currency of reasoning under uncertainty.
Think first A forecaster says "70% chance of rain", and the day stays dry. Was the forecast wrong?
Not necessarily. One day cannot confirm or refute a probability. What you can check is calibration: across all the days she says 70%, it should rain on about 70% of them.
Bayes' rule and the base rate
Back to the question on the cover. A disease affects 1% of people. A test detects 90% of cases, but it also flags 9% of healthy people.
You test positive. What is the probability that you have the disease?
Show the answer
9 %
Picture 1,000 people. 10 have the disease, and 9 of them test positive. Of the 990 healthy people, 9% (about 89) test positive too. So only 9 of roughly 98 positives are ill: about 9%. The disease's low starting probability, the base rate, dominates. Ignoring it is the base-rate fallacy.
Bayes' rule does this bookkeeping in general. Your prior belief is reweighted by the likelihood , how well the hypothesis predicts the evidence, to give the posterior:
Networks of causes
Real problems involve many variables. A full probability table over true/false variables needs numbers, hopeless beyond a few dozen. A Bayesian network records only direct influences instead: an arrow from each cause to its effect, and at every node a small conditional probability table given its parents.
In the classic alarm network, a burglary or an earthquake can set off your alarm, and two neighbors, John and Mary, may call you when it rings. The full table for these five variables needs numbers; the network needs 10.
Burglaries and earthquakes are independent: learning about one tells you nothing about the other. Once you know the alarm is ringing, are they still independent?
- Yes: independent causes stay independent
- No: given the alarm, evidence for one cause lowers the probability of the other
- No: given the alarm, evidence for one cause raises the probability of the other
- It depends on whether John calls
Show the answer
B. No: given the alarm, evidence for one cause lowers the probability of the other
Observing a common effect makes its causes compete to explain it. An earthquake accounts for the alarm, so a burglary becomes less likely: in the network, about 37% fell to about 0.3%.
Beliefs that move through time
Many hidden quantities change over time: a robot's position, a patient's condition, the word being spoken. A hidden Markov model assumes the hidden state evolves step by step, each step depending only on the one before (the Markov property), and that every step produces a noisy observation.
Filtering maintains a belief about the hidden state with a two-beat loop. Predict: push the belief through the motion model, which blurs it. Update: weigh each state by how well it explains the new observation, which sharpens it.
Deciding: utility and exploration
Beliefs are only half the story: an agent must also act. Decision theory gives each outcome a utility and picks the action with the highest expected utility, the probability-weighted average over outcomes.
When outcomes unfold over many steps, this becomes a Markov decision process. When the probabilities themselves are unknown, the agent faces the exploration–exploitation dilemma. A multi-armed bandit is its purest form: slot machines with hidden payout rates. Do you exploit the best machine so far, or explore to learn more?
After 3 pulls each, arm A has paid out twice and arm B once. A purely greedy agent will now…
- keep pulling A while its average stays ahead, so it may never learn that B is actually better
- alternate between A and B to be fair
- switch to B, since B is due a win
- stop playing, since it cannot be sure which arm is better
Show the answer
A. keep pulling A while its average stays ahead, so it may never learn that B is actually better
Greedy always picks the best average so far. B's estimate of 1/3 never changes, because B is never pulled again. Exploration strategies such as ε-greedy and UCB1 fix this by trying uncertain arms on purpose.
Where to go next
Probability gives AI one common language: beliefs are distributions, evidence is conditioning, and choices maximize expected utility. The same few ideas power spam filters, speech recognizers, robot localization and medical decision support.
| Hidden Layer lesson | What you will do there |
|---|---|
| Bayesian Reasoning | Update beliefs with priors and likelihoods, use conjugate priors, and see why correlation is not causation. |
| Bayesian Networks | Read independence off a graph with d-separation, and compute posteriors exactly and by sampling. |
| Markov Chains, HMMs & Filters | Derive the forward, forward–backward and Viterbi algorithms, and track continuous states with Kalman and particle filters. |
| Decisions: Utility, MDPs & Bandits | Solve MDPs by value and policy iteration, plan when the state is hidden, and make exploration precise with regret bounds. |
Key takeaways
- Probability lets an agent hold degrees of belief when rules have exceptions and information is incomplete.
- Bayes' rule turns a prior into a posterior, . Ignoring the prior, the base rate, leads to badly wrong conclusions.
- Bayesian networks represent large joint distributions compactly through conditional independence. Evidence flows in every direction, and causes can explain each other away.
- Filtering tracks a hidden state over time by alternating predict (blur) and update (sharpen), as in HMMs, Kalman filters and particle filters.
- Rational choices maximize expected utility. When the odds are unknown, agents must balance exploring and exploiting.
Check yourself
In the interactive lesson these are scored with your confidence, and what you miss comes back for review.
A condition affects 1% of people. A test detects 99% of cases and wrongly flags 1% of healthy people. Someone tests positive. About how likely is it that they have the condition?
- 99%
- About 50%
- About 9%
- 1%
Show the answer
B. About 50%
Out of 10,000 people, 100 have the condition and 99 of them test positive. Of the 9,900 healthy people, 1% (99) test positive too. Half the positives are ill: .
Your burglar alarm is ringing. Then the radio reports an earthquake nearby. What happens to the probability of a burglary?
- It rises: two pieces of evidence point to trouble
- It stays the same: earthquakes and burglaries are independent
- It falls: the earthquake already explains the alarm
- It drops to exactly zero
Show the answer
C. It falls: the earthquake already explains the alarm
Given the alarm, the two causes compete to explain it. The earthquake accounts for the alarm, so a burglary becomes much less likely. This is explaining away.
In HMM filtering, what does the predict step do to the belief?
- Spreads it out, because motion is uncertain
- Sharpens it, because time has passed
- Replaces it with the single most likely state
- Weighs it by the latest sensor reading
Show the answer
A. Spreads it out, because motion is uncertain
Predicting pushes the belief through an uncertain motion model, so probability leaks into neighboring states. Only the update step, using a new observation, can sharpen it.
Why does a Bayesian network need far fewer numbers than a full probability table?
- It rounds probabilities to fewer digits
- It stores only the most likely outcomes
- It assumes every variable is independent of every other
- Each variable depends directly on only a few parents, so each table is small
Show the answer
D. Each variable depends directly on only a few parents, so each table is small
The network writes the joint distribution as a product of small tables, one per variable given its parents. That works because each variable depends directly only on its parents: given them, it is conditionally independent of its other non-descendants.
Action A wins 10 points with probability 0.5 and nothing otherwise. Action B gives 4 points for sure. For an agent whose utility is simply its points, which statement is correct?
- B is better: a sure thing always beats a gamble
- A is better: its expected utility is 5, versus 4 for B
- They are equally good: each can deliver at least 4 points
- They cannot be compared until the outcome is known
Show the answer
B. A is better: its expected utility is 5, versus 4 for B
and . A rational agent with these utilities picks A, even though A sometimes yields nothing.
Go deeper
- Bayesian Reasoning · Priors, likelihoods and updating beliefs with evidence.
- Bayesian Networks · Compact joint distributions from conditional independence.
- Markov Chains, HMMs & Filters · Reasoning about processes that unfold in time.
- Decisions: Utility, MDPs & Bandits · Choosing well when outcomes are uncertain.
Further reading
- S. Russell & P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. (Pearson, 2020), chapters 12–14, 16 and 17.
- J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, 1988).