Probability

Logic separates worlds that satisfy a statement from worlds that do not. Real agents often need a graded belief because sensors are noisy, evidence is incomplete, outcomes are random, and environments change.

Probability provides a coherent calculus for belief under uncertainty. It does not eliminate uncertainty; it constrains how beliefs should combine and change when evidence arrives.

Uncertainty Sources

Uncertainty enters an AI system in different ways:

• partial observability: the relevant state cannot be seen directly;
• sensor noise: an observation may be inaccurate;
• stochastic effects: the same action can have different outcomes;
• model uncertainty: the agent does not know the correct parameters or structure;
• data uncertainty: measurements are missing, delayed, or biased;
• strategic uncertainty: other agents may change their behavior.

These causes should not be collapsed into one unexplained confidence score. A probability of failure estimated from repeated trials means something different from uncertainty caused by an unfamiliar operating environment. The numerical calculus may be shared, but validation and mitigation differ.

Possible Worlds

A possible world is one complete way the relevant variables could be. With two Boolean variables, Rain and Traffic, there are four worlds:

A probability model assigns each world a non-negative number P(wᵢ) and requires:

Σᵢ P(wᵢ) = 1

The probability of a statement is the sum of probabilities of worlds where it is true. For example:

P(Rain) = P(w₁) + P(w₂)
P(Heavy) = P(w₁) + P(w₃)
P(Rain ∧ Heavy) = P(w₁)

This semantics explains probability rules rather than treating them as disconnected formulas. It also exposes the difficulty of full joint models: n Boolean variables create 2ⁿ possible worlds. Graphical models later reduce this burden by representing conditional structure.

Random Variables

A random variable represents an uncertain quantity. It may be:

• Boolean: Rain ∈ {true, false};
• categorical: Weather ∈ {sunny, cloudy, rainy};
• discrete numeric: number of failed servers;
• continuous: temperature or travel time.

A probability distribution assigns non-negative probabilities summing or integrating to one.

For a Boolean variable:

P(Rain) + P(¬Rain) = 1

Probability is not the same as fuzzy membership. P(Rain)=0.7 expresses uncertainty about whether rain occurs. Membership μWarm(28°C)=0.7 expresses the degree to which a known temperature fits the concept “warm.”

Probability Forms

A joint distribution assigns probabilities to combinations:

P(Weather, Traffic)

A marginal sums out variables:

P(Traffic=heavy) = Σ_weather P(weather, heavy)

Conditional probability updates the sample space after evidence:

P(A | B) = P(A ∩ B) / P(B), when P(B) > 0.

The product rule follows:

P(A ∩ B) = P(A | B)P(B)

and symmetrically:

P(A ∩ B) = P(B | A)P(A)

Probability Rules

Several rules follow directly from the possible-world interpretation.

Complement:

P(¬A) = 1 - P(A)

Addition:

P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

The intersection is subtracted because it was counted once with A and once with B. If the events are mutually exclusive, the intersection is zero.

Marginalization:

P(A) = P(A, B) + P(A, ¬B)

This partitions all worlds where A is true according to whether B is true.

Total probability:

P(A) = P(A | B)P(B) + P(A | ¬B)P(¬B)

More generally, if B₁, ..., Bₖ form mutually exclusive and exhaustive cases:

P(A) = Σᵢ P(A | Bᵢ)P(Bᵢ)

Total probability connects conditional models to unconditional predictions. For example, a system can predict the overall failure rate by weighting separate failure rates for new and old machines by how common each machine group is.

Joint Example

Consider 100 recorded days:

From the table:

P(Rain, Heavy) = 24/100
P(Heavy) = 45/100
P(Rain | Heavy) = 24/45
P(Heavy | Rain) = 24/30

The last two quantities differ because their reference populations differ. The table makes the denominator visible and prevents conditional reversal.

These frequencies estimate probabilities only if the recorded days represent the deployment setting. A roadwork period or seasonal sample can change the relationship. Probability calculation may be exact while probability estimation remains biased.

Bayes’ Rule

Equating the two product rules gives:

P(A | B) = P(B | A)P(A) / P(B)

Interpretation:

• P(A) is the prior belief;
• P(B | A) is the likelihood of evidence under the hypothesis;
• P(B) normalizes across possible causes;
• P(A | B) is the posterior belief.

Diagnostic Example

Suppose:

• 1% of devices have a fault;
• a test is positive for 95% of faulty devices;
• it is also positive for 5% of healthy devices.

For 10,000 devices:

• faulty: 100, with about 95 positive;
• healthy: 9,900, with about 495 positive.

Among 590 positive tests, only 95 indicate a true fault:

P(Fault | Positive) = 95 / 590 ≈ 0.161

Despite a sensitive test, a positive result implies only about 16% fault probability because faults are rare. Ignoring the prior is the base-rate fallacy.

The same calculation can be implemented without concealing the denominator:

def binary_bayes(
    prior,
    likelihood_if_true,
    likelihood_if_false,
):
    values = (prior, likelihood_if_true, likelihood_if_false)
    if any(not 0 <= value <= 1 for value in values):
        raise ValueError("probabilities must be between zero and one")

    true_weight = likelihood_if_true * prior
    false_weight = likelihood_if_false * (1 - prior)
    evidence_probability = true_weight + false_weight
    if evidence_probability == 0:
        raise ValueError("evidence has zero probability")
    return true_weight / evidence_probability

fault_after_positive = binary_bayes(
    prior=0.01,
    likelihood_if_true=0.95,
    likelihood_if_false=0.05,
)
assert round(fault_after_positive, 3) == 0.161

The two unnormalized weights correspond to the true-positive and false-positive branches of a frequency tree. Normalization divides each branch by their total. Keeping these intermediate quantities visible makes a reversed conditional easier to detect.

Odds Form

Bayes’ rule can also be written as:

posterior odds = likelihood ratio × prior odds

The likelihood ratio compares how expected the evidence is under two hypotheses:

LR = P(e | H) / P(e | ¬H)

An LR greater than one supports H; an LR below one supports ¬H; an LR near one carries little diagnostic information. This form makes sequential evidence intuitive: each sufficiently independent observation multiplies the current odds.

Evidence cannot be multiplied repeatedly when observations share the same cause. Treating correlated signals as independent counts the same information more than once and produces overconfident posteriors.

Independence

A and B are independent when:

P(A, B) = P(A)P(B)

Conditional independence is more common:

A ⟂ B | C

means that after C is known, learning B adds no information about A.

Two symptoms may be correlated because they share a disease. Conditioned on the disease, they may become independent. This structure allows a compact model instead of an enormous joint table.

Independence is an assumption to justify or test, not a default convenience.

Decisions

The expected value of random variable X is:

E[X] = Σ_x xP(X=x)

Expected utility evaluates actions:

EU(a | e) = Σ_s P(s | a,e)U(s)

Choose an action maximizing expected utility, subject to constraints.

An expected value can hide risk. Two actions may share the same mean while one has extreme outcomes. Utility can represent risk attitude, and safety constraints can forbid unacceptable outcomes regardless of average benefit.

The value of information asks whether an additional observation could improve the decision enough to justify its cost. A diagnostic test is useful not merely when it is accurate, but when its result can change which action has the highest expected utility. Information that never changes the decision has no decision value in that context.

def expected_utility(
    outcomes,
    utilities,
):
    if outcomes.keys() != utilities.keys():
        raise ValueError("outcomes and utilities must match")
    if abs(sum(outcomes.values()) - 1.0) > 1e-9:
        raise ValueError("outcome probabilities must sum to one")
    return sum(
        probability * utilities[outcome]
        for outcome, probability in outcomes.items()
    )

inspect = expected_utility(
    {"fault": 0.16, "healthy": 0.84},
    {"fault": -5, "healthy": -5},
)
continue_running = expected_utility(
    {"fault": 0.16, "healthy": 0.84},
    {"fault": -100, "healthy": 2},
)
assert inspect > continue_running

The numerical utilities encode consequences, not probabilities. Changing them can change the rational action while beliefs remain fixed. This separation prevents a common modelling error in which “unlikely” is treated as equivalent to “unimportant.”

Calibration

A probabilistic forecast is calibrated when events assigned probability p occur about a fraction p of the time over comparable cases. Among many predictions near 0.8, roughly 80% should be correct.

Calibration is different from accuracy. A classifier can make many correct hard decisions while its confidence values are unreliable. Reliability diagrams group predictions into probability bins and compare predicted confidence with observed frequency. Proper scoring rules such as log loss or the Brier score reward both useful discrimination and honest uncertainty.

Evaluate calibration on data matching deployment conditions. Dataset shift can invalidate a previously calibrated model, especially when base rates change.

Uncertainty Types

Aleatoric uncertainty describes variation treated as inherent in the process, such as a noisy physical outcome. Epistemic uncertainty reflects limited knowledge of the model or its parameters. More data may reduce epistemic uncertainty but cannot necessarily remove aleatoric variation.

The distinction is model-dependent. A die roll may be aleatoric for an agent that cannot observe the throw, yet largely predictable for a sufficiently detailed physical model. State what information the agent has before classifying uncertainty.

Probabilities may be estimated from frequencies, elicited from experts, or produced by a learned model. Each origin requires different evidence. Historical frequency assumes relevant stability; expert judgement needs calibration and aggregation; model scores need out-of-sample validation.

Applications

• diagnosis;
• risk assessment;
• sensor fusion;
• forecasting;
• decision support;
• anomaly scoring;
• uncertain classification.

Common Mistakes

• Reversing P(A|B) and P(B|A).
• Ignoring base rates.
• Treating correlation as causation.
• Assuming independence for convenience.
• Counting correlated evidence more than once.
• Using invalid probability tables.
• Confusing uncertainty with fuzzy membership.
• Reporting expected value without risk.
• Treating confidence as calibrated without testing it.

Exercises

1. Build a joint distribution for two Boolean variables and compute both marginals.
2. Solve a diagnostic problem with Bayes’ rule and a frequency table.
3. Give variables that are dependent but conditionally independent.
4. Compare two actions with equal expected value but different risk.
5. Normalize an unscaled probability vector and test invalid inputs.
6. Explain why a positive result from an accurate test may still imply a low posterior.
7. Compute posterior odds from a prior and two independent likelihood ratios.
8. Design a reliability diagram for a set of probabilistic forecasts.
9. Give an example where a highly accurate test has no value for a decision.
10. Separate aleatoric and epistemic uncertainty in a sensor application.
11. Change the utilities in the maintenance example and find the decision threshold.