Probabilistic Decision Support System: Bayesian Networks
Bayesian networks (also called belief networks, Bayesian belief networks, causal probabilistic networks, or causal networks) (Pearl 1988) are directed acyclic graphs in which nodes represent random variables and arcs represent direct probabilistic dependencies among them. The structure of a Bayesian network is a graphical, qualitative illustration of the interactions among the set of variables that it models. The structure of the directed graph can mimic the causal structure of the modeled domain, although this is not necessary. When the structure is causal, it gives a useful, modular insight into the interactions among the variables and allows for prediction of effects of external manipulation.
Nodes of a Bayesian network are usually drawn as circles or ovals. The following simple Bayesian network represents two variables, Success and Forecast, and expresses the fact that they are directly dependent on each other.
A simple Bayesian network
A Bayesian network also represents the quantitative relationships among the modeled variables. Numerically, it represents the joint probability distribution among them. This distribution is described efficiently, exploring probabilistic independencies among the modeled variables. Each node is described by a probability distribution conditional on its direct predecessors. Nodes with no predecessors are described by prior probability distributions. For example, node Success in the example network above will be described by the prior probability distribution over its two outcomes: Success and Failure.
Node Forecast will be described by a probability distribution over its outcomes (Good, Moderate, Poor) conditional on the outcomes of its predecessor (node Success, outcomes Success and Failure).
Both the structure and the numerical parameters of a Bayesian network can be elicited from an expert. They can also be learned from data, as the structure of a Bayesian network is simply a representation of independencies in the data and the numbers are a representation of the joint probability distributions that can be inferred from the data. Finally, both the structure and the numerical probabilities can be a mixture of expert knowledge and measurements and objective frequency data.
The name Bayesian originates from the fact that the joint probability distribution represented by a Bayesian network is subjective (please recall that they are sometimes called belief networks; Bayesian approach is often used as a synonym for subjective view on probability) and this subjective probability distribution can be updated in light of new evidence using Bayes theorem.