The Bayesian Approach

The essence of the debate between the Frequentists (A statistical approach for assessing the likelihood that a hypothesis is correct, by assessing the strength of the data that supports the hypothesis and the number of hypotheses that are tested. ) and the Bayesian rests on the acceptability of the subjectivist notion of probability. Once one is willing to view probability in this way, the advantages of the Bayesian approach are compelling. But most practitioners, even though they have no strong aversion to the subjectivist notion of probability, do not choose to adopt the Bayesian Approach. The reasons are practical in nature.
1. Formalizing prior beliefs into a prior distribution is not an easy task;
2. The mechanics of finding the posterior distribution are formidable (feel fear and/or respect for something, because they are impressive or powerful, or because they seem very difficult).
3. Convincing others of the validity of Bayesian results is difficult because they view those results as being "contaminated" by personal beliefs.


Following the subjective notion of probability, it is easy to imagine that before looking at the data the researcher could have a "prior" density function for β, reflecting the odds that he or she would give, before looking at the data, if asked to take bets on the true value of β. This prior distribution, when combined with the data via Bayes' Theorem, produces the posterior distribution referred to above. This posterior density function is in essence a Weighted Average of the prior density and the likelihood (or "conditional density", conditional on the data).

Generally, the Bayesian Approach consists of three steps:

1. A prior distribution is formalized, reflecting the researcher's beliefs about the parameters in question before looking at the data.

2. This prior is combined with the data, via Bayes' theorem, to produce the posterior distribution, the main output of a Bayesian analysis.

3. This posterior is combined with a loss or utility function to allow a decision to be made on the basis of minimizing expected loss or maximizing expected utility, this third step is optional.

The Bayesian approach claims several advantages over the classical approach:

1. The Bayesian approach is concerned with how information in data modifies a researcher's beliefs about parameter values and allows computation of probabilities associated with alternative hypotheses or models; this corresponds directly to the approach to these problems taken by most researchers.

2. Extraneous information is routinely incorporated in a consistent fashion in the Bayesian method through the formulation of the prior; in the classical approach such information is more likely to be ignored, and when incorporated is usually done so in ad hoc (arranged or happening when necessary and not planned in advance) ways.

3. The Bayesian approach can tailor the estimate to the purpose of the study, through selection of the loss function; in general, its compatibility with decision analysis is a decided advantage.

4. There is no need to justify the estimating procedure in terms of the awkward concept of the performance of the estimator in hypothetical (based on situations or ideas which are possible and imagined rather than real and true) repeated samples; the Bayesian approach is justified solely on the basis of the prior and the sample data.