Hypothesis testing

Much of statistical analysis is concerned with inferring from measured data in a sample some properties of a population; this is usually achieved making comparisons of sample data with other data of known properties, e.g. generated assuming some theory. In most cases, statistic has to be calculated, based on which the compatibility (expressed as confidence levels) with different hypotheses can be established. Note that a ``simpler hypothesis (often referred to as null hypothesis) is usually to be preferred over a ``more complicated one; introducing many free parameters will typically make the apparent compatibility look better, but introduces a lessening of a ``confidence level not expressed in numbers, if there is no physical reason to introduce the parameters.

In statistical terminology, one refers to a type-I error if a null hypothesis is wrongly rejected, and one calls type-II error when a null hypothesis is accepted when in fact it is false. This corresponds, in event classification, to losses and contamination ( Neyman-Pearson Diagram).