Conditional dependence

From HandWiki
A Bayesian network illustrating conditional dependence

In probability theory, conditional dependence is a relationship between two or more events that are dependent when a third event occurs.[1][2] For example, if [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] are two events that individually increase the probability of a third event [math]\displaystyle{ C, }[/math] and do not directly affect each other, then initially (when it has not been observed whether or not the event [math]\displaystyle{ C }[/math] occurs)[3][4] [math]\displaystyle{ \operatorname{P}(A \mid B) = \operatorname{P}(A) \quad \text{ and } \quad \operatorname{P}(B \mid A) = \operatorname{P}(B) }[/math] ([math]\displaystyle{ A \text{ and } B }[/math] are independent).

But suppose that now [math]\displaystyle{ C }[/math] is observed to occur. If event [math]\displaystyle{ B }[/math] occurs then the probability of occurrence of the event [math]\displaystyle{ A }[/math] will decrease because its positive relation to [math]\displaystyle{ C }[/math] is less necessary as an explanation for the occurrence of [math]\displaystyle{ C }[/math] (similarly, event [math]\displaystyle{ A }[/math] occurring will decrease the probability of occurrence of [math]\displaystyle{ B }[/math]). Hence, now the two events [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] are conditionally negatively dependent on each other because the probability of occurrence of each is negatively dependent on whether the other occurs. We have[5] [math]\displaystyle{ \operatorname{P}(A \mid C \text{ and } B) \lt \operatorname{P}(A \mid C). }[/math]

Conditional dependence is different from conditional independence. In conditional independence two events (which may be dependent or not) become independent given the occurrence of a third event.[6]

Example

In essence probability is influenced by a person's information about the possible occurrence of an event. For example, let the event [math]\displaystyle{ A }[/math] be 'I have a new phone'; event [math]\displaystyle{ B }[/math] be 'I have a new watch'; and event [math]\displaystyle{ C }[/math] be 'I am happy'; and suppose that having either a new phone or a new watch increases the probability of my being happy. Let us assume that the event [math]\displaystyle{ C }[/math] has occurred – meaning 'I am happy'. Now if another person sees my new watch, he/she will reason that my likelihood of being happy was increased by my new watch, so there is less need to attribute my happiness to a new phone.

To make the example more numerically specific, suppose that there are four possible states [math]\displaystyle{ \Omega = \left\{ s_1, s_2, s_3, s_4 \right\}, }[/math] given in the middle four columns of the following table, in which the occurrence of event [math]\displaystyle{ A }[/math] is signified by a [math]\displaystyle{ 1 }[/math] in row [math]\displaystyle{ A }[/math] and its non-occurrence is signified by a [math]\displaystyle{ 0, }[/math] and likewise for [math]\displaystyle{ B }[/math] and [math]\displaystyle{ C. }[/math] That is, [math]\displaystyle{ A = \left\{ s_2, s_4 \right\}, B = \left\{ s_3, s_4 \right\}, }[/math] and [math]\displaystyle{ C = \left\{ s_2, s_3, s_4 \right\}. }[/math] The probability of [math]\displaystyle{ s_i }[/math] is [math]\displaystyle{ 1/4 }[/math] for every [math]\displaystyle{ i. }[/math]

Event [math]\displaystyle{ \operatorname{P}(s_1)=1/4 }[/math] [math]\displaystyle{ \operatorname{P}(s_2)=1/4 }[/math] [math]\displaystyle{ \operatorname{P}(s_3)=1/4 }[/math] [math]\displaystyle{ \operatorname{P}(s_4)=1/4 }[/math] Probability of event
[math]\displaystyle{ A }[/math] 0 1 0 1 [math]\displaystyle{ \tfrac{1}{2} }[/math]
[math]\displaystyle{ B }[/math] 0 0 1 1 [math]\displaystyle{ \tfrac{1}{2} }[/math]
[math]\displaystyle{ C }[/math] 0 1 1 1 [math]\displaystyle{ \tfrac{3}{4} }[/math]

and so

Event [math]\displaystyle{ s_1 }[/math] [math]\displaystyle{ s_2 }[/math] [math]\displaystyle{ s_3 }[/math] [math]\displaystyle{ s_4 }[/math] Probability of event
[math]\displaystyle{ A \cap B }[/math] 0 0 0 1 [math]\displaystyle{ \tfrac{1}{4} }[/math]
[math]\displaystyle{ A \cap C }[/math] 0 1 0 1 [math]\displaystyle{ \tfrac{1}{2} }[/math]
[math]\displaystyle{ B \cap C }[/math] 0 0 1 1 [math]\displaystyle{ \tfrac{1}{2} }[/math]
[math]\displaystyle{ A \cap B \cap C }[/math] 0 0 0 1 [math]\displaystyle{ \tfrac{1}{4} }[/math]

In this example, [math]\displaystyle{ C }[/math] occurs if and only if at least one of [math]\displaystyle{ A, B }[/math] occurs. Unconditionally (that is, without reference to [math]\displaystyle{ C }[/math]), [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] are independent of each other because [math]\displaystyle{ \operatorname{P}(A) }[/math]—the sum of the probabilities associated with a [math]\displaystyle{ 1 }[/math] in row [math]\displaystyle{ A }[/math]—is [math]\displaystyle{ \tfrac{1}{2}, }[/math] while [math]\displaystyle{ \operatorname{P}(A\mid B) = \operatorname{P}(A \text{ and } B) / \operatorname{P}(B) = \tfrac{1/4}{1/2} = \tfrac{1}{2} = \operatorname{P}(A). }[/math] But conditional on [math]\displaystyle{ C }[/math] having occurred (the last three columns in the table), we have [math]\displaystyle{ \operatorname{P}(A \mid C) = \operatorname{P}(A \text{ and } C) / \operatorname{P}(C) = \tfrac{1/2}{3/4} = \tfrac{2}{3} }[/math] while [math]\displaystyle{ \operatorname{P}(A \mid C \text{ and } B) = \operatorname{P}(A \text{ and } C \text{ and } B) / \operatorname{P}(C \text{ and } B) = \tfrac{1/4}{1/2} = \tfrac{1}{2} \lt \operatorname{P}(A \mid C). }[/math] Since in the presence of [math]\displaystyle{ C }[/math] the probability of [math]\displaystyle{ A }[/math] is affected by the presence or absence of [math]\displaystyle{ B, A }[/math] and [math]\displaystyle{ B }[/math] are mutually dependent conditional on [math]\displaystyle{ C. }[/math]

See also

References

  1. Introduction to Artificial Intelligence by Sebastian Thrun and Peter Norvig, 2011 "Unit 3: Conditional Dependence"
  2. Introduction to learning Bayesian Networks from Data by Dirk Husmeier [1] "Introduction to Learning Bayesian Networks from Data -Dirk Husmeier"
  3. Conditional Independence in Statistical theory "Conditional Independence in Statistical Theory", A. P. Dawid"
  4. Probabilistic independence on Britannica "Probability->Applications of conditional probability->independence (equation 7) "
  5. Introduction to Artificial Intelligence by Sebastian Thrun and Peter Norvig, 2011 "Unit 3: Explaining Away"
  6. Conditional Independence in Statistical theory "Conditional Independence in Statistical Theory", A. P. Dawid