Lecture

Decision Rules

Decision Rules: a mapping from data to hypothesis

Type I Error: When $H_0$ negative result is true, we delcare $H_1$ positive result. (false positive: you tested positive, but is false - false alarm)

$\text{significance of a test} = Pr\{\text{declare } H_1 | H_0 \} = Pr\{\cup \text{some events that let you declare } H_1 | H_0 \}$

Type II Error: When $H_1$ positive result is true, we delcare $H_0$ negative result. (false negative: you tested negative, but is false - miss)

$(1 - \text{power}) = Pr\{\text{declare } H_0 | H_1 \} = Pr\{\cup \text{some events that let you declare } H_0 | H_1 \}$

If you decrease one type of error, it usually increase another type of error.

We view $H_0$ or $H_1$ as latent variable.

Maximim Likelihood (ML) Decision Rule and Likelihood Ratio Tests (LRT)

Likelihood Ratio Test: if $\land(k) \geq \tau$ , we declear $H_0$ , otherwise $H_1$ .

$H_0 \iff \land(k) = \frac{Pr\{X = k | H_0\}}{Pr\{X = k | H_1\}} > \tau$

Example: $X = x$ are COVID-19 detected level (the higher level is, usually the more likely you have COVID-19), $H_0$ is you actually don't have COVID-19.

$Pr\{X=k \\| H_*\}$	$X=0$	$X=1$	$X=2$	$X=3$
$H_0$	0.6	0.3	$0.1$	$0.0$
$H_1$	$0.1$	$0.2$	0.3	0.4

Maximim Likelihood (ML): special case of Likelihood Ratio Tests (LRT) when $\tau = 1$ .

MAP Decision Rule

If $Pr\{H_0 | X = k\} \geq^? Pr\{H_1 | X = k\}$ , then declear $H_0$ , otherwise $H_1$ . This is essentially equal to:

$Pr\{X = k | H_0\} Pr\{H_0\} \geq^? Pr\{X = k | H_1\}Pr\{H_1\}$

Example: given $Pr\{H_0\} = 0.8, Pr\{H_1\} = 0.2$ , we transform the table to joint distribution.

$Pr\{X=k\cap H_*\}$	$X=0$	$X=1$	$X=2$	$X=3$
$H_0$	0.48	0.24	0.08	$0$
$H_1$	$0.02$	$0.04$	$0.06$	0.08

$MAP \to LRT$ : This is equivalent to set $\tau = \frac{Pr\{H_1\}}{Pr\{H_0\}}$ . Also you see MAP is skewed by the prior probability, which is not good. (People in non-COVID cluster might have lower chance to be detected.)

Error Probability

Average Error Probability (AEP)

$Pr\{Error\} = Pr\{\text{Declare } H_1 | H_0\} \cdot Pr\{H_0\} + Pr\{\text{Declare } H_0 | H_1\} \cdot Pr\{H_1\}$

It is essentially the sum of two joint probability in case we are wrong.

Theorem: MAP minimizes AEP.

Proof: AFSOC $(\exists \sigma)(\sigma \text{ is not MAP } \land \sigma \text{ achieves lowest AEP})$ . Since $\sigma$ is not MAP, then there exists $X = k'$ such that it is either:

$\sigma \to H_0$ , but $Pr\{\{X = k'\}\} < Pr\{\{X = k'\} \cap H_1\}$
$\sigma \to H_1$ , but $Pr\{\{X = k'\}\} > Pr\{\{X = k'\} \cap H_1\}$

WLOG, we are in case 1.

$\text{AEP}_\sigma = \sum_{k \in \{\sigma \to H_0\}} Pr\{\{X = k\} \cap H_1\} + \sum_{k \in \{\sigma \to H_1\}} Pr\{\{X = k\} \cap H_0\}$

But then there exists another policy $\sigma'$ that has AEP of following

$\text{AEP}_{\sigma'} = \sum_{k \in \{\sigma \to H_0\} - \{k'\}} Pr\{\{X = k\} \cap H_1\} + \sum_{k \in \{\sigma \to H_1 \cap \{k'\}\}} Pr\{\{X = k\} \cap H_0\} < \text{AEP}_{\sigma}$

Cost Model

$E[\text{Cost}] = E[\text{Cost} | \text{Declare}H_1 \land H_0]Pr\{\text{Declare}H_1 | H_0\}Pr\{H_0\} + E[\text{Cost} | \text{Declare }H_0 \land H_1]Pr\{\text{Declare}H_0 | H_1\}Pr\{H_1\}$

This is equivalent to set

$\tau = \frac{Pr\{H_1\} E[\text{Cost} | \text{Declare }H_1 \land H_0]}{Pr\{H_0\} E[\text{Cost} | \text{Declare }H_0 \land H_1]} = \frac{Pr\{H_1\} E[\text{T1 Cost}]}{Pr\{H_0\} E[\text{T2 Cost}]}$

Continuous Random Variable

We just change likelihood: $f_{X | H_0}(x), f_{X | H_1}(x)$ . However, the prior is still discrete because they are prior for hypothesis.

Differential Privacy via Laplace Noise

Table of Content