\n"
],
"text/plain": [
"\u001b[1m{\u001b[0m\n",
"\u001b[2;32m│ \u001b[0m\u001b[32m'odds_ratio'\u001b[0m: \u001b[1;36m1.9583333333333333\u001b[0m,\n",
"\u001b[2;32m│ \u001b[0m\u001b[32m'pvalue'\u001b[0m: \u001b[1;36m0.48734165612568525\u001b[0m,\n",
"\u001b[2;32m│ \u001b[0m\u001b[32m'reject_alpha'\u001b[0m: \u001b[3;91mFalse\u001b[0m\n",
"\u001b[1m}\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Fishers exact test\n",
"pprint(rTZ[\"global_tests\"][\"fisher\"], expand_all=True)"
]
},
{
"cell_type": "markdown",
"id": "089bf384",
"metadata": {},
"source": [
"### Conclusion\n",
"\n",
"To assess whether the binary variables $T$ and $Z$ are independent, we applied both **Pearson’s chi-square test** and **Fisher’s exact test** to the $2 \\times 2$ contingency table.\n",
"\n",
"The obtained p-values were:\n",
"\n",
"- Pearson’s chi-square test: $p = 0.569$\n",
"- Fisher’s exact test: $p = 0.487$\n",
"\n",
"At significance level $\\alpha = 0.05$, both p-values are greater than $\\alpha$, so we **fail to reject** the null hypothesis\n",
"\n",
"$$\n",
"H_0:\\; T \\perp Z.\n",
"$$\n",
"\n",
"Thus, the data do **not** provide statistically significant evidence of an association between $T$ and $Z$. In other words, based on this sample, we do not have evidence that the distribution of $T$ depends on $Z$.\n",
"\n",
"For a binary variable $T$, the null hypothesis of independence can equivalently be written as\n",
"\n",
"$$\n",
"P(T=1 \\mid Z=0) = P(T=1 \\mid Z=1).\n",
"$$\n",
"\n",
"One expected cell count is very small (approximately $0.603$), which means the chi-square approximation may be unreliable. Therefore, **Fisher’s exact test is the more appropriate test in this case**. Since Fisher’s exact test is also not significant, the final conclusion remains:\n",
"\n",
"**There is insufficient evidence to reject independence between $T$ and $Z$ at the 5\\% significance level.**\n"
]
},
{
"cell_type": "markdown",
"id": "6b46ab90",
"metadata": {},
"source": [
"## Prentice Criteria (PC) evaluation framework\n",
"\n",
"It was by Google AI. In practice, these are tested using a nested series of regression models:\n",
"\n",
"1. **Model for PC 1 (Effect of $Z$):**\n",
"\n",
"$$\n",
" \\operatorname{logit}(P(T=1)) = \\mu_1 + \\beta Z\n",
"$$ \n",
"\n",
"*Requirement:* $\\beta \\neq 0$\n",
"\n",
"3. **Model for PC 2 (Effect of $Z$ on $S$):**\n",
"\n",
"$$\n",
" E[S] = \\mu_2 + \\alpha Z\n",
"$$ \n",
"\n",
"*Requirement:* $\\alpha \\neq 0$\n",
"\n",
"5. **Model for PC 3 (The Full Model):**\n",
"\n",
"$$\n",
" \\operatorname{logit}(P(T=1)) = \\mu_3 + \\gamma S + \\beta_S Z\n",
"$$\n",
"\n",
"*Requirement (Criterion 4):* $\\gamma \\neq 0$ and $\\beta_S \\to 0$\n",
"\n",
"**Note:** PC 4 is notoriously difficult to satisfy perfectly. Researchers often calculate the **Proportion Explained (PE)**:\n",
"$$\n",
" PE = \\frac{\\beta - \\beta_S}{\\beta}\n",
"$$\n",
"\n",
"Ideally $PE \\approx 1$ as $\\beta_S \\approx 0$. \n",
"\n",
"Note: $\\beta_S$ and $\\beta$ is sometimes called $\\beta_{\\rm un-adjusted}$, $\\beta_{\\rm adjusted}$."
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "30841dae",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.microsoft.datawrangler.viewer.v0+json": {
"columns": [
{
"name": "index",
"rawType": "int64",
"type": "integer"
},
{
"name": "Criterion",
"rawType": "object",
"type": "string"
},
{
"name": "Method",
"rawType": "object",
"type": "string"
},
{
"name": "Model",
"rawType": "object",
"type": "string"
},
{
"name": "Estimate",
"rawType": "float64",
"type": "float"
},
{
"name": "P-value",
"rawType": "object",
"type": "unknown"
},
{
"name": "Pass",
"rawType": "boolean",
"type": "boolean"
}
],
"ref": "e26dba4c-dc0c-4b6f-855a-e61accb0f5a1",
"rows": [
[
"0",
"1. Treatment -> True",
"Pearson's Chi-squared test",
"Z x T (2x2 table)",
"0.6720937713621131",
"0.569024824759762",
"False"
],
[
"1",
"2. Treatment -> Surrogate",
"Kolmogorov–Smirnov test",
"S by Z groups",
"0.28851540616246496",
"0.5879664894834571",
"False"
],
[
"2",
"3. Surrogate -> True",
"Logistic",
"T ~ S",
"10.73815933915853",
"0.004114055500597262",
"True"
],
[
"3",
"4. Full Mediation",
"Logistic",
"T ~ S + Z",
"1.0343270483046114",
"0.6441919482170702",
"True"
],
[
"4",
"Proportion Explained (PE)",
"Logistic coefficients",
"Compare Z in (T ~ Z) vs (T ~ S + Z).",
"-0.5389624072967832",
null,
"False"
]
],
"shape": {
"columns": 6,
"rows": 5
}
},
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
Criterion
\n",
"
Method
\n",
"
Model
\n",
"
Estimate
\n",
"
P-value
\n",
"
Pass
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
1. Treatment -> True
\n",
"
Pearson's Chi-squared test
\n",
"
Z x T (2x2 table)
\n",
"
0.672094
\n",
"
0.569025
\n",
"
False
\n",
"
\n",
"
\n",
"
1
\n",
"
2. Treatment -> Surrogate
\n",
"
Kolmogorov–Smirnov test
\n",
"
S by Z groups
\n",
"
0.288515
\n",
"
0.587966
\n",
"
False
\n",
"
\n",
"
\n",
"
2
\n",
"
3. Surrogate -> True
\n",
"
Logistic
\n",
"
T ~ S
\n",
"
10.738159
\n",
"
0.004114
\n",
"
True
\n",
"
\n",
"
\n",
"
3
\n",
"
4. Full Mediation
\n",
"
Logistic
\n",
"
T ~ S + Z
\n",
"
1.034327
\n",
"
0.644192
\n",
"
True
\n",
"
\n",
"
\n",
"
4
\n",
"
Proportion Explained (PE)
\n",
"
Logistic coefficients
\n",
"
Compare Z in (T ~ Z) vs (T ~ S + Z).
\n",
"
-0.538962
\n",
"
<NA>
\n",
"
False
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" Criterion Method \\\n",
"0 1. Treatment -> True Pearson's Chi-squared test \n",
"1 2. Treatment -> Surrogate Kolmogorov–Smirnov test \n",
"2 3. Surrogate -> True Logistic \n",
"3 4. Full Mediation Logistic \n",
"4 Proportion Explained (PE) Logistic coefficients \n",
"\n",
" Model Estimate P-value Pass \n",
"0 Z x T (2x2 table) 0.672094 0.569025 False \n",
"1 S by Z groups 0.288515 0.587966 False \n",
"2 T ~ S 10.738159 0.004114 True \n",
"3 T ~ S + Z 1.034327 0.644192 True \n",
"4 Compare Z in (T ~ Z) vs (T ~ S + Z). -0.538962 False "
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df_prentice = utils.prentice_criteria(\n",
" df_work,\n",
" endpoint=\"T\",\n",
" surrogate=\"S\",\n",
" treatment=\"Z\",\n",
" alpha=0.05,\n",
" return_models=False,\n",
" criterion1_test=\"chi2\",\n",
" criterion2_test=\"ks\",\n",
")\n",
"\n",
"df_prentice"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "469afd91",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "base (3.12.3)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}