Akshay Agrawal commited on
Commit
1530e43
·
unverified ·
2 Parent(s): 3cd7fd0 23a3f9a

Merge pull request #84 from marimo-team/haleshot/18_clt

Browse files
probability/18_central_limit_theorem.py ADDED
@@ -0,0 +1,943 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # /// script
2
+ # requires-python = ">=3.10"
3
+ # dependencies = [
4
+ # "marimo",
5
+ # "matplotlib==3.10.1",
6
+ # "scipy==1.15.2",
7
+ # "numpy==2.2.4",
8
+ # "plotly==5.18.0",
9
+ # ]
10
+ # ///
11
+
12
+ import marimo
13
+
14
+ __generated_with = "0.11.30"
15
+ app = marimo.App(width="medium", app_title="Central Limit Theorem")
16
+
17
+
18
+ @app.cell(hide_code=True)
19
+ def _(mo):
20
+ mo.md(
21
+ r"""
22
+ # Central Limit Theorem
23
+
24
+ _This notebook is a computational companion to ["Probability for Computer Scientists"](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/clt/), by Stanford professor Chris Piech._
25
+
26
+ The Central Limit Theorem (CLT) is one of the most important concepts in probability theory and statistics. It explains why many real-world distributions tend to be normal, even when the underlying processes are not.
27
+ """
28
+ )
29
+ return
30
+
31
+
32
+ @app.cell(hide_code=True)
33
+ def _(mo):
34
+ mo.md(
35
+ r"""
36
+ ## Central Limit Theorem Statement
37
+
38
+ There are two ways to state the central limit theorem:
39
+
40
+ ### Sum Version
41
+
42
+ Let $X_1, X_2, \dots, X_n$ be independent and identically distributed random variables. The sum of these random variables approaches a normal distribution as $n \rightarrow \infty$:
43
+
44
+ $n∑i=1Xi∼N(n⋅μ,n⋅σ2)\sum_{i=1}^{n}X_i \sim \mathcal{N}(n \cdot \mu, n \cdot \sigma^2)$
45
+
46
+ Where $\mu = E[X_i]$ and $\sigma^2 = \text{Var}(X_i)$. Since each $X_i$ is identically distributed, they share the same expectation and variance.
47
+
48
+ ### Average Version
49
+
50
+ Let $X_1, X_2, \dots, X_n$ be independent and identically distributed random variables. The average of these random variables approaches a normal distribution as $n \rightarrow \infty$:
51
+
52
+ $\frac{1}{n} ∑i=1Xi∼N(μ,σ2n)\frac{1}{n}\sum_{i=1}^{n}X_i \sim \mathcal{N}(\mu, \frac{\sigma^2}{n})$
53
+
54
+ Where $\mu = E[X_i]$ and $\sigma^2 = \text{Var}(X_i)$.
55
+
56
+ The CLT is incredible because it applies to almost any distribution (as long as it has a finite mean and variance), regardless of its shape.
57
+ """
58
+ )
59
+ return
60
+
61
+
62
+ @app.cell(hide_code=True)
63
+ def _(mo):
64
+ mo.md(
65
+ r"""
66
+ ## Central Limit Theorem Intuition
67
+
68
+ Let's explore what happens when you add random variables together. For example, what if we add 100 different uniform random variables?
69
+
70
+ ```python
71
+ from random import random
72
+
73
+ def add_100_uniforms():
74
+ total = 0
75
+ for i in range(100):
76
+ # returns a sample from uniform(0, 1)
77
+ x_i = random()
78
+ total += x_i
79
+ return total
80
+ ```
81
+
82
+ The value returned by this function will be a random variable. Click the button below to run the function and observe the resulting value of total:
83
+ """
84
+ )
85
+ return
86
+
87
+
88
+ @app.cell(hide_code=True)
89
+ def _(mo):
90
+ run_button = mo.ui.run_button(label="Run add_100_uniforms()")
91
+
92
+ run_button.center()
93
+ return (run_button,)
94
+
95
+
96
+ @app.cell(hide_code=True)
97
+ def _(mo, random, run_button):
98
+ def add_100_uniforms():
99
+ total = 0
100
+ for i in range(100):
101
+ # returns a sample from uniform(0, 1)
102
+ x_i = random.random()
103
+ total += x_i
104
+ return total
105
+
106
+ # Display the result when the button is clicked
107
+ if run_button.value:
108
+ uniform_result = add_100_uniforms()
109
+ display = mo.md(f"**total**: {uniform_result:.5f}")
110
+ else:
111
+ display = mo.md("")
112
+
113
+ display
114
+ return add_100_uniforms, display, uniform_result
115
+
116
+
117
+ @app.cell(hide_code=True)
118
+ def _(mo):
119
+ mo.md(r"""What does total look like as a distribution? Let's calculate total many times and visualize the histogram of values it produces.""")
120
+ return
121
+
122
+
123
+ @app.cell(hide_code=True)
124
+ def _(mo):
125
+ # Simulation control
126
+ run_simulation_button = mo.ui.button(
127
+ value=0,
128
+ on_click=lambda value: value + 1,
129
+ label="Run 10,000 more samples",
130
+ kind="warn"
131
+ )
132
+
133
+ run_simulation_button.center()
134
+ return (run_simulation_button,)
135
+
136
+
137
+ @app.cell(hide_code=True)
138
+ def _(add_100_uniforms, go, mo, np, run_simulation_button, stats, time):
139
+ # store the results
140
+ def get_simulation_results():
141
+ if not hasattr(get_simulation_results, "results"):
142
+ get_simulation_results.results = []
143
+ get_simulation_results.last_button_value = -1 # track button clicks
144
+ return get_simulation_results
145
+
146
+ # grab the results
147
+ sim_storage = get_simulation_results()
148
+ simulation_results = sim_storage.results
149
+
150
+ # Check if button was clicked (value changed)
151
+ if run_simulation_button.value != sim_storage.last_button_value:
152
+ # Update the last seen button value
153
+ sim_storage.last_button_value = run_simulation_button.value
154
+
155
+ with mo.status.spinner(title="Running simulation...") as progress_status:
156
+ sim_count = 10000
157
+ new_results = []
158
+ for _ in mo.status.progress_bar(range(sim_count)):
159
+ sim_result = add_100_uniforms()
160
+ new_results.append(sim_result)
161
+ time.sleep(0.0001) # tiny pause
162
+
163
+ simulation_results.extend(new_results)
164
+
165
+ progress_status.update(f"✅ Added {sim_count:,} samples (total: {len(simulation_results):,})")
166
+
167
+ if simulation_results:
168
+ # Numbers
169
+ mean = np.mean(simulation_results)
170
+ std_dev = np.std(simulation_results)
171
+
172
+ theoretical_mean = 100 * 0.5 # = 50
173
+ theoretical_variance = 100 * (1/12) # = 8.33...
174
+ theoretical_std = np.sqrt(theoretical_variance) # ≈ 2.89
175
+
176
+ # should be 10k times the click number (mainly for the y-axis label)
177
+ total_samples = run_simulation_button.value * 10000
178
+
179
+ fig = go.Figure()
180
+
181
+ # histogram of samples
182
+ fig.add_trace(go.Histogram(
183
+ x=simulation_results,
184
+ histnorm='probability density',
185
+ name='Sum Distribution',
186
+ marker_color='royalblue',
187
+ opacity=0.7
188
+ ))
189
+
190
+ x_vals = np.linspace(min(simulation_results), max(simulation_results), 1000)
191
+ y_vals = stats.norm.pdf(x_vals, theoretical_mean, theoretical_std)
192
+
193
+ fig.add_trace(go.Scatter(
194
+ x=x_vals,
195
+ y=y_vals,
196
+ mode='lines',
197
+ name='Normal approximation',
198
+ line=dict(color='red', width=2)
199
+ ))
200
+
201
+ fig.add_vline(
202
+ x=mean,
203
+ line_dash="dash",
204
+ line_width=1.5,
205
+ line_color="green",
206
+ annotation_text=f"Sample Mean: {mean:.2f}",
207
+ annotation_position="top right"
208
+ )
209
+
210
+ # some notes
211
+ fig.add_annotation(
212
+ x=0.02, y=0.95,
213
+ xref="paper", yref="paper",
214
+ text=f"Sum of 100 Uniform(0,1) variables<br>" +
215
+ f"Sample size: {total_samples:,}<br>" +
216
+ f"Sample mean: {mean:.2f} (expected: {theoretical_mean})<br>" +
217
+ f"Sample std: {std_dev:.2f} (expected: {theoretical_std:.2f})<br>" +
218
+ f"According to CLT: Normal({theoretical_mean}, {theoretical_variance:.2f})",
219
+ showarrow=False,
220
+ align="left",
221
+ bgcolor="white",
222
+ opacity=0.8
223
+ )
224
+
225
+ fig.update_layout(
226
+ title=f'Distribution of Sum of 100 Uniforms (Click #{run_simulation_button.value})',
227
+ xaxis_title='Values',
228
+ yaxis_title=f'Probability Density ({total_samples:,} runs)',
229
+ template='plotly_white',
230
+ height=500
231
+ )
232
+
233
+ # show
234
+ histogram = mo.ui.plotly(fig)
235
+ else:
236
+ histogram = mo.md("Click the button to run the simulation!")
237
+
238
+ # display
239
+ histogram
240
+ return (
241
+ fig,
242
+ get_simulation_results,
243
+ histogram,
244
+ mean,
245
+ new_results,
246
+ progress_status,
247
+ sim_count,
248
+ sim_result,
249
+ sim_storage,
250
+ simulation_results,
251
+ std_dev,
252
+ theoretical_mean,
253
+ theoretical_std,
254
+ theoretical_variance,
255
+ total_samples,
256
+ x_vals,
257
+ y_vals,
258
+ )
259
+
260
+
261
+ @app.cell(hide_code=True)
262
+ def _(mo):
263
+ mo.md(
264
+ r"""
265
+ That is interesting! The sum of 100 independent uniforms looks normal. Is that a special property of uniforms? No! It turns out to work for almost any type of distribution (as long as the distribution has finite mean and variance).
266
+
267
+ - Sum of 40 $X_i$ where $X_i \sim \text{Beta}(a = 5, b = 4)$? Normal.
268
+ - Sum of 90 $X_i$ where $X_i \sim \text{Poisson}(\lambda = 4)$? Normal.
269
+ - Sum of 50 dice-rolls? Normal.
270
+ - Average of 10000 $X_i$ where $X_i \sim \text{Exp}(\lambda = 8)$? Normal.
271
+
272
+ For any distribution, the sum or average of a sufficiently large number of independent, identically distributed random variables will be approximately normally distributed.
273
+ """
274
+ )
275
+ return
276
+
277
+
278
+ @app.cell(hide_code=True)
279
+ def _(mo):
280
+ mo.md(
281
+ r"""
282
+ ## Continuity Correction
283
+
284
+ When using the Central Limit Theorem with discrete random variables (like a Binomial or Poisson), we need to apply a continuity correction. This is because we're approximating a discrete distribution with a continuous one (normal).
285
+
286
+ The continuity correction involves adjusting the boundaries in probability calculations by ±0.5 to account for the discrete nature of the original variable.
287
+
288
+ You should use a continuity correction any time your normal is approximating a discrete random variable. The rules for a general continuity correction are the same as the rules for the [binomial-approximation continuity correction](http://marimo.app/https://github.com/marimo-team/learn/blob/main/probability/14_binomial_distribution.py).
289
+
290
+ In our example above, where we added 100 uniforms, a continuity correction isn't needed because the sum of uniforms is continuous. However, in examples with dice or other discrete distributions, a continuity correction would be necessary.
291
+ """
292
+ )
293
+ return
294
+
295
+
296
+ @app.cell(hide_code=True)
297
+ def _(mo):
298
+ mo.md(
299
+ r"""
300
+ ## Examples
301
+
302
+ Let's work through some practical examples to see how the Central Limit Theorem is applied.
303
+ """
304
+ )
305
+ return
306
+
307
+
308
+ @app.cell(hide_code=True)
309
+ def _(mo):
310
+ mo.md(
311
+ r"""
312
+ ### Example 1: Dice Game
313
+
314
+ You will roll a 6-sided dice 10 times. Let $X$ be the total value of all 10 dice: $X = X_1 + X_2 + \dots + X_{10}$. You win the game if $X \leq 25$ or $X \geq 45$. Use the central limit theorem to calculate the probability that you win.
315
+
316
+ Recall that for a single die roll $X_i$:
317
+
318
+ - $E[X_i] = 3.5$
319
+ - $\text{Var}(X_i) = \frac{35}{12}$
320
+
321
+ **Solution:**
322
+
323
+ Let $Y$ be the approximating normal distribution. By the Central Limit Theorem:
324
+
325
+ $Y∼N(10⋅E[Xi],10⋅Var(Xi))Y \sim \mathcal{N}(10 \cdot E[X_i], 10 \cdot \text{Var}(X_i))$
326
+
327
+ Substituting in the known values:
328
+
329
+ $Y∼N(10⋅3.5,10⋅3512)=N(35,29.2)Y \sim \mathcal{N}(10 \cdot 3.5, 10 \cdot \frac{35}{12}) = \mathcal{N}(35, 29.2)$
330
+
331
+ Now we calculate the probability:
332
+
333
+ $P(X≤25 or X≥45)P(X \leq 25 \text{ or } X \geq 45)$
334
+
335
+ $=P(X≤25)+P(X≥45)= P(X \leq 25) + P(X \geq 45)$
336
+
337
+ $≈P(Y<25.5)+P(Y>44.5) (Continuity Correction)\approx P(Y < 25.5) + P(Y > 44.5) \text{ (Continuity Correction)}$
338
+
339
+ $≈P(Y<25.5)+[1−P(Y<44.5)]\approx P(Y < 25.5) + [1 - P(Y < 44.5)]$
340
+
341
+ $≈Φ(25.5−35√29.2)+[1−Φ(44.5−35√29.2)]\approx \Phi\left(\frac{25.5 - 35}{\sqrt{29.2}}\right) + \left[1 - \Phi\left(\frac{44.5 - 35}{\sqrt{29.2}}\right)\right]$
342
+
343
+ $≈Φ(−1.76)+[1−Φ(1.76)]\approx \Phi(-1.76) + [1 - \Phi(1.76)]$
344
+
345
+ $≈0.039+(1−0.961)\approx 0.039 + (1 - 0.961)$
346
+
347
+ $≈0.078\approx 0.078$
348
+ So, the probability of winning the game is approximately 7.8%.
349
+ """
350
+ )
351
+ return
352
+
353
+
354
+ @app.cell(hide_code=True)
355
+ def _(create_dice_game_visualization, fig_to_image, mo):
356
+ # Display visualization
357
+ dice_game_fig = create_dice_game_visualization()
358
+ dice_game_image = mo.image(fig_to_image(dice_game_fig), width="100%")
359
+
360
+ dice_explanation = mo.md(
361
+ r"""
362
+ **Visualization Explanation:**
363
+
364
+ The graph shows the distribution of the sum of 10 dice rolls. The blue bars represent the actual probability mass function (PMF), while the red curve shows the normal approximation using the Central Limit Theorem.
365
+
366
+ The winning regions are shaded in orange:
367
+ - The left region where $X \leq 25$
368
+ - The right region where $X \geq 45$
369
+
370
+ The total probability of these regions is approximately 0.078 or 7.8%.
371
+
372
+ Notice how the normal approximation provides a good fit to the discrete distribution, demonstrating the power of the Central Limit Theorem.
373
+ """
374
+ )
375
+
376
+ mo.vstack([dice_game_image, dice_explanation])
377
+ return dice_explanation, dice_game_fig, dice_game_image
378
+
379
+
380
+ @app.cell(hide_code=True)
381
+ def _(mo):
382
+ mo.md(
383
+ r"""
384
+ ### Example 2: Algorithm Runtime Estimation
385
+
386
+ Say you have a new algorithm and you want to test its running time. You know the variance of the algorithm's run time is $\sigma^2 = 4 \text{ sec}^2$, but you want to estimate the mean run time $t$ in seconds.
387
+
388
+ You can run the algorithm repeatedly (IID trials). How many trials do you have to run so that your estimated runtime is within ±0.5 seconds of $t$ with 95% certainty?
389
+
390
+ Let $X_i$ be the run time of the $i$-th run (for $1 \leq i \leq n$).
391
+
392
+ **Solution:**
393
+
394
+ We need to find $n$ such that:
395
+
396
+ $0.95=P(−0.5≤∑ni=1Xin−t≤0.5)0.95 = P\left(-0.5 \leq \frac{\sum_{i=1}^n X_i}{n} - t \leq 0.5\right)$
397
+
398
+ By the central limit theorem, the sample mean follows a normal distribution.
399
+ We can standardize this to work with the standard normal:
400
+
401
+ $Z=(∑ni=1Xi)−nμσ√nZ = \frac{\left(\sum_{i=1}^n X_i\right) - n\mu}{\sigma \sqrt{n}}$
402
+
403
+ $=(∑ni=1Xi)−nt2√n= \frac{\left(\sum_{i=1}^n X_i\right) - nt}{2 \sqrt{n}}$
404
+
405
+ Rewriting our probability inequality so that the central term is $Z$:
406
+
407
+ $0.95=P(−0.5≤∑ni=1Xin−t≤0.5)0.95 = P\left(-0.5 \leq \frac{\sum_{i=1}^n X_i}{n} - t \leq 0.5\right)$
408
+
409
+ $=P(−0.5√n2≤Z≤0.5√n2)= P\left(\frac{-0.5 \sqrt{n}}{2} \leq Z \leq \frac{0.5 \sqrt{n}}{2}\right)$
410
+
411
+ And now we find the value of $n$ that makes this equation hold:
412
+
413
+ $0.95=Φ(√n4)−Φ(−√n4)0.95 = \Phi\left(\frac{\sqrt{n}}{4}\right) - \Phi\left(-\frac{\sqrt{n}}{4}\right)$
414
+
415
+ $4=Φ(√n4)−(1−Φ(√n4))= \Phi\left(\frac{\sqrt{n}}{4}\right) - \left(1 - \Phi\left(\frac{\sqrt{n}}{4}\right)\right)$
416
+
417
+ $=2Φ(√n4)−1= 2\Phi\left(\frac{\sqrt{n}}{4}\right) - 1$
418
+
419
+ Solving for $\Phi\left(\frac{\sqrt{n}}{4}\right)$:
420
+
421
+ $0.975=Φ(√n4)0.975 = \Phi\left(\frac{\sqrt{n}}{4}\right)$
422
+
423
+ $Φ−1(0.975)=√n4\Phi^{-1}(0.975) = \frac{\sqrt{n}}{4}$
424
+
425
+ $1.96=√n41.96 = \frac{\sqrt{n}}{4}$
426
+
427
+ $n=61.4n = 61.4$
428
+
429
+ Therefore, we need to run the algorithm 62 times to estimate the mean runtime within ±0.5 seconds with 95% confidence.
430
+ """
431
+ )
432
+ return
433
+
434
+
435
+ @app.cell(hide_code=True)
436
+ def _(create_algorithm_runtime_visualization, fig_to_image, mo):
437
+ # Display visualization
438
+ runtime_fig = create_algorithm_runtime_visualization()
439
+ runtime_image = mo.image(fig_to_image(runtime_fig), width="100%")
440
+
441
+ runtime_explanation = mo.md(
442
+ r"""
443
+ **Visualization Explanation:**
444
+
445
+ The graph illustrates how the standard error of the mean (SEM) decreases as the number of trials increases. The standard error is calculated as $\frac{\sigma}{\sqrt{n}}$.
446
+
447
+ - When we conduct 62 trials, the standard error is approximately 0.254 seconds.
448
+ - With a 95% confidence level, this gives us a margin of error of about ±0.5 seconds (1.96 × 0.254 ≈ 0.5).
449
+ - The shaded region shows how the confidence interval narrows as the number of trials increases.
450
+
451
+ This demonstrates why 62 trials are sufficient to meet our requirements of estimating the mean runtime within ±0.5 seconds with 95% confidence.
452
+ """
453
+ )
454
+
455
+ mo.vstack([runtime_image, runtime_explanation])
456
+ return runtime_explanation, runtime_fig, runtime_image
457
+
458
+
459
+ @app.cell(hide_code=True)
460
+ def _(mo):
461
+ mo.md(
462
+ r"""
463
+ ## Interactive CLT Explorer
464
+
465
+ Let's explore how the Central Limit Theorem works with different underlying distributions. You can select a distribution type and see how the distribution of the sample mean changes as the sample size increases.
466
+ """
467
+ )
468
+ return
469
+
470
+
471
+ @app.cell(hide_code=True)
472
+ def _(controls):
473
+ controls
474
+ return
475
+
476
+
477
+ @app.cell(hide_code=True)
478
+ def _(
479
+ distribution_type,
480
+ fig_to_image,
481
+ mo,
482
+ np,
483
+ plt,
484
+ run_explorer_button,
485
+ sample_size,
486
+ sim_count_slider,
487
+ stats,
488
+ ):
489
+ # Run simulation when button is clicked
490
+ if run_explorer_button.value:
491
+ # Set distribution parameters based on selection
492
+ if distribution_type.value == "uniform":
493
+ dist_name = "Uniform(0, 1)"
494
+ # For uniform(0,1): mean = 0.5, variance = 1/12
495
+ true_mean = 0.5
496
+ true_var = 1/12
497
+
498
+ # generate samples
499
+ def generate_sample():
500
+ return np.random.uniform(0, 1, sample_size.value)
501
+
502
+ elif distribution_type.value == "exponential":
503
+ rate = 1.0
504
+ dist_name = f"Exponential(λ={rate})"
505
+ # For exponential(λ): mean = 1/λ, variance = 1/λ²
506
+ true_mean = 1/rate
507
+ true_var = 1/(rate**2)
508
+
509
+ def generate_sample():
510
+ return np.random.exponential(1/rate, sample_size.value)
511
+
512
+ elif distribution_type.value == "binomial":
513
+ n_param, p = 10, 0.3
514
+ dist_name = f"Binomial(n={n_param}, p={p})"
515
+ # For binomial(n,p): mean = np, variance = np(1-p)
516
+ true_mean = n_param * p
517
+ true_var = n_param * p * (1-p)
518
+
519
+ def generate_sample():
520
+ return np.random.binomial(n_param, p, sample_size.value)
521
+
522
+ elif distribution_type.value == "poisson":
523
+ rate = 3.0
524
+ dist_name = f"Poisson(λ={rate})"
525
+ # For poisson(λ): mean = λ, variance = λ
526
+ true_mean = rate
527
+ true_var = rate
528
+
529
+ def generate_sample():
530
+ return np.random.poisson(rate, sample_size.value)
531
+
532
+ # Generate the simulation data using a spinner for progress
533
+ with mo.status.spinner(title="Running simulation...") as explorer_progress:
534
+ sample_means = []
535
+ original_samples = []
536
+
537
+ # Run simulations
538
+ for _ in mo.status.progress_bar(range(sim_count_slider.value)):
539
+ sample = generate_sample()
540
+
541
+ # Store the first simulation's individual values for visualizing original distribution
542
+ if len(original_samples) < 1000: # limit to prevent memory issues
543
+ original_samples.extend(sample)
544
+
545
+ # sample mean
546
+ sample_means.append(np.mean(sample))
547
+
548
+ # progress
549
+ explorer_progress.update(f"✅ Completed {sim_count_slider.value:,} simulations")
550
+
551
+ # Create visualization
552
+ explorer_fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))
553
+
554
+ # Original distribution histogram
555
+ ax1.hist(original_samples, bins=30, density=True, alpha=0.7, color='royalblue')
556
+ ax1.set_title(f"Original Distribution: {dist_name}")
557
+
558
+ # Theoretical mean line
559
+ ax1.axvline(x=true_mean, color='red', linestyle='--',
560
+ label=f'True Mean = {true_mean:.3f}')
561
+
562
+ ax1.set_xlabel("Value")
563
+ ax1.set_ylabel("Density")
564
+ ax1.legend()
565
+
566
+ # Sample means histogram and normal approximation
567
+ sample_mean_mean = np.mean(sample_means)
568
+ sample_mean_std = np.std(sample_means)
569
+ expected_std = np.sqrt(true_var / sample_size.value) # CLT prediction
570
+
571
+ ax2.hist(sample_means, bins=30, density=True, alpha=0.7, color='forestgreen',
572
+ label=f'Sample Size = {sample_size.value}')
573
+
574
+ # Normal approximation from CLT
575
+ explorer_x = np.linspace(min(sample_means), max(sample_means), 1000)
576
+ explorer_y = stats.norm.pdf(explorer_x, true_mean, expected_std)
577
+ ax2.plot(explorer_x, explorer_y, 'r-', linewidth=2, label='CLT Normal Approximation')
578
+
579
+ # Add mean line
580
+ ax2.axvline(x=true_mean, color='purple', linestyle='--',
581
+ label=f'True Mean = {true_mean:.3f}')
582
+
583
+ ax2.set_title(f"Distribution of Sample Means\n(CLT Prediction: N({true_mean:.3f}, {true_var/sample_size.value:.5f}))")
584
+ ax2.set_xlabel("Sample Mean")
585
+ ax2.set_ylabel("Density")
586
+ ax2.legend()
587
+
588
+ # Add CLT description
589
+ explorer_fig.text(0.5, 0.01,
590
+ f"Central Limit Theorem: As sample size increases, the distribution of sample means approaches\n" +
591
+ f"a normal distribution with mean = {true_mean:.3f} and variance = {true_var:.3f}/{sample_size.value} = {true_var/sample_size.value:.5f}",
592
+ ha='center', fontsize=10, bbox=dict(facecolor='white', alpha=0.8))
593
+
594
+ plt.tight_layout(rect=[0, 0.05, 1, 1])
595
+
596
+ # Display plot
597
+ explorer_image = mo.image(fig_to_image(explorer_fig), width="100%")
598
+ else:
599
+ explorer_image = mo.md("Click the 'Run Simulation' button to see how the Central Limit Theorem works.")
600
+
601
+ explorer_image
602
+ return (
603
+ ax1,
604
+ ax2,
605
+ dist_name,
606
+ expected_std,
607
+ explorer_fig,
608
+ explorer_image,
609
+ explorer_progress,
610
+ explorer_x,
611
+ explorer_y,
612
+ generate_sample,
613
+ n_param,
614
+ original_samples,
615
+ p,
616
+ rate,
617
+ sample,
618
+ sample_mean_mean,
619
+ sample_mean_std,
620
+ sample_means,
621
+ true_mean,
622
+ true_var,
623
+ )
624
+
625
+
626
+ @app.cell(hide_code=True)
627
+ def _(mo):
628
+ mo.md(
629
+ r"""
630
+ ## 🤔 Test Your Understanding
631
+
632
+ /// details | What is the shape of the distribution of the sum of many independent random variables?
633
+ The sum of many independent random variables approaches a normal distribution, regardless of the shape of the original distributions (as long as they have finite mean and variance). This is the essence of the Central Limit Theorem.
634
+ ///
635
+
636
+ /// details | If $X_1, X_2, \dots, X_{100}$ are IID random variables with $E[X_i] = 5$ and $Var(X_i) = 9$, what is the distribution of their sum?
637
+ By the Central Limit Theorem, the sum $S = X_1 + X_2 + \dots + X_{100}$ follows a normal distribution with:
638
+
639
+ - Mean: $E[S] = 100 \cdot E[X_i] = 100 \cdot 5 = 500$
640
+ - Variance: $Var(S) = 100 \cdot Var(X_i) = 100 \cdot 9 = 900$
641
+
642
+ Therefore, $S \sim \mathcal{N}(500, 900)$, or equivalently $S \sim \mathcal{N}(500, 30^2)$.
643
+ ///
644
+
645
+ /// details | When do you need to apply a continuity correction when using the Central Limit Theorem?
646
+ You need to apply a continuity correction when you're using the normal approximation (through CLT) for a discrete random variable.
647
+
648
+ For example, when approximating a binomial or Poisson distribution with a normal distribution, you should adjust boundaries by ±0.5 to account for the discrete nature of the original variable. This makes the approximation more accurate.
649
+ ///
650
+
651
+ /// details | If $X_1, X_2, \dots, X_{n}$ are IID random variables, how does the variance of their sample mean $\bar{X} = \frac{1}{n}\sum_{i=1}^{n}X_i$ change as $n$ increases?
652
+ The variance of the sample mean decreases as the sample size $n$ increases. Specifically:
653
+
654
+ $Var(\bar{X}) = \frac{Var(X_i)}{n}$
655
+
656
+ This means that as we take more samples, the sample mean becomes more concentrated around the true mean of the distribution. This is why larger samples give more precise estimates.
657
+ ///
658
+
659
+ /// details | Why is the Central Limit Theorem so important in statistics?
660
+ The Central Limit Theorem is foundational in statistics because:
661
+
662
+ 1. It allows us to make inferences about population parameters using sample statistics, regardless of the population's distribution.
663
+ 2. It explains why the normal distribution appears so frequently in natural phenomena.
664
+ 3. It enables the construction of confidence intervals and hypothesis tests for means, even when the underlying population distribution is unknown.
665
+ 4. It justifies many statistical methods that assume normality, even when working with non-normal data, provided the sample size is large enough.
666
+
667
+ In essence, the CLT provides the theoretical justification for much of statistical inference.
668
+ ///
669
+ """
670
+ )
671
+ return
672
+
673
+
674
+ @app.cell(hide_code=True)
675
+ def _(mo):
676
+ mo.md(r"""## Appendix (helper code and functions)""")
677
+ return
678
+
679
+
680
+ @app.cell
681
+ def _():
682
+ import marimo as mo
683
+ return (mo,)
684
+
685
+
686
+ @app.cell(hide_code=True)
687
+ def _():
688
+ from wigglystuff import TangleSlider
689
+ return (TangleSlider,)
690
+
691
+
692
+ @app.cell(hide_code=True)
693
+ def _():
694
+ # Import libraries
695
+ import numpy as np
696
+ import matplotlib.pyplot as plt
697
+ from scipy import stats
698
+ import io
699
+ import base64
700
+ import random
701
+ import time
702
+ import plotly.graph_objects as go
703
+ import plotly.io as pio
704
+ return base64, go, io, np, pio, plt, random, stats, time
705
+
706
+
707
+ @app.cell(hide_code=True)
708
+ def _(base64, io):
709
+ from matplotlib.figure import Figure
710
+
711
+ # Helper function to convert matplotlib figures to images
712
+ def fig_to_image(fig):
713
+ buf = io.BytesIO()
714
+ fig.savefig(buf, format='png', bbox_inches='tight')
715
+ buf.seek(0)
716
+ img_str = base64.b64encode(buf.getvalue()).decode('utf-8')
717
+ return f"data:image/png;base64,{img_str}"
718
+ return Figure, fig_to_image
719
+
720
+
721
+ @app.cell(hide_code=True)
722
+ def _(np, plt, stats):
723
+ def create_dice_game_visualization():
724
+ """Create a visualization for the dice game example."""
725
+ # Parameters
726
+ n_dice = 10
727
+ dice_values = np.arange(1, 7) # 1 to 6
728
+
729
+ # Theoretical values
730
+ single_die_mean = np.mean(dice_values) # 3.5
731
+ single_die_var = np.var(dice_values) # 35/12
732
+
733
+ # Sum distribution parameters
734
+ sum_mean = n_dice * single_die_mean
735
+ sum_var = n_dice * single_die_var
736
+ sum_std = np.sqrt(sum_var)
737
+
738
+ # Possible outcomes for the sum of 10 dice
739
+ min_sum = n_dice * min(dice_values) # 10
740
+ max_sum = n_dice * max(dice_values) # 60
741
+ sum_values = np.arange(min_sum, max_sum + 1)
742
+
743
+ # Create figure
744
+ fig, ax = plt.subplots(figsize=(10, 6))
745
+
746
+ # Calculate PMF through convolution
747
+ # For one die
748
+ single_pmf = np.ones(6) / 6
749
+
750
+ sum_pmf = single_pmf.copy()
751
+ for _ in range(n_dice - 1):
752
+ sum_pmf = np.convolve(sum_pmf, single_pmf)
753
+
754
+ # Plot the PMF
755
+ ax.bar(sum_values, sum_pmf, alpha=0.7, color='royalblue', label='Exact PMF')
756
+
757
+ # Normal approximation
758
+ x = np.linspace(min_sum - 5, max_sum + 5, 1000)
759
+ y = stats.norm.pdf(x, sum_mean, sum_std)
760
+ ax.plot(x, y, 'r-', linewidth=2, label='Normal Approximation')
761
+
762
+ # Win conditions (x ≤ 25 or x ≥ 45)
763
+ win_region_left = sum_values <= 25
764
+ win_region_right = sum_values >= 45
765
+
766
+ # Shade win regions
767
+ ax.bar(sum_values[win_region_left], sum_pmf[win_region_left],
768
+ color='darkorange', alpha=0.7, label='Win Region')
769
+ ax.bar(sum_values[win_region_right], sum_pmf[win_region_right],
770
+ color='darkorange', alpha=0.7)
771
+
772
+ # Calculate win probability
773
+ win_prob = np.sum(sum_pmf[win_region_left]) + np.sum(sum_pmf[win_region_right])
774
+
775
+ # Add vertical lines for critical values
776
+ ax.axvline(x=25.5, color='red', linestyle='--', linewidth=1.5, label='Critical Points')
777
+ ax.axvline(x=44.5, color='red', linestyle='--', linewidth=1.5)
778
+
779
+ # Add mean line
780
+ ax.axvline(x=sum_mean, color='green', linestyle='--', linewidth=1.5,
781
+ label=f'Mean = {sum_mean}')
782
+
783
+ # Text box with relevant information
784
+ textstr = '\n'.join((
785
+ f'Number of dice: {n_dice}',
786
+ f'Sum Mean: {sum_mean}',
787
+ f'Sum Std Dev: {sum_std:.2f}',
788
+ f'Win Probability: {win_prob:.4f}',
789
+ f'CLT Approximation: {0.078:.4f}'
790
+ ))
791
+ props = dict(boxstyle='round', facecolor='wheat', alpha=0.5)
792
+ ax.text(0.05, 0.95, textstr, transform=ax.transAxes, fontsize=10,
793
+ verticalalignment='top', bbox=props)
794
+
795
+ # Formatting
796
+ ax.set_xlabel('Sum of 10 Dice')
797
+ ax.set_ylabel('Probability')
798
+ ax.set_title('Central Limit Theorem: Dice Game Example')
799
+ ax.legend()
800
+ ax.grid(alpha=0.3)
801
+
802
+ plt.tight_layout()
803
+ plt.gca()
804
+ return fig
805
+ return (create_dice_game_visualization,)
806
+
807
+
808
+ @app.cell(hide_code=True)
809
+ def _(np, plt):
810
+ def create_algorithm_runtime_visualization():
811
+ """Create a visualization for the algorithm runtime example."""
812
+ # Parameters
813
+ variance = 4 # σ² = 4 sec²
814
+ std_dev = np.sqrt(variance) # σ = 2 sec
815
+ confidence_level = 0.95
816
+ z_score = 1.96 # for 95% confidence
817
+ target_error = 0.5 # ±0.5 seconds
818
+
819
+ # Calculate n needed for desired precision
820
+ n_required = int(np.ceil((z_score * std_dev / target_error) ** 2)) # ≈ 62
821
+
822
+ n_values = np.arange(1, 100)
823
+
824
+ # standard error
825
+ standard_errors = std_dev / np.sqrt(n_values)
826
+
827
+ # margin of error
828
+ margins_of_error = z_score * standard_errors
829
+
830
+ # Create figure
831
+ fig, ax = plt.subplots(figsize=(10, 6))
832
+
833
+ # standard error vs sample size plot
834
+ ax.plot(n_values, standard_errors, 'b-', linewidth=2, label='Standard Error of Mean')
835
+
836
+ # Plot margin of error vs sample size
837
+ ax.plot(n_values, margins_of_error, 'r--', linewidth=2,
838
+ label=f'{confidence_level*100}% Margin of Error')
839
+
840
+ ax.axvline(x=n_required, color='green', linestyle='-', linewidth=1.5,
841
+ label=f'Required n = {n_required}')
842
+
843
+ ax.axhline(y=target_error, color='purple', linestyle='--', linewidth=1.5,
844
+ label=f'Target Error = ±{target_error} sec')
845
+
846
+ # Shade the region below target error
847
+ ax.fill_between(n_values, 0, target_error, alpha=0.2, color='green')
848
+
849
+ # intersection point
850
+ ax.plot(n_required, target_error, 'ro', markersize=8)
851
+ ax.annotate(f'({n_required}, {target_error} sec)',
852
+ xy=(n_required, target_error),
853
+ xytext=(n_required + 5, target_error + 0.1),
854
+ arrowprops=dict(facecolor='black', shrink=0.05, width=1))
855
+
856
+ # Text box with appropriate information
857
+ textstr = '\n'.join((
858
+ f'Algorithm Variance: {variance} sec²',
859
+ f'Standard Deviation: {std_dev} sec',
860
+ f'Confidence Level: {confidence_level*100}%',
861
+ f'Z-score: {z_score}',
862
+ f'Target Error: ±{target_error} sec',
863
+ f'Required Sample Size: {n_required}'
864
+ ))
865
+ props = dict(boxstyle='round', facecolor='wheat', alpha=0.5)
866
+ ax.text(0.05, 0.95, textstr, transform=ax.transAxes, fontsize=10,
867
+ verticalalignment='top', bbox=props)
868
+
869
+ # Formatting
870
+ ax.set_xlabel('Sample Size (n)')
871
+ ax.set_ylabel('Error (seconds)')
872
+ ax.set_title('Sample Size Determination for Algorithm Runtime Estimation')
873
+ ax.set_xlim(0, 100)
874
+ ax.set_ylim(0, 2)
875
+ ax.legend()
876
+ ax.grid(alpha=0.3)
877
+
878
+ plt.tight_layout()
879
+ return fig
880
+ return (create_algorithm_runtime_visualization,)
881
+
882
+
883
+ @app.cell(hide_code=True)
884
+ def _(mo):
885
+ mo.md(
886
+ r"""
887
+ ## Summary
888
+
889
+ The Central Limit Theorem is truly one of the most remarkable ideas in all of statistics. It tells us that when we add up many independent random variables, their sum will follow a normal distribution, regardless of what the original distributions looked like. This is why we see normal distributions so often in real life – many natural phenomena are the result of numerous small, independent factors adding up.
890
+
891
+ What makes the CLT so powerful is its universality. Whether we're working with dice rolls, measurement errors, or stock market returns, as long as we have enough independent samples, their average or sum will be approximately normal. For sums, the distribution will be $\mathcal{N}(n\mu, n\sigma^2)$, and for averages, it's $\mathcal{N}(\mu, \frac{\sigma^2}{n})$.
892
+
893
+ The CLT gives us the foundation for confidence intervals, hypothesis testing, and many other statistical tools. Without it, we'd have a much harder time making sense of data when we don't know the underlying population distribution. Just remember that if you're working with discrete distributions, you'll need to apply a continuity correction to get more accurate results.
894
+
895
+ Next time you see a normal distribution in data, think about the Central Limit Theorem – it might be the reason behind that familiar bell curve!
896
+ """
897
+ )
898
+ return
899
+
900
+
901
+ @app.cell(hide_code=True)
902
+ def _(mo):
903
+ # controls for the interactive explorer
904
+ distribution_type = mo.ui.dropdown(
905
+ options=["uniform", "exponential", "binomial", "poisson"],
906
+ value="uniform",
907
+ label="Distribution Type"
908
+ )
909
+
910
+ sample_size = mo.ui.slider(
911
+ start =1,
912
+ stop =100,
913
+ step=1,
914
+ value=30,
915
+ label="Sample Size (n)"
916
+ )
917
+
918
+ sim_count_slider = mo.ui.slider(
919
+ start =100,
920
+ stop =10000,
921
+ step=100,
922
+ value=1000,
923
+ label="Number of Simulations"
924
+ )
925
+
926
+ run_explorer_button = mo.ui.run_button(label="Run Simulation", kind="warn")
927
+
928
+ controls = mo.hstack([
929
+ mo.vstack([distribution_type, sample_size, sim_count_slider]),
930
+ run_explorer_button
931
+ ], justify='space-around')
932
+
933
+ return (
934
+ controls,
935
+ distribution_type,
936
+ run_explorer_button,
937
+ sample_size,
938
+ sim_count_slider,
939
+ )
940
+
941
+
942
+ if __name__ == "__main__":
943
+ app.run()