Distributions and Basic Statistics
Before modeling, look at the shape of the data. These posts cover the tools for that: histogram bin rules, kernel density estimates and their bandwidth, when the median describes a column better than the mean, and what a normality test can and cannot tell you at different sample sizes.
- How to check if data is normally distributed in Python
Q-Q plots, Shapiro-Wilk and D'Agostino tests in SciPy, and why the same near-normal data passes at n=300 but fails at n=3,000. Tested with SciPy 1.18.
- How can you explore histograms and KDE interactively in Python?
A small pygame app to change bins and KDE bandwidth on the same sample and watch what happens. Three hands-on experiments: 5 vs 200 vs auto bins, bandwidth 0.05 vs Scott vs 0.8, and a KDE that leaks below zero on positive data.
- Mean or median: which should you report for skewed data?
When the mean and median disagree, which one tells the truth? Three quick NumPy experiments: skewed salaries, one extreme value, and two groups where the mean describes almost nobody. Tested with NumPy 2.5 and SciPy 1.18.
- Histogram vs KDE: how do you choose the bandwidth?
A KDE is a smooth histogram, and its bandwidth works like the bin width. Tested with SciPy: what Scott's rule gets right, how a KDE leaks below zero, and when a histogram shows more.
- How many bins should a histogram have? Sturges, √n and Freedman–Diaconis compared
For 1,000 normal values the common rules give 11 (Sturges), 20 (Rice, Scott), 27 (Freedman–Diaconis, NumPy auto) or 32 (square root) bins. Start with NumPy's auto rule, then check one coarser and one finer setting.
- What is a histogram?
What a histogram shows, how to read its shape (symmetric, skewed, bimodal), how it differs from a bar chart, and when to use something else. Includes a short Python example with real output.
