Summarizing Process Data

Summarizing process data is simply boiling a long list of readings down to a few numbers that describe it fairly. You already know most of the tools from descriptive statistics; the new part is choosing the right one for production data. The formulas are small: $\text{CV}$ (the spread as a percent of the average), $\bar R/d_2$ (a quick way to get σ from the ranges of small groups), the percentile rule (sort, then interpolate between neighbors) and the fences with $\text{IQR}/1.349$ (an outlier screen and a spread estimate that a glitch can't fool). Four small tools, each with one job.

A column of numbers from the line. A filler puts 500 g of sauce in each jar, and the checkweigher logs twelve jars: 501.2, 499.8, 500.6, 498.9, 500.3, 501.0, 499.5, 50.04, 500.9, 499.2, 500.1 and 500.7 g. One reading is obviously a typo (500.4 keyed in as 50.04), but a spreadsheet doesn't know that. The mean comes out 462.7 g and the standard deviation 130 g, a disaster on paper. The median, 500.2 g, and the IQR, 1.45 g, barely notice. The center and spread lessons define these numbers; here's how engineers choose among them.

The summaries and their jobs:

• Mean and SD are the right summaries for a stable, roughly symmetric process, and they feed every normal-model calculation (ppm out of spec, capability). One wild value ruins both.
• Median and IQR are resistant. Use them for skewed data, or whenever glitches are possible, which in production data means always.
• Range (max − min) is the quickest spread. For small subgroups from a normal process the average range is a fixed multiple of σ, $E(R) = d_2\sigma$, so $\hat\sigma = \bar R/d_2$. That's how X̄ and R charts estimate σ on the shop floor. For one big sample the range is wasteful: it uses only two of the values.
• The coefficient of variation compares spread relative to size. A 250 g jar line with $s = 1.5$ g (CV 0.6%) is less consistent than a 1,000 g line with $s = 4.0$ g (CV 0.4%). Use it only for quantities with a true zero, like mass, length or resistance, never for °C.
• Percentiles describe tails, and specs live in the tails: "95% of valve closures under 120 ms", "the 10th percentile of weld strength". Sort, find the position $1 + p(n - 1)$ and interpolate. Spreadsheets do exactly this. For the quartiles of a small sample it can differ slightly from the median-of-halves method in the box plot lesson; both are fine, so say which one you used.

Fences, and why z-scores make poor outlier detectors. The fences $Q_1 - 1.5\,\text{IQR}$ and $Q_3 + 1.5\,\text{IQR}$ flag the typo instantly. A rule like "flag $|z| > 3$" sounds just as good, but the outlier inflates $s$, which shrinks its own z-score. Push that to the limit: in a sample of $n$, let $n - 1$ values equal $0$ and one equal $a$. Then $\bar x = a/n$, the odd value sits $a(n - 1)/n$ above the mean, and

So the odd value's z-score is $\frac{a(n - 1)/n}{a/\sqrt n} = \frac{n - 1}{\sqrt n}$, no matter how huge $a$ is. For $n = 10$ that's 2.85, and no sample of 10 can do better, so a $|z| > 3$ rule can never fire. The outlier hides behind the spread it creates; this is called masking. Fences are built from quartiles, which one outlier can't drag.

A robust SD. For normal data $Q_1$ and $Q_3$ sit at $\mu \pm 0.6745\sigma$ (see the normal distribution), so $\text{IQR} = 1.349\sigma$, and $\text{IQR}/1.349$ estimates σ even with a glitch in the data. For the jars that's $1.45/1.349 \approx 1.07$ g, a sane number next to the absurd 130 g.

Look at the order, too. A summary throws away time. A filler that drifts upward through the shift and one that's steady but noisier can have the same mean and SD. Plot readings in time order before trusting any summary; that habit grows into statistical process control.

A common mistake. Once a value is flagged, it's tempting to delete it and move on. Don't delete outliers on sight. A flagged value is a question. A typo gets fixed; a sensor glitch gets documented and excluded; a real event, like a clogged nozzle, may be the most valuable data point you have.

Means and SDs tell the whole story only when the data are close to normal, and the next lesson, probability plots, checks that. Gauge noise from measurement error is part of every SD you compute here.

Certificates of analysis that ship with materials report the mean, SD, minimum and maximum of each property; packaging rules in many countries require the average fill to meet the label while limiting how many packages fall below a tolerable shortfall, which is a statement about a mean and a low percentile at once. Monitoring software screens sensor streams with IQR fences and robust SDs, because a single dropout reading would wreck a mean-and-SD rule.