Chapter 3: Propagation of Error and ANOVA

Bias and Single-Sample Uncertainty

Precision error can be calculated using statistical methods for repeat-sampled data. However, bias error cannot be uncovered using statistical methods. The only direct method is via comparison with measurements made using another more accurate apparatus. However, this is practically impossible. It is much too costly and time consuming. Instead, we will estimate the bias error using experience and knowledge of the device. Then we will obtain an estimate of the precision and accuracy of our experimental apparatus and determine if we must redesign our apparatus to measure a particular quantity of interest.

Bias Error: Measuring Volume in a Cylinder

What measurements do we need to measure the volume of a cylinder?

$$
V = \pi r^2 l
\]

where \( r \) is the radius and \( l \) is the length. Uncertainty originates from the independent variables \( r \) and \( l \). Precision error in these measurements may come from temperature fluctuation, thermal expansion, etc. Bias error comes from the type of measurement device (yard stick vs. micrometer vs. quantum laser).

Propagating Uncertainty

Usually we measure several quantities and use those measurements to calculate a desired quantity. Each measurement has some associated uncertainty which creates uncertainty in the calculated result.

We propagate the uncertainty.

Let \( y \) be a function of independent variables \( x_i \) with standard deviation \( \sigma_i \). Then:

$$
\sigma_y = \sqrt{\left( \frac{\partial y}{\partial x_1} \sigma_1 \right)^2 + \left( \frac{\partial y}{\partial x_2} \sigma_2 \right)^2 + \ldots + \left( \frac{\partial y}{\partial x_n} \sigma_n \right)^2}
\]

Assuming small uncertainties and linearization via Taylor expansion:

$$
y(x_1 + u_1, ..., x_n + u_n) \approx y(x_1, ..., x_n) + \frac{\partial y}{\partial x_1} u_1 + \ldots + \frac{\partial y}{\partial x_n} u_n
\]

So:

$$
u_y = \sqrt{\left( \frac{\partial y}{\partial x_1} u_1 \right)^2 + \ldots + \left( \frac{\partial y}{\partial x_n} u_n \right)^2}
\]

Assumes all uncertainties are independent.

Example: Volume of a Cylinder

Given:
Length = 50 m ± 0.5 m
Radius = 15 m ± 0.1 m

No precision error information, so only bias error applies.

The equation:

$$
V = \pi r^2 l
\]

Then:

$$
B_V = \sqrt{ \left( \frac{\partial V}{\partial r} u_r \right)^2 + \left( \frac{\partial V}{\partial l} u_l \right)^2 }
\]

Where:

$$
\frac{\partial V}{\partial r} = 2 \pi r l, \quad \frac{\partial V}{\partial l} = \pi r^2
\]

Plug in:

$$
B_V = 589.05 \text{ m}^3
\]

Since no precision uncertainty:

$$
U_V = \sqrt{B_V^2 + 0^2 + 0^2} = B_V
\]

Convert to fractional uncertainty:

$$
\frac{B_V}{V} = \sqrt{ \left( \frac{2 u_r}{r} \right)^2 + \left( \frac{u_l}{l} \right)^2 } = 0.01667
\]

Percent = 1.667%.
Nominal volume = 35343 m³
Check: 35343 × 0.01667 = 589.05 m³

Example: Uniaxial Tension Test

Given:
- Force: mean 48.9 N, std dev 3.24, bias ±1 N
- Width: mean 0.1 m, std dev 0.01
- Length: mean 0.5 m, std dev 0.02
- Micrometer used for L and W
- 14 measurements of each

Equation:

$$
\sigma = \frac{F}{l \cdot w}
\]

Precision errors:

$$
P_F = t_{0.025,13} \frac{S_F}{\sqrt{n}}, \quad \frac{P_F}{F} = 0.04 \\
P_l = t_{0.025,13} \frac{S_l}{\sqrt{n}}, \quad \frac{P_l}{l} = 0.023 \\
P_w = t_{0.025,13} \frac{S_w}{\sqrt{n}}, \quad \frac{P_w}{w} = 0.06
\]

Bias uncertainty in fractional form:

$$
\frac{B_\sigma}{\sigma} = \sqrt{ \left( \frac{u_F}{F} \right)^2 + \left( \frac{u_l}{l} \right)^2 + \left( \frac{u_w}{w} \right)^2 } = 0.02
\]

Total uncertainty in percent:

$$
U_\sigma = \sqrt{ 0.02^2 + 0.04^2 + 0.023^2 + 0.06^2 } \cdot 100 = 7.93\%
$$

Suggested Procedure for Calculating Total Uncertainty

1. Determine if bias, precision, or both are involved
2. Write the equation and identify independent variables
3. List known values: means, uncertainties, sample size
4. Start with precision error and convert to percent/fraction
5. Take partial derivatives and compute bias error
6. Combine to compute total uncertainty

Minimizing Error in Designing Experiments

Best time to minimize uncertainty is during design:

- Avoid subtracting large similar numbers
- Amplify signal strength
- Use null designs
- Avoid large correction factors
- Minimize sensor influence
- Calibrate the system

When in doubt about a data point, temporarily exclude it and refit. If the fit improves, the point may be an outlier — or it could indicate new behavior.

ANOVA

You may have heard of ANOVA analysis in some engineering or even more likely in the biological context. ANOVA is analysis of variance and it is an extremely useful statistical analytical tool.

ANOVA is particularly useful in experimental designs where there are multiple variables being varied simultaneously to see if they will have an effect on the measured outcome of the experiment. For example, perhaps I am creating a new high strength alloy and I want to design the new alloy material to maximize the ultimate tensile strength. I can vary a number of parameters when processing this material such as annealing time, solute concentration, amount of work hardening, and many more parameters. One question you may want to answer is: do any of these variables have a statistically significant effect on the ultimate tensile strength? Which parameter has the largest effect? Are there factors that have a combined effect?

Some of these questions we can answer using the tools we know about but others are not as easy to answer. So, before we jump into the deep end so to speak, let's start with developing a solid foundation and build a framework to analyze a single variable and perform a One-Way ANOVA.

One-Way ANOVA

Let us consider that we run an experiment with \( k \) independent random samples from \( k \) different populations and there will be a number of observations of measurands \( y \), and we can classify them as \( y_{ij} \) where \( i \) varies from 1 to \( k \), and \( j \) indicates the number of observations. Therefore, we will have \( k \) sample means \( \overline{y}_k \).

Now in this analysis, there are several critical values that we will need to calculate. The first of which is the total sum of all observations:

$$
T = \sum_{i=1}^{k} \sum_{j=1}^{n_i} y_{ij}
\]

where \( n_i \) is the number of observations in group \( i \). The total sample size is:

$$
N = \sum_{i=1}^{k} n_i
\]

The overall sample mean is:

$$
\overline{y} = \frac{T}{N}
\]

The treatment sum of squares is:

$$
SS_a = \sum_{i=1}^{k} n_i(\overline{y}_i - \overline{y})^2
\]

The error sum of squares is:

$$
ESS_a = \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \overline{y}_i)^2
\]

The degrees of freedom are:

$$
df_a = \sum_{i=1}^{k} n_i - 1 = k - 1 + \sum_{i=1}^{k} n_i - k
\]

The mean square is defined as:

$$
\frac{SS_a}{df_a}
\]

The F-ratio for treatments is:

$$
F_a = \frac{ \frac{SS_a}{k - 1} }{ \frac{ESS_a}{N - k} }
\]

The F-ratio is a key parameter in performing ANOVA analysis. A larger F value indicates large differences between sample means and thus large differences between groups.

We will also obtain a P-value from this analysis.

All of these values appear in the standard ANOVA table. Let’s examine an example analyzing crop yields as a function of fertilizer type.

Here we can see the P-value is less than 0.05. Therefore, we can conclude that fertilizer type has a statistically significant effect on the crop yield.

Often we want to go beyond the ANOVA table and perform a post-hoc test. This lets us identify differences between specific group pairs. One common method is the Tukey post-hoc test, but it assumes equal variances. To be more general, we will use a Games-Howell post-hoc test.

Looking at the P-values, there is no statistically significant difference between fertilizer types 1 and 2. However, there are significant differences between types 1 and 3 and between types 2 and 3. This insight can guide further experiments and optimization.

Another valuable metric is the effect size. We can quantify this using \( \eta^2 \), calculated as:

$$
\eta^2 = \frac{SS_a}{SS_a + SS_{error}}
\]

where \( SS_a \) is the treatment sum of squares. Interpretation:

- \( \eta^2 \geq 0.14 \): large effect
- \( 0.06 \leq \eta^2 < 0.14 \): medium effect
- \( \eta^2 < 0.01 \): small effect

This confirms that fertilizer has a large effect on crop yield.

Two-Way ANOVA

We can also run a Two-Way ANOVA. The total sum of squares can be expressed as:

$$
\sum_{i=1}^a \sum_{j=1}^b (y_{ij} - \overline{y}_{..})^2 = \sum_{i=1}^a \sum_{j=1}^b (y_{ij} - \overline{y}_{i.} - \overline{y}_{.j} + \overline{y}_{..})^2 + b \sum_{i=1}^a (\overline{y}_{i.} - \overline{y}_{..})^2 + a \sum_{j=1}^b (\overline{y}_{.j} - \overline{y}_{..})^2
\]

We define:

- Treatment sum of squares:

$$
SS = b \sum_{i=1}^a (\overline{y}_{i.} - \overline{y}_{..})^2
\]

- Block sum of squares:

$$
BSS = a \sum_{j=1}^b (\overline{y}_{.j} - \overline{y}_{..})^2
\]

- Error sum of squares:

$$
ESS = \sum_{i=1}^a \sum_{j=1}^b (y_{ij} - \overline{y}_{i.} - \overline{y}_{.j} + \overline{y}_{..})^2
\]

- Total sum of squares:

$$
TSS = \sum_{i=1}^a \sum_{j=1}^b (y_{ij} - \overline{y}_{..})^2
\]

Degrees of freedom:

$$
df = ab - a = (a - 1) + (b - 1) + (a - 1)(b - 1)
\]

F-ratios:

$$
F_a = \frac{ \frac{TSS}{a - 1} }{ \frac{ESS}{(a - 1)(b - 1)} }, \quad
F_b = \frac{ \frac{BSS}{b - 1} }{ \frac{ESS}{(a - 1)(b - 1)} }
\]

Now let’s look at a real example.

Here we see fertilizer and crop density both have statistically significant effects on yield. The interaction between fertilizer and density does not.

Effect sizes:

- Fertilizer: large
- Density: medium
- Interaction: small

A post-hoc Games-Howell test confirms the fertilizer differences and shows that density types 1 and 2 are also significantly different.

Three-Way ANOVA

We can extend this to three independent variables.

This shows:

- Fertilizer and density have statistically significant effects
- Block location does not
- Interaction between density and fertilizer is not significant
- Interaction between density and block is significant
- Three-way interaction is significant

Effect sizes:

- Fertilizer: large
- Density: medium
- Block: small
- Density and fertilizer: small
- Fertilizer and block: small

Only blocks 2 and 3 show a statistically significant difference.

ANOVA is an incredibly useful tool.