2.3: Precision, significant digits and errors

When designing new systems, it is essential to be sure that they satisfy the required specifications. Therefore, when working with numerical values, it is essential to be aware of the precision of these numbers. Neglecting the consideration of the precision can lead to big problems, like parts of machines that do not fit together. Uncertainties in quantities with respect to the average value can occur both due to actual variations (in time) in the quantity that is measured and due to uncertainties in the measurement technique that is used to determine the quantity.

2.3.1 Precision and significant digits

The most common method to determine the uncertainty in a quantity is to measure the quantity \(q\) a large number of times and determine the average value \(q_{\text {avg }}\) and standard deviation \(\sigma_{q}\). For example we measure the diameter of a 1 euro cent coin \(N\) times with a calliper, analyse the results and obtain \(d=d_{\text {avg }} \pm \sigma_{d}=(16.20 \pm 0.05) \mathrm{mm}\). The standard deviation \(\sigma_{d}\) is a measure of the uncertainty (imprecision) in a single measurement \({ }^{1}\) of the value of the coin diameter \(d\). If many different euro cent coins are measured variations between the euro cent coins can also increase the uncertainty, thus both measurement uncertainty and object variations can contribute to the total uncertainty. Assuming a Gaussian distribution of the measurement values, the value of the standard deviation gives a probability of \(68 \%\) that the diameter of the coin is between \(16.15 \mathrm{~mm}\) and \(16.25 \mathrm{~mm}\).

Significant digits (also called significant figures) are an approximate way to indicate these kinds of uncertainties in a value. Since adding or removing one significant digit changes the precision of the value by a factor 10 , it is a quite rough indicator of the precision. Nevertheless, even this rough estimate can protect you from making grave mistakes. For instance, you buy a piece of wood with a width of \(w=100 \mathrm{~cm}\) to fix a door in your house. When installing it you notice that it is only \(99.6 \mathrm{~cm}\) such that you have an ugly gap of \(0.4 \mathrm{~cm}\) (or you might find the door doesn’t close because \(w=100.4 \mathrm{~cm}\) ). If you go back to the salesman you might not get your money back, since \(100 \mathrm{~cm}\) means that the width can have any value between \(99.5 \mathrm{~cm}\) and \(100.5 \mathrm{~cm}\), instead you should have bought a piece of wood with a width of \(w=100.0 \mathrm{~cm}\).

In the example of the euro cent coin, the diameter of the coin would be indicated as \(d=16.2 \mathrm{~mm}\). The last digit indicates the uncertainty, showing that with reasonable certainty the value will be between \(16.3 \mathrm{~mm}\) and 16.1 \(\mathrm{mm}\). Alternatively you can continue working with the full expression ( \(16.20 \pm\) \(0.05) \mathrm{mm}\), and use error analysis to determine the error and precision. These methods also can be used to determine the error propagation, for instance in case a quantity \(q\) is determined from \(d\) by a function \(q=f(d)\), an error in \(d\) will also result in an error in \(q\). If we determine the area of the euro cent coin using the equation \(A(d)=f(d)=\frac{1}{4} \pi d^{2}\) we find, using the Taylor expansion for small errors \(\sigma_{d}\), that \(A=f\left(d_{\text {avg }} \pm \sigma_{d}\right) \approx f\left(d_{\text {avg }}\right) \pm \sigma_{d} \frac{\partial f}{\partial d}=\frac{1}{4} \pi\left(d_{\text {avg }}^{2} \pm 2 d_{\text {avg }} \sigma_{d}\right)\), such that the error (standard deviation) in \(A\) can be determined by: \(\sigma_{A} \approx \sigma_{d} \frac{\partial f}{\partial d}\). However, although very important, further discussion of these methods is out of the scope of this textbook.

When reporting measured quantities in engineering, the number of significant digits should always be estimated and reported \({ }^{2}\). In case estimating precision is difficult, a conservative estimate of the number of significant digits is recommended. Significant digits can sometimes only be properly indicated using scientific notation, which is notation of a number by a single digit before the decimal point and an appropriate exponent. For example, if there are two significant digits, a mass should be written as \(1.2 \times 10^{2} \mathrm{~kg}\) and not as 120 \(\mathrm{kg}\) (which indicates three significant digits). The most important point to note is that trailing zeros, like in 1.000 , right of the last non-zero digit are all significant, while the leading zeros like in 0.001 , left of the first non-zero digit, are not significant. Some examples:
- \(1.2 \times 10^{2} \mathrm{~kg} \neq 120 \mathrm{~kg}\)
- \(2.000 \mathrm{~kg}=2000 \mathrm{~g}=2.000 \times 10^{3} \mathrm{~g}\)
- \(1.8 \times 10^{-3} \mathrm{~m}=0.0018 \mathrm{~m}=1.8 \mathrm{~mm}\)

In the first case the quantity on the left has two significant digits, while the quantity on the right has three significant digits, because the zero is a trailing zero, just like the zeros in the second case. In the third case, the zeros in 0.0018 \(\mathrm{m}\) are not significant, because they are leading zeros, such that there are only two significant digits.

2.3.2 Making calculations with significant digits

When making calculations based on quantities with significant digits, the number of significant digits can change. If the number of significant digits reduces, the value is calculated and rounded to the nearest number with the determined number of significant digits. The following rules are most important to determine the number of significant digits of a calculated value:

• There is no uncertainty in countable, integer numbers, or mathematical quantities like \(\pi\). They should be treated as having an infinite number of significant digits.
• To facilitate working with significant digits, write the number in scientific notation to remove all zeros on the left, but keep all trailing zeros on the right of the value: \(d=0.0162 \mathrm{~m}=1.62 \times 10^{-2} \mathrm{~m}\). Then count the number of significant digits, in this case 3.
• In multiplication or divisions, the resulting number of significant digits is equal to the figure with the fewest significant digits. E.g. in \(F=m \cdot a=\) \(1.0 \mathrm{~kg} \times 3.124 \mathrm{~m} / \mathrm{s}^{2}=3.1 \mathrm{~N}\), the resulting number of significant digits is two, equal to that of the mass, which has the lowest number of significant digits.

• In additions or subtractions, the last significant digit of the calculated result should be equal to the last (smallest) significant digit of the least
  significant quantity from which it is determined. E.g. \(\Delta x=x_{2}-x_{1}=\) \(2.156 \mathrm{~m}-2.13 \mathrm{~m}=0.03 \mathrm{~m}\). The result of the calculation is \(0.026 \mathrm{~m}\), but the last significant digit in \(2.13 \mathrm{~m}\) (the 3 ) is at two positions right of the decimal point, such that only 1 significant digit remains in the final rounded result.

• When applying functions to quantities, like \(\tan \frac{x_{1}}{x_{2}}=\tan (0.988)=1.52\), the number of significant digits is kept the same as the argument. Note that his rule, although often sufficient, is not always completely accurate, e.g. when working with power functions. More advanced methods are outside the scope of this textbook.
• Only determine significant digits at the end of a calculation. For intermediate results in the calculation one should keep a sufficiently large number of digits to prevent additional imprecision due to rounding.

S Example 2.6 Which of these expressions make correct use of significant digits and units, for mass \(m\), time \(t\), force \(F\), angular velocity \(\omega\), and angle \(\theta\) ?
A. \(\quad F=0.0020 \mathrm{~kg} \cdot 300 \mathrm{~m} / \mathrm{s}^{2}=0.60 \mathrm{~N}\)
B. \(\quad t=0.10 \mathrm{~s}-1.0 \mathrm{~ms}=99 \mathrm{~ms}\)
C. \(\quad \theta=\pi \times 10^{-3.0 \mathrm{~ms} /(1.0 \mathrm{~ms})} \mathrm{rad}=3.1 \times 10^{-3} \mathrm{rad}\)

Exemplary solution

\(F=0.60 \mathrm{~N}\)

\(t \neq 99 \mathrm{~ms}\)

\(\theta=3.1 \times 10^{-3} \mathrm{rad}\) correct! 2 significant digits are kept in multiplication.

\(t=0.10 \mathrm{~s}\). Most significant figure in first value is \(10 \mathrm{~ms}\), same should hold for final result after subtraction.

correct! \(10^{-3.0} \approx 1.0 \times 10^{-3}\).