Skip to main content
Engineering LibreTexts

Chapter 10: Fitting, Regressions, and Confidence Bands

  • Page ID
    99734
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

    ( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\id}{\mathrm{id}}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\kernel}{\mathrm{null}\,}\)

    \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\)

    \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\)

    \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    \( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

    \( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

    \( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vectorC}[1]{\textbf{#1}} \)

    \( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

    \( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

    \( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)

    Here you can find a series of pre-recorded lecture videos that cover this content: https://youtube.com/playlist?list=PLSKNWIzCsmjBxk5lvZ_LaUX2hBGSEaxpV&si=nOGeAaWzNwuNlz5X

    As we touched on at the beginning of this course and have emphasized throughout is visualizing and plotting data and analyzing trend to find statistically significant results.

    Often this may involve fitting data which we have only briefly touched upon up onto this point with the exception of our \(\chi^2\) goodness of fit test which can indeed be utilized to analyze distribution shapes and fits.

    One of the most common methods to analyze fits and curves is with regression analysis and one of the most popular techniques is regression analysis via the method of least squares.

    Method of Least Squares

    In the method of least squares we will be working with two variables x, which will be considered the independent or input variable and y, the dependent or response variable. We will assume that the total uncertainty in y will be much greater than x which in most experimental conditions is a fairly reasonable assumption and we will start by analyzing the regression curve of y on x as a linear regression curve

    \begin{equation}
    y = a +bx
    \end{equation}

    where a and b are fitting constants. Then the error ei in predicting a y value will be given by

    \begin{equation}
    e_i = y_i - y(x_i)
    \end{equation}

    We will thereby select a and b such that \(\sum_{i=1}^n e_i\) is as close to 0 as possible to minimize the error.

    So the principle of least squares will give us

    \begin{equation}
    \sum_{i=1}^n e_i = \sum_{i=1}^n [y_{i} - y(x_{i})]^{2}
    \end{equation}

    Before we can proceed with this process it will be extremely convenient here to define some critical values specifically

    \begin{equation}
    S_{xx} = \sum_{i=1}^n (x_i-\overline{x})^2
    \end{equation}

    and similarly

    \begin{equation}
    S_{yy} = \sum_{i=1}^n (y_i-\overline{y})^2
    \end{equation}

    and

    \begin{equation}
    S_{xy} = \sum_{i=1}^n (x_i-\overline{x})(y_i-\overline{y})
    \end{equation}

    Then we can calculate the error sum of squares which we seek to minimize as

    \begin{equation}
    SSE = S_{yy} - \frac{S_{xy}^2}{S_{xx}}
    \end{equation}

    This calculation will give us fitting parameters but we can also obtain estimates of the confidence intervals of those fitting parameters as well.

    Let’s look at some plots of raw data and then we can perform some linear fits

    clipboard_e60f7c438374fa0bc57f5f374d7c8ef63.png

    Figure \(\PageIndex{1}\): R Raw Data Plots.

    Here we can see some example of raw data and we can now proceed to attempt to fit the data based on functions that we believe will best fit the values. We set the function and then proceed to allow the program to run through the above calculations to determine the fitting parameters in the linear function

    We can see the fits here and we can also plot the 95% confidence interval for these fits as well as seen here

    Non-Linear Fitting:

    We can extend these tools to also perform non-linear fits that can range from exponential expressions, logarithmic expressions, polynomials, and beyond.

    Here we can see some nonlinear fits of the previous data set

    clipboard_e355110c9342821d4531ee190ea22cf4f.png

    Figure \(\PageIndex{2}\): R Linear Fit.

    clipboard_e6337c7afa640fc6cfe6819f8eb1d9b7c.png

    Figure \(\PageIndex{3}\): R 95% Confidence Interval Fit.

    Non-Linear Fitting:

    We can extend these tools to also perform non-linear fits that can range from exponential expressions, logarithmic expressions, polynomials, and beyond.

    Here we can see some nonlinear fits of the previous data set

    clipboard_e355110c9342821d4531ee190ea22cf4f.png

    Figure \(\PageIndex{2}\): R Linear Fit.

    clipboard_e6337c7afa640fc6cfe6819f8eb1d9b7c.png

    Figure \(\PageIndex{3}\): R 95% Confidence Interval Fit.

    Correlation

    So far we have assumed that the total uncertainty in x was extremely small and close to 0, while this holds for my real life scenarios there are many many many scenarios where this is not the case in which not true.

    In which case the purpose of plotting x vs y in the most general sense is to observe some type of correlation between the variables. One can quantitatively calculate this via the sample correlation coefficient r which is defined as

    \begin{equation}
    r= \frac{S_{xy}}{\sqrt{S_{xx} \cdot S_{yy}}}
    \end{equation}

    here if r = 1 we have perfect positive correlation if r = −1 we have perfect negative correlation, and if r = 0 we have no correlation, i.e. there is no linear association. Additionally the limits of r is between −1 and 1.

    We can also calculate our r2 value which varies from 0-1 and is defined as

    \begin{equation}
    r^2 = \frac{S_{xy}^2}{S_{xx}S_{yy}}
    \end{equation}

    We can see these values tabulated here for our previous fits to show correlations, both r and r2 values.

    Looking at our previous fits we can run regression analysis to investigate the correlations as well.

    Critical Consideration When Fitting

    Regardless of the statistical tool and measure that you are using when fitting value be sure that you do not lose the physical intuition of the system that you are investigating. There are very very few physical systems that can be represented by 12th order polynomial function despite the fact that that particular function may yield the strongest correlation factor. Instead, a more physically intuitive function that yields a lower correlation coefficient may be more appropriate and one may spend more time analyzing why this experimental system deviated from this theoretical function/representation.


    Chapter 10: Fitting, Regressions, and Confidence Bands is shared under a CC BY-NC-SA 4.0 license and was authored, remixed, and/or curated by LibreTexts.

    • Was this article helpful?