# Chapter 10: Fitting, Regressions, and Confidence Bands

- Page ID
- 99734

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Here you can find a series of pre-recorded lecture videos that cover this content: https://youtube.com/playlist?list=PLSKNWIzCsmjBxk5lvZ_LaUX2hBGSEaxpV&si=nOGeAaWzNwuNlz5X

As we touched on at the beginning of this course and have emphasized throughout is visualizing and plotting data and analyzing trend to find statistically significant results.

Often this may involve fitting data which we have only briefly touched upon up onto this point with the exception of our \(\chi^2\)^{ }goodness of fit test which can indeed be utilized to analyze distribution shapes and fits.

One of the most common methods to analyze fits and curves is with **regression analysis **and one of the most popular techniques is regression analysis via the **method of least squares**.

**Method of Least Squares**

In the method of least squares we will be working with two variables *x*, which will be considered the **independent **or **input **variable and *y*, the **dependent **or **response **variable. We will assume that the total uncertainty in *y *will be much greater than *x *which in most experimental conditions is a fairly reasonable assumption and we will start by analyzing the regression curve of *y *on *x *as a linear regression curve

\begin{equation}

y = a +bx

\end{equation}

where *a *and *b *are fitting constants. Then the error *e _{i }*in predicting a

*y*value will be given by

\begin{equation}

e_i = y_i - y(x_i)

\end{equation}

We will thereby select *a *and *b *such that \(\sum_{i=1}^n e_i\) is as close to 0 as possible to minimize the error.

So the principle of least squares will give us

\begin{equation}

\sum_{i=1}^n e_i = \sum_{i=1}^n [y_{i} - y(x_{i})]^{2}

\end{equation}

Before we can proceed with this process it will be extremely convenient here to define some critical values specifically

\begin{equation}

S_{xx} = \sum_{i=1}^n (x_i-\overline{x})^2

\end{equation}

and similarly

\begin{equation}

S_{yy} = \sum_{i=1}^n (y_i-\overline{y})^2

\end{equation}

and

\begin{equation}

S_{xy} = \sum_{i=1}^n (x_i-\overline{x})(y_i-\overline{y})

\end{equation}

Then we can calculate the error sum of squares which we seek to minimize as

\begin{equation}

SSE = S_{yy} - \frac{S_{xy}^2}{S_{xx}}

\end{equation}

This calculation will give us fitting parameters but we can also obtain estimates of the confidence intervals of those fitting parameters as well.

Let’s look at some plots of raw data and then we can perform some linear fits

Here we can see some example of raw data and we can now proceed to attempt to fit the data based on functions that we believe will best fit the values. We set the function and then proceed to allow the program to run through the above calculations to determine the fitting parameters in the linear function

We can see the fits here and we can also plot the 95% confidence interval for these fits as well as seen here

__Non-Linear Fitting:__

We can extend these tools to also perform non-linear fits that can range from exponential expressions, logarithmic expressions, polynomials, and beyond.

Here we can see some nonlinear fits of the previous data set

__Non-Linear Fitting:__

We can extend these tools to also perform non-linear fits that can range from exponential expressions, logarithmic expressions, polynomials, and beyond.

Here we can see some nonlinear fits of the previous data set

__Correlation__

So far we have assumed that the total uncertainty in *x *was extremely small and close to 0, while this holds for my real life scenarios there are many many many scenarios where this is not the case in which not true.

In which case the purpose of plotting *x *vs *y *in the most general sense is to observe some type of **correlation **between the variables. One can quantitatively calculate this via the **sample correlation coefficient ***r *which is defined as

\begin{equation}

r= \frac{S_{xy}}{\sqrt{S_{xx} \cdot S_{yy}}}

\end{equation}

here if *r *= 1 we have perfect positive correlation if *r *= −1 we have perfect negative correlation, and if *r *= 0 we have no correlation, i.e. there is no linear association. Additionally the limits of *r *is between −1 and 1.

We can also calculate our *r*^{2 }value which varies from 0-1 and is defined as

\begin{equation}

r^2 = \frac{S_{xy}^2}{S_{xx}S_{yy}}

\end{equation}

We can see these values tabulated here for our previous fits to show correlations, both *r *and *r*^{2 }values.

Looking at our previous fits we can run regression analysis to investigate the correlations as well.

**Critical Consideration When Fitting**

Regardless of the statistical tool and measure that you are using when fitting value be sure that you do not lose the physical intuition of the system that you are investigating. There are very very few physical systems that can be represented by 12th order polynomial function despite the fact that that particular function may yield the strongest correlation factor. Instead, a more physically intuitive function that yields a lower correlation coefficient may be more appropriate and one may spend more time analyzing why this experimental system deviated from this theoretical function/representation.