18.1: Analysis Methods

$$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$$ $$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$$$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}[1]{\| #1 \|}$$ $$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}[1]{\| #1 \|}$$ $$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$

In engineering and science, analysis is important for proper design. Most analysis and modeling techniques are covered in numerical methods which has been sacrificed in a number of programs for signals and systems. Some programs include numerical methods by including a section or two in signals and systems and various programming courses. It is important for a student to make sure they have some background in numerical methods. Here we will review some essentials ideas in numerical methods in computer science. This will not be a complete picture.

Signals and Systems and Numerical Methods

A system is a process that can accomplish a task or is a process that by nature accomplishes a task. The term system is so generic it can be used in the sciences, engineering, social, computer science, music, etc. In biology for instance we have a digestive system that takes in food and processes to output materials the body needs and waste, in computer system you have system calls that input a command you wish the computer system to do and outputs the action of the computers system (which is a system in itself), in engineering you may have a LED which you input a potential and the LED outputs light, and so on. Think of all the terms where system is used...a government system, a railroad system, a system of stars, a musical system (staves), natural system, social system, etc. This makes a consistent definition of system a little problematic, however we will try to define it to fit more of a science and engineering purpose.

A system is a process that has an input and has an output. You can see that in the examples above. In engineering we add another term to this complicated definition and that is signal. Therefore, a system is a process that has an input signal or signals and an output signal or signals. The output signal is usually different but in certain instances and be the same as the input. A subset of this is a control system that controls or regulates a system. This now leaves the question of what is a signal?

A signal is the representation of a physical "wave." That is normally expressed as a variable in time-space, for instance x(t). Examples of signals might be the voltage or current of an electrical circuit, the force in a mechanical circuit, heat flow in a thermal circuit, hydraulic flow in a fluid circuit, biological mass flow in a digestive circuit, etc. This definition gives us a basis to move to a more mathematical approach to systems and signals which can lead to computational methods.

A system can be expressed mathematically as a transfer function or an operator. A simple example of an operator could be an integrator that takes a signal and integrates it over time or it could be as simple as multiplying the signal by a constant. Transfer functions are specific to engineering and have the notation of a Laplace transform, where as operators are more specific to physics (and other sciences) and have the notation of mathematics.

How we express these ideas tangible is through different mathematical techniques or methods. The system methods that are used are generally the same methods that a student would learn in linear algebra. The signal methods that are used are generally the ones learned in numerical methods. This makes both methods important in engineering so we will give a quick bread crumb run down of the various methods starting with numerical methods.

Numerical Methods

Technically we already did one type of numerical methods in parachute person. This method is the Euler's method for numerically solving differential equations. There are better methods to numerical solving differential equation but it the most intuitive.

¬ What type of problems do engineers/scientists solve
• We solve problem of balance, like balancing forces or balancing voltages (Kirchhoff's voltage law)
• Any balanced equation can be written such that the function equals zero
• These are problems of finding a root or roots
• Root solving methods
• We solve optimization problems like minimum energy, minimum volume, or maximized efficiency
• Mathematically this requires taking a derivative and solving the resulting function when it is set to zero
• Solving for a system set to zero is a root problem so root solving methods and optimization methods are related
• Though related methods they are not the same because of slightly different goals
• Optimization methods
¬ What other tasks do engineers/scientists perform on a routine basis
• Analysis for experimental data
• Fitting experimental data to a curve (see sections in laboratory chapter)
• Filtering experimental data or filtering signals (which produces the same results as fitting)
• Fourier analysis (or other related analysis)
• Fourier fast transform (FFT)
• Wavelets
• Filter methods (or fitting methods)
• Transform methods (operators)
• Simulation
• Circuit analysis
• Finite difference analysis
• Finite element analysis
• Finite volume analysis
• Monte Carlo analysis
• "Molecular" dynamic analysis
• Simulation methods

Conservation Laws

The idea of balanced equations comes to us from the conservation laws of physics. Here is a brief list of the laws with notes.

From a theorem by Emmy Noether1 -- Symmetry implies conservation (breaking symmetry is a phase change)

• Conservation of Energy
• Historically promoted by engineers over the objections of physicists
• First Law of Thermodynamics – Chemical Engineering
• Voltage Balance – Electrical Engineering
• Invariant under time translation
• Conservation of linear momentum (or just momentum)
• Force balance -- Civil and Mechanical Engineering
• No external force is required
• Invariant under translation
• Conservation of angular momentum
• Angular analogy to linear momentum – Civil and Mechanical Engineering
• No external torque is required
• Invariant under rotations
• Conservation of electric charge
• Current balance – Electrical Engineering
• Charge in a note must eventually leave that node
• Conservation of mass
• Mass balance equation – Chemical Engineering
• Mass into a system must equal the mass out of a system plus any accumulated mass in the system
• Conservation fails in certain instances if the equivalence of energy and mass is not considered
• Does not fail in reality...just in our understanding of what is going on

From these laws we get a number of the laws (Kirchhoff's law, etc.) that we have already discussed in this course/book. The equations produced here can then be programmed into a computer and solved for the problem you wish to solve. Of for that more advance languages some of the laws maybe already incorporated into the language itself (like say SPICE for circuits).

Finding the roots of an equation

• Bracketing methods
• Converges but can go really slow in certain instances
• Need to points to bound the root
• Bisection method
• Similar to what you do when someone says guess a number between 1 and 50
• Just keep dividing by 2, right?
• Regula Falsi method
• Improves bisection method by speeding it up
• The speed up can result in issues itself
• Open methods
• Finds solutions fast but may not converge
• Only need one point
• Fixed-point iteration method
• Isn't used by itself but maybe combined with another method
• Newton-Raphson method (Newton method)
• Most popular method
• Need derivative of function
• Uses Taylor's series expansion
• Secant method
• Similar to Newton method
• Uses two points so it does not require a derivative like Newton's method does
• There are hundreds of rooting finding methods which attest to it's importance
• In general root finding methods in higher level languages such as Octave employ a combination of the previous methods (or similar methods)

Here is a bisection method example. Review it and see if you can understand what this does. This uses a feature of Octave called handles.

% Uses the bisection method to calculate a root of the function
%
% SYNTAX: root=bisect(function_to_find_root,bounds_of_roots,tolerance)
% PURPOSE: To get the root of a function using the bisection method
% EXPLANATION: Uses bisection method to solve for a root
% CALLING SEQUENCE EXAMPLE: root = bisect(@myfunc,[0,4],1e-6)
% INPUTS: The function, fin, the bounds to find the root, bounds, and the tolerance, tol
% OUTPUTS: The root of the function
% DEPENDENCIES: feval
% NOTES: Class example with limited error checks
%
% Author: Scott D. Johnson as an example to understanding Octave/MATLAB  2019
%
function root = bisect(fin,bounds,tol)
xl = bounds(1);
xu = bounds(2);
f1 = feval(fin,xl);
f2 = feval(fin,xu);
if (f1*f2 > 0)
return
else
printf("Bounds are good let's proceed\n");
endif
xnew = (xl+xu)/2.;
while (abs(f1*f2) > tol)
f1 = feval(fin,xl);
f2 = feval(fin,xnew);
if (f1*f2 < 0)
xu = xnew;
elseif (f1*f2 > 0)
xl = xnew;
endif
if (abs(f1*f2) <= tol)
root = xnew;
printf("Root has been found\n");
endif
xnew = (xl+xu)/2.;
endwhile
return
end

% A function to use to test root finding methods
% This is an example of a function to be used in either MATLAB or octave.
% There is a difference between the two programs though they are generally
% alike.
%
function ansfunc = myfunc(x)
%
ansfunc = (x-4).^4-3.*x+2;
return
end

root = bisect(@myfunc,[0,4],1e-6)

Optimization methods

• Golden search method
• Uses golden ratio
• Just like in art
• Like bracketing methods, converges but is slow
• Unfortunately slow in optimization is not good given the some very complicated optimization problems
• Newton's method
• Non-gradient methods like Powell's method
• Don't use these as they are slow and not very efficient
• Levenberg-Marquardt method (one of the best)
• Octave routine, leasqr in the package optim
• "Simulation" methods
• Predictor-corrector methods
• Idea extends what we did in parachute person
• Predictor is what we did in parachute person, corrector is the improvement of what we did
• Simulated Annealing
• Neural Nets
• Genetic Algorithms
• Linear programming
• Simplex method
• There are hundreds of optimization methods which attest to it's importance

Fitting data

• Least square method
• This method is typically available on most engineering/science oriented higher level languages
• Octave, polyfit or wpolyfit (from optim package)...see discussion in laboratory pages
• Splines
• This is a very useful method of fitting data
• There are a number of different spline methods that are typically available on most engineering/science oriented higher level languages
• Digital filters (remember filtering is fitting and fitting is filtering)

Differential analysis methods

• Finite element analysis
• Solve partial differential equations on a grid (many points) and interpolate rather than a continuous system (infinite points)
• Use extensively in engineering
• Finite difference analysis
• Finite volume analysis

These methods and more are normally discussed in numerical methods or in signals and systems (assuming a section on numerical methods is included). This course is essential for engineering and required in many Universities (names might be slightly different depending on discipline).

While there are other topics to discuss here the intent is for you to see what is out there with this bread crumb chapter, not to actually teach the course. So with that let us move on to the next section.

1Emmy Noether was a famous German mathematician during the time period of World War II. She helped many scientists, in particular physicists, solve problems, including Albert Einstein.

18.1: Analysis Methods is shared under a CC BY-NC-SA license and was authored, remixed, and/or curated by LibreTexts.