18.1: Analysis Methods
 Page ID
 50311
In engineering and science, analysis is important for proper design. Most analysis and modeling techniques are covered in numerical methods which has been sacrificed in a number of programs for signals and systems. Some programs include numerical methods by including a section or two in signals and systems and various programming courses. It is important for a student to make sure they have some background in numerical methods. Here we will review some essentials ideas in numerical methods in computer science. This will not be a complete picture.
Signals and Systems and Numerical Methods
A system is a process that can accomplish a task or is a process that by nature accomplishes a task. The term system is so generic it can be used in the sciences, engineering, social, computer science, music, etc. In biology for instance we have a digestive system that takes in food and processes to output materials the body needs and waste, in computer system you have system calls that input a command you wish the computer system to do and outputs the action of the computers system (which is a system in itself), in engineering you may have a LED which you input a potential and the LED outputs light, and so on. Think of all the terms where system is used...a government system, a railroad system, a system of stars, a musical system (staves), natural system, social system, etc. This makes a consistent definition of system a little problematic, however we will try to define it to fit more of a science and engineering purpose.
A system is a process that has an input and has an output. You can see that in the examples above. In engineering we add another term to this complicated definition and that is signal. Therefore, a system is a process that has an input signal or signals and an output signal or signals. The output signal is usually different but in certain instances and be the same as the input. A subset of this is a control system that controls or regulates a system. This now leaves the question of what is a signal?
A signal is the representation of a physical "wave." That is normally expressed as a variable in timespace, for instance x(t). Examples of signals might be the voltage or current of an electrical circuit, the force in a mechanical circuit, heat flow in a thermal circuit, hydraulic flow in a fluid circuit, biological mass flow in a digestive circuit, etc. This definition gives us a basis to move to a more mathematical approach to systems and signals which can lead to computational methods.
A system can be expressed mathematically as a transfer function or an operator. A simple example of an operator could be an integrator that takes a signal and integrates it over time or it could be as simple as multiplying the signal by a constant. Transfer functions are specific to engineering and have the notation of a Laplace transform, where as operators are more specific to physics (and other sciences) and have the notation of mathematics.
How we express these ideas tangible is through different mathematical techniques or methods. The system methods that are used are generally the same methods that a student would learn in linear algebra. The signal methods that are used are generally the ones learned in numerical methods. This makes both methods important in engineering so we will give a quick bread crumb run down of the various methods starting with numerical methods.
Numerical Methods
Technically we already did one type of numerical methods in parachute person. This method is the Euler's method for numerically solving differential equations. There are better methods to numerical solving differential equation but it the most intuitive.
 ¬ What type of problems do engineers/scientists solve

 We solve problem of balance, like balancing forces or balancing voltages (Kirchhoff's voltage law)
 Any balanced equation can be written such that the function equals zero
 These are problems of finding a root or roots
 Root solving methods
 We solve optimization problems like minimum energy, minimum volume, or maximized efficiency
 Mathematically this requires taking a derivative and solving the resulting function when it is set to zero
 Solving for a system set to zero is a root problem so root solving methods and optimization methods are related
 Though related methods they are not the same because of slightly different goals
 Optimization methods
 We solve problem of balance, like balancing forces or balancing voltages (Kirchhoff's voltage law)
 ¬ What other tasks do engineers/scientists perform on a routine basis

 Analysis for experimental data
 Fitting experimental data to a curve (see sections in laboratory chapter)
 Filtering experimental data or filtering signals (which produces the same results as fitting)
 Fourier analysis (or other related analysis)
 Fourier fast transform (FFT)
 Wavelets
 Filter methods (or fitting methods)
 Transform methods (operators)
 Simulation
 Circuit analysis
 Finite difference analysis
 Finite element analysis
 Finite volume analysis
 Monte Carlo analysis
 "Molecular" dynamic analysis
 Simulation methods
 Analysis for experimental data
Conservation Laws
The idea of balanced equations comes to us from the conservation laws of physics. Here is a brief list of the laws with notes.
From a theorem by Emmy Noether^{1}  Symmetry implies conservation (breaking symmetry is a phase change)
 Conservation of Energy
 Historically promoted by engineers over the objections of physicists
 First Law of Thermodynamics – Chemical Engineering
 Voltage Balance – Electrical Engineering
 Invariant under time translation
 Conservation of linear momentum (or just momentum)
 Force balance  Civil and Mechanical Engineering
 No external force is required
 Invariant under translation
 Conservation of angular momentum
 Angular analogy to linear momentum – Civil and Mechanical Engineering
 No external torque is required
 Invariant under rotations
 Conservation of electric charge
 Current balance – Electrical Engineering
 Charge in a note must eventually leave that node
 Conservation of mass
 Mass balance equation – Chemical Engineering
 Mass into a system must equal the mass out of a system plus any accumulated mass in the system
 Conservation fails in certain instances if the equivalence of energy and mass is not considered
 Does not fail in reality...just in our understanding of what is going on
From these laws we get a number of the laws (Kirchhoff's law, etc.) that we have already discussed in this course/book. The equations produced here can then be programmed into a computer and solved for the problem you wish to solve. Of for that more advance languages some of the laws maybe already incorporated into the language itself (like say SPICE for circuits).
Finding the roots of an equation
 Bracketing methods
 Converges but can go really slow in certain instances
 Need to points to bound the root
 Bisection method
 Similar to what you do when someone says guess a number between 1 and 50
 Just keep dividing by 2, right?
 Regula Falsi method
 Improves bisection method by speeding it up
 The speed up can result in issues itself
 Open methods
 Finds solutions fast but may not converge
 Only need one point
 Fixedpoint iteration method
 Isn't used by itself but maybe combined with another method
 NewtonRaphson method (Newton method)
 Most popular method
 Need derivative of function
 Uses Taylor's series expansion
 Secant method
 Similar to Newton method
 Uses two points so it does not require a derivative like Newton's method does
 There are hundreds of rooting finding methods which attest to it's importance
 In general root finding methods in higher level languages such as Octave employ a combination of the previous methods (or similar methods)
Here is a bisection method example. Review it and see if you can understand what this does. This uses a feature of Octave called handles.
Optimization methods
 Golden search method
 Uses golden ratio
 Just like in art
 Like bracketing methods, converges but is slow
 Unfortunately slow in optimization is not good given the some very complicated optimization problems
 Newton's method
 Nongradient methods like Powell's method
 Don't use these as they are slow and not very efficient
 Gradient methods
 LevenbergMarquardt method (one of the best)
 Octave routine, leasqr in the package optim
 "Simulation" methods
 Predictorcorrector methods
 Idea extends what we did in parachute person
 Predictor is what we did in parachute person, corrector is the improvement of what we did
 Simulated Annealing
 Neural Nets
 Genetic Algorithms
 Predictorcorrector methods
 Linear programming
 Simplex method
 There are hundreds of optimization methods which attest to it's importance
Fitting data
 Least square method
 This method is typically available on most engineering/science oriented higher level languages
 Octave, polyfit or wpolyfit (from optim package)...see discussion in laboratory pages
 Splines
 This is a very useful method of fitting data
 There are a number of different spline methods that are typically available on most engineering/science oriented higher level languages
 Digital filters (remember filtering is fitting and fitting is filtering)
Differential analysis methods
 Finite element analysis
 Solve partial differential equations on a grid (many points) and interpolate rather than a continuous system (infinite points)
 Use extensively in engineering
 Finite difference analysis
 Finite volume analysis
These methods and more are normally discussed in numerical methods or in signals and systems (assuming a section on numerical methods is included). This course is essential for engineering and required in many Universities (names might be slightly different depending on discipline).
While there are other topics to discuss here the intent is for you to see what is out there with this bread crumb chapter, not to actually teach the course. So with that let us move on to the next section.
^{1}Emmy Noether was a famous German mathematician during the time period of World War II. She helped many scientists, in particular physicists, solve problems, including Albert Einstein.