Numerical Methods using MATLAB

In this chapter, we will talk about the basics of MATLAB and how to get started with it. We expect the reader to have basic programming skills, and therefore we will not cover any MATLAB programming concepts in detail. When discussing any topic, we will try not to go into such specifics that cause us to deviate from our main goal. Instead, we will provide proper references which can be consulted for more information.

Recall that MATLAB is an abbreviation for Matrix Laboratory. Everything in MATLAB is a matrix, be it a scalar, variable or an image. The primary motivation behind building an entire piece of software for matrices was the fact that matrices are very important in engineering. Almost everyone has seen matrices in mathematics, where they are used for linear transformations, but matrices are also used in a wide range of applications in engineering where they represent a variety of different notions. In this section, we will see different representations of matrices, their applications, and learn different operations on them. Another important aspect is the trick known as vectorization. Since we will be dealing with big matrices, we will see that it is important to avoid loops over matrices using this trick.

In practical real life engineering designs, we often come across problems which consist of numerical operations such as integration or differentiation. For example, in a standard tracing of a robotic arm, the computation of the total error involves integration of instantaneous errors. While mathematical problems require analytic solutions which are sometimes very hard or impossible to obtain, most engineering problems require only numerical solutions. Sometimes the variation of values with parameters is considered sufficient instead of actually finding the closed form relation in terms of parameters. Such a method which involves the numerical computation of the solution is known as a numerical method. These methods form the backbone of engineering and are widely available in MATLAB. In the present chapter, we will study a few basic numerical techniques with which almost all numerical analysis can be done.

Visualization is an important part of any engineering application and can be used to provide insights into design problems or to examine the performance of any solution method. In this chapter, we will talk about basic plotting techniques in MATLAB and how to visualize complex results using them, including animation. MATLAB provides a wide range of functions for 2D and 3D plots of various kinds and also enables you to annotate or decorate the plots with auxiliary functions.

In many cases, to understand the dynamics of a real world system or a function of a physical system, we need to model it and imitate the step by step operation of the system using a computational tool. This step by step imitation of a real world process is known as simulation. The systems can include any physical phenomena such as Brownian motion or the path of a projectile; or man-made devices such as robots or motors. Computational tools developed for simulating a system can be based on hardware or software. In this chapter, we will learn how to simulate in a software-based computation tool, in particular in MATLAB.

Most real world processes have some random components in them. That is why we have to investigate the statistical properties of such systems/processes. For example, we may be interested in computing the expected performance of a system or the expected distance in a random walk process, or the mean power of a received signal or the expected number of people in a waiting room (see the last example in Chapter 5). Many mathematical tools have been derived to compute such metrics, however in most cases the systems are too complex to derive any tractable results. In such scenarios, simulations with random sampling, also known as Monte Carlo simulations, can provide the desired analysis. This technique is based on the assumption that if we perform random sampling of a random variable sufficiently many times, we can derive the approximate statistical properties of it. On the other hand, we sometimes add the random component into a computation or algorithm deliberately when deterministic computation is very extensive or difficult. Monte Carlo simulations are also very useful in such cases.

An important aspect of any engineering design problem is to achieve efficiency. This can be in terms of energy consumption, performance, time complexity, etc. In many cases, we have multiple solutions to a problem and we have to select the most efficient solution. Finding the optimal solution from all the available options is known as optimization. Optimization plays an important role in various engineering applications such as design optimization, transportation, computation, manufacturing, economics and game theory. There is a broad body of work involving the mathematical theories behind optimization, aimed at designing efficient algorithms. In this chapter, we will concentrate on the implementation of numerical methods for optimization.

In Chapter 7, we discussed many optimization algorithms and inbuilt MATLAB functions, and we saw examples of their use. However, almost all of these algorithms are gradient-based and return only local minima. If the objective function has many local minima and we start from an initial guess close to one such minimum, the algorithm will most probably output the local minimum closest to the initial guess. In such cases, we would never reach the global minimum. Hence the success of the method depends highly on the initial guess. Most real world objectives are multivariate functions and contain multiple minima. Therefore, new heuristic search-based algorithms have been devised. These algorithms start with multiple initial guesses and result in solutions which evolve with time. Hence these methods are known as evolutionary computation methods.

In previous chapters, we have discussed methods for simulating a system defined by a particular model, but we haven’t talked about how to build such a model or verify that the model we are using is correct or suitable. Building/finding the optimal model for any system is one of the most important tasks in any engineering design. If the system is simple, we can study the physics behind it and come up with an analytical and mathematical model. But most real world systems are very complex and finding an exact model for them is very difficult. Another way to build a model is by empirical fitting. In this method, we first perform a series of experiments on the system to collect samples of input and output signals. Then we try to find a relation between output and inputs with some basic assumptions about the system. Given any two data sets, the derivation of a relation is known as regression. In this chapter, we will first learn the basic concepts and methods of regression and then use this knowledge to perform model fitting.

In Chapter 5, we saw how we can simulate a continuous process given by the update equations which define the rate of change of the process in terms of the current and previous values of the process. A real world system is a combination of interdependent processes which may be hidden or visible to the outside world. A mathematical model of the system generally consists of the update equations (known as state equations) of all such processes and these processes themselves determine the system dynamics. As discussed earlier, the update equations of the continuous time processes are in the form of differential equations. In this chapter, we will first learn to solve differential equations to compute their behavior at different time samples. Using the previous tools and the tools developed in this chapter, we will then learn to simulate the system dynamics of a system to visualize its dynamics and outputs with varying time.