History of this Course

(1)

Advanced Mathematics for Engineers

Wolfgang Ertel

translated by Elias Drotleff and Richard Cubek October 1, 2012

(2)

Since 2008 this mathematics lecture is offered for the master courses computer science, mechatronics and electrical engineering. After a repetition of basic linear algebra, computer algebra and calculus, we will treat numerical calculus, statistics and function approximation, which are the most important mathematics basic topics for engineers.

We also provide an introduction to Computer Algebra. Mathematica, Matlab and Octave are powerful tools for the Exercises. Event though we favour the open source tool Octave, the student is free to choose either one of the three.

We are looking forward to work with interesting semesters with many motivated and eager students who want to climb up the steep, high and fascinating mountain of engineering mathematics together with us. I assure you that we will do our best to guide you through the sometimes wild, rough and challenging nature of mathematics. I also assure you that all your efforts and your endurance in working on the exercises during nights and weekends will pay off as good marks and most importantly as a lot of fun.

Even though we repeat some undergraduate linear algebra and calculus, the failure rate in the exams is very high, in particular among the foreign students. As a consequence, we strongly recommend all our students to repeat undergraduate linear algebra such as operation on matrices like solution of linear systems, singularity of matrices, inversion, eigenvalue problems, row-, column- and nullspaces. You also should bring decent knowledge of one- dimensional and multidimensional calculus, e.g. differentiation and integration in one and many variables, convergence of sequences and series and finding extrema with constraints of multivariate functions. And basic statistics is also required. To summarize: If you are not able to solve problems (not only know the terms) in these fields, you have very little chances to successfully finish this course.

History of this Course

The first version of this script was created in the winter semester 95/96. I had included in this lecture only Numerics, although I wanted to cover initially Discrete Mathematics too, which is very important for computer scientists. If you want to cover both in a lecture of three semester week hours, it can happen only superficially. Therefore I decided to focus like my colleagues on Numerics. Only then it is possible to impart profound knowledge.

From Numerical Calculus besides the basics, systems of linear equations, various interpola- tion methods, function approximation, and the solution of nonlinear equations will be pre- sented. An excursion into applied research follows, where e.g. in the field of benchmarking of Microprocessors, mathematics (functional equations) is influencing directly the practice of computer scientists.

In summer 1998 a chapter about Statistics was added, because of the weak coverage at our University till then. In the winter semester 1999/2000, the layout and structure were improved, as well some mistakes have been removed.

In the context of changes in the summer semester 2002 in the curriculum of Applied Computer science, statistics was shifted, because of the general relevance for all students, into the lecture Mathematics 2. Instead of Statistics, contents should be included, which are specifically relevant for computer scientists. The generation and verification of random numbers is an important topic, which is finally also covered.

Since summer 2008, this lecture is only offered to Master (Computer Science) students.

Therefore the chapter about random numbers was extended. Maybe other contents will be included in the lecture. For some topics original literature will be handed out, then student

(3)

To the winter semester 2010/11 the lecture has now been completely revised, restructured and some important sections added such as radial basis functions, Gaussian processes and statistics and probability. These changes become necessary with the step from Diploma to Master. I want to thank Markus Schneider and Haitham Bou Ammar who helped me improve the lecture.

To the winter semester 2010/11 the precourse will be integrated in the lecture in order to give the students more time to work on the exercises. Thus, the volume of lecture grows from 6 SWS to 8 SWS and we will now split it into two lectures of 4 SWS each.

In the winter semester 2012/13 we go back to a one semester schedule with 6 hours per week for computer science and mechatronics students. Electrical engineering students will only go for four hours, covering chapters one to six.

Wolfgang Ertel

(4)

Chapter 1 Linear Algebra

1.1 Video Lectures

We use the excellent video lectures from G. Strang, the author of [1], available from: http://

ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010. In particular we show the following lectures:

Lec # Topics

1 The geometry of linear equations (lecture 01) 2 Transposes, Permutations, SpacesRⁿ (lecture 05) 3 Column Space and Nullspace (lecture 06)

4 Solving Ax = 0: Pivot Variables, Special Solutions (lecture 07) 5 Independence, Basis, and Dimension (lecture 09)

6 The Four Fundamental Subspaces (lecture 10) 7 Orthogonal Vectors and Subspaces (lecture 14) 8 Properties of Determinants (lecture 18)

9 Determinant Formulas and Cofactors (lecture 19) 10 Cramer’s rule, inverse matrix, and volume (lecture 20) 11 Eigenvalues and Eigenvectors (lecture 21)

12 Symmetric Matrices and Positive Definiteness (lecture 25) 13 Linear Transformations and Their Matrices (lecture 30)

1.2 Exercises

Exercise 1.1 Solve the nonsingular triangular system

u+v+w=b₁ (1.1)

v+w=b₂ (1.2)

w=b₃ (1.3)

Show that your solution gives a combination of the columns that equals the column on the right.

Exercise 1.2 Explain why the system

u+v+w= 2 (1.4)

u+ 2v+ 3w= 1 (1.5)

v+ 2w= 0 (1.6)

(7)

is singular, by finding a combination of the three equations that adds up to 0 = 1. What value should replace the last zero on the right side, to allow the equations to have solutions, and what is one of the solutions?

Inverses and Transposes

Exercise 1.3 Which properties of a matrix A are preserved by its inverse (assuming A⁻¹ exists)?

(1) A is triangular (2) A is symmetric (3) A is tridiagonal

(4) all entries are whole numbers

(5) all entries are fractions (including whole numbers like ³₁) Exercise 1.4

a) How many entries can be chosen independently, in a symmetric matrix of order n?

b) How many entries can be chosen independently, in a skew-symmetric matrix of ordern?

Permutations and Elimination

Exercise 1.5

a) Find a square 3×3 matrixP, that multiplied from left to any 3×m matrixA exchanges rows 1 and 2.

b) Find a squaren×nmatrixP, that multiplied from left to anyn×m matrixAexchanges rowsi and j.

Exercise 1.6 A permutation is a bijective mapping from a finite set onto itself. Applied to vectors of length n, a permutation arbitrarily changes the order of the vector components. The word “ANGSTBUDE” is a permutation of “BUNDESTAG”. An example of a permutation on vectors of length 5 can be described by

(3,2,1,5,4).

This means component 3 moves to position 1, component 2 stays where it was, component 1 moves to position 3, component 5 moves to position 4 and component 4 moves to position 5.

a) Give a 5×5 matrix P that implements this permutation.

b) How can we come from a permutation matrix to its inverse?

Exercise 1.7

a) Find a 3×3 matrix E, that multiplied from left to any 3×mmatrix Aadds 5 times row 2 to row 1.

b) Describe an×nmatrixE, that multiplied from left to anyn×mmatrixA addsk times rowi to row j.

c) Based on the above answers, prove that the elimination process of a matrix can be realized by successive multiplication with matrices from left.

(8)

Column Spaces and NullSpaces

Exercise 1.8 Which of the following subsets of R³ are actually subspaces?

a) The plane of vectors with first component b₁ = 0.

b) The plane of vectors b with b₁ = 1.

c) The vectors b withb1b2 = 0 (this is the union of two subspaces, the plane b1 = 0 and the planeb₂ = 0).

d) The solitary vectorb = (0,0,0).

e) All combinations of two given vectorsx= (1,1,0) and y= (2,0,1).

f ) The vectors (b₁,b₂, b₃) that satisfyb₃−b₂ + 3b₁ = 0.

Exercise 1.9 Let P be the plane in 3-space with equation x+ 2y+z = 6. What is the equation of the plane P₀ through the origin parallel to P? Are P andP₀ subspaces of R³? Exercise 1.10 Which descriptions are correct? The solutions xof

Ax =

1 1 1 1 0 2



 x₁ x₂ x₃



= 0

0

(1.7)

form a plane, line, point, subspace, nullspace of A, column space of A.

Ax = 0 and Pivot Variables

Exercise 1.11 For the matrix

A=

0 1 4 0 0 2 8 0

(1.8) determine the echelon formU, the basic variables, the free variables, and the general solution to Ax= 0. Then apply elimination toAx=b, with components b₁ and b₂ on the right side;

find the conditions for Ax = b to be consistent (that is, to have a solution) and find the general solution in the same form as Equation (3). What is the rank of A?

Exercise 1.12 Write the general solution to 1 2 2

2 4 5



 u v w



= 1

4

(1.9)

as the sum of a particular solution toAx=band the general solution to Ax= 0, as in (3).

Exercise 1.13 Find the value of c which makes it possible to solve

u+v+ 2w = 2 (1.10)

2u+ 3v −w = 5 (1.11)

3u+ 4v+w=c (1.12)

(9)

Solving Ax = b

Exercise 1.14 Is it true that if v1, v2, v3 are linearly independent, that also the vectors w₁ = v₁+v₂, w₂ = v₁ +v₃, w₃ = v₂ +v₃ are linearly independent? (Hint: Assume some combination c₁w₁+c₂w₂+c₃w₃ = 0, and find which c_i are possible.)

Exercise 1.15 Find a counterexample to the following statement: Ifv₁, v₂, v₃, v₄ is a basis for the vector space R⁴, and if W is a subspace, then some subset of the v’s is a basis for W.

Exercise 1.16 Suppose V is known to have dimension k. Prove that a) any k independent vectors in V form a basis;

b) any k vectors that spanV form a basis.

In other words, if the number of vectors is known to be right, either of the two properties of a basis implies the other.

Exercise 1.17 Prove that if V and W are three-dimensional subspaces of R⁵, then V and W must have a nonzero vector in common. Hint: Start with bases of the two subspaces, making six vectors in all.

The Four Fundamental Subspaces

Exercise 1.18 Find the dimension and construct a basis for the four subspaces associated with each of the matrices

A=

0 1 4 0 0 2 8 0

and U =

0 1 4 0 0 0 0 0

(1.13)

Exercise 1.19 If the product of two matrices is the zero matrix, AB = 0, show that the column space of B is contained in the nullspace of A. (Also the row space of A is the left nullspace of B, since each row of A multipliesB to give a zero row.)

Exercise 1.20 Explain why Ax =b is solvable if and only if rank A = rank A⁰, where A⁰ is formed from A by adding b as an extra column. Hint: The rank is the dimension of the column space; when does adding an extra column leave the dimension unchanged?

Exercise 1.21 Suppose A is an m byn matrix of rankr. Under what conditions on those numbers does

a) A have a two-sided inverse: AA⁻¹ =A⁻¹A=I? b) Ax=b have infinitely many solutions for every b?

Exercise 1.22 IfAx= 0 has a nonzero solution, show thatA^Ty=f fails to be solvable for some right sides f. Construct an example of A and f.

Orthogonality

Exercise 1.23 InR³ find all vectors that are orthogonal to (1, 1, 1) and (1, -1, 0). Produce from these vectors a mutually orthogonal system of unit vectors (an orthogonal system) in

(10)

R³.

Exercise 1.24 Show that x−y is orthogonal to x+y if and only ifkxk=kyk.

Exercise 1.25 LetP be the plane (not a subspace) in 3-space with equationx+ 2y−z = 6.

Find the equation of a plane P⁰ parallel to P but going through the origin. Find also a vector perpendicular to those planes. What matrix has the plane P⁰ as its nullspace, and what matrix hast P⁰ as its row space?

Projections

Exercise 1.26 Suppose A is the 4×4 identity matrix with its last column removed. A is 4×3. Project b = (1,2,3,4) onto the column space of A. What shape is the projection matrix P and what isP?

Determinants

Exercise 1.27 How are det(2A), det(−A), and det(A²) related to det A, when A is n by n?

Exercise 1.28 Find the determinants of:

a) a rank one matrix

A =



 1 4 2





2 −1 2

(1.14)

b) the upper triangular matrix

U =







4 4 8 8 0 1 2 2 0 0 2 6 0 0 0 2







(1.15)

c) the lower triangular matrixU^T; d) the inverse matrix U⁻¹;

e) the “reverse-triangular” matrix that results from row exchanges,

M =







0 0 0 2 0 0 2 6 0 1 2 2 4 4 8 8







(1.16)

Exercise 1.29 If every row of A adds to zero prove that detA = 0. If every row adds to 1 prove that det(A−I) = 0. Show by example that this does not imply detA= 1.

(11)

Properties of Determinants

Exercise 1.30 Suppose An is the n by n tridiagonal matrix with 1’s everywhere on the three diagonals:

A₁ = 1

, A₂ = 1 1

1 1

, A₃ =





1 1 0 1 1 1 0 1 1



, ... (1.17)

Let Dn be the determinant ofAn; we want to find it.

a) Expand in cofactors along the first row ofA_n to show that D_n =Dn−1−Dn−2.

b) Starting from D₁ = 1 and D₂ = 0 find D₃, D₄, ..., D₈. By noticing how these numbers cycle around (with what period?) find D₁₀₀₀.

Exercise 1.31 Explain why a 5 by 5 matrix with a 3 by 3 zero submatrix is sure to be a singular (regardless of the 16 nonzeros marked by x’s):

the determinant of A=







x x x x x x x x x x

0 0 0 x x







is zero. (1.18)

Exercise 1.32 If A ism byn and B is n by m, show that det=

0 A

−B I

= det AB.

Hint: Postmultiply by

I 0 B I

.

(1.19) Do an example with m < n and an example with m > n. Why does the second example have det AB= 0?

Cramers’ rule

Exercise 1.33 The determinant is a linear function of the column 1. It is zero if two columns are equal. When b=Ax=x1a1+x2a2+x3a3 goes into the first column ofA, then the determinant of this matrix B₁ is

|b a₂ a₃|=|x₁a₁+x₂a₂+x₃a₃ a₂ a₃|=x₁|a₁ a₂ a₃|=x₁detA a) What formula for x₁ comes from left side = right side?

b) What steps lead to the middle equation?

Eigenvalues and Eigenvectors

Exercise 1.34 Suppose that λ is an eigenvalue of A, and x is its eigenvector: Ax =λx.

a) Show that this samex is an eigenvector of B =A−7I, and find the eigenvalue.

b) Assumingλ6= 0, show that x is also an eigenvector of A⁻¹ and find the eigenvalue.

(12)

Exercise 1.35 Show that the determinant equals the product of the eigenvalues by imagining that the characteristic polynomial is factored into

det(A−λI) = (λ₁−λ)(λ₂−λ)· · ·(λ_n−λ) (1.20) and making a clever choice of λ.

Exercise 1.36 Show that the trace equals the sum of the eigenvalues, in two steps. First, find the coefficient of (−λ)ⁿ⁻¹ on the right side of (15). Next, look for all the terms in

det(A−λI) = det







a11−λ a12 · · · a1n

a₂₁ a₂₂−λ · · · a_2n

... ... ...

a_n1 a_n2 · · · a_nn −λ







(1.21)

which involve (−λ)ⁿ⁻¹. Explain why they all come from the product down the main diagonal, and find the coefficient of (−λ)ⁿ⁻¹ on the left side of (15). Compare.

Diagonalization of Matrices

Exercise 1.37 Factor the following matrices into SΛS⁻¹: A=

1 1 1 1

and A= 2 1

0 0

. (1.22)

Exercise 1.38 Suppose A=uv^T is a column times a row (a rank-one matrix).

a) By multiplying A timesu show that u is an eigenvector. What isλ?

b) What are the other eigenvalues (and why)?

c) Compute trace(A) = v^Tuin two ways, from the sum on the diagonal and the sum ofλ’s.

Exercise 1.39 If A is diagonalizable, show that the determinant of A = SΛS⁻¹ is the product of the eigenvalues.

Symmetric and Positive Semi-Definite Matrices

Exercise 1.40 If A = QΛQ^T is symmetric positive definite, then R = Q√

ΛQ^T is its symmetric positive definite square root. Why doesR have real eigenvalues? ComputeR and verify R² =A for

A= 2 1

1 2

and A=

10 −6

−6 10

. (1.23)

Exercise 1.41 If A is symmetric positive definite and C is nonsingular, prove that B = C^TAC is also symmetric positive definite.

Exercise 1.42 If A is positive definite and a₁₁ is increased, prove from cofactors that the determinant is increased. Show by example that this can fail if A is indefinite.

(13)

Linear Transformation

Exercise 1.43 Suppose a linear mapping T transforms (1, 1) to (2, 2) and (2, 0) to (0, 0).

Find T(v):

(a) v = (2,2) (b) v = (3,1) (c) v = (−1,1) (d) v = (a, b)

Exercise 1.44 Suppose T is reflection across the 45^o line, and S is reflection across the y axis. If v = (2,1) then T(v) = (1,2). Find S(T(v)) andT(S(v)). This shows that generally ST 6=T S.

Exercise 1.45 Suppose we have two bases v₁, ..., v_n and w₁, ..., w_n for Rⁿ. If a vector has coefficients b_i in one basis and c_i in the other basis, what is the change of basis matrix in b =M c? Start from

b₁v₁+...+b_nv_n=V b=c₁w₁+...+c_nw_n=W c. (1.24) Your answer represents T(v) = v with input basis ofv’s and output basis of w’s. Because of different bases, the matrix is not I.

(14)

Chapter 2 Computer Algebra

Definition 2.1 Computer Algebra = Symbol Processing + Numerics + Graphics

Definition 2.2 Symbol Processing is calculating with symbols (variables, constants, function symbols), as in Mathematics lectures.

Advantages of Symbol Processing:

often considerably less computational effort compared to numerics.

symbolic results (for further calculations), proofs in the strict manner possible.

Disadvantages of Symbol Processing:

often there is no symbolic (closed form) solution, then Numerics will be applied, e.g.:

– Calculation of Antiderivatives

– Solving Nonlinear Equations like: (e^x = sinx) Example 2.1

1. symbolic:

x→∞lim

lnx x+ 1

0

=? (asymptotic behavior) lnx

x+ 1 0

=

1

x(x+ 1)−lnx

(x+ 1)² = 1

(x+ 1)x− lnx (x+ 1)² x→ ∞:

lnx x+ 1

0

→ 1

x² − lnx

x² → lnx x² →0 2. numeric:

x→∞lim f⁰(x) =?

(15)

Example 2.2 Numerical solution of x² = 5 x² = 5, x= 5

x, 2x=x+ 5 x x= 1

2

x+ 5 x

iteration:

x_n+1 = 1 2

x_n+ 5 x_n

n x_n

0 2← Startwert

1 2.25

2 2.236111 3 2.23606798 4 2.23606798

⇒ √

5 = 2.23606798±10⁻⁸ (approximate solution)

2.1 Symbol Processing on the Computer

Example 2.3 Symbolic Computing with natural numbers:

Calculation rules, i.e. Axioms necessary. ⇒ Peano Axioms e.g.:

∀x, y, z :x+y = y+x (2.1)

x+ 0 = x (2.2)

(x+y) +z = x+ (y+z) (2.3)

Out of these rules, e.g. 0 +x=x can be deduced:

0 +x = z }| { (2.1)

x+ 0 = z }| { (2.2)

x

Implementation of symbol processing on the computer by ”Term Rewriting”.

Example 2.4 (Real Numbers) Chain Rule for Differentiation:

[f(g(x))]⁰ ⇒ f⁰(g(x))g⁰(x) sin(lnx+ 2)⁰ = cos(lnx+ 2)1

x Computer: (Pattern matching)

sin(P lus(lnx,2))⁰ = cos(P lus(lnx,2))P lus⁰(lnx,2) sin(P lus(lnx,2))⁰ = cos(P lus(lnx,2))P lus(ln⁰x,2⁰)

(16)

sin(P lus(lnx,2))⁰ = cos(P lus(lnx,2))P lus 1

x,0

sin(P lus(lnx,2))⁰ = cos(P lus(lnx,2))1 x sin(P lus(lnx,2))⁰ = cos(lnx+ 2)

x Effective systems:

Mathematica (S. Wolfram & Co.)

Maple (ETH Zurich + Univ. Waterloo, Kanada)

2.2 Short Introduction to Mathematica

Resources:

• Library: Mathematica Handbook (Wolfram)

• Mathematica Documentation Online: http://reference.wolfram.com

• http://www.hs-weingarten.de/~ertel/vorlesungen/mae/links.html 2.2.0.1 Some examples as jump start

In[1]:= 3 + 2^3 Out[1]= 11 In[2]:= Sqrt[10]

Out[2]= Sqrt[10]

In[3]:= N[Sqrt[10]]

Out[3]= 3.16228

In[4]:= N[Sqrt[10],60]

Out[4]= 3.1622776601683793319988935444327185337195551393252168268575 In[5]:= Integrate[x^2 Sin[x]^2, x]

3 2

4 x - 6 x Cos[2 x] + 3 Sin[2 x] - 6 x Sin[2 x]

Out[5]= --- 24

In[7]:= D[%, x]

2 2

12 x - 12 x Cos[2 x]

Out[7]= --- 24

(17)

In[8]:= Simplify[%]

2 2

Out[8]= x Sin[x]

In[9]:= Series[Exp[x], {x,0,6}]

2 3 4 5 6

x x x x x 7

Out[9]= 1 + x + -- + -- + -- + --- + --- + O[x]

2 6 24 120 720

In[10]:= Expand[(x + 2)^3 + ((x - 5)^2 (x + y)^2)^3]

2 3 6 7 8 9

Out[10]= 8 + 12 x + 6 x + x + 15625 x - 18750 x + 9375 x - 2500 x +

10 11 12 5 6 7

> 375 x - 30 x + x + 93750 x y - 112500 x y + 56250 x y -

8 9 10 11 4 2

> 15000 x y + 2250 x y - 180 x y + 6 x y + 234375 x y -

5 2 6 2 7 2 8 2 9 2

> 281250 x y + 140625 x y - 37500 x y + 5625 x y - 450 x y +

10 2 3 3 4 3 5 3 6 3

> 15 x y + 312500 x y - 375000 x y + 187500 x y - 50000 x y +

7 3 8 3 9 3 2 4 3 4

> 7500 x y - 600 x y + 20 x y + 234375 x y - 281250 x y +

4 4 5 4 6 4 7 4 8 4

> 140625 x y - 37500 x y + 5625 x y - 450 x y + 15 x y +

5 2 5 3 5 4 5 5 5

> 93750 x y - 112500 x y + 56250 x y - 15000 x y + 2250 x y -

6 5 7 5 6 6 2 6 3 6

> 180 x y + 6 x y + 15625 y - 18750 x y + 9375 x y - 2500 x y +

4 6 5 6 6 6

> 375 x y - 30 x y + x y In[11]:= Factor[%]

2 3 4 2 3 2

Out[11]= (2 + x + 25 x - 10 x + x + 50 x y - 20 x y + 2 x y + 25 y -

2 2 2 2 3 4 5 6

> 10 x y + x y ) (4 + 4 x - 49 x - 5 x + 633 x - 501 x + 150 x -

7 8 2 3 4 5

> 20 x + x - 100 x y - 10 x y + 2516 x y - 2002 x y + 600 x y -

6 7 2 2 2 2 3 2

> 80 x y + 4 x y - 50 y - 5 x y + 3758 x y - 3001 x y +

4 2 5 2 6 2 3 2 3 3 3

(18)

> 900 x y - 120 x y + 6 x y + 2500 x y - 2000 x y + 600 x y -

4 3 5 3 4 4 2 4 3 4 4 4

> 80 x y + 4 x y + 625 y - 500 x y + 150 x y - 20 x y + x y )

In[12]:= InputForm[%7]

Out[12]//InputForm= (12*x^2 - 12*x^2*Cos[2*x])/24 In[20]:= Plot[Sin[1/x], {x,0.01,Pi}]

Out[20]= -Graphics-

In[42]:= Plot3D[x^2 + y^2, {x,-1,1}, {y,0,1}]

Out[42]= -SurfaceGraphics-

In[43]:= f[x_,y_] := Sin[(x^2 + y^3)] / (x^2 + y^2) In[44]:= f[2,3]

Sin[31]

Out[44]= --- 13

In[45]:= ContourPlot[x^2 + y^2, {x,-1,1}, {y,-1,1}]

In[46]:= Plot3D[f[x,y], {x,-Pi,Pi}, {y,-Pi,Pi}, PlotPoints -> 30,

PlotLabel -> "Sin[(x^2 + y^3)] / (x^2 + y^2)", PlotRange -> {-1,1}]

Sin[(x^2 + y^3)] / (x^2 + y^2)

-2 -1

0 1

2

-2 -1

0 1

2

-1 -0.5

0 0.5

1

-2 -1

0 1

2

-2 -1 0 1 2

Sin[(x^2 + y^3)] / (x^2 + y^2)

In[47]:= ContourPlot[f[x,y], {x,-2,2}, {y,-2,2}, PlotPoints -> 30, ContourSmoothing -> True, ContourShading -> False,

PlotLabel -> "Sin[(x^2 + y^3)] / (x^2 + y^2)"]

Out[47]= -ContourGraphics-

(19)

In[52]:= Table[x^2, {x, 1, 10}]

Out[52]= {1, 4, 9, 16, 25, 36, 49, 64, 81, 100}

In[53]:= Table[{n, n^2}, {n, 2, 20}]

Out[53]= {{2, 4}, {3, 9}, {4, 16}, {5, 25}, {6, 36}, {7, 49}, {8, 64},

> {9, 81}, {10, 100}, {11, 121}, {12, 144}, {13, 169}, {14, 196},

> {15, 225}, {16, 256}, {17, 289}, {18, 324}, {19, 361}, {20, 400}}

In[54]:= Transpose[%]

Out[54]= {{2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,

> 20}, {4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225, 256,

> 289, 324, 361, 400}}

In[60]:= ListPlot[Table[Random[]+Sin[x/10], {x,0,100}]]

Out[60]= -Graphics-

20 40 60 80 100

-0.5 0.5 1 1.5

In[61]:= x = Table[i, {i,1,6}]

Out[61]= {1, 2, 3, 4, 5, 6}

In[62]:= A = Table[i*j, {i,1,5}, {j,1,6}]

Out[62]= {{1, 2, 3, 4, 5, 6}, {2, 4, 6, 8, 10, 12}, {3, 6, 9, 12, 15, 18},

> {4, 8, 12, 16, 20, 24}, {5, 10, 15, 20, 25, 30}}

In[63]:= A.x

Out[63]= {91, 182, 273, 364, 455}

In[64]:= x.x Out[64]= 91

In[71]:= B = A.Transpose[A]

Out[71]= {{91, 182, 273, 364, 455}, {182, 364, 546, 728, 910},

> {273, 546, 819, 1092, 1365}, {364, 728, 1092, 1456, 1820},

> {455, 910, 1365, 1820, 2275}}

In[72]:= B - IdentityMatrix[5]

Out[72]= {{90, 182, 273, 364, 455}, {182, 363, 546, 728, 910},

(20)

> {273, 546, 818, 1092, 1365}, {364, 728, 1092, 1455, 1820},

> {455, 910, 1365, 1820, 2274}}

% last command

%n nth last command

?f help for function f

??f more help for f

f[x_,y_] := x^2 * Cos[y] define function f(x, y)

a = 5 assign a constant to variable a f = x^2 * Cos[y] assign an expression to variable f

(f is only a placeholder for the expression, not a function!) D[f[x,y],x] derivative of f with respect tox Integrate[f[x,y],y] antiderivative of f with respect tox

Simplify[expr] simplifies an expression Expand[expr] expand an expression Solve[f[x]==g[x]] solves an equation

^C cancel

InputForm[Expr] converts into mathematica input form TeXForm[Expr] converts into the L^ATEXform

FortranForm[Expr] converts into the Fortran form CForm[Expr] converts into the C form ReadList["daten.dat", {Number, Number}] reads 2-column table from file

Table[f[n], {n, n_min, n_max}] generates a list f(nmin), . . . , f(nmax) Plot[f[x],{x,x_min,x_max}] generates a plot of f

ListPlot[Liste] plots a list

Plot3D[f[x,y],{x,x_min,x_max},{y,y_min,y_max}] generates a three-dim. plot of f ContourPlot[f[x,y],{x,x_min,x_max},{y,y_min,y_max}] generates a contour plot of f

Display["Dateiname",%,"EPS"] write to the file in PostScript format Table 2.2: Mathematica – some inportant commands

Example 2.5 (Calculation of Square Roots)

(*********** square root iterative **************) sqrt[a_,genauigk_] := Module[{x, xn, delta, n},

For[{delta=9999999; n = 1; x=a}, delta > 10^(-accuracy), n++, xn = x;

x = 1/2(x + a/x);

delta = Abs[x - xn];

Print["n = ", n, " x = ", N[x,2*accuracy], " delta = ", N[delta]];

];

N[x,genauigk]

]

sqrt::usage = "sqrt[a,n] computes the square root of a to n digits."

Table[sqrt[i,10], {i,1,20}]

(21)

(*********** square root recursive **************) x[n_,a_] := 1/2 (x[n-1,a] + a/x[n-1,a])

x[1,a_] := a

2.3 Gnuplot, a professional Plotting Software

Gnuplot is a powerful plotting programm with a command line interface and a batch interface. Online documentation can be found on www.gnuplot.info.

On the command line we can input plot [0:10] sin(x)

to obtain the graph

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 2 4 6 8 10

sin(x)

Almost arbitrary customization of plots is possible via the batch interface. A simple batch file may contain the lines

set terminal postscript eps color enhanced 26

set label "{/Symbol a}=0.01, {/Symbol g}=5" at 0.5,2.2 set output "bucket3.eps"

plot [b=0.01:1] a=0.01, c= 5, (a-b-c)/(log(a) - log(b)) \

title "({/Symbol a}-{/Symbol b}-{/Symbol g})/(ln{/Symbol a} - ln{/Symbol b})"

producing a EPS file with the graph

1 2 3 4 5 6 7 8

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 ttot

γ α=0.01, γ=5 (α-β-γ)/(lnα - lnβ)

3-dimensional plotting is also possible, e.g. with the commands

set isosamples 50

splot [-pi:pi][-pi:pi] sin((x**2 + y**3) / (x**2 + y**2))

which produces the graph

-3 -2 -1 0 1 2 3 -3-2-1 0 1 2 3 -0.8-1

-0.6-0.4 -0.2 0.2 0.4 0.6 0.8 0 1

sin((x**2 + y**3) / (x**2 + y**2))

(22)

2.4 Short Introduction to MATLAB

Effective systems:

MATLAB & SIMULINK (MathWorks) 2.4.0.2 Some examples as jump start

Out(1)=3+2^3 ans = 11 Out(2)=sqrt(10) ans = 3.1623

Out(3)=vpa(sqrt(10),60)

= 3.16227766016837933199889354443271853371955513932521682685750 syms x

syms y

y=x^2sin(x)^2

2 2

x sin(x) z=int(y,x)

2 2 3

x (- 1/2 cos(x) sin(x) + 1/2 x) - 1/2 x cos(x) + 1/4 cos(x) sin(x) + 1/4 x - 1/3 x Der=diff(z,x)

2 2 2

2 x (- 1/2 cos(x) sin(x) + 1/2 x) + x (1/2 sin(x) - 1/2 cos(x) + 1/2)

2 2 2

- 1/4 cos(x) + x cos(x) sin(x) - 1/4 sin(x) + 1/4 - x Simple=simplify(Der)

2 2

x sin(x) Series=Taylor(exp(x),6,x,0)

2 3 4 5

1 + x + 1/2 x + 1/6 x + 1/24 x + 1/120 x (x+2)^2+((x+5)^2(x+y)^2)^3

2 6 6

(x + 2) + (x - 5) (x + y) Exp_Pol=expand(Pol)

2 6 5 4 2 3 3

4 + 4 x + x + 15625 x + 93750 x y + 234375 x y + 312500 x y

2 4 5 11 10 2 9 3

> + 234375 x y + 93750 x y + 6 x y + 15 x y + 20 x y

8 4 7 5 6 6 10 9 2 8 3

> + 15 x y + 6 x y + x y - 180 x y - 450 x y - 600 x y

7 4 6 5 6 12 11 10 9

> - 450 x y - 180 x y + 15625 y + x - 30 x + 375 x - 2500 x

(23)

8 7 5 6 9 8 2 7 3

> + 9375 x - 18750 x - 30 x y+ 2250 x y + 5625 x y + 7500 x y

6 4 5 5 4 6 8 7 2

> + 5625 x y + 2250 x y + 375 x y - 15000 x y - 37500 x y

6 3 5 4 4 5 3 6 7

> - 50000 x y - 37500 x y - 15000 x y - 2500 x y + 56250 x y

6 2 5 3 4 4 3 5

> + 140625 x y + 187500 x y + 140625 x y + 56250 x y

2 6 6 5 2 4 3

> + 9375 x y - 112500 x y - 281250 x y - 375000 x y

3 4 2 5 6

> - 281250 x y - 112500 x y - 18750 x y t=0:0.01:pi

plot(sin(1./t)) --Plot Mode---

[X,Y]=meshgrid(-1:0.01:1,-1:0.01:1) Z=sin(X.^2+Y.^3)/(X.^2+Y.^2)

surf(X,Y,Z)

x=1:1:10 y(1:10)=x.^2 y =

[ 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

A_1=[1 2 4; 5 6 100; -10.1 23 56]

A_1 =

1.0000 2.0000 4.0000 5.0000 6.0000 100.0000 -10.1000 23.0000 56.0000 A_2=rand(3,4)

(24)

A_2 =

0.2859 0.7157 0.4706 0.7490 0.5437 0.8390 0.5607 0.5039 0.9848 0.4333 0.2691 0.6468 A_2’=

0.3077 0.1387 0.4756 0.3625 0.7881 0.7803 0.6685 0.1335 0.0216 0.5598 0.3008 0.9394 A_1.*A_2=

3.1780 5.9925 5.0491 3.0975 43.5900 94.5714 92.6770 29.3559 26.3095 57.1630 58.7436 17.5258 [U L]=lu(A_1)

U =

-0.0990 0.2460 1.0000

-0.4950 1.0000 0

1.0000 0 0

L =

-10.1000 23.0000 56.0000 0 17.3861 127.7228

0 0 -21.8770

[Q R]=qr(A_1) Q =

-0.0884 -0.2230 0.9708 -0.4419 -0.8647 -0.2388 0.8927 -0.4501 -0.0221 R =

-11.3142 17.7035 5.4445 0 -15.9871 -112.5668

0 0 -21.2384

b=[1;2;3]

x=A_1\b b =

1 2 3 x =

0.3842 0.3481 -0.0201

A_3=[1 2 3; -1 0 5; 8 9 23]

A_3 =

1 2 3

-1 0 5

(25)

8 9 23 Inverse=inv(A_3) Inverse =

-0.8333 -0.3519 0.1852 1.1667 -0.0185 -0.1481 -0.1667 0.1296 0.0370

Example 2.6 (Calculation of Square Roots) (*********** root[2] iterative **************) function [b]=calculate_Sqrt(a,accuracy)

clc;

x=a;

delta=inf;

while delta>=10^-(accuracy) Res(n)=x;

xn=x;

x=0.5*(x+a/x);

delta=abs(x-xn);

end b=Res;

2.5 Short Introduction to GNU Octave

From the Octave homepage: GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments. It also provides extensive graphics capabilities for data visualization and manipulation. Octave is normally used through its interactive command line interface, but it can also be used to write non-interactive programs. The Octave language is quite similar to Matlab so that most programs are easily portable.

Downloads, Docs, FAQ, etc.:

http://www.gnu.org/software/octave/

Nice Introduction/Overview:

http://math.jacobs-university.de/oliver/teaching/iub/resources/octave/octave-intro/octave- intro.html

Plotting in Octave:

http://www.gnu.org/software/octave/doc/interpreter/Plotting.html

(26)

// -> comments

BASICS

======

octave:47> 1 + 1 ans = 2

octave:48> x = 2 * 3 x = 6

// suppress output octave:49> x = 2 * 3;

octave:50>

// help

octave:53> help sin

‘sin’ is a built-in function -- Mapping Function: sin (X)

Compute the sine for each element of X in radians.

...

VECTORS AND MATRICES

====================

// define 2x2 matrix octave:1> A = [1 2; 3 4]

A =

1 2

3 4

// define 3x3 matrix

octave:3> A = [1 2 3; 4 5 6; 7 8 9]

A =

1 2 3

4 5 6

7 8 9

// access single elements octave:4> x = A(2,1) x = 4

octave:17> A(3,3) = 17 A =

1 2 3

4 5 6

7 8 17

// extract submatrices octave:8> A

A =

1 2 3

(27)

4 5 6 7 8 17

octave:9> B = A(1:2,2:3) B =

2 3

5 6

octave:36> b=A(1:3,2) b =

2 5 8

// transpose octave:25> A’

ans =

1 4 7

2 5 8

3 6 17

// determinant octave:26> det(A) ans = -24.000 // solve Ax = b // inverse

octave:22> inv(A) ans =

-1.54167 0.41667 0.12500 1.08333 0.16667 -0.25000 0.12500 -0.25000 0.12500 // define vector b

octave:27> b = [3 7 12]’

b = 3 7 12

// solution x

octave:29> x = inv(A) * b x =

-0.20833 1.41667 0.12500 octave:30> A * x ans =

3.0000 7.0000 12.0000

(28)

// try A\b

// illegal operation octave:31> x * b

error: operator *: nonconformant arguments (op1 is 3x1, op2 is 3x1) // therefore allowed

octave:31> x’ * b ans = 10.792 octave:32> x * b’

ans =

-0.62500 -1.45833 -2.50000 4.25000 9.91667 17.00000 0.37500 0.87500 1.50000 // elementwise operations

octave:11> a = [1 2 3]

a =

1 2 3

octave:10> b = [4 5 6]

b =

4 5 6

octave:12> a*b

error: operator *: nonconformant arguments (op1 is 1x3, op2 is 1x3) octave:12> a.*b

ans =

4 10 18

octave:23> A = [1 2;3 4]

A =

1 2

3 4

octave:24> A^2 ans =

7 10 15 22 octave:25> A.^2 ans =

1 4

9 16

// create special vectors/matrices octave:52> x = [0:1:5]

x =

0 1 2 3 4 5

octave:53> A = zeros(2) A =

0 0

(29)

0 0

octave:54> A = zeros(2,3) A =

0 0 0

octave:55> A = ones(2,3) A =

1 1 1

octave:56> A = eye(4) A =

Diagonal Matrix

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

octave:57> B = A * 5 B =

Diagonal Matrix

5 0 0 0

0 5 0 0

0 0 5 0

0 0 0 5

// vector/matrix size octave:43> size(A) ans =

3 3

octave:44> size(b) ans =

3 1

octave:45> size(b)(1) ans = 3

PLOTTING (2D)

============

octave:35> x = [-2*pi:0.1:2*pi];

octave:36> y = sin(x);

octave:37> plot(x,y) octave:38> z = cos(x);

octave:39> plot(x,z) // two curves in one plot octave:40> plot(x,y) octave:41> hold on octave:42> plot(x,z) // reset plots

(30)

octave:50> close all // plot different styles octave:76> plot(x,z,’r’) octave:77> plot(x,z,’rx’) octave:78> plot(x,z,’go’) octave:89> close all // manipulate plot octave:90> hold on

octave:91> x = [-pi:0.01:pi];

// another linewidth

octave:92> plot(x,sin(x),’linewidth’,2) octave:93> plot(x,cos(x),’r’,’linewidth’,2) // define axes range and aspect ratio octave:94> axis([-pi,pi,-1,1], ’equal’)

-> try ’square’ or ’normal’ instead of ’equal’ (help axis) // legend

octave:95> legend(’sin’,’cos’)

// set parameters (gca = get current axis)

octave:99> set(gca,’keypos’, 2) // legend position (1-4) octave:103> set(gca,’xgrid’,’on’) // show grid in x octave:104> set(gca,’ygrid’,’on’) // show grid in y // title/labels

octave:102> title(’OCTAVE DEMO PLOT’) octave:100> xlabel(’unit circle’) octave:101> ylabel(’trigon. functions’) // store as png

octave:105> print -dpng ’demo_plot.png’

DEFINE FUNCTIONS

(31)

================

sigmoid.m:

---

function S = sigmoid(X) mn = size(X);

S = zeros(mn);

for i = 1:mn(1) for j = 1:mn(2)

S(i,j) = 1 / (1 + e ^ -X(i,j));

end end end --- easier:

---

function S = sigmoid(X) S = 1 ./ (1 .+ e .^ (-X));

end ---

octave:1> sig + [TAB]

sigmoid sigmoid.m octave:1> sigmoid(10) ans = 0.99995

octave:2> sigmoid([1 10])

error: for x^A, A must be square // (if not yet implemented elementwise) error: called from:

error: /home/richard/faculty/adv_math/octave/sigmoid.m at line 3, column 4 ...

octave:2> sigmoid([1 10]) ans =

0.73106 0.99995

octave:3> x = [-10:0.01:10];

octave:5> plot(x,sigmoid(x),’linewidth’,3);

PLOTTING (3D)

============

// meshgrid

octave:54> [X,Y] = meshgrid([1:3],[1:3]) X =

1 2 3

Y =

1 1 1

2 2 2

3 3 3

// meshgrid with higher resolution (suppress output) octave:15> [X,Y] = meshgrid([-4:0.2:4],[-4:0.2:4]);

(32)

// function over x and y, remember that cos and sin // operate on each element, result is matrix again octave:20> Z = cos(X) + sin(1.5*Y);

// plot

octave:21> mesh(X,Y,Z) octave:22> surf(X,Y,Z)

octave:44> contour(X,Y,Z) octave:45> colorbar octave:46> pcolor(X,Y,Z)

RANDOM NUMBERS / HISTOGRAMS

===========================

// equally distributed random numbers octave:4> x=rand(1,5)

x =

0.71696 0.95553 0.17808 0.82110 0.25843 octave:5> x=rand(1,1000);

octave:6> hist(x);

// normally distributed random numbers octave:5> x=randn(1,1000);

octave:6> hist(x);

(33)

// try

octave:5> x=randn(1,10000);

octave:6> hist(x, 25);

2.6 Exercises

Mathematica

Exercise 2.1 Program the factorial function with Mathematica.

a) Write an iterative program that calculates the formulan! =n·(n−1)·. . .·1.

b) Write a recursive program that calculates the formula n! =

n·(n−1)! ifn >1 1 if n = 1 analogously to the root example in the script.

Exercise 2.2

a) Write a Mathematica program that multiplies two arbitrary matrices. Don’t forget to check the dimensions of the two matrices before multiplying. The formula is

C_ij =

n

X

k=1

A_ikB_kj. Try to use the functionsTable, Sum and Length only.

b) Write a Mathematica program that computes the transpose of a matrix using theTable function.

c) Write a Mathematica Program that computes the inverse of a matrix using the function Linear Solve.

MATLAB

Exercise 2.3

a) For a finite geometic series we have the formula Σⁿ_i=0qⁱ = ^1−q_1−qⁿ⁺¹. Write a MATLAB function that takesq and n as inputs and returns the sum.

b) For an infinite geometic series we have the formula Σ^∞_i=0qⁱ = _1−q¹ if the series converges.

Write a MATLAB function that takes q as input and returns the sum. Your function should produce an error if the series diverges.

(34)

Exercise 2.4

a) Create a 5×10 random Matrix A.

b) Compute the mean of each column and assign the results to elements of a vector called avg.

c) Compute the standard deviation of each column and assign the results to the elements of a vector calleds.

Exercise 2.5 Given the row vectorsx= [4, 1, 6, 10, −4, 12, 0.1] andy = [−1, 4, 3, 10, −9, 15, −2.1]

compute the following arrays, a) a_ij =x_iy_j

b) b_ij = ^x_yⁱ

j

c) c_i =x_iy_i, then add up the elements ofc using two different programming approaches.

d) d_ij = _2+x^xⁱ

i+yj

e) Arrange the elements ofxandy in ascending order and calculatee_ij being the reciprocal of the lessxi and yj.

f ) Reverse the order of elements inx and y in one command.

Exercise 2.6 Write a MATLAB function that calculates recursively the square root of a number.

Analysis Repetition

Exercise 2.7 In a bucket with capacity v there is a poisonous liquid with volume αv. The bucket has to be cleaned by repeatedly diluting the liquid with a fixed amount (β −α)v (0 < β < 1− α) of water and then emptying the bucket. After emptying, the bucket always keeps αv of its liquid. Cleaning stops when the concentration cn of the poison after n iterations is reduced from 1 toc_n < >0.

a) Assume α= 0.01, β = 1 and = 10⁻⁹. Compute the number of cleaning-iterations.

b) Compute the total volume of water required for cleaning.

c) Can the total volume be reduced by reducing β? If so, determine the optimal β.

d) Give a formula for the time required for cleaning the bucket.

e) How can the time for cleaning the bucket be minimized?

(35)

Chapter 3 Calculus – Selected Topics

3.1 Sequences and Convergence

Definition 3.1 A function N→R, n7→a_n is called sequence.

Notation: (a_n)n∈N or (a₁, a₂, a₃, ...) Example 3.1

(1,2,3,4, ...) = (n)n∈N

(1,¹₂,¹₃,¹₄, ...) = (_n¹)n∈N

(1,2,4,8,16, ...) = (2ⁿ⁻¹)n∈N

Consider the following sequences:

1. 1,2,3,5,7,11,13,17,19,23,...

2. 1,3,6,10,15,21,28,36,45,55,66,..

3. 1,1,2,3,5,8,13,21,34,55,89,...

4. 8,9,1,-8,-10,-3,6,9,4,-6,-10

5. 1,2,3,4,6,7,9,10,11,13,14,15,16,17,18,19,21,22,23,24,26,27,29,30,31,32,33,34,35,36, 37,..

6. 1,3,5,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,31,33, 35, 37,38,39,41,43,..

Find the next 5 elements of each sequence. If you do not get ahead or want to solve other riddles additionaly, have a look at http://www.oeis.org.

(36)

Definition 3.2 (a_n)n∈N is called bounded, if there is A, B ∈Rwith ∀n A ≤a_n≤B (a_n)n∈N is called monotonically increasing/decreasing, iff ∀n a_n+1 ≥ a_n (a_n+1 ≤ an)