Three Archetypical Linear Problems

We shall now take our newly gained geometrical familiarity with infinite dimensional vector spaces and apply it to each of three fundamental problems which, in linear algebra, have the form

1. $(A-\lambda B)\vec u = 0$
2. $(A-\lambda B)G=I$
3. $(A-\lambda B)\vec u = \vec b$ ,

i.e., the eigenvalue problem, the problem of inverting the matrix $A-\lambda B$ , and the inhomogeneous problem.

The most important of these three is the eigenvalue problem because once it has been solved, the solutions to the others follow directly.

Indeed, assume that we found for the vector space a basis of eigenvectors, say

$$\{ \vec u_1,\dots ,\vec u_N\}$$

as determined by

$$A\vec u = \lambda B\vec u$$

(We are assuming that the matrices $A$ and $B$ are such that an eigenbasis does indeed exist.) In that case, the solutions to problems 2 and 3 are given by

$$G = \sum^N_{i=1}\frac{\vec u_i \vec u^H_i}{\lambda_i -\lambda}$$

and

$$\vec u = \sum^N_{i=1}\frac{\vec u_i\langle \vec u_i,\vec b\rangle}{\lambda_i -\lambda}$$

respectively, as one can readily verify. Here $\vec u^H_i$ refers to the Hermitian adjoint of the vector $\vec u_i$ .

On the other hand, suppose we somehow solved problem 2 and found its solution to be

$$G = G_\lambda\,.$$

Then it turns out that the complex contour integral of that solution, namely

$$\frac{1}{2\pi i} \oint G_\lambda d\lambda = - \sum^N_{i=1} \vec u_i \vec u^H_i\,,$$

yields the sum of the products

$$-\sum^N_{i=1}\vec u_i\vec u^H_i$$

of the eigenvectors $\vec u_i$ ( $i=1,\dots ,N$ ) of the eigenvalue problem 1. Thus solving problem 2 yields the solution to problem 1. It also, of course, yields the solution to problem 3, namely

$$\vec u = G\vec b\,.$$

Thus, in a sense, problem 1 and problem 2 are equally important.

We shall extend our considerations of problems 1-3 from finite dimensional to infinite dimensional vector spaces. We shall do this by letting $A$ be a second order differential operator and $B$ a positive definite function. In this case, problem 1 becomes a homogeneous boundary value problem, most often a so-called Sturm-Liouville problem, which we shall formulate and solve. Problem 2 becomes the problem of finding the so-called Green's function. This will be done in the next chapter. There we shall also formulate and solve the inhomogeneous boundary value problem corresponding to problem 3.

We extend problems 1-3 to infinite dimensions by focussing on second order linear ordinary differential equations and their solutions. They are the most important and they illustrate most of the key ideas.

It is difficult to overstate the importance of Sturm-Liouville theory. Not only does it provide a practical means for dealing with those phenomena (namely wave propagation and vibrations) that underly twentieth century science and technology, but it also provides a very powerful way of reasoning which deals with the qualitative essentials, and not only with the quantitative details.

A Sturm-Liouville eigenvalue problem gives rise to eigenfunctions. It is extremely beneficial to view them as basis vectors which span an inner product space. Doing so places the theory of linear d.e.'s into the framework of Linear Algebra, thus yielding an easy panoramic view of the field. In particular, it allows us to apply our geometrical mode of reasoning to the Sturm-Liouville problem.