Linear algebra and matrix theory have long been fundamental tools in mathematical disciplines as well as fertile fields for research in their own right. In this book, and in the companion volume, Topics in Matrix Analysis, we present classical and recent results of matrix analysis that have proved to be important to applied mathematics. The bookmay be used as an undergraduate or graduate text and as a self-contained reference for avariety of audiences. We assume background equivalent to a one-semester elementarylinear algebra course and knowledge of rudimentary analytical concepts. We beginwith the notions of eigenvalues and eigenvectors; no prior knowledge of these conceptsis assumed.
Facts about matrices, beyond those found in an elementary linear algebra course ,are necessary to understand virtually any area of mathematical science, whether it be differential equations; probability and statistics; optimization; or applications in theoretical and applied economics, the engineering disciplines, or operations research,to name only a few. But until recently, much of the necessary material has occurred sporadically (or not at all) in the undergraduate and graduate curricula. As interestin applied mathematics has grown and more courses have been devoted to advancedmatrix theory, the need for a text offering a broad selection of topics has become moreapparent, as has the need for a modern reference on the subject.
There are several well-loved classics in matrix theory, but they are not well suitedfor general classroom use, nor for systematic individual study. A lack of problems,applications, and motivation; an inadequate index; and a dated approach are amongthe difficulties confronting readers of some traditional references. More recent bookstend to be either elementary texts or treatises devoted to special topics. Our goalwas to write a book that would be a useful modern treatment of a broad range oftopics.
One view of “matrix analysis” is that it consists of those topics in linear algebrathat have arisen out of the needs of mathematical analysis, such as multivariablecalculus, complex variables, differential equations, optimization, and approximationtheory. Another view is that matrix analysis is an approach to real and complex linear algebraic problems that does not hesitate to use notions from analysis – such as limits,continuity, and power series – when these seem more efficient or natural than a purelyalgebraic approach. Both views of matrix analysis are reflected in the choice andtreatment of topics in this book. We prefer the term matrix analysis to linear algebraas an accurate reflection of the broad scope and methodology of the field.
For review and convenience in reference, Chapter 0 contains a summary ofnecessary facts from elementary linear algebra, as well as other useful, though notnecessarily elementary, facts. Chapters 1, 2, and 3 contain mainly core material likelyto be included in any second course in linear algebra or matrix theory: a basic treatmentof eigenvalues, eigenvectors, and similarity; unitary similarity, Schur triangularizationand its implications, and normal matrices; and canonical forms and factorizations,including the Jordan form, LU factorization,QRfactorization, and companion matrices.Beyond this, each chapter is developed substantially independently and treats in somedepth a major topic:
Hermitian and complex symmetric matrices (Chapter 4).We give special emphasisto variational methods for studying eigenvalues of Hermitian matrices and includean introduction to the notion of majorization.
Norms on vectors and matrices (Chapter 5) are essential for error analyses ofnumerical linear algebraic algorithms and for the study of matrix power series anditerative processes. We discuss the algebraic, geometric, and analytic propertiesof norms in some detail and make a careful distinction between those norm resultsfor matrices that depend on the submultiplicativity axiom for matrix norms andthose that do not.
Eigenvalue location and perturbation results (Chapter 6) for general (not necessarilyHermitian) matrices are important for many applications.We give a detailedtreatment of the theory of Gerˇsgorin regions, and some of its modern refinements,and of relevant graph theoretic concepts.
Positive definite matrices (Chapter 7) and their applications, including inequalities,are considered at some length. A discussion of the polar and singular value decompositionsis included, along with applications to matrix approximation problems.
Entry-wise nonnegative and positive matrices (Chapter 8) arise in many applicationsin which nonnegative quantities necessarily occur (probability, economics,engineering, etc.), and their remarkable theory reflects the applications. Our developmentof the theory of nonnegative, positive, primitive, and irreducible matricesproceeds in elementary steps based on the use of norms.
In the companion volume, further topics of similar interest are treated: the fieldof values and generalizations; inertia, stable matrices, M-matrices and related specialclasses; matrix equations, Kronecker and Hadamard products; and various ways inwhich functions and matrices may be linked.
This book provides the basis for a variety of one- or two-semester courses throughselection of chapters and sections appropriate to a particular audience.We recommendthat an instructor make a careful preselection of sections and portions of sections of thebook for the needs of a particular course. This would probably include Chapter 1, muchof Chapters 2 and 3, and facts about Hermitian matrices and norms from Chapters 4and 5.
Most chapters contain some relatively specialized or nontraditional material. Forexample, Chapter 2 includes not only Schur’s basic theorem on unitary triangularizationof a single matrix but also a discussion of simultaneous triangularization of families ofmatrices. In the section on unitary equivalence, our presentation of the usual facts isfollowed by a discussion of trace conditions for two matrices to be unitarily equivalent.A discussion of complex symmetric matrices in Chapter 4 provides a counterpoint tothe development of the classical theory of Hermitian matrices. Basic aspects of a topicappear in the initial sections of each chapter, while more elaborate discussions occur atthe ends of sections or in later sections. This strategy has the advantage of presentingtopics in a sequence that enhances the book’s utility as a reference. It also provides arich variety of options to the instructor.
Many of the results discussed are valid or can be generalized to be valid formatrices over other fields or in some broader algebraic setting. However, we deliberatelyconfine our domain to the real and complex fields where familiar methods of classicalanalysis as well as formal algebraic techniques may be employed.
Though we generally consider matrices to have complex entries, most examplesare confined to real matrices, and no deep knowledge of complex analysis is required.Acquaintance with the arithmetic of complex numbers is necessary for an understandingof matrix analysis and is covered to the extent necessary in an appendix. Other briefappendices cover several peripheral, but essential, topics such asWeierstrass’s theoremand convexity.
We have included many exercises and problems because we feel these areessential to the development of an understanding of the subject and its implications.The exercises occur throughout as part of the development of each section; they aregenerally elementary and of immediate use in understanding the concepts. We recommendthat the reader work at least a broad selection of these. Problems are listed(in no particular order) at the end of each section; they cover a range of difficultiesand types (from theoretical to computational) and they may extend the topic, developspecial aspects, or suggest alternate proofs of major ideas. Significant hints are givenfor the more difficult problems. The results of some problems are referred to in otherproblems or in the text itself. We cannot overemphasize the importance of the reader’sactive involvement in carrying out the exercises and solving problems.
While the book itself is not about applications, we have, for motivational purposes,begun each chapter with a section outlining a few applications to introduce the topicof the chapter.
Readers who wish to consult alternative treatments of a topic for additionalinformation are referred to the books listed in the References section following theappendices.
The list of book references is not exhaustive. As a practical concession to thelimits of space in a general multitopic book, we have minimized the number of citationsin the text. A small selection of references to papers – such as those we have explicitlyused – does occur at the end of most sections accompanied by a brief discussion, butwe have made no attempt to collect historical references to classical results. Extensivebibliographies are provided in the more specialized books we have referenced.
We appreciate the helpful suggestions of our colleagues and students who havetaken the time to convey their reactions to the class notes and preliminary manuscripts that were the precursors of the book. They includeWayne Barrett, Leroy Beasley, BryanCain, David Carlson, Dipa Choudhury, Risana Chowdhury, Yoo Pyo Hong, DmitryKrass, Dale Olesky, Stephen Pierce, Leiba Rodman, and Pauline van den Driessche.