Notes on Linear Algebra
David K. Zhang
Last Modified 2022-03-15
Let ⟨F;0,1,−F,+F,⋅F⟩ be a field. A vector space over F is an algebraic structure ⟨V;0,−V,+V,⋅V⟩ consisting of:
- a set V, called the underlying set;
- a distinguished element 0∈V, called the zero vector;
- a unary operation −V:V→V, written as v↦−v, called vector negation;
- a binary operation +V:V×V→V, written as (v,w)↦v+w, called vector addition;
- a binary operation ⋅V:F×V→V, written as (a,v)↦a⋅v, called scalar multiplication;
satisfying the following requirements:
- Additive structure: ⟨V;0,−V,+V⟩ is an abelian group.
- Identity: 1⋅v=v for all v∈V.
- Multiplicative compatibility: (a⋅b)⋅v=a⋅(b⋅v) for all a,b∈F and v∈V.
- Left distributivity: a⋅(v+w)=(a⋅v)+(a⋅w) for all a∈F and v,w∈V.
- Right distributivity: (a+b)⋅v=(a⋅v)+(b⋅v) for all a,b∈F and v∈V.
The elements of V are called vectors, and the elements of F are called scalars.
In these notes, we will use boldface upright letters, such as v and w, to denote vectors. On the other hand, scalars will be denoted by italic letters, such as a and b. This will allow us to distinguish vector variables from scalar variables at a glance. (This convention is commonly adopted in physics and engineering, but is uncommon in pure mathematics.)
Adopting this convention also provides another benefit: namely, we no longer need to distinguish between the operations of scalar addition +F:F×F→F and vector addition +V:V×V→V. Instead, we proceed with the implicit understanding that whenever the + sign occurs between two scalars, as in a+b, it refers to scalar addition. Likewise, whenever it occurs between two vectors, as in v+w, it refers to vector addition.
The same goes for the two multiplication operations ⋅F:F×F→F and ⋅V:F×V→V. Moving forward, these operations will be denoted by juxtaposition, as in ab and av. The requirement of multiplicative compatibility allows us to write abv without ambiguity, and the identity property allows us to freely insert and remove factors of 1.
As with groups, rings, and fields, it is common to denote a vector space ⟨V;0V,−V,+V,⋅V⟩ simply by the name of its underlying set V. Thus, we will often say “let V be a vector space over a field F.”
Let V be a vector space over a field F, and let n∈N. A linear combination of a finite sequence of vectors v1,v2,…,vn∈V is a vector of the form
a1v1+a2v2+⋯+anvnwhere a1,a2,…,an∈F. We call the sequence of scalars a1,a2,…,an the coefficients of the linear combination. For n=0, we define the unique linear combination of the empty sequence of vectors to be 0.
Let V be a vector space over a field F. The span of a subset S⊆V, denoted by spanS, is the set of all linear combinations of all finite sequences of vectors in S.
spanS:={a1v1+a2v2+⋯+anvn:n∈N, a1,a2,…,an∈F, v1,v2,…,vn∈S}
Because we defined 0 to be the unique linear combination of an empty set of vectors, we always have 0∈spanS for any subset S⊆V, including the empty set.
Let V be a vector space over a field F. A linear subspace of V is a subset S⊆V that is closed under taking linear combinations, i.e., every linear combination of every finite sequence of vectors in S is itself in S.
Note that the empty set ∅⊆V is not a linear subspace! The requirement of closure under linear combinations includes the empty linear combination, which the empty set fails to satisfy, since 0∈/∅. Because of this requirement, a linear subspace must always contain the zero vector.
Theorem: Let V be a vector space over a field F. A subset S⊆V is a linear subspace of V if and only if S has the following two properties:
- Contains the zero vector: 0∈S.
- Closed under pairwise linear combinations: For all a,b∈F and v,w∈S, we have av+bw∈S.
Proof sketch: We prove by induction on n∈N that S is closed under taking n-ary linear combinations. The first property handles the base case n=0, and the second property handles the inductive step, using the fact that an n-ary linear combination can be written as a binary linear combination of a vector and an (n−1)-ary linear combination. □
Let V be a vector space over a field F, and let n∈N. An affine combination of a finite sequence of vectors v1,v2,…,vn∈V is a vector of the form a1v1+a2v2+⋯+anvn where the coefficients a1,a2,…,an∈F satisfy a1+a2+⋯+an=1.
Note that, unlike linear combinations, there is no such thing as an empty affine combination. The sum of an empty sequence of scalars is 0, and by the definition of a field, 0=1.
Let V be a vector space over a field F. The affine span or affine hull of a subset S⊆V, denoted by affS, is the set of all affine combinations of all finite sequences of vectors in S.
Let V be a vector space over a field F. An affine set in V is a subset S⊆V that is closed under taking affine combinations, i.e., every affine combination of every finite sequence of vectors in S is itself in S.
Because there are no empty affine combinations, the empty set ∅∈V does (vacuously) qualify as an affine set.
Let V be a vector space over an ordered field (F,<), and let n∈N. A conic combination of a finite sequence of vectors v1,v2,…,vn∈V is a vector of the form a1v1+a2v2+⋯+anvn where the coefficients a1,a2,…,an∈F are all nonnegative, i.e., ai≥0 for all i∈{1,2,…,n}.
Let V be a vector space over an ordered field (F,<). The conic hull of a set S⊆V, denoted by coneS, is the set of all conic combinations of all finite sequences of vectors in S.
Let V be a vector space over an ordered field (F,<). A cone in V is a subset S⊆V that is closed under scalar multiplication by nonnegative scalars, i.e., if a∈F satisfies a≥0 and v∈S, then av∈S.
Note that a cone is not necessarily closed under taking conic combinations!
Let V be a vector space over an ordered field (F,<), and let n∈N. A convex combination of a finite sequence of vectors v1,v2,…,vn∈V is a vector of the form a1v1+a2v2+⋯+anvn where the coefficients a1,a2,…,an∈F satisfy a1+a2+⋯+an=1 and are all nonnegative, i.e., ai≥0 for all i∈{1,2,…,n}.
Let V be a vector space over an ordered field (F,<). The convex hull of a set S⊆V, denoted by convS, is the set of all convex combinations of all finite sequences of vectors in S.
Let V be a vector space over an ordered field (F,<). A convex set in V is a subset S⊆V that is closed under taking convex combinations, i.e., every convex combination of every finite sequence of vectors in S is itself in S.
A real vector space is a vector space over the field R of real numbers. Similarly, a complex vector space is a vector space over the field C of complex numbers.