# Notes on Linear Algebra

David K. Zhang

Definition: Vector Space, Vector, Scalar

Let $\alg{F; 0, 1, -_F, +_F, \cdot_F}$ be a field. A vector space over $F$ is an algebraic structure $\alg{V; \vo, -_V, +_V, \cdot_V}$ consisting of:

• a set $V$, called the underlying set;
• a distinguished element $\vo \in V$, called the zero vector;
• a unary operation $-_V: V \to V$, written as $\vv \mapsto -\vv$, called vector negation;
• a binary operation $+_V: V \times V \to V$, written as $(\vv, \vw) \mapsto \vv + \vw$, called vector addition;
• a binary operation $\cdot_V: F \times V \to V$, written as $(a, \vv) \mapsto a \cdot \vv$, called scalar multiplication;

satisfying the following requirements:

• Additive structure: $\alg{V; \vo, -_V, +_V}$ is an abelian group.
• Identity: $1 \cdot \vv = \vv$ for all $\vv \in V$.
• Multiplicative compatibility: $(a \cdot b) \cdot \vv = a \cdot (b \cdot \vv)$ for all $a, b \in F$ and $\vv \in V$.
• Left distributivity: $a \cdot (\vv + \vw) = (a \cdot \vv) + (a \cdot \vw)$ for all $a \in F$ and $\vv, \vw \in V$.
• Right distributivity: $(a + b) \cdot \vv = (a \cdot \vv) + (b \cdot \vv)$ for all $a, b \in F$ and $\vv \in V$.

The elements of $V$ are called vectors, and the elements of $F$ are called scalars.

In these notes, we will use boldface upright letters, such as $\vv$ and $\vw$, to denote vectors. On the other hand, scalars will be denoted by italic letters, such as $a$ and $b$. This will allow us to distinguish vector variables from scalar variables at a glance. (This convention is commonly adopted in physics and engineering, but is uncommon in pure mathematics.)

Adopting this convention also provides another benefit: namely, we no longer need to distinguish between the operations of scalar addition $+_F: F \times F \to F$ and vector addition $+_V: V \times V \to V$. Instead, we proceed with the implicit understanding that whenever the $+$ sign occurs between two scalars, as in $a + b$, it refers to scalar addition. Likewise, whenever it occurs between two vectors, as in $\vv + \vw$, it refers to vector addition.

The same goes for the two multiplication operations $\cdot_F: F \times F \to F$ and $\cdot_V: F \times V \to V$. Moving forward, these operations will be denoted by juxtaposition, as in $ab$ and $a\vv$. The requirement of multiplicative compatibility allows us to write $ab\vv$ without ambiguity, and the identity property allows us to freely insert and remove factors of $1$.

As with groups, rings, and fields, it is common to denote a vector space $\alg{V; \vo_V, -_V, +_V, \cdot_V}$ simply by the name of its underlying set $V$. Thus, we will often say “let $V$ be a vector space over a field $F$.”

Definition: Linear Combination

Let $V$ be a vector space over a field $F$, and let $n \in \N$. A linear combination of a finite sequence of vectors $\vv_1, \vv_2, \dots, \vv_n \in V$ is a vector of the form

$a_1 \vv_1 + a_2 \vv_2 + \cdots + a_n \vv_n$

where $a_1, a_2, \dots, a_n \in F$. We call the sequence of scalars $a_1, a_2, \dots, a_n$ the coefficients of the linear combination. For $n = 0$, we define the unique linear combination of the empty sequence of vectors to be $\vo$.

Definition: Span

Let $V$ be a vector space over a field $F$. The span of a subset $S \subseteq V$, denoted by $\vspan S$, is the set of all linear combinations of all finite sequences of vectors in $S$.

$\vspan S \coloneqq \{ a_1 \vv_1 + a_2 \vv_2 + \cdots + a_n \vv_n : n \in \N,\ a_1, a_2, \dots, a_n \in F,\ \vv_1, \vv_2, \dots, \vv_n \in S \}$

Because we defined $\vo$ to be the unique linear combination of an empty set of vectors, we always have $\vo \in \vspan S$ for any subset $S \subseteq V$, including the empty set.

Definition: Linear Subspace

Let $V$ be a vector space over a field $F$. A linear subspace of $V$ is a subset $S \subseteq V$ that is closed under taking linear combinations, i.e., every linear combination of every finite sequence of vectors in $S$ is itself in $S$.

Note that the empty set $\varnothing \subseteq V$ is not a linear subspace! The requirement of closure under linear combinations includes the empty linear combination, which the empty set fails to satisfy, since $\vo \notin \varnothing$. Because of this requirement, a linear subspace must always contain the zero vector.

Pairwise Linear Combinations Suffice

Theorem: Let $V$ be a vector space over a field $F$. A subset $S \subseteq V$ is a linear subspace of $V$ if and only if $S$ has the following two properties:

• Contains the zero vector: $\vo \in S$.
• Closed under pairwise linear combinations: For all $a, b \in F$ and $\vv, \vw \in S$, we have $a\vv + b\vw \in S$.

Proof sketch: We prove by induction on $n \in \N$ that $S$ is closed under taking $n$-ary linear combinations. The first property handles the base case $n = 0$, and the second property handles the inductive step, using the fact that an $n$-ary linear combination can be written as a binary linear combination of a vector and an $(n-1)$-ary linear combination.

Definition: Affine Combination

Let $V$ be a vector space over a field $F$, and let $n \in \N$. An affine combination of a finite sequence of vectors $\vv_1, \vv_2, \dots, \vv_n \in V$ is a vector of the form $a_1 \vv_1 + a_2 \vv_2 + \cdots + a_n \vv_n$ where the coefficients $a_1, a_2, \dots, a_n \in F$ satisfy $a_1 + a_2 + \cdots + a_n = 1$.

Note that, unlike linear combinations, there is no such thing as an empty affine combination. The sum of an empty sequence of scalars is $0$, and by the definition of a field, $0 \ne 1$.

Definition: Affine Span, Affine Hull

Let $V$ be a vector space over a field $F$. The affine span or affine hull of a subset $S \subseteq V$, denoted by $\aff S$, is the set of all affine combinations of all finite sequences of vectors in $S$.

Definition: Affine Set

Let $V$ be a vector space over a field $F$. An affine set in $V$ is a subset $S \subseteq V$ that is closed under taking affine combinations, i.e., every affine combination of every finite sequence of vectors in $S$ is itself in $S$.

Because there are no empty affine combinations, the empty set $\varnothing \in V$ does (vacuously) qualify as an affine set.

Definition: Conic Combination

Let $V$ be a vector space over an ordered field $(F, <)$, and let $n \in \N$. A conic combination of a finite sequence of vectors $\vv_1, \vv_2, \dots, \vv_n \in V$ is a vector of the form $a_1 \vv_1 + a_2 \vv_2 + \cdots + a_n \vv_n$ where the coefficients $a_1, a_2, \dots, a_n \in F$ are all nonnegative, i.e., $a_i \ge 0$ for all $i \in \{1, 2, \dots, n \}$.

Definition: Conic Hull

Let $V$ be a vector space over an ordered field $(F, <)$. The conic hull of a set $S \subseteq V$, denoted by $\cone S$, is the set of all conic combinations of all finite sequences of vectors in $S$.

Definition: Cone

Let $V$ be a vector space over an ordered field $(F, <)$. A cone in $V$ is a subset $S \subseteq V$ that is closed under scalar multiplication by nonnegative scalars, i.e., if $a \in F$ satisfies $a \ge 0$ and $\vv \in S$, then $a\vv \in S$.

Note that a cone is not necessarily closed under taking conic combinations!

Definition: Convex Combination

Let $V$ be a vector space over an ordered field $(F, <)$, and let $n \in \N$. A convex combination of a finite sequence of vectors $\vv_1, \vv_2, \dots, \vv_n \in V$ is a vector of the form $a_1 \vv_1 + a_2 \vv_2 + \cdots + a_n \vv_n$ where the coefficients $a_1, a_2, \dots, a_n \in F$ satisfy $a_1 + a_2 + \cdots + a_n = 1$ and are all nonnegative, i.e., $a_i \ge 0$ for all $i \in \{1, 2, \dots, n \}$.

Definition: Convex Hull

Let $V$ be a vector space over an ordered field $(F, <)$. The convex hull of a set $S \subseteq V$, denoted by $\conv S$, is the set of all convex combinations of all finite sequences of vectors in $S$.

Definition: Convex Set

Let $V$ be a vector space over an ordered field $(F, <)$. A convex set in $V$ is a subset $S \subseteq V$ that is closed under taking convex combinations, i.e., every convex combination of every finite sequence of vectors in $S$ is itself in $S$.

Definition: Real Vector Space, Complex Vector Space

A real vector space is a vector space over the field $\R$ of real numbers. Similarly, a complex vector space is a vector space over the field $\C$ of complex numbers.