Vector spaces

Vector Space

Definition

A vector space over a Field $F$ is a non-empty set $V$ together with a binary operation and a binary function that satisfy the eight axioms listed below.

• In this context, the elements of $V$ are commonly called vectors, and the elements of $F$ are called scalars.
• The binary operation, called vector addition or simply addition assigns to any two vectors $v$ and $w$ in $V$ a third vector in $V$ which is commonly written as $v+w$, and called the sum of these two vectors.
• The binary function, called scalar multiplication, assigns to any scalar $a$ in $F$ and any vector $v$ in $V$ another vector in $V$, which is denoted $av$

	Vector space axioms
Vector Addition	Associativity	$u+(v+w)=(u+v)+w$
	Commutativity	$u+v=v+u$
	Identity element
	Inverse elements
Scalar Multiplication	Distributivity (vector addition)	$a(u+v)=au+av$
	Distributivity (field addition)	$(a+b)v=av+bv$
	Compatibility with field multiplication	$a(bv)=(ab)v$
	Identity element

it has to add closure property (for vector addition and scalar mul.) depending on definition of binary operation

An equivalent definition of a vector space can be given: the first four axioms (related to vector addition) say that a vector space is an abelian group under addition, and the four remaining axioms (related to the scalar multiplication) say that this operation defines a ring homomorphism from the field F into the endomorphism ring of this group

Properties

(7.2)

• $0\mathbf{u}=0$
• $k0=0$
• $(-1)\mathbf{u}=-\mathbf{u}$
• $(-k)\mathbf{u}=k(-\mathbf{u})=-(k\mathbf{u})$
• $k\mathbf{u}=0⟹k=0\vee \mathbf{u}=0$
• $\mathbf{u}+\mathbf{u}=\mathbf{u}⟹\mathbf{u}=0$
• $-(-\mathbf{u})=\mathbf{u}$
• $-(\mathbf{u}+\mathbf{v})=(-\mathbf{u})+(-\mathbf{v})$
• $\mathbf{u}-\mathbf{v}=0⟹\mathbf{u}=\mathbf{v}$

Operations

• Scalar Multiplication ($\text{scalar}\cdot \mathbf{vector}=\mathbf{vector}$)
• Vector Addition ($\mathbf{vector}+\mathbf{vector}=\mathbf{vector}$)
• Dot Product ($\mathbf{vector}\cdot \mathbf{vector}=\text{saclar}$)
• Cross Product ($\mathbf{vector}\times \mathbf{vector}=\mathbf{vector}$)

Dot Product

also scalar product or Euclidean inner product

Definition:

• Coordinate definition: $\mathbf{a}\cdot \mathbf{b}=\sum _{i=1}^n{a}_i{b}_i=a_1{b}_1+a_2{b}_2+\cdots +a_n{b}_n$
• Geometric definition: $\mathbf{a}\cdot \mathbf{b}=\Vert \mathbf{a}\Vert \Vert \mathbf{b}\Vert \cos \theta$

Properties: (12.1.2)

• Symmetry $\text{a}\cdot \text{b}=\text{b}\cdot \text{a}$
• Distributive $(\text{a}+\text{b})\cdot \text{c}=\text{a}\cdot \text{c}+\text{b}\cdot \text{c}$
• Homogeneity $(t\text{a})\cdot \text{b}=t(\text{a}\cdot \text{b})$
• Positivity $\text{a}\cdot \text{a}\ge 0$
  • $\text{a}\cdot \text{a}=0⟺\text{a}=\text{0}$
• $\text{0}\cdot \text{a}=\text{a}\cdot \text{0}=0$

Cross Product

• $\mathbf{a}\times \mathbf{b}=\Vert \mathbf{a}\Vert \Vert \mathbf{b}\Vert \sin (\theta )\mathbf{n}$not-in-course

Subspaces

• A subset $W$ of a vector space $V$ is called a subspace of $V$ if $W$ is itself a vector space under the addition and scalar multiplication defined on $V$
• nonempty subset $W$ of $V$ is a subspace of $V$ if and only if it is closed under addition and scalar multiplication

Aritmetic

$U$ and $W$ are subspaces of $V$.

• (q7.6.2) $U\cup W$ is subspace of $V$, if and only if, $U\subseteq W\vee W\subseteq U$

Sum

$U+W:=\{u+w\mid u\in U,w\in W\}$

Properties:

• (q7.6.3a) $U+W=W+U$
• (q7.6.3b) $U_1+(U_2+U_3)=(U_1+U_2)+U_3$
• (q7.6.5) $U+W=U⟺W\subseteq U$
• (q7.6.6) $\text{Sp}(U_1\cup \cdots \cup U_k)=U_1+\cdots +U_k$
• (q7.6.7) $U+W=U\cup W$, if and only if, $U\subseteq W\vee W\subseteq U$
• (q7.6.8) let $K,S\subseteq V$ non-empty sets, then $\text{Sp}(K)+\text{Sp}(S)=\text{Sp}(K\cup S)$

Direct sum

Let $U+W=V$, then the following statements are equivalent:

• $U\oplus W=V$
• (7.7.1) every vector in $V$ can be expressed in exactly one way as $u+w=v$
• (7.7.2) $U\cap W=\{0\}$
• (8.3.7) $\dim U+\dim W=\dim V$

Equality

• (7.5.12) If $A$ and $B$ are row equivalent matrices, then $\text{row-space}(A)=\text{row-space}(B)$
• (8.3.4a) $U\subseteq V⟹\dim U\le \dim V$.
• (8.3.4b) if $U\subseteq V$ then, $U=V⟺\dim V=\dim U$.

Isomorphic Subspaces

Isomorphic is an equivalence relation

(Assumption: the spaces are on the same filed $F$)

Definition; The following statements are equivalent:

• $V$ and $W$ are isomorphic: $V\cong W$
• There exists isomorphism $T:V\to W$
• (9.5.7, 9.5.9) $\dim V=\dim W$

Theorems:

• (9.5.8) $\dim V=n⟹V\cong F^n$

Dimension

• (8.3.6) $\dim (U+W)=\dim U+\dim W-\dim (U\cap W)$
• (8.3.7) if $V=U+W$, then $V=U\oplus W⟺\dim V=\dim U+\dim W$
• (9.5.9) $V\cong W⟺\text{dim}V=\text{dim}W$ (on the same field, and finite dim.)
• (9.6.1) Rank–nullity theorem $\text{dim}V=\text{dim}(\text{Ker}T)+\text{dim}(\text{Im}T)$. ($T:V\to W$ is lin. trans.)
• (d8.5.4) $\rho (A)=\dim (\text{row-space(A)})=\dim (\text{column-space(A)})$

Orthogonality

• (d12.2.2) $\mathbf{v}\perp U$ if for all vectors $\mathbf{u}\in U$, $\mathbf{v}\cdot \mathbf{u}=0$
• (12.2.3) let $\text{Sp}(K)=U$, then $\mathbf{v}\perp U⟺\mathbf{v}\perp K$

Orthogonal Complement

• (d12.2.4) The Orthogonal Complement - $U^{\perp }=\{\mathbf{v}\in {\mathbb{R}}^n\mid \mathbf{v}\cdot \mathbf{u}=0,\forall \mathbf{u}\in U\}$
  • (q12.2.7) $U^{\perp }\cap U=\{0\}$
• (12.3.1) $\text{row-space}(A)\perp \text{null}(A)$
  • $\text{column-space}(A)\perp \text{null}(A^t)$
• (12.3.2) The Orthogonal Decomposition of the Euclidean space of dimension $n$.
  • (12.3.2a) $U^{\perp }\oplus U={\mathbb{R}}^n$
  • (12.3.2a) $\dim U^{\perp }+\dim U=n$
  • (12.3.2b) $(U^{\perp }{)}^{\perp }=U$
• ${(U+W)}^{\perp }=U^{\perp }\cap W^{\perp }$

Orthogonal Projection

• Definition: the orthogonal projection of $\mathbf{a}$ onto $\mathbf{u}$ is $\frac{\mathbf{a}\cdot \mathbf{u}}{\mathbf{u}\cdot \mathbf{u}}\mathbf{u}$.

• Definition: the orthogonal projection of $\mathbf{a}$ onto $U\subseteq {\mathbb{R}}^n$ is the vector $\text{u}={\sum }_{i=1}^k\frac{\mathbf{a}\cdot {\mathbf{u}}_i}{\Vert {\text{u}}_i{\Vert }^2}{\text{u}}_i={\text{proj}}_U\mathbf{a}$

  • (where $({\mathbf{u}}_1,\dots ,{\mathbf{u}}_k)$ is orthogonal basis of $U$, and $0\ne \mathbf{a}\in {\mathbb{R}}^n$, and $\mathbf{a}=\mathbf{u}+\mathbf{v}$ where $\mathbf{u}\in U$ and $\mathbf{v}\in U^{\perp }$) (after12.3.2, 12.4.6)
  • $\mathbf{a}=\mathbf{u}+\mathbf{v}$ is the uniqe form of $\mathbf{a}$ as vector in $U$ and vector in $U^{\perp }$
  • The orthogonal projection of $\mathbf{a}$ onto $U$ is the closest vector to $\mathbf{a}$ in $U$
  • if $({\mathbf{u}}_1,\dots ,{\mathbf{u}}_k)$ is orthonormal basis of $U$, then $\text{u}={\sum }_{i=1}^k(\mathbf{a}\cdot {\mathbf{u}}_i){\text{u}}_i={\text{proj}}_U\mathbf{a}$

• Projection Theorem - if $\mathbf{v}$ and $\mathbf{u}$ are vectors in ${\mathbb{R}}^n$, and $\mathbf{u}\ne 0$, then $\mathbf{v}$ can be expressed in excatly one way in the form $\mathbf{v}=t\mathbf{u}+\mathbf{w}$, where $t$ is a scalar, and $\mathbf{w}\perp \mathbf{u}$.

• (12.5.1) $K=\{{\mathbf{u}}_1,\dots ,{\mathbf{u}}_k\}$ is orthogonal set, and $U=\text{Sp}(K)$ and $\mathbf{v}=\mathbf{a}-{\text{proj}}_U\mathbf{a}$, then:

  1. $\mathbf{v}\perp K$, and $\mathbf{v}\perp U$
  2. $\mathbf{v}=0⟺\mathbf{a}\in U$

Methods

• given subspace $U$, find the orthogonal projection of $\mathbf{a}$ on $U^{\perp }$. (e2023a.85.q1b)
  • find basis for $U$
  • find basis for $U^{\perp }$
  • normalize this $U^{\perp }$ basis into $({\mathbf{u}}_1,\dots ,{\mathbf{u}}_k)$
  • then $\text{u}={\sum }_{i=1}^k(\mathbf{a}\cdot {\mathbf{u}}_i){\text{u}}_i={\text{proj}}_{U^{\perp }}\mathbf{a}$