Vectors set

$K=\{v_1,\dots ,v_m\}\subseteq V,\dim V=n$

• $A$ is $K$‘s vectors as rows
• $A^{\top }$ is $K$‘s vectors as comluns
• $r=\text{rank}(A)$
• $B$ is some basis of $V$

Linear independence

Definitions of linearly independent. The following statements are equivalent:

• $K$ is linearly independent
• ${\lambda }_1{v}_1+{\lambda }_2{v}_2+\cdots +{\lambda }_m{v}_m=0⟹{\lambda }_1,\dots ,{\lambda }_m=0$
• $A^{\top }\mathbf{x}=0⟹\mathbf{x}=0$
• $\text{rank}(A)=m\le n$
• (8.4.4) $\{[v_1{]}_B,\dots ,[v_m{]}_B\}\text{ is linearly independent}$
• $\text{REF}(A^{\top })$ has a pivot position in every columntodo

Theorems:

• $0\in K⟹K\text{ is linearly dependent}$

• $m>n⟹K\text{ is linearly dependent}$

• Dimension of the Span:

  • $K\text{ is linearly dependent}⟸\dim \text{Sp}(K)<m$
  • $K\text{ is linearly independent}⟹\dim \text{Sp}(K)=m$

• if $m=n$, then:

  • $A$ or $A^{\top }$ is invertible matrix, if and only if, $K\text{ is linearly independent}$
  • $K\text{ is linearly independent}$, if and only if, $\det (A)\ne 0$,

• (11.2.4) Eigenvectors corresponding to distinct eigenvalues are linearly independent

• let $L,K\subseteq V$, and $K\subseteq \text{Sp}(L)$, and $L$ is linearly dependent, then $K$ is also linearly dependent (by 7.5.1, 8.3.4)

Span

Definitions. The following statements are equivalent:

• $K\text{ spans }V$, that is $\text{Sp}(K)=V$
• $\text{rank}(A)=\dim \text{Sp(K)}$. ($A$ is $K$‘s vectors as rows)
• $A\mathbf{x}=\mathbf{b}\text{ is consistent}$. ($A$ is $K$‘s vectors as columns)

Theorems:

• $\text{Sp}(K)$ is subspace
• $B\subseteq \text{Sp}(K)⟹K\text{ spans }V$. (where $\text{Sp}(B)=V$)
• $m<n⟹K\text{ is not span }V$
• (7.5.4) $\text{Sp}(K)=\text{Sp}(L)⟺K\subseteq \text{Sp}(L)\wedge L\subseteq \text{Sp}(K)$
• (7.5.1, q7.5.16b) $K\subseteq \text{Sp}(L)⟹\text{Sp}(K)\subseteq \text{Sp}(L)$
• (q7.5.16a) $K\subseteq L⟹\text{Sp}(K)\subseteq \text{Sp}(L)$
• (q7.5.17a) $\text{Sp}(K)\cup \text{Sp}(L)\subseteq \text{Sp}(K\cup L)$
• (q7.5.17b) $\text{Sp}(K\cap L)\subseteq \text{Sp}(K)\cap \text{Sp}(L)$

Basis

Definitions of basis. The following statements are equivalent:

• $K$ is a basis of $V$
• (8.2.5) every element of $V$ may be written in a unique way as a finite linear combination of elements of $K$.
• (8.3.2) Two out of three are fulfilled
  1. $K\text{ is linearly independent}$
  2. $\text{Sp}(K)=V$ (K spans V)
  3. $m=\dim V$
• (8.4.5) $m=\dim V$, and the transition matrix from some basis $B$ to $K$ is invertible

Bases for the Fundamental Spaces

• A basis of $\text{row-space}(A)=\text{Sp}(K)$

  • The non-zero rows of $\text{REF}(A)$ (q8.5.2b)
  • The $r$ columns in $A^t$, such that in $\text{RREF}(A^t)$ are contain a pivot.

• A basis of $\text{column-space}(A)$.

  • The non-zero rows of $\text{REF}(A^t)$
  • The $r$ columns in $A$, such that in $\text{RREF}(A)$ are contain a pivot.

• A basis of the $\text{null}(A)$

  • The $n-r$ vectors that span the solution space of $(\text{RREF}(A))\mathbf{x}=0$.

• A basis of the $\text{null}(A^t)=\text{left-null}(A^t)$

  • The $m-r$ vectors that span the solution space of $(\text{RREF}(A^t))\mathbf{x}=0$.

• todo Let $K$ is linearly independent, then $K\cup \{{\mathbf{e}}_i\mid i\text{th column is not a pivot column in REF}(A)\}$ forms a basis for $V$.

Orthogonality

(12.2.3) The following statements are equivalent:

• $\mathbf{u}$ orthogonal to $K$
• $\forall \mathbf{v}\in K:\mathbf{u}\cdot \mathbf{v}=0$
• $\mathbf{u}\perp K$
• $\mathbf{u}\perp \text{Sp}(K)$
• $\mathbf{u}\in (\text{Sp}(K){)}^{\perp }$
• $\mathbf{u}\in \text{null}(A)$
• $A\mathbf{u}=0$

Orthogonal set

Definition:

• (d12.4.1a) let $K=\{{\text{u}}_1,\dots ,{\text{u}}_k\}\subseteq {\mathbb{R}}^n$. we say that $K$ is a orthogonal set, if $\text{0}\notin K$, and $\forall j\ne i:{\text{u}}_i\cdot {\text{u}}_j=0$

Properties: $K=\{{\text{u}}_1,\dots ,{\text{u}}_k\}\subseteq {\mathbb{R}}^n$ is orthogonal set. then:

• (12.4.2) $K$ is independent set
• (q12.4.3a) $K$ has at most $n$ vectors
• $K$ is a basis of $\text{Sp}(K)$

Orthogonal basis

Orthonormal set

• (d12.4.1b) $K=\{{\text{u}}_1,\dots ,{\text{u}}_k\}$ is orthonormal set if, $\Vert {\text{u}}_i\Vert =1$ for each $i$, ($1\le i\le k$)
• (q12.4.2) Normalizing - if $K=\{{\mathbf{u}}_1,\dots ,{\text{u}}_k\}$ is orthogonal set, then $L=\{\frac{{\text{u}}_1}{\Vert {\text{u}}_1\Vert },\dots ,\frac{{\text{u}}_k}{\Vert {\text{u}}_k\Vert }\}$ is orthonormal set, and $\text{Sp}(K)=\text{Sp}(L)$
• The normalized vector $\hat{\mathbf{u}}$ of a non-zero vector $\mathbf{u}$ is the unit vector in the direction of $\mathbf{u}$. i.e. $\hat{\mathbf{u}}=\frac{\mathbf{u}}{\Vert \mathbf{u}\Vert }$
• A Unit vector is a vector $\text{v}$ such that $\Vert \text{v}\Vert =1$

Orthonormal basis

• (12.4.5) Let $B=(u_1,\dots ,u_n)$ ordered basis of ${\mathbb{R}}^n$, then the following properties are equivalence:
  • $B$ is orthonormal basis
  • $\text{a}={\sum }_{i=1}^n({\mathbf{a}\cdot \mathbf{u}}_i)\cdot {\mathbf{u}}_i$
  • $\mathbf{a}\cdot \mathbf{b}={\sum }_{i=1}^n({\mathbf{a}\cdot \mathbf{u}}_i)\cdot ({\mathbf{b}\cdot \mathbf{u}}_i)$
  • $\Vert \text{a}{\Vert }^2={\sum }_{i=1}^n({\mathbf{a}\cdot \mathbf{u}}_i{)}^2$
• (q12.4.10)todo generalition of 12.4.5 for orthogonal bases

Orthogonality of Sets

• $\forall a\in A,b\in B:a\perp b⟺A\perp B$

Gram–Schmidt process (12.5.2)

Convert a basis $\{{\mathbf{u}}_1,\dots ,{\mathbf{u}}_k\}$ into an orthogonal basis $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_k\}$:

${\mathbf{v}}_1={\mathbf{u}}_1$ ${\mathbf{v}}_i={\mathbf{u}}_i-\frac{{\mathbf{u}}_i\cdot {\mathbf{v}}_1}{\Vert {\mathbf{v}}_1{\Vert }^2}{\mathbf{v}}_1-\frac{{\mathbf{u}}_i\cdot {\mathbf{v}}_2}{\Vert {\mathbf{v}}_2{\Vert }^2}{\mathbf{v}}_2-\cdots -\frac{{\mathbf{u}}_i\cdot {\mathbf{v}}_{i-1}}{\Vert {\mathbf{v}}_{i-1}{\Vert }^2}{\mathbf{v}}_{i-1}$

• during the computation you can multiple ${\mathbf{v}}_i$ by a scalar (note before q12.5.4)
• To convert the orthogonal basis into an orthonormal basis see (q12.4.2)
• for dependent set see q12.5.3
• expanding orthogonal set of $k<n$ vectors into orthogonal basis see q12.5.4