Orthogonality

$K=\{v_1,\dots ,v_m\}\subseteq V,\dim V=n$

• $A$ is $K$‘s vectors as rows
• $A^{\top }$ is $K$‘s vectors as comluns
• $r=\text{rank}(A)$
• $B$ is some basis of $V$

Orthogonal Vector

(12.2.3) The following statements are equivalent:

• $\mathbf{u}$ orthogonal to $K$
• $\forall \mathbf{v}\in K:\mathbf{u}\cdot \mathbf{v}=0$
• $\mathbf{u}\perp K$
• $\mathbf{u}\perp \text{Sp}(K)$
• $\mathbf{u}\in (\text{Sp}(K){)}^{\perp }$
• $\mathbf{u}\in \text{null}(A)$
• $A\mathbf{u}=0$

Orthogonal set

Definition:

• (d12.4.1a) let $K=\{{\text{u}}_1,\dots ,{\text{u}}_k\}\subseteq {\mathbb{R}}^n$. we say that $K$ is a orthogonal set, if $\text{0}\notin K$, and $\forall j\ne i:{\text{u}}_i\cdot {\text{u}}_j=0$

Properties: $K=\{{\text{u}}_1,\dots ,{\text{u}}_k\}\subseteq {\mathbb{R}}^n$ is orthogonal set. then:

• (12.4.2) $K$ is independent set
• (q12.4.3a) $K$ has at most $n$ vectors
• $K$ is a basis of $\text{Sp}(K)$

Orthogonal basis

Orthonormal set

• (d12.4.1b) $K=\{{\text{u}}_1,\dots ,{\text{u}}_k\}$ is orthonormal set if, $\Vert {\text{u}}_i\Vert =1$ for each $i$, ($1\le i\le k$)
• (q12.4.2) Normalizing - if $K=\{{\mathbf{u}}_1,\dots ,{\text{u}}_k\}$ is orthogonal set, then $L=\{\frac{{\text{u}}_1}{\Vert {\text{u}}_1\Vert },\dots ,\frac{{\text{u}}_k}{\Vert {\text{u}}_k\Vert }\}$ is orthonormal set, and $\text{Sp}(K)=\text{Sp}(L)$
• The normalized vector $\hat{\mathbf{u}}$ of a non-zero vector $\mathbf{u}$ is the unit vector in the direction of $\mathbf{u}$. i.e. $\hat{\mathbf{u}}=\frac{\mathbf{u}}{\Vert \mathbf{u}\Vert }$
• A Unit vector is a vector $\text{v}$ such that $\Vert \text{v}\Vert =1$

Orthonormal basis

• (12.4.5) Let $B=(u_1,\dots ,u_n)$ ordered basis of ${\mathbb{R}}^n$, then the following properties are equivalence:
  • $B$ is orthonormal basis
  • $\text{a}={\sum }_{i=1}^n({\mathbf{a}\cdot \mathbf{u}}_i)\cdot {\mathbf{u}}_i$
  • $\mathbf{a}\cdot \mathbf{b}={\sum }_{i=1}^n({\mathbf{a}\cdot \mathbf{u}}_i)\cdot ({\mathbf{b}\cdot \mathbf{u}}_i)$
  • $\Vert \text{a}{\Vert }^2={\sum }_{i=1}^n({\mathbf{a}\cdot \mathbf{u}}_i{)}^2$
• (q12.4.10)todo generalition of 12.4.5 for orthogonal bases

Orthogonality of Sets

• $\forall a\in A,b\in B:a\perp b⟺A\perp B$

Gram–Schmidt process (12.5.2)

Convert a basis $\{{\mathbf{u}}_1,\dots ,{\mathbf{u}}_k\}$ into an orthogonal basis $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_k\}$:

${\mathbf{v}}_1={\mathbf{u}}_1$ ${\mathbf{v}}_i={\mathbf{u}}_i-\frac{{\mathbf{u}}_i\cdot {\mathbf{v}}_1}{\Vert {\mathbf{v}}_1{\Vert }^2}{\mathbf{v}}_1-\frac{{\mathbf{u}}_i\cdot {\mathbf{v}}_2}{\Vert {\mathbf{v}}_2{\Vert }^2}{\mathbf{v}}_2-\cdots -\frac{{\mathbf{u}}_i\cdot {\mathbf{v}}_{i-1}}{\Vert {\mathbf{v}}_{i-1}{\Vert }^2}{\mathbf{v}}_{i-1}$

• during the computation you can multiple ${\mathbf{v}}_i$ by a scalar (note before q12.5.4)
• To convert the orthogonal basis into an orthonormal basis see (q12.4.2)
• for dependent set see q12.5.3
• expanding orthogonal set of $k<n$ vectors into orthogonal basis see q12.5.4