Click and drag to move the nearest vector:
- The goal of this tool is to get an intuitive sense of three linear algebra operations: orthogonalization, normalization, and matrix multiplication.
- Linear algebra sounds scary. I hope you come away from this feeling more optimistic about it! There's a lot of complex looking descriptions, but there's a core to it that's not at all difficult. Remember: in math, "linear" means "easy". As long as you can multiply and add, you've got this.
- The fundamental unit of linear algebra is a vector, which just means a list of numbers that each have some meaning. The set of all possible values for those numbers is a space. An (x,y) pair is a vector in height/width space. A color is a vector in RGB or CMYK space. The ingredients table in a recipe is a vector in food space.
- Of the three operations, normalization is the most obvious: it stretches a vector to lie on the unit circle (radius 1). Often when we multiply vectors they can get really big or really small. Normalizing keeps them in a constant region.
- Orthogonalization is a little more complicated, but has a nice visual interpretation. In 2D, "orthogonal" means "at right angles". If we subtract a scaled version of vector a from vector b, the result will be orthogonal to vector a. As we saw, repeatedly multiplying two different vectors by the same matrix can cause the vectors to become very similar (although possibly negated). Orthogonalizing is a good way to keep vectors distinct.
- Matrix multiplication is more tricky, because different matrices have different effects. Multiplying a square matrix times a vector of the same length creates a new vector the same length as the old one. In our 2D case, you can think of the top row of the 2x2 matrix as the rule for creating the new x value: take this much of the old x and add this much of the old y. The bottom row is the same, but for creating the new y.
- An identity matrix [[1,0],[0,1]] is the equivalent of 1. Multiplying it times any vector has no effect. You can see why: the new x value is 1 times the old x plus 0 times the old y, and the new y is 0 times the old x plus 1 times the old y.
- In class we tried changing the 1s in the identity matrix to 1.2s, and then to 0.8s, and then to 1.2 and 0.8. The first two scale the two vectors up and down, the last one scales the x value up and the y value down, so the vectors become very large and very close to the x-axis.
- The default matrix when you load the page has the same effect, but rotated. In both cases what we're finding is the "dominant eigenvector", or its negative. No matter where you start, if you keep multiplying by this matrix, you will always get the one of these vectors.
- Now try a loop: multiply, orthogonalize, normalize. These three operations in sequence allow us to pick up the second eigenvector, which is orthogonal to the first.