For the question (b) below, I agree that W is positive definite since it's a diagonal matrix, so the eigen values are the values on the diagonal, i.e. the wi which are all > 0. But then, why X_TWX is invertible if X is full column rank? Where does that condition come from?
Thank you for your help!
The proof goes in the same as to why \(X^\top X\) would be invertible if X is full column rank.
Basically \(null(X^\top X)= null(X)\), to show this pick \(w\in null(X^\top X)\), this means \(w^\top X^\top X w=0\), this is the same as \( \Vert Xw\Vert^2=0\) which means \(Xw=0\) i.e. \(w\in null(X)\) , the other implication is straightforward.
So now if \(X\) is full column rank i.e. the columns of \(X\) are linearly independent, then \(rank(X)=D\), using the rank equality \(rank(X) + dim null(X)=D\) we get that \(dim null(X)=0\) which means dim \(null(X^\top X)=0\) which means \(rank(X^\top X)=D\) and that \(X^\top X\) is invertible.
Now with the matrix (symmetric) positif definite matrix W added,we still have \(null(X^\top W X)= null(X)\), now if \(w\in null(X^\top W X\)) then \(w^\top X^\top W X w=0\) which means \( \Vert W^{1/2}Xw\Vert^2=0\) which implies \(W^{1/2}Xw=0\) implies \(Xw=0\) which concludes the proof.
The proof goes in the same as to why \(X^\top X\) would be invertible if X is full column rank.
Basically \(null(X^\top X)= null(X)\), to show this pick \(w\in null(X^\top X)\), this means \(w^\top X^\top X w=0\), this is the same as \( \Vert Xw\Vert^2=0\) which means \(Xw=0\) i.e. \(w\in null(X)\) , the other implication is straightforward.
So now if \(X\) is full column rank i.e. the columns of \(X\) are linearly independent, then \(rank(X)=D\), using the rank equality \(rank(X) + dim null(X)=D\) we get that \(dim null(X)=0\) which means dim \(null(X^\top X)=0\) which means \(rank(X^\top X)=D\) and that \(X^\top X\) is invertible.
Now with the matrix (symmetric) positif definite matrix W added,we still have \(null(X^\top W X)= null(X)\), now if \(w\in null(X^\top W X\)) then \(w^\top X^\top W X w=0\) which means \( \Vert W^{1/2}Xw\Vert^2=0\) which implies \(W^{1/2}Xw=0\) implies \(Xw=0\) which concludes the proof.
Mock exam 2014 invertible matrix conditions
Hello,
For the question (b) below, I agree that W is positive definite since it's a diagonal matrix, so the eigen values are the values on the diagonal, i.e. the wi which are all > 0. But then, why X_TWX is invertible if X is full column rank? Where does that condition come from?
Thank you for your help!
The proof goes in the same as to why \(X^\top X\) would be invertible if X is full column rank.
Basically \(null(X^\top X)= null(X)\), to show this pick \(w\in null(X^\top X)\), this means \(w^\top X^\top X w=0\), this is the same as \( \Vert Xw\Vert^2=0\) which means \(Xw=0\) i.e. \(w\in null(X)\) , the other implication is straightforward.
So now if \(X\) is full column rank i.e. the columns of \(X\) are linearly independent, then \(rank(X)=D\), using the rank equality \(rank(X) + dim null(X)=D\) we get that \(dim null(X)=0\) which means dim \(null(X^\top X)=0\) which means \(rank(X^\top X)=D\) and that \(X^\top X\) is invertible.
Now with the matrix (symmetric) positif definite matrix W added,we still have \(null(X^\top W X)= null(X)\), now if \(w\in null(X^\top W X\)) then \(w^\top X^\top W X w=0\) which means \( \Vert W^{1/2}Xw\Vert^2=0\) which implies \(W^{1/2}Xw=0\) implies \(Xw=0\) which concludes the proof.
1
Hi, where can you find these older mock exams? They are not on the course website?
Yes they are in the github in the folder mock midterm exams :)
The proof goes in the same as to why \(X^\top X\) would be invertible if X is full column rank.
Basically \(null(X^\top X)= null(X)\), to show this pick \(w\in null(X^\top X)\), this means \(w^\top X^\top X w=0\), this is the same as \( \Vert Xw\Vert^2=0\) which means \(Xw=0\) i.e. \(w\in null(X)\) , the other implication is straightforward.
So now if \(X\) is full column rank i.e. the columns of \(X\) are linearly independent, then \(rank(X)=D\), using the rank equality \(rank(X) + dim null(X)=D\) we get that \(dim null(X)=0\) which means dim \(null(X^\top X)=0\) which means \(rank(X^\top X)=D\) and that \(X^\top X\) is invertible.
Now with the matrix (symmetric) positif definite matrix W added,we still have \(null(X^\top W X)= null(X)\), now if \(w\in null(X^\top W X\)) then \(w^\top X^\top W X w=0\) which means \( \Vert W^{1/2}Xw\Vert^2=0\) which implies \(W^{1/2}Xw=0\) implies \(Xw=0\) which concludes the proof.
1
Add comment