Here are my notes on information theory. I will write as I go through the articles/books.
A good video for setting the scene. This video gives an intuitive reasioning for what cross entropy is and why we use it. This will help us understand the next section.
Notes on A Mathematical Theory of Communication. This is the paper that started it all. By Claude Shannon. I hope to find the bare bones idea for information theory, and why it was created, before going into other resources. This will be my notes on the subject.