Code: The Hidden Language of Computer Hardware and Software is a wonderful book explaining how computers work. I've read the section on Computer Architecture in Gödel Escher Bach, but that did not stay in my mind. This book, however, explains how computers work from the most basic level; it starts with circuits and builds up! I have not finished it, but I'm over halfway and Petzold has built up a machine with memory and a calculator function, which can load items from memory and perform operations upon them.

At times it is a bit difficult to follow, but I think it has been written, overall, very well. I really have found it a fascinating book. Highly recommended.