Acknowledgments
This book would not have been possible without the contributions of many people.
We would like to thank those who commented on our proposal for the book
and helped plan its contents and organization: Hugo Larochelle, Guillaume Alain,
Kyunghyun Cho, C¸ a˘glar G¨ul¸cehre, Razvan Pascanu, David Krueger and Thomas
Roh´ee.
We would like to thank the people who offered feedback on the content of
the book itself. Some offered feedback on many chapters: Julian Serban, Laurent
Dinh, Guillaume Alain, Kelvin Xu, Meire Fortunato, Ilya Sutskever, Vincent
Vanhoucke, David Warde-Farley, Augustus Q. Odena, Matko Boˇsnjak, Stephan
Dreseitl, Jurgen Van Gael, Dustin Webb, Johannes Roith, Ion Androutsopoulos,
Stephan Dreiseitl, Karl Pichotta, Pawel Chilinski, Halis Sak, Fr´ed´eric Francis,
Jonathan Hunt, and Grigory Sapunov.
We would also like to thank those who provided us with useful feedback on
individual chapters:
• Chapter 1, Introduction: Johannes Roith, Eric Morris, Samira Ebrahimi,
Ozan C¸a˘glayan, Mart´ın Abadi, and Sebastien Bratieres.
• Chapter 2, Linear Algebra: Pierre Luc Carrier, Li Yao, Thomas Roh´ee,
Colby Toland, Amjad Almahairi, Sergey Oreshkov, Istv´an Petr´as, Dennis
Prangle, and Alessandro Vitale.
• Chapter 3, Probability and Information Theory: Rasmus Antti, Stephan
Gouws, Vincent Dumoulin, Artem Oboturov, Li Yao, John Philip Anderson,
and Kai Arulkumaran.
• Chapter 4, Numerical Computation: Tran Lam An.
• Chapter 5, Machine Learning Basics: Dzmitry Bahdanau, and Zheng Sun.
• Chapter 6, Feedforward Deep Networks: David Krueger.
• Chapter 8, Optimization for Training Deep Models: James Martens and
Marcel Ackermann.
vii
CONTENTS
• Chapter 9, Convolutional Networks: Mehdi Mirza, C¸ a˘glar G¨ul¸cehre, and
Mart´ın Arjovsky.
• Chapter 10, Sequence Modeling: Recurrent and Recursive Nets: Mihaela
Rosca, Razvan Pascanu, Dmitriy Serdyuk, and Dongyu Shi.
• Chapter 18, Confronting the Partition Function: Sam Bowman, and Ozan
C¸ a˘glayan.
• Chapter 20, Deep Generative Models: Fady Medhat
• Bibliography, Leslie N. Smith.
We also want to thank those who allowed us to reproduce images, figures
or data from their publications: David Warde-Farley, Matthew D. Zeiler, Rob
Fergus, Chris Olah, Jason Yosinski, Nicolas Chapados and James Bergstra. We
indicate their contributions in the figure captions throughout the text.
Finally, we would like to thank Google for allowing Ian Goodfellow to work
on the book as his 20% project while working at Google. In particular, we would
like to thank Ian’s former manager, Greg Corrado, and his subsequent manager,
Samy Bengio, for their support of this effort.
viii
