Acknowledgments

This book would not have been possible without the contributions of many people.

We would like to thank those who commented on our proposal for the book

and helped plan its contents and organization: Hugo Larochelle, Guillaume Alain,

Kyunghyun Cho, C¸ a˘glar G¨ul¸cehre, Razvan Pascanu, David Krueger and Thomas

Roh´ee.

We would like to thank the people who oﬀered feedback on the content of

the book itself. Some oﬀered feedback on many chapters: Julian Serban, Laurent

Dinh, Guillaume Alain, Kelvin Xu, Ilya Sutskever, Vincent Vanhoucke, David

Warde-Farley, Jurgen Van Gael, Dustin Webb, Johannes Roith, Ion Androut-

sopoulos, Pawel Chilinski, Halis Sak, Fr´ed´eric Francis, Jonathan Hunt, and Grig-

ory Sapunov.

We would also like to thank those who provided us with useful feedback on

individual chapters:

• Chapter 1, Introduction: Johannes Roith, Eric Morris, Samira Ebrahimi,

Ozan C¸a˘glayan, Mart´ın Abadi, and Sebastien Bratieres.

• Chapter 2, Linear Algebra: Pierre Luc Carrier, Li Yao, Thomas Roh´ee,

Colby Toland, Amjad Almahairi, Sergey Oreshkov, Istv´an Petr´as, and Den-

nis Prangle.

• Chapter 3, Probability and Information Theory: Rasmus Antti, Stephan

Gouws, Vincent Dumoulin, Artem Oboturov, Li Yao, John Philip Anderson,

and Kai Arulkumaran.

• Chapter 4, Numerical Computation: Meire Fortunato, and Tran Lam An.

• Chapter 5, Machine Learning Basics: Dzmitry Bahdanau, and Meire For-

tunato.

• Chapter 8, Optimization for Training Deep Models: Marcel Ackermann.

• Chapter 9, Convolutional Networks: Mehdi Mirza, C¸ a˘glar G¨ul¸cehre, and

Mart´ın Arjovsky.

vii

CONTENTS

• Chapter 10, Sequence Modeling: Recurrent and Recursive Nets: Dmitriy

Serdyuk, and Dongyu Shi.

• Chapter 18, Confronting the Partition Function: Sam Bowman, and Ozan

C¸ a˘glayan.

• Bibliography, Leslie N. Smith.

We also want to thank those who allowed us to reproduce images, ﬁgures

or data from their publications: David Warde-Farley, Matthew D. Zeiler, Rob

Fergus, Chris Olah, Jason Yosinski, Nicolas Chapados, and James Bergstra. We

indicate their contributions in the ﬁgure captions throughout the text.

Finally, we would like to thank Google for allowing Ian Goodfellow to work

on the book as his 20% project while working at Google. In particular, we would

like to thank Ian’s former manager, Greg Corrado, and his subsequent manager,

Samy Bengio, for their support of this eﬀort.

viii