Part II
Modern Practical Deep
Networks
147
This part of the book summarizes the state of modern deep learning as it is
used to solve practical applications.
Deep learning has a long history and many aspirations. Several approaches
have been proposed that have yet to entirely bear fruit. Several ambitious goals
have yet to be realized. These less-developed branches of deep learning appear in
the final part of the book.
This part focuses only on those approaches that are essentially working tech-
nologies that are already used heavily in industry.
Modern deep learning provides a very powerful framework for supervised learn-
ing. By adding more layers and more units within a layer, a deep network can
represent functions of increasing complexity. Most tasks that consist of mapping
an input vector to an output vector, and that are easy for a person to do rapidly,
can be accomplished via deep learning, given sufficiently large model and dataset
of labeled training examples. Other tasks, that can not be described as associat-
ing one vector to another, or that are difficult enough that to do them a person
would require time to think and reflect, remain beyond the scope of deep learning
for now.
This part of the book describes the core parametric function approximation
technology that is behind nearly all modern practical applications of deep learn-
ing. Our description includes details such as, how to efficiently model specific
kinds of inputs, how to process image inputs with convolutional networks as well
as how to process sequence inputs with recurrent and recursive networks. More-
over, we provide guidance for how to preprocess the data for various tasks and
how to choose the values of the various settings that govern the behavior of these
algorithms.
These chapters are the most important for a practitioner – someone who wants
to begin implementing and using deep learning algorithms to solve real-world
problems today.
148
