# Course contents

## The course covers the theoretical basis of a wide range of different network types and their implementation in TensorFlow. To get an overview about the course content, you can have a look at the following list.

### Session 1: Introduction

*Part 1:* Brains and Neurons

*Part 2:* The Perceptron

*Part 3:* Deep Learning and Big Data

*Downloads:* Slides

### Session 2: Multilayer Perceptron

*Part 1:* Improving the Model

*Part 2:* Linear and Non-linear Transformations

*Part 3:* Multilayer Perceptron

*Part 4:* Types of Learning

*Part 5:* Output Activation Functions

*Downloads:* Slides

### Session 3: Gradient Descent and Backpropagation

*Part 1:* The Crucial Insight

*Part 2:* The Error Surface

*Part 3:* Numerical Optimization

*Part 4:* Derivatives, Partial Derivatives and Gradients

*Part 5:* The Backpropagation Algorithm

*Part 6:* Gradient Descent

*Part 7:* Non-Convex Functions

*Part 8:* Batch, Mini-Batch and Stochastic Learning

*Part 9:* Optimizing Standard Gradient Descent

*Downloads:* Slides, Homework Assignment, Solution

### Session 4: TensorFlow

*Part 1:* Deep Learning Frameworks

*Part 2:* GPU Acceleration

*Part 3:* Data Flow Graphs

*Part 4:* Writing Code with TensorFlow

*Part 5:* TensorFlow and Automatic Differentiation

*Part 6:* Advanced TensorFlow Code

*Downloads:* Slides, Homework Assignment, Solution

### Session 5: Convolutional Neural Networks

*Part 1:* Visual Information Processing in the Human Brain

*Part 2:* Cross Corelation

*Part 3:* Convolutional Neural Networks

*Part 4:* Forward Propagation in CNNs

*Part 5:* Efficiency of CNNs

*Part 6:* CNN Applications

*Downloads:* Slides, Homework Assignment, Solution

### Session 6: Training Artificial Neural Networks

*Part 1:* Improving the Network Structure

*Part 2:* Vanishing and Exploding Gradients

*Part 3:* Activation Functions

*Part 4:* Weight Initialization

*Part 5:* Data Preparation and Batch Normalization

*Part 6:* Learning Rates

*Part 7:* Tools Against Overfitting

### Session 7: Advanced Convolutional Neural Networks

*Part 1:* Overview

*Part 2:* Comparing Networks

*Part 3:* AlexNet and VGG

*Part 4:* 1x1 Convolutions

*Part 5:* Inception and Residual Networks

*Part 6:* Densely Connected Networks

*Downloads:* Slides, Homework Assignment, Solution

### Session 8: Word Embeddings

*Part 1:* Statistical Language Models

*Part 2:* Vector Space Models

*Part 3:* Distributional Hypothesis

*Part 4:* Continuous Bag-of-Words Model

*Part 5:* Skip-Gram Model

*Part 6:* Negative Sampling

*Part 7:* Noise-Contrastive Estimation

*Part 8:* Evaluation and Visualization of Word Embeddings

*Downloads:* Slides, Homework Assignment, Solution

### Session 9: Data and Training Visualization

*Part 1:* Deep Dream

*Part 2:* Visualization of Higher Order Features

*Part 3:* Adversarial Examples

*Part 4:* Dimensionality Reduction with t-SNE

*Part 5:* TensorBoard

### Session 10: Recurrent Neural Networks

*Part 1:* Sequences of Data

*Part 2:* Recurrent Neural Networks

*Part 3:* Generative Recurrent Neural Networks

*Part 4:* Backpropagation through Time

*Part 5:* Batch Generation and Training

*Part 6:* Long Short-Term Memory

*Downloads:* Slides, Homework Assignment, Solution

### Session 11: Advanced Recurrent Neural Networks

### Session 12: Reinforcement Learning 1

*Part 1:* What is Reinforcement Learning?

*Part 2:* Agent and Environment

*Part 3:* Rewards and Returns

*Part 4:* State Transition Matrix

*Part 5:* Markov Decision Process

*Part 6:* Components of an RL Agent

*Part 7:* Bellman Equation

*Part 8:* Optimality and Learning

### Session 13: Deep Reinforcement Learning 2

*Part 1:* Iterative Policy Evaluation

*Part 2:* Q-Learning

*Part 3:* Deep Q-Learning

*Part 4:* Policy Gradients

*Part 5:* Human vs. Reinforcement Learning

*Downloads:* Slides, Homework Assignment, Solution

### Session 14: Generative Adversarial Networks

*Part 1:* Generative Models

*Part 2:* Generative Adversarial Networks

*Part 3:* Training Generative Adversarial Networks

*Part 4:* Advanced Generative Adversarial Networks

*Downloads:* Slides, Homework Assignment, Solution

### Session 15: Echo State Networks 1

### Session 16: Echo State Networks 2

*Downloads:* Slides, Homework Assignment, Solution

### Session 17: Spiking Neural Networks 1

*Part 1:* Rate vs. Temporal Coding

*Part 2:* Electricity and Neurons

*Part 3:* Neurodynamics

*Part 4:* Differential Equations

*Part 5:* Leaky Integrate and Fire Neuron

*Downloads:* Slides, Homework Assignment, Solution

### Session 18: Spiking Neural Networks 2

*Part 1:* Liquid State Machines

*Part 2:* Plasticity

*Part 3:* Backpropagating Errors in Spiking Neural Networks

*Downloads:* Slides, Homework Assignment, Solution

### General

*Part 1:* Further Reading

*Part 2:* Machine Learning and Ethics

*Part 3:* The Small Artificial Neural Networks Dictionary