Lab homeEdit page

Next lab meeting: Feb 28, 2018

2018-Feb-28, Stochastic Backpropagation and Approximate Inference in Deep Generative Models

One of the first derivations of the Variational Autoencoder, an alternative to the GAN for generating high-quality images.

Candidate papers

Notes from past discussions

2017-Nov-20, Progressive Growing of GANs for Quality, Stability, and Variation

  • Primary contribution: Iteratively trained by progressively upscaling the resolution with each new layer.
  • Avoid "shocking" the system by slowly interpolating in new layers with a knob on a skip connection.
  • They've got some way of dynamically tuning the learning rate or weights based on a He 2015 paper.
  • Awesome video and some nice pics to go with the paper.

2017-Nov-06, DISCO Nets: DISsimilarity COefficient Networks

  • This trains a NN to model a probability distribution.
  • (It seems a lot like jitter, but somewhat more intelligent. The uncertainty is built-in.)
  • Uses two terms: one makes it fit the target values, one makes it not ignore the random inputs.
  • Impressive that they published in NIPS without a lot of validation.
  • Strong theoretical basis. The whole paper is described by just a few equations.
  • Kind of the opposite of a GAN. GAN samples a distribution. This models a distribution.

2017-Oct-23, Neural Architecture Search with Reinforcement Learning

  • Used reinforcement learning to predict NN architectures.
  • Not clear why reinforcement learning is important. Could possibly use function approximation to accomplish the same goal.
  • Does beat a random search for topologies.
  • Works really well for certain datasets.
  • Not efficient or general or reproducible.

2017-Sep-25, Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

  • Not clear how they encode each word. We talked about a potential encoding made from applying dimensionality reduction to WordNet distances.
  • They used residual connections to make their network deeper.
  • Bi-directional LSTM layers on the first layer. We discussed doing it on more layers.
  • We couldn't figure out the attention module.
  • We considered using a generative approach with a similar model.
  • We think their training set is all divided into sentences and uses tokenized values.

2017-Sep-11, A parameterized activation function for learning fuzzy logic operations in deep neural networks

2017-Mar-01, Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  • Summary: Normalize input to layers, and you get training with fewer epochs! (Enables larger learning rate, and more robust to initialization, and it implicitly regularizes.)
  • Internal Covariate Shift is a problem where the inputs to a layer have changing distributions.
  • Normalize the input to every layer and we can fix this problem. Subtract the mean, divide by the variance, and scale and shift.
  • This is used with mini-batches to get a better idea of the mean and variance of the batch.

2017-Feb-15, Deep Residual Learning for Image Recognition

  • Summary: Train a network using the residual function, and then you can build very deep powerful neural networks.
  • Insight 1: The shortcut connection on page 2 is how they add the residual back in.
  • Insight 2: Probably don't want to do this on the input layer. That layer needs to sort of solve the problem a bit before we can begin stacking.
  • Insight 3: Error travels along the shortcut connections back through the network, mostly eliminating the vanishing gradient problem.
  • Insight 4: 1x1 convolutions are equivalent to a fully connected layer.
  • Contribution: A simple residual block that can be used to build deep neural networks, based on a shortcut or skip connection.
  • Contribution 2: Their networks performed so much better that they won several awards on image recognition and so forth.

2016-Nov-15, Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

  • Summary: Super-scaling images with generative adversarial networks.
  • Insight 1: Perhaps we could use generative adversarian networks
  • Contribution: Used a different loss function in combination with generative adversarial networks. No longer is it simple mean-squared error as the loss function.

2016-Nov-1, Full Resolution Image Compression with Recurrent Neural Networks

  • Summary: This paper presents an autoencoder using recurrent connections and convolution to compress images.
  • Insight 1: Recurrent connections and Convolution can give a lot of power to a neural network.
  • Insight 2: Gated Recurrent Units are interesting, and will have less weights than LSTM.
  • Insight 3: Does as well as JPG with an auto-encoder.
  • Insight 4: Multiplication in a neural network might be very important, or provide a lot of extra information.
  • Contribution: Works on any size image due to the image patches, and has recurrent connections between those patches.

2016-Oct-4, WaveNet: A Generative Model of Raw Audio

  • Summary: This paper presents a deep neural network that can generate natural sounding audio.
  • Insight 1: We believe that in figure 3, the bottom input is time.
  • Insight 2: The Gated Activation Unit is essentially choosing how much of x to allow through a gate. We also believe that the added value from these GAU's is the fact that they use multiplication, which can possibly add extra power to the neural network. The paper that created these GAU's has similar claims.
  • Results: This is a great paper, and an interesting approach with a lot of ideas that we could use. There are some strange things to it however.

2016-Sep-20, Generative Adversarial Imitation Learning

  • Summary: Paper does a lot of math to go from Inverse Reinforcement Learning to Generative Adversarial networks. They use a strange gradient to train the networks. They use networks of only two layers with 100 units each.
  • Insight 1: "We first generated expert behavior for these tasks by running TRPO on these true cost functions to create expert policies." This essentially means their "expert" was generated by another algorithm (TRPO).
  • Results: The results appear good, but given Insight 1 might be more of a strawman argument.

2016-Sep-06, From Pixels to Torques: Policy Learning with Deep Dynamical Models

  • Summary: Did an autoencoder and transition model approach to learn how to control a "robotic arm".
  • This paper starts with a lot of literature review covering reinforcement learning, and more; and is pretty thorough.
  • Problem Set Up: Minimize the cost according to some function.
  • Insight 1: They use the last k timesteps as input to the transition function (which they call the prediction model).
  • Equation 3a: Feed in the observations to the encoder to get the intrinsic beliefs(z). Equation 4: The future predicted error is the difference between what you see in the next time step (observation) and what you predicted.
  • Equation 5: Essentially minimize the sum squared error. Interesting notation as they were able to pull three equations out of it.
  • Insight 2: Simultaneously train the transition and observation models together.
  • Very similar to Stephen and Mike's robot arm work.
  • Insight 3: Regularization seems to help. Regularizing it can help it settle into a simple representation.
  • Results: Only tested on very simple simulations including a pendulum experiment.

2016-Aug-11, Continuous Control With Deep Reinforcement Learning

  • Summary: Did a deep Q-network with continuous actions to control simulated robots.
  • This paper starts with a lot of general equations describing the Bellman equation and Q-learning.
  • Insight 1: (Equation 6) backpropagate from the critic to train the actor. (The paper admits that this was not really a novel contribution.)
  • Insight 2: Avoid divergence by using a copy of the actor and critic that lags or slowly tracks the updated ones.
  • Insight 3: Use batch normalization.
  • Insight 4: Do exploration by adding noise to what the actor said to do.
  • We decided that insight 2 is probably the main contribution of this paper.