1. Failed attempts at graduated optimization

    Since the Dogs vs. Cats project is coming at an end, I'll summarize my (failed) attempts at speeding up training using the idea of graduated optimization (or, continuation). In particular, I've been looking at the ideas presented in this paper that was recently uploaded to ArXiv: Hazan, E. Levy, K…


  2. Bagging and ensembles

    With the new Fuel processing server I figured I would approach the problem like a Kaggler would: Make use of the speed by throwing a whole bunch of models at the problem. So here's the setup: Take Alexandre's high-capacity model and train 8 of them it with random hyperparameters: Dropout…


  3. Fuel data processing server

    As I mentioned in my previous post, I have been wanting to write a user-friendly data processing server in Fuel. In my experiments, around 75 seconds per epoch where spent reading data and extracting a random patch. If you would perform data augmentation, this number would only significantly increase, taking…


  4. Blocks/Fuel development for Dogs vs. Cats

    A most overdue update on my work for the Dogs vs. Cats project. A while ago I succesfully reproduced Alexandre's results with Blocks and Fuel. A lot more time the last few weeks has been spent on developing Blocks and Fuel further to allow for easier experimentation, especially considering several…


  5. Fuel + Convolutional layers in Blocks

    With some help of Harm and Dima a series of PRs (#221, #282, and #304) were merged that introduce support for convolutional neural networks into Blocks. Meanwhile, work has started on factoring out the dataset management from Blocks into a new framework, christened Fuel. I wrote a dataset wrapper for…