Simon's Blog

I study Data Science at ETH Zurich and work at First Momentum Ventures, an early stage VC. I write a lot of Python code and do research into probabilistic machine learning.

• ---

  The Normalizing Flow Network

  August 8, 2019
  

  The Normalizing Flow Network (NFN) is a normalizing-flow based regression model, great at modelling complex conditional densities. Look at our recent paper on noise regularization for conditional density estimation for some results of using the NFN on real-world and benchmark regression datasets.

  Here I’ll explain the structure of the NFN and go through some of the math. Implementations can be found in our Open-Source Python package as well as in my repo.

• ---

  Hyperparameter tuning for Keras models (Part 1)

  June 8, 2019
  

  UPDATE: Since this guide came out the keras team has released an official Python package for hyperparameter tuning. While the code here still works, you should check out that package as well.

  When encountering a new dataset, you use your own (helpful) inductive biases and come up with an initial model structure that you think will fit the data well. That’s only the first step to finding a good model since now you’re left with a bunch of hyperparameters and often only a vague idea of how to set them. Thorough tuning of those parameters can greatly increase the performance of your model.

  Here I compiled two ways to do this for your Keras model, without running into memory problems nor having to rely on single-thread execution.This guide is mostly for smaller models that train well on CPU, since that’s what I’ve been working on recently in the context of density estimation. The first option uses scikit-learn, the second option is more advanced and flexible by using Python multiprocessing directly.

• ---

  Hyperparameter tuning for Keras models (Part 2)

  June 8, 2019
  

  In the first part of this post we used scikit-learn and it’s implementation of grid search and random search to find the optimal settings for our hyperparameters. While that was quick to implement, it lacked some flexibility, for example in what metrics could be recorded and still wasn’t completely secure against memory overflow issues. To improve on this, here I provide a second approach using Python’s multiprocessing library and my own implementation of grid search. To illustrate, we’ll do some regression and fit a Mixture Density Network to some bimodal data.

• ---

  Fighting misinformation with Machine Learning

  March 20, 2019
  

  This essay was originally written as part of an application to TUM’s Data Analytics Master’s program Machine learning is a double edged sword when it comes to fighting the spread of misinformation. On one hand it enables scalable, accurate detection of deceptive content and can adapt automatically to new threats and challenges [6]. On the other hand, machine learning methods facilitate the generation of fake texts, images and videos with a quality and cost-effectiveness that vastly supersedes previous manual methods [5]. Artificial intelligence can stop an ill-motivated conspiracy theory from spreading on social media within minutes of being posted [3][4]. Artificial intelligence can also create seemingly real video or audio of a person of public interest, making them say whatever is in the bad actor’s best interest [0][1][2]. Combating misinformation requires academics researching machine learning to do two things: Develop effective methods of detecting deceptive content while showing foresight when it comes to malign applications of the technology they create.