### LogSumExp

This is a short note to show the difference between softmax and softargmax. It is common to use a “Softmax” layer as an activation function for a neural network used for classification. But this is actually a wrong name. The...

This is a short note to show the difference between softmax and softargmax. It is common to use a “Softmax” layer as an activation function for a neural network used for classification. But this is actually a wrong name. The...

This is a summary of the lecture Stochastic Information Processing which I attended in the winter semester 2019/2020. The content has overlaps with LMA (localization of mobile agents), i.e. Kalman Filter. In SI it is taught how to convert a...

This post explains iDistance and R-Trees for k Nearest Neighbour (kNN) matching. Both are using regular database indices to solve the problem. Problem Assume you want to use kNN in a live system with many data points. The data points...

What is the Partitioned Normal Form (PNF) in the context of databases? NF² NF² = NFNF = ¬NF = Non-First-Normal-Form. The First Normal Form (1NF) is too restrictive. NF² allows attributes themselves to be relations. Attribute values can be sets....

Motivation This is just a short piece of code which I have needed multiple times when training a LSTM with masked inputs. Setup: Seq2seq training with additional loss Problem: How to get the last time step which is not masked...

This post is a loose collection of JavaScript demos I made while studying for the course “Generating Behavior for Vehicles”. The following two gifs show you how two of the demos work: **Planning a path through a city map with...

In this post I am going to try to explain the basic concepts of localization of mobile agents which I have learned in the lecture “Lokalisierung mobiler Agenten” at the KIT. The main outline: Wheel and vehicles (one lane kinematic...

This article tries to explain Boltzmann machines, RBMs, DBN, a little bit about energy models and contains a python example which shows how a generative model can be trained with contrastive divergence. Boltzmann Machines I have explained Binary Hopfield Networks...

This article explains Hopfield networks, simulates one and contains the relation to the Ising model. Hopfield Nets Let’s assume you have a classification task for images where all images are known. The optimal solution would be to store all images...

I recently read the book “Infinite Powers” by Steven Strogatz. In chapter 10 he told the story of Joseph Fourier and why he initially cared about approximating functions with a mixture of sine and cosine waves. His objective was to...

List of car datasets This is a simple lookup page to answer questions like “In which dataset are images of the Renault Clio?” The lists are extracted from the datasets either by reading the directory structure or the accompanying matlab...

Optimization Search for the optimum Assume we define a function f which takes a profession and returns the annual salary. It sounds naturally to search for the optimum. Maybe we even have some constraints like “the job should be in...

Dead reckoning The following notebook contains a simple implementation of dead reckoning with the Explicit and Implicit Euler Method for a bi-cycle given the steering angle and velocity at every timestep t. import math import numpy as np import matplotlib.pyplot...

This is an interactive demo for the Harris Corner Detector running in your browser with tensorflow.js. The goal is to find points in an image which are corners. These points have Hessians with large, similar eigenvalues. If only one of...

Whether it is cryptography, automatic speech recognition or computer vision: Information theory seems to be everywhere in computer science. This post is a collection of intuitions and formulas. Information theory reasons about channels and how to optimally encode messages on...

For human babies we know that: Depth perception, which is the ability to judge if objects are nearer or farther away than other objects, is not present at birth. It is not until around the fifth month that the eyes...

A fellow student counted the most frequent exam questions for the course “Grundlagen der Automatischen Spracherkennung” as I am preparing for the oral exam I thought that a visualization of this would be cool. About the course I attended the...

Discrete HMM We can use a markov chain to model a stochastic process. A popular example is a weather model. We assume that the past can be summed up to states which describe enough to decide which state will be...

In the last post I wrote about inference, marginalization and such for SPNs. Now let us take a look at parameter learning for SPNs. Toy example In order to motivate ourselves let’s try to solve a motivating example: We try...

This post aims to give an overview of Sum-Product Networks with a graphical explanation. It also contains an introduction on inference. TL;DR Sum-Product Network is a group of probabilistic graphical models with the nice property that inference is linear in...

Visualizing joint angles of grasps In this post I am going to explain how to use the master motor map (MMM) in order to visualize grasping movements with joint angle data I downloaded from handcorpus.org. The MMM has a Whole-body...

Some impressions of LSTM architectures for simple math functions: seq2seq, seq2vec and then seq-seq-autoencoder. In particular the last part is an experiment of reconstructing sinoid waves with phase displacement from a single latent parameter. Sequence to sequence X-Y I generated...

The odyssey of writing a YOLO-like detection model for German traffic signs. Simply said: Give a CNN an image and let it draw boxes around the traffic signs. Getting started slowly Before detecting concrete boxes I thought it would be...

I have been using tensorflow for some examples in this blog. Building a sequential model and evaluating some metrics was easy. But when I wanted to do a little bit more (for example image data set augmentation) I got really...

Review of the video course When looking up resources on how to get a good overview of neural networks for computer vision you almost always get pointed to the youtube videos of the CS231n course held at Stanford. I watched...