Suppose you have a neural network with a function as its last layer that squashes its outputs to a fixed interval [a,b]. Popoviciu’s inequality can give us an upper bound to the sample variance. If we knew the distribution where...

In visual localization, the accuracy of a system is determined by comparing predicted poses to ground truth poses. The position in R³ is a euclidean space. But the orientations can be given in multiple ways, like axis and angle, rotations...

This post is a summary of my expenses during two years of studying at Karlsruhe Institute of Technology (KIT). I collected the data with Splitwise because I am sharing expenses with my girlfriend. It contains my regular meals at the...

This is an illustrative explanation of Jensen’s inequality applied to probability. Jensen’s inequality Remember that a convex function is a function whose area above the graph (called epi graph) is a convex set. Informally speaking, a convex function can be...

If you want to calculate the expected value of a continuous random variable which was transformed by a monotonic function, then the law of the unconscious statistician provides a convenient shortcut. Transforming a random variable We have a continuous random...

This is a short note to show the difference between softmax and softargmax. It is common to use a “Softmax” layer as an activation function for a neural network used for classification. But this is actually a wrong name. The...

This is a summary of the lecture Stochastic Information Processing which I attended in the winter semester 2019/2020. The content has overlaps with LMA (localization of mobile agents), i.e. Kalman Filter. In SI it is taught how to convert a...

This post explains iDistance and R-Trees for k Nearest Neighbour (kNN) matching. Both are using regular database indices to solve the problem. Problem Assume you want to use kNN in a live system with many data points. The data points...

What is the Partitioned Normal Form (PNF) in the context of databases? NF² NF² = NFNF = ¬NF = Non-First-Normal-Form. The First Normal Form (1NF) is too restrictive. NF² allows attributes themselves to be relations. Attribute values can be sets....

Motivation This is just a short piece of code which I have needed multiple times when training a LSTM with masked inputs. Setup: Seq2seq training with additional loss Problem: How to get the last time step which is not masked...

This post is a loose collection of JavaScript demos I made while studying for the course “Generating Behavior for Vehicles”. The following two gifs show you how two of the demos work: **Planning a path through a city map with...

In this post I am going to try to explain the basic concepts of localization of mobile agents which I have learned in the lecture “Lokalisierung mobiler Agenten” at the KIT. The main outline: Wheel and vehicles (one lane kinematic...

This article tries to explain Boltzmann machines, RBMs, DBN, a little bit about energy models and contains a python example which shows how a generative model can be trained with contrastive divergence. Boltzmann Machines I have explained Binary Hopfield Networks...

This article explains Hopfield networks, simulates one and contains the relation to the Ising model. Hopfield Nets Let’s assume you have a classification task for images where all images are known. The optimal solution would be to store all images...

I recently read the book “Infinite Powers” by Steven Strogatz. In chapter 10 he told the story of Joseph Fourier and why he initially cared about approximating functions with a mixture of sine and cosine waves. His objective was to...

List of car datasets This is a simple lookup page to answer questions like “In which dataset are images of the Renault Clio?” The lists are extracted from the datasets either by reading the directory structure or the accompanying matlab...

Optimization Search for the optimum Assume we define a function f which takes a profession and returns the annual salary. It sounds naturally to search for the optimum. Maybe we even have some constraints like “the job should be in...

Dead reckoning The following notebook contains a simple implementation of dead reckoning with the Explicit and Implicit Euler Method for a bi-cycle given the steering angle and velocity at every timestep t. import math import numpy as np import matplotlib.pyplot...

This is an interactive demo for the Harris Corner Detector running in your browser with tensorflow.js. The goal is to find points in an image which are corners. These points have Hessians with large, similar eigenvalues. If only one of...

Whether it is cryptography, automatic speech recognition or computer vision: Information theory seems to be everywhere in computer science. This post is a collection of intuitions and formulas. Information theory reasons about channels and how to optimally encode messages on...

For human babies we know that: Depth perception, which is the ability to judge if objects are nearer or farther away than other objects, is not present at birth. It is not until around the fifth month that the eyes...

A fellow student counted the most frequent exam questions for the course “Grundlagen der Automatischen Spracherkennung” as I am preparing for the oral exam I thought that a visualization of this would be cool. About the course I attended the...

Discrete HMM We can use a markov chain to model a stochastic process. A popular example is a weather model. We assume that the past can be summed up to states which describe enough to decide which state will be...

In the last post I wrote about inference, marginalization and such for SPNs. Now let us take a look at parameter learning for SPNs. Toy example In order to motivate ourselves let’s try to solve a motivating example: We try...

This post aims to give an overview of Sum-Product Networks with a graphical explanation. It also contains an introduction on inference. TL;DR Sum-Product Network is a group of probabilistic graphical models with the nice property that inference is linear in...