Notes

Popoviciu's inequality

2021-01-14T00:00:00+01:00

Suppose you have a neural network with a function as its last layer that squashes its outputs to a fixed interval [a,b]. Popoviciu’s inequality can give us an upper bound to the sample variance.

If we knew the distribution where the samples stem from, then it would be easy to answer this question. Consider the most simple case of a uniform distribution: $p(x) \sim U(a,b), Var[p(x)]=\frac{1}{12}(b-a)^2$

The upper bound derived from Popoviciu’s inequality is $\frac{1}{4}(b-a)^2$. Interestingly, Mihaly Bencze and Florin Popovici write in their proof that “Popoviciu’s inequality is a refinement of Jensen’s inequality”.

In the following, I am going to adapt the proof by Zen with minor notational changes:

Let $g(x) = E[(X - t)^2]$ where X is our random variable and $t$ to be determined such that we find the minimum of g.

Therefore we derive g with respect to t: $\frac{d}{dt} g(x) = \frac{d}{dt} E[X^2 - 2Xt + t^2]$

\[= \frac{d}{dt} E[X^2] - E[2Xt] + E[t^2] = 0 - \frac{d}{dt} E[2Xt] + \frac{d}{dt} E[t^2]\] \[= -2 E[X] + 2t\]

… and set it to zero:

\[-2 E[X] + 2t \stackrel{!}{=} 0\] \[t = E[X]\]

The second derivative is $2$ which is larger than zero. $E[X]$ is therefore the global minimum. $g(\cdot)$ was constructed such that $g(E[X]) = Var[X]$

Now we choose $t = \frac{a+b}{2}$ seemingly arbitrary in the middle of our interval but of course it is chosen on purpose.

Since $g(\cdot)$ has its global minimum at $E[X]$, we know that our chosen $t$ cannot make the function smaller:

\[g(E[X]) \le g(\frac{a+b}{2})\]

By having a closer look at $g(\frac{a+b}{2})$ we can derive an upper bound for it:

\[g(\frac{a+b}{2}) = E[(X - \frac{a+b}{2})^2]\] \[= E[(\frac{1}{2}(2X - (a + b))^2] = E[\frac{1}{4}(2X - (a - b))^2]\] \[= \frac{1}{4} E[(X+X - a - b)^2] = \frac{1}{4} E[(X - a + X - b)^2]\] \[= \frac{1}{4} E[(\underbrace{(X - a)}_{\ge 0} + \underbrace{(X - b)}_{\le 0})^2]\]

Since every sample from X has to be greater than $a$ and smaller than $b$, these observations are true. While the left part increases the value, the right part decreases it. Our upper bound can thus be constructed by simply flipping the sign such that the right part is not subtracted but added:

\[\frac{1}{4} E[((X - a) + (X - b))^2] \le \frac{1}{4} E[((X - a) - (X - b))^2]\]

Modifying the right hand side:

\[\frac{1}{4} E[((X - a) + (X - b))^2] \le \frac{1}{4} E[(X - a - X + b))^2]\]

The X is eradicated:

\[\frac{1}{4} E[((X - a) + (X - b))^2] \le \frac{1}{4} E[(b-a)^2]\] \[\underbrace{\frac{1}{4} E[((X - a) + (X - b))^2]}_{g(\frac{a+b}{2})} \le \frac{1}{4} (b-a)^2\]

Recalling that $Var[X] = g(E[X]) \le g(\frac{a+b}{2})$ we see that:

\[Var[X] \le \frac{1}{4} (b-a)^2\]

Popoviciu's inequality was originally published by Daniel Pollithy at Notes on January 14, 2021.

Deviation between rotations

2021-01-13T00:00:00+01:00

In visual localization, the accuracy of a system is determined by comparing predicted poses to ground truth poses.

The position in R³ is a euclidean space. But the orientations can be given in multiple ways, like axis and angle, rotations matrices or quaternions.

Du Q. Huynh describes in Metrics for 3D Rotations: Comparison and Analysis five metrics on SO(3):

Euclidean Distance between Euler Angles: This is only a metric on SO(3) if further restrictions are made.
Norm of the Difference of Quaternions: Since every rotation is covered twice by unit-quaternions, one property of metrics does not hold. Which is the property that states, if the distance between two entities is zero then the two entities are the same. This is not true for the positive and the negative quaternion who have a rotational difference of zero. This makes it a pseudometric which can be alleviated by adding a further restriction.
Inner Product of Unit Quaternions: Also pseudometrics on the unit quaternion.
Deviation from the Identity Matrix: $\lVert I - R_1 R_2^T \rVert_F$
Geodesic on the Unit Sphere: the magnitude of the angle of the rotation matrix necessary to transform $R_1$ into $R_2$

Deviation between rotations was originally published by Daniel Pollithy at Notes on January 13, 2021.

Expenses during Master studies

2021-01-01T00:00:00+01:00

This post is a summary of my expenses during two years of studying at Karlsruhe Institute of Technology (KIT). I collected the data with Splitwise because I am sharing expenses with my girlfriend. It contains my regular meals at the university’s canteen, my share of the rent, regular visits at a coffee-store, personal gifts, travels and vacations. I am sure that some expenses are missing but the broad picture should be correct.

I spent a total of € 20,965.40. This corresponds to € 873.55 per month. I assume that the missing expenses won’t increase this number to more than € 900. Just for the reference, a student could apply for a maximum BAföG of € 861 in 2020 if she or he is not living at parent’s house and has to pay insurances. Source

The COVID-19 pandemic does not become visible in the expenses although I could not go to the canteen in the last seven months and did order less at restaurants etc.

The total euros spent on each category can be read from the following chart:

Expenses during Master studies was originally published by Daniel Pollithy at Notes on January 01, 2021.

Jensen's inequality

2020-09-09T00:00:00+02:00

This is an illustrative explanation of Jensen’s inequality applied to probability.

Jensen’s inequality

Remember that a convex function is a function whose area above the graph (called epi graph) is a convex set. Informally speaking, a convex function can be identified by having positive curvature everywhere. A concave function has the opposite attributes.

Jensen’s inequality states for $\theta \in [0,1]$ and $f$ convex:

\[f(\theta x_1 + (1 - \theta) x_2) \le \theta f(x_1) + (1 - \theta) f(x_2)\]

The following image illustrates the inequality:

This is usable to define and show convexity of functions analogously to the definition of convex sets.

In probability

If we have a random variable X with probability distribution p(x) over x, then we could set $x_1=x_2=E[X]$. For this case the Jensen’s inequality displays itself as:

\[f(E[X]) \le E[f(X)]\]

For affine functions f the equality holds. Which tells us that they are concave and convex.

Derivation of the ELBO

$log p(X)$ is the log evidence, X the observed variables and Z the latent variables. q denotes a variational approximation of p(Z|X):

\[log p(X) = log \int_{Z} p(X,Z)\] \[= log \int_{Z} p(X,Z) \frac{q(Z)}{q(Z)} = log \int_{Z} \frac{p(X,Z)}{q(Z)} q(Z)\]

This is the definition of the expected value wrt q(Z):

\[= log(E[\frac{p(X,Z)}{q(Z)}])\]

Here comes the inverted Jensen’s inequality in to play (it holds for concave functions). We can confirm ourselves easily that the log() is a concave function by noting that the area under its curve is convex.

\[log(E[\frac{p(X,Z)}{q(Z)}]) \ge E[log( \frac{p(X,Z)}{q(Z)} )]\]

This can be brought into a nicer form by realizing that the definition of entropy is $H(p) = E_{x\sim p(x)}[log(\frac{1}{p(x)})] = -E_{x\sim p(x)}[log ~ p(X)]$

\[E[log( \frac{p(X,Z)}{q(Z)} )] = E[log(p(X,Z)) - log(q(Z))]\] \[E[log(p(X,Z)) - log(q(Z))] = E[log(p(X,Z))] - E[log(q(Z))]\] \[E[log(p(X,Z))] - E[log(q(Z))] = E[log(p(X,Z))] + H(q(Z))\]

Proof Kullback-Leibler divergence is always >= 0

The KL between p(x) and q(x) is defined as:

\[KL(p(x);q(x)) = -E_{x \sim p(x)}[log(\frac{q(x)}{p(x)})]\]

We can apply the concave version of Jensen’s inequality. q/p is the distribution and log the concave function.

\[-E_{x \sim p(x)}[log(\frac{q(x)}{p(x)})] \le - log(E_{x \sim p(x)}[\frac{q(x)}{p(x)}])\]

Multiply by (-1):

\[E_{x \sim p(x)}[log(\frac{q(x)}{p(x)})] \ge log(E_{x \sim p(x)}[\frac{q(x)}{p(x)}])\] \[log(E_{x \sim p(x)}[\frac{q(x)}{p(x)}]) = log(\int p(x)\frac{q(x)}{p(x)} ~ dx) = log(\int q(x) ~ dx)\]

q is a pdf therefore this integral has to be 1. log(1) = 0. As a result:

\[E_{x \sim p(x)}[log(\frac{q(x)}{p(x)})] \ge 0\]

Jensen's inequality was originally published by Daniel Pollithy at Notes on September 09, 2020.

LOTUS

2020-08-20T00:00:00+02:00

If you want to calculate the expected value of a continuous random variable which was transformed by a monotonic function, then the law of the unconscious statistician provides a convenient shortcut.

Transforming a random variable

We have a continuous random variable $X$ with a probability density function $f_{X}(x)$. This could for example be our last knowledge about the position of a robot.

Now, we apply a continuous, monotonic function $g(\cdot)$ to the random variable. This could be a simple motion model for the robot.

If we want to calculate $E[g(X)]$ then LOTUS tells us that we don’t have to solve $g(X)$ but instead we can write:

\[E[g(X)] = \int_{+\infty}^{+\infty}{g(x) \cdot f_{X}(x) dx}\]

Proof

For the proof let’s assume $g$ to be strictly increasing (decreasing would also be possible). And due to its continuity, $g(x)$ therefore has a positive derivative for every x. This results in $g(\cdot)$ being bijective therefore g is invertible. We call its inverse $g^{-1}$. With $g^{-1}: Y \rightarrow X$

Prepare change of variable

The derivative of a function and its inverse are related. They are reciprocal:

\[\frac{dx}{dy} \cdot \frac{dy}{dx} = 1\] \[\frac{dx}{dy} = \frac{1}{\frac{dy}{dx}}\]

First we replace $y$ with $g(x)$ on the right side:

\[\frac{dx}{dy}= \frac{1}{\frac{d g(x)}{dx}}\]

Second we replace $x$ with $g^{-1}(y)$ on the right side:

\[\frac{dx}{dy}= \frac{1}{\frac{d g(g^{-1}(y))}{dx}}\]

Multiplying with dy on both sides:

\[dx = \frac{1}{\frac{d g(g^{-1}(y))}{dx}} dy\]

Expected value with exchanged variable

We start with the result:

\[\int_{+\infty}^{+\infty}{g(x) \cdot f_{X}(x) dx}\]

And now we can switch from x to y. First, replace g(x) with y. Second, replace $f_{X}(x)$ with $f_{X}(g^{-1}(y))$. And third, replace dx with the right hand side from “dx = …” above:

\[\int_{+\infty}^{+\infty}{g(x) \cdot f_{X}(x) dx} = \int_{+\infty}^{+\infty}{y \cdot f_{X}(g^{-1}(y)) \frac{1}{\frac{d g(g^{-1}(y))}{dx}} dy}\]

We have now switched to integrating over y.

Cumulative density function

\[F_{Y}(y) = Pr(Y \le y)\]

Apply g:

\[F_{Y}(y) = Pr(g(X) \le y)\]

Apply $g^{-1}$ on both sides

\[F_{Y}(y) = Pr(X \le g^{-1}(y))\] \[F_{Y}(y) = F_{X}(g^{-1}(y))\]

Derivative of CDF

Now we can get the derivative of $F_{Y}(y)$ for y in order to get $f_{Y}(y)$. The chain rule is used. Note that this is the place where we need the derivative of $g^{-1}(y)$ which is $\frac{1}{\frac{d g(g^{-1}(y))}{dx}}$ !

Apply the CDF solution from above:

\[f_{Y}(y) = \frac{d}{dy} F_{Y}(y) = \frac{d}{dy} F_{X}(g^{-1}(y))\]

Apply the chain rule of derivation:

\[= f_{x}(g^{-1}(y)) \cdot \frac{d}{dy} g^{-1}(y)\] \[= f_{X}(g^{-1}(y)) \cdot \frac{1}{\frac{d g(g^{-1}(y))}{dx}} =\]

Plug-in

Two sections before, we got to this point:

\[E[g(X)] = \int_{+\infty}^{+\infty}{y \cdot f_{Y}(y) dy} = \int_{+\infty}^{+\infty}{y \cdot f_{X}(g^{-1}(y)) \frac{1}{\frac{d g(g^{-1}(y))}{dx}} dy}\]

Looking at the last formula from the section “Prepare change of variable”, we find that $f_{X}(g^{-1}(y)) \cdot \frac{1}{\frac{d g(g^{-1}(y))}{dx}} dy$ to be the same as $f_{X}(g^{-1}(y)) dx$

\[E[g(X)] = \int_{+\infty}^{+\infty}{y \cdot f_{X}(g^{-1}(y)) dx}\]

Per definition $g^{-1}(y)$ can be replaced by x.

\[E[g(X)] = \int_{+\infty}^{+\infty}{y \cdot f_{X}(x) dx}\]

And also $y$ can be replaced by g(x).

\[E[g(X)] = \int_{+\infty}^{+\infty}{g(x) \cdot f_{X}(x) dx}\]

LOTUS was originally published by Daniel Pollithy at Notes on August 20, 2020.

LogSumExp

2020-04-19T00:00:00+02:00

This is a short note to show the difference between softmax and softargmax.

It is common to use a “Softmax” layer as an activation function for a neural network used for classification. But this is actually a wrong name. The layer should be called softargmax.

Softmax

Given a vector x = [20, 5, 0.1, 300]. The softmax (also called LogSumExp) is a approximation to the maximum of the vector. Let’s apply the three steps in the name:

exp: exp([20, 5, 0.1, 300]) -> [4.8e+008, 1.4e+002, 1.1e+000, 1.9e+130]
sum: ~1.9e+130
log: 299.97

Note: If there were two “300” in the input vector, then the sum in the second step was 2 * 1.9e+130. Third step: log(2 * 1.9e+130) = log(2) + log(1.9e+130) = 300.67

Softargmax

The softargmax divides each entry of the input vector x by the softmax. Therefore it can be seen as a probability measure. The probability of each entry to be the maximum of the vector.

Calculate softmax s=299.97
x’ = x/s = [20, 5, 0.1, 300] / 299.97 ~= [0.066, 0.016, 0.0003, 1.0001]

In the classification framework we would choose the fourth class as a prediction.

Addendum

Maximum rewritten:$max(x) = log(exp(max(x)))$
Softargmax definition: $logsumexp(x) = log(exp(x_1) + ... + exp(x_n))$
Upper bound for Softargmax: $log(exp(x_1) + ... + exp(x_n)) \le log(exp(max(x)) + ... + exp(max(x)))$
Factor out n: $log(exp(max(x)) + ... + exp(max(x))) = log(n \cdot exp(max(x)))$
Log rule: $log(n \cdot exp(max(x))) = log(n) + log(exp(max(x)))$
Insert defintion of max: $log(n) + log(exp(max(x))) = log(n) + max(x)$
Result: $logsumexp(x) \le max(x) + log(n)$

LogSumExp was originally published by Daniel Pollithy at Notes on April 19, 2020.

Stochastic Information Processing

2020-04-17T00:00:00+02:00

This is a summary of the lecture Stochastic Information Processing which I attended in the winter semester 2019/2020. The content has overlaps with LMA (localization of mobile agents), i.e. Kalman Filter. In SI it is taught how to convert a generative model into a probabilistic one and how to confront parametric integrals by means of approximations.

SI

Unfortunately, I wrote this in German. But the derivations could be interesting nevertheless.

Statische Systeme

Statische Systeme sind besonders einfache wertediskrete Systeme, in denen die Ausgabe y nur von der Eingabe u abhängt. Wobei u und y diskrete Zufallsvariablen sind. Ein statisches System S stellt eine bedingte Wahrscheinlichkeitsverteilung dar: $P(y=i|u=j)$. Von der Verteilung u zur Verteilung y. Da beide diskreten ZV auf endlichen Mengen definiert sind, können wir S als stochastische Matrix A darstellen. Diese enthält per Konvention $(P(y=i|u=j))_{i,j}$, also pro Zeile die Verteilung für ein konkretes u. A enthält nur nicht negative Einträge, jede Zeilensumme ist eins. Wenn zusätzlich auch noch die Spaltensummen ein ergeben, dann spricht man von einer doppelt-stochastischen Matrix.

Möchte man nun y aus u errechnen, so kann man so vorgehen:

\[P(y=i) \stackrel{(1)}{=} \sum_{j=1}^{J}{P(y=i, u=j)} \stackrel{(2)}{=} \sum_{j=1}^{J}{ P(y=i|u=j) \cdot P(u=j)} \stackrel{(3)}{=} A^T \cdot P(u)\]

(1) Satz der totalen W’keit. (2) Konditionierung. (3) Summe als Matrix.

Man sieht also, dass das System nicht deterministisch ist. Unsicherheiten werden weiter propagiert.

Dynamische Systeme (diskret)

Der Ausgang $\boldsymbol{y_k}$ hängt von Eingabe $\boldsymbol{u_k}$ und Zustand $\boldsymbol{x_k}$ ab. Der Zustand fasst die gesamte Historie zusammen. Man unterteilt dyn. Systeme in die Systemabbildung (dynamischer Teil) und die statische Messabbildung. Diskrete ZV $\boldsymbol{x_k}$ ist Markov-Kette erster Ordnung, falls gilt:

Systemabbildung

\[P(\boldsymbol{x_{k+1}}|\boldsymbol{x_{k}}, ... \boldsymbol{x_{0}}, \boldsymbol{u_{k}}, ...\boldsymbol{x_{0}}) = P(\boldsymbol{x_{k+1}}|\boldsymbol{x_{k}})\]

(1. Markoweigenschaft)

Man sagt: $\boldsymbol{x_{k+1}}$ ist bedingt unabhängig von $\boldsymbol{x_{k-1}}$ … $\boldsymbol{x_{0}}$. Wobei die Bedingung ist, dass $\boldsymbol{x_{k}}$ bekannt ist.

Achtung! $\boldsymbol{x_{k+1}}$ ist nicht unabhängig von $\boldsymbol{x_{0}}$. Nur bedingt unabhängig. Es folgt also, dass die zukünftige Entwicklung bedingt unabhängig von alten Zuständen ist, falls der aktuelle Zustand bekannt ist. Man betrachte zum Beispiel einen fallenden Ball. Kennt man den Zustand (Position, Geschwindigkeit, Beschleunigung, etc.) des Balles, so kann man seinen nächsten Zustand ohne Wissen über die Vergangenheit prädizieren.

Messabbildung

Der Zustand ist meist nicht verfügbar (Wortwahl ist Absicht). Man spricht von einer “latenten Variable”. Stattdessen kann man meist modellieren, wie $y_k$ sein müsste, wenn man einen gewissen Zustand $x_k$ und einen gewissen Eingang $u_k$ hätte. Diese Abbildung kann man ebenfalls als bedingte W’keit formulieren: $P(y_k=j|x_k=i, u_k=m)$. Der Einfluss von $u_k$ wird als “Durchgriff” bezeichnet, aber oft weggelassen. Wenn $y_k$ bedingt unabhängig von allen anderen ZV ist, falls $x_k$ gegeben ist, so nennt man dies die 2. Markoweigenschaft.

Die diskrete Messabbildung wird als Messmatrix B(i,j) bezeichnet (analog zu A.).

Hidden Markov Model (HMM)

Mit folgenden Zutaten kann nun also ein HMM beschrieben werden:

Initialer Zustandsvektor
Systemabbildung
Messabbildung

Prädiktion, Filterung und Glättung

Die Aufgabe eines Schätzers in einem solchen System läge zum Beispiel darin, gegeben die Messungen bis zu einem Zeitpunkt t1, einen einzelnen oder mehrere Zustände zu einem beliebigen Zeitpunkt t2 zu schätzen.

Dabei unterscheidet man:

Prädiktion: t2 > t1
Filterung: t2 = t1 (Man prädiziert von (t2-1) zum aktuellen Zeitschritt und korriegiert dann seine Schätzung mit der ebenfalls bekannten Messung)
Glättung: t2 < t1.
- “Fixed-Lag smoothing” ähnlich zu Filterung bloß kleiner Lag.
- “Fixed-Point Smoothing”: Der Zeitpunkt, der geschätzt werden soll, steht fest. Es kommen aber immer noch mehr Daten hinein.
- “Fixed-interval smoothing”: Man schätzt alle Zustände bis zum t2=t1-1, verwendet aber nur die Messungen, die bei den aktuellen Zeitpunkten zur Verfügung standen.
- “Fixed-interval prediction”: Man hat nur die Messungen in einem festen Interval, es kommt nicht neues mehr dazu. Diese feste Anzahl an Punkten verwendet man dann um beliebige Prädiktionen zu machen.

Wonham Filter

Komplettes rekursives, diskretes Filter unter Verwendung von Einschrittprädiktion.

Dynamikmodell: $x_{t+1} = A^T x_t$
Messmodell: $y_t = B^T x_t$

Aus einer Messung auf den Zustand zu schließen, funktioniert wie folgt:

\[P(x_k|y_{0:k}) \stackrel{(1)}{=} P(x_k|y_k, y_{0:{k-1}}) \\ \stackrel{(2)}{=} \frac{P(y_k|x_k, y_{0:{k-1}}) P(x_k|y_{0:{k-1}})}{P(y_k|y_{0:{k-1}})} \\ \stackrel{(3)}{=} \frac{P(y_k|x_k, y_{0:{k-1}}) P(x_k|y_{0:{k-1}})}{\sum_{i}{P(y_k, x_k=i|y_{0:{k-1}})}} \\ \stackrel{(4)}{=} \frac{P(y_k|x_k, y_{0:{k-1}}) P(x_k|y_{0:{k-1}})}{\sum_{i}{P(y_k|x_k=i,y_{0:{k-1}}) P(x_k=i|y_{0:{k-1}})}} \\ \stackrel{(5)}{=} \frac{P(y_k|x_k) P(x_k|y_{0:{k-1}})}{\sum_{i}{P(y_k|x_k=i) P(x_k=i|y_{0:{k-1}})}} \\\]

(1) Nur andere Schreibweise

(2) Bayes Regel

(3) Satz der totalen W’keit

(4) Konditionierung

(5) 2. Markovannahme

Für ein konkret beobachtetes Symbol y’ bedeutet das, dass man aus der Matrix B die beobachtete Spalte entnimmt. Diese enthält die Likelihood von y’ unter allen möglichen Zuständen. Diesen multipliziert man dann mit vom Systemmodell prädizierten Zuständen und normiert. So erhält man einen durch die Beobachtungen verbesserte Schätzung über den Zustand.

# Beispiel: Wie viele Sterne (1-5) hat ein Buch wirklich verdient?
#           Böse Bots bewerten Bücher entweder mit 5 oder 1 Stern.
#           Menschen tendieren zu "nicht extremen" Bewertungen.
#              Also lieber 4 als 5 Sterne geben, obwohl das Buch sehr gut war.
#              Und wenn ein Buch schon gut bewertet wird, dann ist man umso kritischer.


# Starte gleichverteilt
x = np.array([0.2, 0.2, 0.2, 0.2, 0.2])[:, np.newaxis]

A = np.identity(5)

B = np.array([
    [0.60, 0.20, 0.1, 0.01, 0.09],  # Messungen, wenn das Buch 5 Sterne hätte
    [0.10, 0.70, 0.1, 0.01, 0.09],  # Messungen, wenn das Buch 4 Sterne hätte
    [0.09, 0.01, 0.8, 0.01, 0.09],  # Messungen, wenn das Buch 3 Sterne hätte
    [0.09, 0.01, 0.1, 0.70, 0.10],  # Messungen, wenn das Buch 2 Sterne hätte
    [0.09, 0.01, 0.1, 0.20, 0.60],  # Messungen, wenn das Buch 1 Sterne hätte
])

# Wie viele Sterne sollen wir auf unserer Website darstellen, wenn wir folgende
#    Bewertungen erhalten haben
star_ratings= [5, 1, 3, 5, 3, 4, 4, 1, 3, 4, 2]

# Rekursive Filterung

print("Kunde bewertet: \t\t 1 Stern \t 2 Sterne \t 3 Sterne \t 4 Sterne \t 5 Sterne")
print("-"*110)

for star_rating in star_ratings:    
    # step prediction
    x = A.T @ x

    # filter.
    # select the column of B for our observed measurement
    B_col = B[:, star_rating-1]

    x = (np.diag(B_col) @ x) / (np.ones(5) @  (np.diag(B_col) @ x))  

    print("{} Sterne \t\t->\t {:.2f} \t\t {:.2f} \t\t {:.2f} \t\t {:.2f} \t\t {:.2f}".format(
        star_rating, x[0,0], x[1,0], x[2,0], x[3,0], x[4,0])
         )

Kunde bewertet: 		 1 Stern 	 2 Sterne 	 3 Sterne 	 4 Sterne 	 5 Sterne
--------------------------------------------------------------------------------------------------------------
Sterne 		->	 0.09 		 0.09 		 0.09 		 0.10 		 0.62
Sterne 		->	 0.40 		 0.07 		 0.06 		 0.07 		 0.40
Sterne 		->	 0.28 		 0.05 		 0.34 		 0.05 		 0.28
Sterne 		->	 0.11 		 0.02 		 0.13 		 0.02 		 0.72
Sterne 		->	 0.06 		 0.01 		 0.54 		 0.01 		 0.38
Sterne 		->	 0.01 		 0.00 		 0.06 		 0.08 		 0.85
Sterne 		->	 0.00 		 0.00 		 0.00 		 0.25 		 0.74
Sterne 		->	 0.00 		 0.00 		 0.00 		 0.25 		 0.74
Sterne 		->	 0.00 		 0.00 		 0.02 		 0.25 		 0.73
Sterne 		->	 0.00 		 0.00 		 0.00 		 0.54 		 0.46
Sterne 		->	 0.00 		 0.00 		 0.00 		 0.54 		 0.46

Wertekontinuierliche Systeme

Wertekontinuierliche lineare Systeme

Linearität eines System ($A:x\rightarrow y$) besteht dann, wenn folgende zwei Eigenschaften vorliegen:

Skalierung: $A(c \cdot x) = c \cdot A(x)$
Superposition: $A(x_1 + x_2) = A(x_1) + A(x_2)$

In der Realität existieren lineare Systeme nur in gewissen Grenzen.

Statisches System

$ y_k = f(u_k) $, wobei $u_k \in R^{P}$ und $y_k \in R^{M}$

Für $f$ linear kann man schreiben: $y_k = A_k u_k$ mit $A \in R^{MxP}$.

Dieses generative Modell wird stochastisch, wenn $u_k$ eine W’keitsverteilung ist oder man endogenes Rauschen hinzufügt (also Rauschen im System). Dafür erweitert man $u_k$ zu $u_{k}^{*} = [u_k, w_k]$, wobei $w_k$ das Rauschen zusammenfasst.

Die Unsicherheiten in $u_k$ und $y_k$ beschreibt man oft mit den ersten beiden Momenten dieser vektoriellen ZV (Erwartungswert und Kovarianzmatrix).

Wir definieren $\hat{u}_k = E[u_k]$ und $C_k^u = Cov[u_k]$

Wichtige Erinnerung

Die Linearität des Erwartungswertes liefert für ein lineares System: $E[y_k] = E[A_k u_k] = A_k E[u_k] = A_k \hat{u}_k$.

Und die Kovarianz: $Cov[y_k] = Cov[y_k] = E[(E[y_k] - y_k)^2] = E[ (\hat{y}_k - y_k)^2 ] = E[ (A \hat{u}_k - A u_k)^2 ] = E[(A (\hat{u}_k - u_k))^2] = E[A ( \hat{u}_k - u_k)^2 A^T] = A E[ ( \hat{u}_k - u_k)^2 ] A^T = A Cov_k^u A^T$

Dynamische Systeme

Lineare Entwicklung des Zustandes: $x_{k+1} = A_k x_k + B_k u_k$ (Markovkette 1. Ordnung, zeitvariant, ohne k wäre es zeitinvariant).

Additives Rauschen: $u_k = \hat{u}_k + w_k$, wobei $w_k$ ein Zufallsvektor mit $E[w_k]=0$ und $Cov[w_k]=C_k^w$

Messmodell: Zustand nicht verfügbar. Achtung: Verfügbarkeit heißt, dass man eine Größe nicht messen kann, wobei Beobachtbarkeit bedeutet, dass man mit Messungen auf die Größe schließen kann. Der verdeckte Zustand ist nicht verfügbar, aber beobachtbar. Beispiel: Position eines Autos in Foto ist verfügbar, Geschwindigkeit nur über mehrere Frames beobachtbar.

Lineare Messabbilung: $y_k = H_k x_k + v_k$ wobei $v_k$ wieder additives Rauschen ist.

Kalman Filter

Prädiktionsschritt

$x_{k+1} = A_k x_k + B_k (\hat{u}_k + w_k)$

$E[x_{k+1}] = E[A_k x_k + B_k (\hat{u}_k + w_k)] = E[A_k x_k] + E[B_k (\hat{u}_k + w_k)] = A_k \hat{x}_k + B_k E[\hat{u}_k + w_k] = A_k \hat{x}_k + B_k E[\hat{u}_k + w_k] = A_k \hat{x}_k + B_k \hat{u}$

Notiz: $E[w_k] = 0$ per Definition. Wenn er nicht null wäre, aber bekannt, so könnte man ihn von $w_k$ abziehen. Wenn er nicht bekannt wäre, dann müsste man diesen ebenfalls schätzen.

$Cov[x_{k+1}] = Cov[A_k x_k + B_k (\hat{u}_k + w_k)] = A_k C_k^x A_k^T + B_k C_k^w B_k^T$

Trick:

Um die Kovarianz zu berechnen, definiert man $A_k^{} = [A_k, B_k]$ und $x_k^{} = [x_k, u_k]^T$.

Rauschen sei unabhängig von Zustand. Dann sieht $Cov[x_k^{*}]$ wie folgt aus: $\begin{bmatrix}C_k^x & 0\0 & C_k^w\end{bmatrix}$

Dann geht:

\[Cov[x_{k+1}^{*}] = E[(x_k^{*} - \hat{x_k^{*}})^2] = E[(x_k^{*} - \hat{x_k^{*}}) (x_k^{*} - \hat{x_k^{*}})^T]\] \[= E[A_k^{*} \cdot \begin{bmatrix} x_k - \hat{x_k} \\ u_k - \hat{u_k} \end{bmatrix} \cdot [(x_k - \hat{x_k})^T , (u_k - \hat{u_k})^T] \cdot {A_k^{*}}^T ]\] \[= A_k^{*} \cdot E[\begin{bmatrix} x_k - \hat{x_k} \\ u_k - \hat{u_k} \end{bmatrix} \cdot [(x_k - \hat{x_k})^T , (u_k - \hat{u_k})^T] ] \cdot {A_k^{*}}^T\]

Da die Kovarianz mittelwertbefreit ist, folgt:

\[= A_k^{*} \cdot Cov[ \begin{bmatrix} x_k \\ u_k \end{bmatrix} ] \cdot {A_k^{*}}^T\] \[= A_k^{*} \cdot \begin{bmatrix}C_k^x & 0\\0 & C_k^w\end{bmatrix} \cdot {A_k^{*}}^T\] \[= [A_k, B_k] \cdot \begin{bmatrix}C_k^x & 0\\0 & C_k^w\end{bmatrix} \cdot \begin{bmatrix}A_k^T \\ B_k^T \end{bmatrix}\] \[= A_k C_k^x A_k^T + B_k C_k^w B_k^T\]

Kalman Filterschritt

Lineare Messabbildung: $y_k = H_k x_k + v_k$ “Was für eine Messung erwarte ich?”

Die Kombination von Vorwissen und Messung ermöglicht es, auf einen höherdimensionalen verdeckten Zustand zu schließen:

$x_k^p = K_k^1 x_k + K_k^2 y_k$ (Der Filterschritt ist eine gewichtete Linearkombination)

Gesucht ist das BLUE filter (Best linear unbiased estimator). Linear ist er ja schon.

Unbiased: $E[x_k] = \tilde{x_k}$ Wobei $\tilde{x_k}$ die Grundwahrheit ist.

\[E[x_k^p] = E[K_k^1 x_k + K_k^2 y_k] = K_k^1 E[x_k] + K_k^2 E[y_k] = K_k^1 E[x_k] + K_k^2 H_k E[x_k] + E[v_k] = K_k^1 E[x_k] + K_k^2 H_k E[y_k] = K_k^1 \tilde{x_k} + K_k^2 H_k \tilde{x_k}\] \[K_k^1 \tilde{x_k} + K_k^2 H_k \tilde{x_k} = \tilde{x_k}\] \[K_k^1 + K_k^2 H_k = I\] \[K_k^1 = I - K_k^2 H_k\]

=> Dadurch kann man die Linearkombination mit nur einer Matrix $K_k$ ausdrücken:

\[x_k^p = (I - K_k^2 H_k) x_k + K_k^2 y_k = (I - K_k H_k) x_k + K_k y_k\]

Die Frage ist nun, wie bestimmen wir das optimale K? Wir suchen das K, für das die prädizierte Kovarianz $C_k^e$ minimal wird.

\[C_k^e = Cov[x_k^p] = (I - K_k H_k) C_k^x (I - K_k H_k)^T + K_k C_k^y K_k^T\]

$C_k^e$ ist abhängig von $K_k$. Daher schreiben wir: $C_k^p(K_k)$

$C_k^p(K_k)$ ist positiv definit und symmetrisch, das heißt, es handelt sich um eine Kovarianzmatrix. Wie minimiert man das?

Man könnte $det(C_k^p(K_k))$ oder $trace(C_k^p(K_k))$ minimieren. Beide sind proportional zum Flächeninhalt der Kovarianzellipse. Die Determinante ist 1/4 des umschließenden Rechteckes. Die Spur hat die Größe des größten Quaders, der in die Ellipse hineinpasst.

Wenn man die Determinante minimiert, dann ist es theoretisch möglich, dass eine Richtung 0 wird. Dadurch wird der Flächeninhalt null und die Covarianz ist minimal. Das ist unschön, weil man die Unsicherheit in alle Richtungen minimieren will.

Daher verwenden wir die Projektion der Matrix auf alle möglichen Einheitsvektoren e. Diese Projektion P(K) liefert ein skalares Gütemaß, welches wir minimieren.

$P(K_k) = e^T C_k^e(K_k) e$.

Ableitungsregeln für Matrizen:

Matrix C:

$ d/dC (a^T C b) = a^T b $ ($a^T b$ ist diadisches Produkt, also eine singuläre Matrix. Rang=1. det=0. nicht invertierbar.)
$ d/dK (a^T K C K^T b) = ab^T K C^T + ba^T K C $ (KCK^T ist ein lineare Propagation, evtl. Basiswechsel. a^T … b ist eine Projektion.)
wenn a=b=e und C symm., dann: $ d/dK (e^T K C K^T e) = 2 ee^T KC $

jetzt können wir $P(K_k)$ ableiten:

\[\frac{d}{dK_k} e^T C_k^e(K_k) e =\] \[\frac{d}{dK_k} e^T ((I - K_k H_k) C_k^x (I - K_k H_k)^T + K_k C_k^y K_k^T) e =\] \[\frac{d}{dK_k} e^T ( (I C_k^x I^T) - (I C_k^x H_k^T K_k^T) - (K_k H_k C_k^x I^T) + (K_k H_k C_k^x H_k^T K_k^T) + K_k C_k^y K_k^T) e =\]

Erster Summand fällt weg, weil er kein K enthält.

\[\frac{d}{dK_k} e^T ( - (I C_k^x H_k^T K_k^T) - (K_k H_k C_k^x I^T) + (K_k H_k C_k^x H_k^T K_k^T) + K_k C_k^y K_k^T) e =\]

Projektion auf alle Summanden anwenden:

\[\frac{d}{dK_k} - e^T (I C_k^x H_k^T K_k^T) e - e^T (K_k H_k C_k^x I^T) e + e^T (K_k H_k C_k^x H_k^T K_k^T) e + e^T ( K_k C_k^y K_k^T) e\]

Summand (Mit $D^T = C_k^x H_k^T$ und) $\frac{d}{dK_k} - e^T (I C_k^x H_k^T K_k^T) e =$
\[(\frac{d}{dK_k^T} - (e^T (D^T K_k^T) e))^T = - (D ee^T)^T = - (H_k C_k^x ee^T)^T\]

2 Summand: $- ee^T (H_k C_k^x)^T$
3 Summand: $2 ee^T K_k H_k C_k^x H_k^T$
4 Summand: $2 ee^T K_k C_k^y$

Daraus folgt:

\[\frac{d}{dK_k} - e^T (I C_k^x H_k^T K_k^T) e - e^T (K_k H_k C_k^x I^T) e + e^T (K_k H_k C_k^x H_k^T K_k^T) e + e^T ( K_k C_k^y K_k^T) e\] \[= - (H_k C_k^x ee^T)^T - ee^T (H_k C_k^x)^T + 2 ee^T K_k H_k C_k^x H_k^T + 2 ee^T K_k C_k^y\]

Setzen wir Null:

\[- (H_k C_k^x ee^T)^T - ee^T (H_k C_k^x)^T + 2 ee^T K_k H_k C_k^x H_k^T + 2 ee^T K_k C_k^y \stackrel{!}{=} 0\] \[= ee^T [-{C_k^x}^T H_k^T - {C_k^x}^T H_k^T + 2 K_k H_k C_k^x H_k^T + 2 K_k C_k^y ]\] \[= ee^T [-2{C_k^x}^T H_k^T + 2 K_k H_k C_k^x H_k^T + 2 K_k C_k^y ]\]

Wir multiplizieren beidseitig mit $({ee^T})^{-1}$:

\[-2{C_k^x}^T H_k^T + 2 K_k H_k C_k^x H_k^T + 2 K_k C_k^y = 0\] \[+ 2 K_k H_k C_k^x H_k^T + 2 K_k C_k^y = +2{C_k^x}^T H_k^T\]

Teilen durch 2:

\[K_k H_k C_k^x H_k^T + K_k C_k^y = {C_k^x}^T H_k^T\]

Klammern $K_k$ aus:

\[K_k (H_k C_k^x H_k^T + C_k^y) = {C_k^x}^T H_k^T\]

Und multiplizieren rechtseitig mit $(H_k C_k^x H_k^T + C_k^y)^{-1}$, um $K_k$ alleine zu stellen:

\[K_k = (C_k^x)^T H_k^T (H_k C_k^x H_k^T + C_k^y)^{-1}\]

Das Transponierte der Kovarianzmatrix ist immernoch die Kovarianzmatrix, da sie symmetrisch ist:

\[K_k = C_k^x H_k^T (H_k C_k^x H_k^T + C_k^y)^{-1}\]

$K_k$ heißt Kalman Gain und ist optimal, wenn:

das Filter linear ist und
die Zufallsvariablen normalverteilt sind (zum Beispiel additives, normalverteiltes Rauschen)

Somit ergibt sich:

\[E[x_k^e] = (I - K_k H_k) E[x_k^p] + K_k E[y_k]\]

Und:

$C_k^e = Cov[x_k^p] = (I - K_k H_k) C_k^x (I - K_k H_k)^T + K_k C_k^y K_k^T$ $...$ $C_k^e = C_k^p - K_k H_k C_k^p$

Für intuitive Anschauungen eignet es sich die Feedback Form zu betrachten:

\[\hat{x_k^e} = \hat{x_k^p} + K_k (\hat{y_k} - H_k \hat{x_k^p})\]

Hieran lässt sich erkennen, dass der geschätzte Zustand nach einem Messupdate eine Kombination des priori Zustandes mit dem neuen Wissen aus der Messung ist. Wenn das Kalman Gain hoch ist, dann wird die Messung mit geringer Unsicherheit gesehen und somit werden starke Abweichungen der Messung von der erwarteten Messung stark auf den neuen Zustand einfließen. Wenn das Messrauschen unendlich groß wäre, dann wäre $K_k=0.$

Allgemeine Systeme

Statische Systeme

Generatives Modell (Werte abgebildet auf Werte): $\underline{\mathbf{y}_k} = a(\underline{\mathbf{u}_k})$

Jetzt sind aber $u_k$ und $y_k$ durch die W’keitsverteilungsdichtefunktionen (kurz “Dichten”) $f_k^u(\underline{\mathbf{u}_k})$ und $f_k^y(\underline{\mathbf{y}_k})$

Würde man nun einfach trotzdem mit Erwartungswert und Kovarianz weiterrechnen, anstatt die Dichten zu betrachten, dann wäre das eine Linearisierung, wie sie beim EKF vorgenommen wird.

Gesucht ist also die Dichte $f_k^y(\underline{\mathbf{y}_k})$ zur gegebenen Dichte $f_k^u(\underline{\mathbf{u}_k})$. Ein Abbildung von einer Dichte zu einer anderen Dichte nennen wir probabilistisches Modell.

Dynamische Systeme

Diese zerfallen wieder in das Beobachtungsmodell und die Systemabbildung.

Ein “einfacheres” nichtlineares Systemmodell: $x_{k+1} = a_k(x_k, \hat{u}_k, w_k)$

Es wäre schwieriger, wenn $x_{x+1}$ auch auf der rechten Seite vorkäme, dann spricht man von einem impliziten System. Dieser Fall wurde in LMA behandelt.)

Für additives Rauschen kommt man leicht vom generativen zu einem probabilistischen Modell:

\[x_{k+1} = a(x_k, \hat{u}_k) + w_k\]

Das Systemrauschen $w_k$ ist hier additiv und wird beschrieben durch die Dichte $f_k^w(w_k)$.

Zusätzlich gehen wir davon aus, dass das Systemrauschen:

Gaussverteil ist mit bekannten Parametern.
$f_k^w(w_k)$ für unterschiedliche k unabhängig voneinander sind. Im Falle von Gaußverteilung reicht unkorreliert, da in diesem Fall unabhängig daraus folgt. Dann gilt für die gemeinsame Verteilung zweier Rauschdichten: $f_{1,2}^w(w_1, w_2) = f_1^w(w_1) \cdot f_2^w(w_2)$

Bei der Messabbildung kann genauso verfahren werden: $y_k = h_k(x_k) + v_k$

Was für Dichten können wir verwenden?

Kontinuierliche analytische Dichten
- Gaußdichte
- GMM
- Exponentialdichte (Exponenten komplexer machen)
- Edgeworth-Entwicklungen (Gaußdichte mit Polynom multipliziert. Nichts globales, aber lokal bekommt man ein paar Hügel rein).
Diskrete Dichten auf kontinuierlichen Domänen
- Dirac-Mixture Densities

Blick auf Gaußdichte

2d: $f_{x,y}(x,y) = N(\begin{bmatrix}x\\y\end{bmatrix} - \begin{bmatrix}\hat{x}\\\hat{y}\end{bmatrix}, \begin{bmatrix}\sigma_x^2 & \rho \sigma_x \sigma_y \\ \rho \sigma_x \sigma_y & \sigma_y^2 \end{bmatrix})$

Wobei $\rho \in [-1, +1]$ als Korrelationskoeffizient bezeichnet wird. Wenn dieser =+1 ist, dann lieft eine deterministische Abhängigkeit vor.

Man schreibt manchmal: $f_{x,y}(x,y) = \frac{1}{2 \pi \sigma_x \sigma_y \sqrt{(1-\rho^2)}} \exp(-\frac{1}{2}Q(x,y))$

mit $Q(x,y) = \frac{1}{1-\rho} \Big( \frac{(x-\hat{x})^2}{\sigma_x^2} + \frac{(y-\hat{y})^2}{\sigma_y^2} - 2 \rho \frac{x-\hat{x}}{\sigma_x} \frac{y-\hat{y}}{\sigma_y} \Big)$

Maßgrößen für Kovarianzen:

die Determinante von Cov det(C) ist proportional zum Volumen der Kovarianz-Körpers, da die Determinante das Produkt der Eigenwerte ist. Somit ist im 2d-Fall zum Beispiel 4*det(C) die Fläche des umschließenden Rechtecks.
Spur (Summe der Quadrate der Diagonale): Nach Pythagoras ist die Spur einer diagonalen Kovarianzmatrix das Quadrat der der Länge … -> Wurzel … Vorteil: Spur invariant ggü. unitärer Rotation. Selbst wenn ein Eigenwert mal 0 sein sollte, dann ist die Spur immernoch nicht null.

Moment Matching

Berechne die Verteilung einer Population anhand ihrer Momente. Angenommen wir haben eine Stichprobe aus der Population genommen. Nun könnten wir zum Beispiel die empirischen 1. und 2. Momente berechnen. Empirischer Mittelwert und empirische Varianz. Nun wählen wir eine beliebige Verteilungsfunktion. Jetzt können wir Parameter für die Verteilung bestimmen, die empirischen Momente liefern. Diese Methode ist besser per Hand zu rechnen und kann zum Beispiel auch als Initialisierung für eine Maximum-Likelihood-Schätzung verwendet werden.

Theorem: Für eine gegebene Dichte $\tilde{f}(x)$ führt die Wahl von $m=E_{\tilde{f}}[x]$ und $\sigma^2=Cov_{\tilde{f}}[x]$, zu einer Minimierung der Kullback-Leibler-Divergenz zwischen $\tilde{f}$ und der Normalverteilung.

\[KL = \int_{R} { \tilde{f}(x) log\big( \frac{\tilde{f}(x)}{f(x)} \big) } dx\]

Offene Frage: Warum $\tilde{f}(x)$ als Vorfaktor?

Das i-te zentrale Momente einer Gaußdichte lautet:

\[C_i = E[(x-x)^{i}]\]

Ein zentrales Moment ist das gleiche, wie ein “normales” Moment, bloß, dass man die Dichte um Null zentriert hat.

ToDo: Beschreibe, wie man Momente generell berechnen kann.

GMM: $f(x) = \sum_{i=1}^{L}{w_i N(\hat{x}_i, \sigma_i)}$

Anzahl Moden ist ungleich Anzahl Komponenten -> Maximum bestimmen nicht trivial
Wir akzeptieren nur $w_i > 0$. Theoretisch geht es aber auch anders. Und Summe der Gewichte muss 1 sein.
Ein GMM mit L Komponenten hat 3L-1 Freiheitsgrade

Diracsche Deltafunktion

$\delta(x-x_0) = $ …

nicht definiert, für $x=x_0$
0, für alles andere

Mit $\int_{R}{\delta(x-x_0) dx} = 1$

Sie besitzt die Ausblendeigenschaft:

$\int f(x) \delta(x-x_0) dx = f(x_0)$
Auch: $f(x) \cdot \delta(x-x_0) = f(x_0) \cdot \delta(x-x_0)$
- Fall $x=x_0$: $f(x_0) = f(x_0)$
- Sonst: 0

Allgemein gilt:

\[\delta(f(x)) = \sum_{i=1}^{N}{\frac{1}{|f'(x_i)|} \delta(x-x_i) }\]

$x_i$ sind die N Nullstellen der Funktion f(x). $f(x) \ne 0$

Beweis dazu: https://math.stackexchange.com/questions/276583/dirac-delta-function-of-a-function.

(Eine sehr umfassende Sammlung von Eigenschaften der diracschen Delta-Funktion: https://mathworld.wolfram.com/DeltaFunction.html)

Beispiele generatives Modell zu bedingter Dichte:

$y = 3 * x$ —–> $f(y

x) = \delta(y-3x)$

$y = a(x) + w$ —–> $f(y

x,w) = \delta(y-a(x)-w)$

Bei den bedingten Dichten handelt es sich also um dirac distributionen, die nur dort ungleich Null sind, wo die Funktion wahr ist.

Additives Rauschen

\[x_{k+1} = a_k(x_k) + w_k\] \[f^{x_{k+1}}(x_{k+1}| x_k, w_k) = \delta(a_k(x_k) + w_k - x_{k+1})\] \[f^{x_{k+1}}(x_{k+1}| x_k) = \int{f^{x_{k+1}, w_k}(x_{k+1}, w_k| x_k) d w_k}\] \[f^{x_{k+1}}(x_{k+1}| x_k) = \int{f^{x_{k+1}}(x_{k+1}|x_k, w_k) \cdot f^w(w_k|x_k) d w_k}\]

Hier nehmen wir jetzt an, dass das Rauschen unabhängig von den Daten ist.

\[f^{x_{k+1}}(x_{k+1}| x_k) = \int{f^{x_{k+1}}(x_{k+1}|x_k, w_k) \cdot f^w(w_k) d w_k}\]

Jetzt Dirac einsetzen:

\[f^{x_{k+1}}(x_{k+1}| x_k) = \int{\delta(a_k(x_k) + w_k - x_{k+1}) \cdot f^w(w_k) d w_k}\]

Für welches $w_k$ ist der Dirac-Stoß ungleich Null? (vgl. $\delta(w_k - w_{k,0})$)

\[a_k(x_k) + w_k - x_{k+1} = 0\] \[w_k = x_{k+1} - a_k(x_k)\]

Durch Siebeigenschaft des Dirac folgt:

\[f^{x_{k+1}}(x_{k+1}| x_k) = f^w(x_{k+1} - a_k(x_k) )\]

Was eine bequeme Form für additives Rauschen ist. Wenn das Rauschen zum Beispiel normalverteilt ist, dann ist die Transitionsdichte für ein gegebens x_k ebenfalls normalverteilt.

Multiplikatives Rauschen

$ x_{k+1} = x_k \cdot w_k $ mit $w_k \sim f_k^w(w_k)$

\[f(x_{k+1}|x_k, w_k) = \delta(g(w_k))\] \[g(w_k) = x_k w_k - x_{k+1}\]

Finde Nullstellen von g:

\[x_k w_k - x_{k+1} = 0\] \[x_k w_k = x_{k+1}\] \[w_k = \frac{x_{k+1}}{x_k}\] \[\frac{d}{d w_k} g(w_k) = x_k\]

Daraus folgt:

\[\delta(g(w_k)) = \frac{1}{|x_k|} \delta(w_{k} - \frac{x_{k+1}}{x_k})\] \[f(x_{k+1}|x_k) = \int{f(x_{k+1}|x_k, w_k) f_k^w(w_k) d w_k}\] \[f(x_{k+1}|x_k) = \int{\delta(g(w_k)) f_k^w(w_k) d w_k}\] \[f(x_{k+1}|x_k) = \int{\frac{1}{|x_k|} \delta(w_{k} - \frac{x_{k+1}}{x_k}) f_k^w(w_k) d w_k}\] \[f(x_{k+1}|x_k) = \frac{1}{|x_k|} f_k^w(\frac{x_{k+1}}{x_k})\]

Visualisierung der Transitionsdichte

y = a(x) + w

y = x + N(0, 1)

\[f(y|x) = \int{f(y|x,w) \cdot f(w) dw} = f^w(y - a(x)) = f^w(y - x)\] \[f(y|x) = N(y - x)\]

from scipy.stats import norm

%matplotlib notebook

step = 0.5

x = np.arange(0, 10, step)
y = np.arange(0, 10, step*0.1)
z = np.arange(0, 10, step)

X, Y = np.meshgrid(x, y)

f = norm.pdf(Y-X, 0.1)



fig = plt.figure()
ax = fig.gca(projection='3d')

surf = ax.plot_surface(X, Y, f, cmap=cm.coolwarm,
                       linewidth=0, antialiased=True)

# Add a color bar which maps values to colors.
fig.colorbar(surf, shrink=0.5, aspect=5)

plt.xlabel("$x_k$")
plt.ylabel("$y_k$")
ax.set_zlabel("$P(y|x)$")

plt.show()

<IPython.core.display.Javascript object>

Dieser Plot ist so zu sehen, dass für jedes gegeben $\hat{x}_k$ eine Normalverteilung vorliegt. Die Verbundwahrscheinlichkeit von $P(x,y)$ im Diskreten oder $f^{x,y}(x,y)$ im Kontinuierlichen hängt von $x_k$ ab. Im Diskreten kann man folgenden Satz nutzen: $P(X \cap Y) = P(X|Y) \cdot P(Y) = P(Y|X) \cdot P(X)$.

Chapman-Kolmogorov-Integral (CKI)

Betrachten wir einen stochastischen Prozess $(x)_k$ bei dem wir die 1. Markov-Eigenschaft annehmen.

\[f(x_{k+1}) = \int_{R}{ f(x_{k+1}|x_k) \cdot f(x_k) d x }\]

Wir berechnen die Transitionsdichte zunächst weiterhin mit der dirachsen Delta-Funktion. Für den Trivialfall, dass $f(x_k)$ deterministisch (also dirac-verteilt) ist, löst sich das Intergral noch von selbst auf.

Das Hauptproblem ist, dass es sich beim CKI um ein Parameterintegral handelt. Das heißt, es ist nicht nur abhängig von der Integriervariable $x_k$, sondern auch noch von anderen “freien” Variablen. In diesem Fall $x_{k+1}$.

Beobachtungen einfließen lassen

Gegeben einer Beobachtung $\hat{y}$ müssen wir uns entscheiden, was für eine Art von Schätzer wir haben wollen:

Maximum-Likelihood: $$arg_x max f(\hat{y}	x)$$
Maximum-A-Posteriori: $$arg_x max f(x	\hat{y})$$

Angenommen wir haben das folgende generative Modell: $y = x^2 + v$ Dann erhalten wir $f(y|x,v) = \delta(y-x^2-v)$

Interessieren wir uns für $f(x

\hat{y})$:

\[f(x|\hat{y}) = \frac{f(x, \hat{y})}{f(\hat{y})} = \frac{f(\hat{y}|x) f(x)}{f(\hat{y})}\]

Stochastic Information Processing was originally published by Daniel Pollithy at Notes on April 17, 2020.

Nearest Neighbour in DBs

2020-02-22T00:00:00+01:00

This post explains iDistance and R-Trees for k Nearest Neighbour (kNN) matching. Both are using regular database indices to solve the problem.

Problem

Assume you want to use kNN in a live system with many data points.
The data points are rows in your relational database.
Problem: The classic kNN algorithm would have to scan your whole table.
Solution: Build an index.

In the following I am going to explain two possible solutions using indices which are implemented in most database systems.

R-Trees

We have the following point cloud:

Now, depending on the exact tree implementation (for example x-tree), the space get separated into multiple layers of containing rectangles. Every rectangle is a minimum bounding rectangle for either another rectangle or points of a lower layer.

The recursive structure of the rectangles containing rectangles or points forms a tree: the R-tree.

Now let us say that you are here:

And want to find the k=4 nearest points.

We start by calculating the distance to the closest objects in the highest layer. The highest layer only contains rectangles, therefore we calculate the shortest distance to the rectangles.

We traverse the rectangles by ascending distance until a rectangle is more distant than our furthest nearest neighbour. All of the points in a rectangle are scanned sequentially.

The next image shows the best k=4 candidates.

The largest distance within the group of candidates determines whether we can exclude rectangles. The distance is illustrated as a red circle in the next image.

Having finished the right rectangle. We proceed to the left, small rectangle. The nearest neighbours change, because there are closer ones. This results in a red circle with a smaller diameter.

Finished! The yellow rectangle is out of reach. It was not processed but it could neither supply a point close enough to become a nearest neighbour. This “exclusion” is what saves us from doing a whole sequential scan.

The problem with R trees

According to the curse of dimensionality: With increasing dimensionality, the distance between two points grows exponentially.

This results in larger minimum bounding rectangles which tend to span the whole space diagonal. But these rectangles are not a good partitioning of the space anymore. To many intersections between them result in scanning most of them. This can become even worse than the sequential scan, because now we use an index which takes up space, but only supports sequential scans.

iDistance

The main idea of iDistance is to partition the space into spheres. And then reduce the points which have to be scanned by using the distance of the points to their closest sphere center in a B+ tree.

We visualize the algorithm with the following point cloud again. This time we look for k=3.

First of all we have to partition the data. For example using k-Means or DBSCAN.

Every point is only part of one partition (sphere). For every partition we calculate the distance to the center and use this information as an index for the partition. This is where the B+ tree is used. (Note: It is also possible to use a single B+ tree for all partitions. It just necessary to shift the centers with an offset large enough to disable intersections. The space diagonal should be enough.)

It is time for a new query at the same position:

Opposed to the procedure of the R trees which shrank the radius of the red circle, iDistance starts with a small radius and increases it incrementally. These are two hyperparameters: initial_radius and radius_growth_factor.

The initial_radius was too small. We found no partition. As long as we don’t have enough points, we increase the initial_radius by multiplying it with the radius_growth_factor.

We find three intersections with three partitions. Here comes the “trick”. For every partition we can calculate the ring which intersects with the red query circle. Only the points in the ring can be in the intersection. The rest of the partition does not have to be scanned (marked with black). This saves us time.

In the case of our example for k=3 we would already be done:

Attention: All points in the ring have to be scanned.

In the case of the left, small circle, another speedup comes in handy, too. The B+ tree of this partition allows us to early abort the search. We can order the points in descending distance from the center. This way we start with the closest point first. Fortunately it is also closest to our point and we can stop the search. Note however, that it does not have to be that way. The point with the largest distance to the center of this partition could also have been on the left side of the center.

iDistance is not a mitigation to the problem introduced by the curse of dimensionality but it can restrict the search space within a partition. This leads to a better behaviour compared to the R tree.

Nearest Neighbour in DBs was originally published by Daniel Pollithy at Notes on February 22, 2020.

Partitioned Normal Form

2020-02-21T00:00:00+01:00

What is the Partitioned Normal Form (PNF) in the context of databases?

NF²

NF² = NFNF = ¬NF = Non-First-Normal-Form.

The First Normal Form (1NF) is too restrictive. NF² allows attributes themselves to be relations. Attribute values can be sets.

A	D
A	B	C
1	2	3
	4	2
2	1	1
	4	1
3	1	1

(Table 1: NF² and PNF)

The construction of an algebra which has the same expressive strength as relational algebra is more possible, but more complicated and it employs the operators $nest_{...}{(Relation)}$ and $unnest_{...}{(Relation)}$. Each of them is generally not commuative.

PNF

PNF is a subset of NF² with nicer properties. The construction of the algebra is easier.

The special properties are:

In the partitioned normal form the nest operator is commutative!
The PNF can be unnested into a 1NF.
On every nesting layer there exists a flat key. (In table 1 the key is the attribute A)

The table above is also in PNF.

More formally expressed:

Recursive definition of PNF:

A relation R has a set of unnested (atomic) attributes A. A is not the empty set.
The relation can also have nested (composite) attributes X. X can be the empty set.
There exists a functional dependency: A -> X
for every tuple in R:
- for every complex attribute x in X:
  - value of the complex attribute t.x is in PNF again.

The last point means, that we can construct a PNF from PNFs.

NF² but not PNF?

Constructing a relation, which is not in PNF but in NF², can be achieved by breaking the recursive construction rules. The easiest is the “flat key” rule. If we change the values of the column A from (1,2,3) to (1,1,2) then we removed the total functional dependency between A and D.

A -> D is now an incorrect statement, therefore the relation is not in PNF anymore. Due to the fact that we have not changed anything about the structure of the relation, it is still in NF².

A	D
A	B	C
1	2	3
	4	2
1	1	3
	4	1
2	1	1

(Table 2)

Partitioned Normal Form was originally published by Daniel Pollithy at Notes on February 21, 2020.

Last timestep in masked sequence

2020-01-26T00:00:00+01:00

Motivation

This is just a short piece of code which I have needed multiple times when training a LSTM with masked inputs.

Setup: Seq2seq training with additional loss

Problem: How to get the last time step which is not masked

Why: Calculating loss only at this timestep

Example data

The input batches are masked like with 0.0:

input = [ # mini-batch
	[ # 1st time series
    	[0.1, 0.00],
        [0.2, 0.01],
        [0.3, 0.01], # <--- we want this timestep
        [0.0, 0.0 ],
        [0.0, 0.0 ]
    ], 
	[ # 2nd time series
    	[0.5, 0.32], # <--- we want this timestep
        [0.0, 0.0 ],
        [0.0, 0.0 ],
        [0.0, 0.0 ],
        [0.0, 0.0 ]
    ], 
    ...
]

The labels for the seq2seq training are the inputs but shifted one time step.

Solution

masking_value = 0.
timesteps = inp.shape[1]
point_dim = inp.shape[-1]  # example: is 2 for (x,y) points 

# 1. Find entries which are equal to the masking_value 
equal_zero = inp == K.cast([masking_value], tf.float64)

# 2. Count the zeros in the row
zeros_per_row = K.sum(equal_zero, axis=-1)

# 3. Find the rows which only contain masking_values
correct_rows = K.cast(zeros_per_row == [point_dim], tf.float64)

# (The following code is argmin which is forced to return the first occurence)

# 4. Set the wrong rows to the maximum value (-> not selected by argmin)
#        (timesteps**2 is an upper bound for step 5)
wrong_rows = (-1 * correct_rows + 1) * (timesteps**2)

# 5. Build enumerations for every track
range_np = np.array(list(range(timesteps)))  # [0,1,2,3...]
range_np_batch = np.tile(range_np, (batch_size, 1)).reshape([batch_size, timesteps])
range_keras = K.variable(range_np_batch)
# [
#   [0,1,2,3...],
#   [0,1,2,3...],
#   ...
# ]

# 6. Enumerate the correct values ascending + add the infinities of step 4
search_space = correct_rows * range_keras + wrong_rows
last_timesteps = K.argmin(search_space, axis=1) - 1

# 7. Build a mask for selection
mask_last_step = tf.one_hot(last_timesteps, depth=timesteps, dtype=tf.bool, on_value=True, off_value=False)
mask_last_step = K.cast(mask_last_step, tf.float64)

# 8. Use the mask for loss calc
loss_per_track = tf.keras.losses.mean_squared_error(y_label, y_hat) * mask_last_step
loss = K.sum(loss_per_track)

Why not only argmin?

The tensorflow documentation for tf.math.argmin (doc) states:

“Note that in case of ties the identity of the return value is not guaranteed.”

Notes

range_np, range_np_batch, range_keras should only be created once.
Insted of using the ultimate mask a solution with tf.gather_scatter_nd is possible
The sum in step 2 could be replaced by K.all(K.equal(inp, mask_value), axis=-1)

Last timestep in masked sequence was originally published by Daniel Pollithy at Notes on January 26, 2020.

Generating Behavior for Vehicles

2019-11-06T00:00:00+01:00

This post is a loose collection of JavaScript demos I made while studying for the course “Generating Behavior for Vehicles”.

The following two gifs show you how two of the demos work:

**Planning a path through a city map with A star **

A drifting car

Kinematic One Track Model

A car is controlled by its steering angle $\delta$ and its acceleration. If we want a vehicle to perform any kind of maneuver then we have to relate the input commands (steering and acceleration) with future positions of the car.

For this purpose we use dead reckoning. In the following example we assume to have a car with constant velocity. You can control its steering angle with the slider below. It starts to drive once you change the steering angle a little bit.

Steering angle:

The model is called kinematic because we only model the geometry and not the dynamics. This will be our next step. It is called One-Track because we assume that the two wheels on both axis can be modelled by averaged wheels in the center of both axis.

The simple differential equations are:

\[\dot{x} = v \cdot \frac{\cos(\delta)}{l}\] \[\dot{y} = v \cdot \frac{\sin(\delta)}{l}\] \[\dot{\theta} = v \cdot \frac{\tan(\delta)}{l}\]

Assuming constant inputs we use the Forward Euler to calculate the new state:

\[x = x + \Delta x\] \[y = y + \Delta y\] \[\theta = \theta + \Delta \theta\]

This model is in general unrealistic because it assumes that the car can change its direction instantaneously. But it works good enough as long as the velocity of the car is low. Therefore we could use it to build a parking assistant.

Linearized Dynamic One Track Model

Now we consider a car which is driving with a regular speed like 50 km/h. In this model we accept that there can be a difference between the steering angle and the real angle in which direction the mass of the vehicle moves. Usually we want this discrepancy to be as low as possible because it is difficult to control the car then. Think of a drifting car.

In the following demo you can control the steering angle again (velocity is constant). Because it is easy to loose control and leave the screen, the car will come back into the screen if that happens.

Steering angle:

This linearized dynamic one track model can be used in ESP systems to estimate the yaw rate of the car. If it is higher than the driver might expect it then it is becoming dangerous because the control over the car can be lost easily. Starting from this the ESP regulator can break tires or accelerate others to reduce the yaw rate to keep the car controllable.

ToDo: Add equations for dynamic one track model and show how to linearize

Planning

We want our car to perform a maneuver. But what should it actually do? Let’s think about the scenario when a car wants to drive from one point in a city to another point. We assume that we have an HD (high definition) city map and we want to plan an optimal path.

First we should discretize the world to make the planning feasible and assume that the robot is a point:

building a Voronoi graph around the obstacles
polygon fitting into the free room
Growing a tree structure into the space (Quad/Octree, something like Rapidly Exploring Random Trees, although they already search)

Then we convert the problem into a graph search with Dijkstra or A. This results in a trajectory. In the following demo every pixel is a node in the graph. You can click on the left canvas to draw obstacles (black). The robot starts in the left upper position and wants to drive to the right lower corner. It will search for the shortest path with A. All the expanded pixels are colored in blue.

Left canvas: Map input. Right canvas: A* visualization

Optimal control

Given the trajectory from the last step, how do we get the steering angle and acceleration necessary to drive it?

One option would have been to plan in a Markov Decision Process. Instead of assuming a point robot we could frame it as follows:

Our car is in an initial state (steering angle, acceleration, position). It can take actions (steer and accelerate) which leads to different states. The different actions (lets assume they are discrete) lead to new states which we calculate using the motion model from before.

The different action values determine the branching factor of our state tree. Using the Bellman optimality equation we could iterate backwards starting at the goal in order to determine the “goodness” of each state at different time steps. Once the “goodness” has converged (the order of the values doesn’t change anymore or the values of the values either) we can greedily search through the graph in order to maximize the “goodness” at every time step.

This is called value iteration. Actually Dijkstra and A* are also based on the Bellman Recursion but the time index is replaced by the sorting index of the next expansion candidate. Dijkstra sorts by cost-so-far and A* by cost-so-far plus cost-to-go (a heuristic which has to be admissible, which means that it underestimates the real cost-to-go).

The problem with this discrete approach is that it would happily jump between accelerations and steering angles although in reality these values are continuous.

No matter where our target trajectory comes from and the controls for that we somehow have to “smoothen” them. We want to optimize them. Imagine the trajectory makes an abrupt corner. To directly follow it we would have to stop at the corner turn the wheels and accelerate again. We note that there is a trade off between following the trajectory and following the control inputs.

Finding the optimal solution depends on what we want. We can formulate a cost functional (a functional is a function of functions) which penalizes both deviations of the trajectory and the input controls. According to our needs we add weightings to the parts to balance it or stress that we have to reach the target.

The cost functional look as follows:

\[J = x^T(k_f) Q_f x(k_f) + \sum_{k=1}^{k_{f}-1} x^T(k) Q_f x(k) + \sum_{k=0}^{k_{f}-1} u^T(k) R_f u(k)\]

$Q_f$ is the weighting of the final state deviation. $Q$ is the weighting of the state deviation. $R$ is the weighting of the control input deviation.

We can solve this problem by applying the Bellman Recursion Formula backwards. But this time the amount of control values and state values is infinite because we are in a continuous domain. The cost functional between two time steps is a quadratic equation dependent on a weighting matrix P. We differentiate and solve for P which gives us the optimal P(k) for the time step k. We repeat this process until we reach the beginning. The formula to calculate P(k) is a Riccati difference equation which can be solved analytically.

Using P(k) we can calculate the optimal Gain matrix K* which we use to calculate the optimal control input u(k) = -K(k) x(k) for every time step.

If we don’t have a fixed target at time $k_f$ and our system is controllable and the system is time invariant (really?) then we can find an optimal P(k) which is optimal for all time steps. We call is $P_{\infty}$ and obtain it by solving the riccati differences equation until convergence. This leads to a time independent optimal control u* = -K* x(t) which is the LQR (linear quadratic regulator). As opposed to the classical creation of a regulator by determining eigenvalues, the advantages of the LQR are intuitive weighting with Q/R and no additional unwanted degrees of freedom.

ToDo: Clean this up and add the formulas.

Unfortunately I don’t have JavaScript example for this because I could not find a nice matrix package which helps me, therefore I will show some python code.

import numpy as np
import matplotlib.pyplot as plt


def build_batch_matrices(A, B, k_f):
   # Batch Matrices
   A_batch = np.vstack( [ np.linalg.matrix_power(A, n) for n in range(0, k_f) ] )

   B_batch = []

   for k in range(1, k_f+1):
      # build row
      row = []
      for m in range(0, k-1):
         row.append(np.linalg.matrix_power(A, k-m-1) @ B)
      row.append(B)

      while len(row) < k_f:
         row.append(np.zeros(B.shape))

      B_batch.append(np.hstack(row))

   B_batch = np.vstack(B_batch)

   return A_batch, B_batch


def build_batch_cost(Q, R, k_f):
   Q_batch = []
   R_batch = []

   for k in range(0, k_f):
      # build row
      row_q = []
      row_r = []

      for k2 in range(0, k_f):
         if k == k2:
            row_q.append(Q)
            row_r.append(R)
         else:
            row_q.append(np.zeros(Q.shape))
            row_r.append(np.zeros(R.shape))

      Q_batch.append(np.hstack(row_q))
      R_batch.append(np.hstack(row_r))

   Q_batch = np.vstack(Q_batch)
   R_batch = np.vstack(R_batch)

   return Q_batch, R_batch


def build_H_F(A_batch, B_batch, Q_batch, R_batch, k_f):
   H_batch = B_batch.T @ Q_batch @ B_batch + R_batch
   F_batch = A_batch.T @ Q_batch @ B_batch

   return H_batch, F_batch

# how many time steps
k_f = 100

# target trajectory
x = np.arange(0, 100, 1)
y = np.zeros(x.shape)
# different values for y are more interesting
# y = np.sin(x)

x_traj = np.stack([x,y]).T
kappa = np.zeros(x.shape)

A = np.array(
   [
      [1, 1],
      [0, 1]
   ]
)

B = np.array(
   [
      [0, 0],
      [0, 1]
   ]
)

# x(k+1) = A x(k) + B u(k)
# u(k) = [u, u]


# Riccati

# Q_f = P(k_f)
Q_f = np.array(
   [
      [100, 0],
      [0, 1]
   ]
)

#  Q
Q = np.array(
   [
      [10, 0],
      [0, 1]
   ]
)

#  R
R = np.array(
   [
      [1, 0],
      [0, 1]
   ]
)

Ps = [Q_f]
Ks = []

# Loese Riccati-Differenzengleichung rückwärts
for k in reversed(range(0, k_f-1)):
   P_kp1 = Ps[-1]
   P_k = Q + A.T @ P_kp1 @ A - A.T @ P_kp1 @ B.T @ np.linalg.pinv( B.T @ P_kp1 @ B.T + R) @ B.T @ P_kp1 @ A.T
   K_k = np.linalg.pinv( B.T @ P_kp1 @ B.T + R) @ B.T @ P_kp1 @ A.T
   Ps.append(P_k)
   Ks.append(K_k)

Ps = Ps[::-1]
Ks = Ks[::-1]

# Optimale Stellgroesse u(k) = -K(k) * x(k)

# state space
x_0 = np.array([
   3, # d
   0, # delta_psi
])

X = [x_0]

# euclidean
X_world = [(0, 3)]

# -------------------------
# The following lines contain a naive regulator which does not work as good
# as the riccati regulator
us = []

for k in range(0, k_f-1):
   u_k = -Ks[k] @ X[k]

   u_k[1] = max(-np.pi/2.0, min(u_k[1], np.pi/2.0))

   us.append(u_k)


   # next time step
   x_p = A @ X[k] + B @ u_k
   # normalize delta_psi
   # x_p[1] = ((x_p[1] + np.pi ) % 2*np.pi) - np.pi   

   # print(x_traj[k].copy())
   x_world = x_traj[k].copy()
   x_world[0] += 0
   x_world[1] += x_p[0]
   X_world.append(x_world)

   x_n = x_world - x_traj[k]
   # x_n += np.random.normal(0, 0.01) # add some randomness

   X.append(x_p)

# end of suboptimal regulator
# ----------------------------


A_batch, B_batch = build_batch_matrices(A, B, k_f-1)


us = np.hstack(us).T

x_batch_traj = A_batch @ x_0 + B_batch @ us

# u.T H u + 2 x_0.T F u

Q_batch, R_batch = build_batch_cost(Q, R, k_f-1)
H_batch, F_batch = build_H_F(A_batch, B_batch, Q_batch, R_batch, k_f-1)

u_opt = -np.linalg.inv(H_batch) @ F_batch.T @ x_0
x_opt_traj = A_batch @ x_0 + B_batch @ u_opt


plt.plot(x_batch_traj.reshape([-1, 2])[:, 0], label="Suboptimal", color="red")
plt.plot(x_opt_traj.reshape([-1, 2])[:, 0], label="Optimal", color="blue")

plt.legend()
plt.show()

Geometric modeling of a trailer

The following simulation models a car with an attached trailer. This is a differential equation grounded on geometric considerations.

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as patches

timesteps = 500

# variables
delta = 0.1 # steering angle

time = 0


# constants
car_length = 4
car_width = 2
trailer_length = 2
distance_car_trailer_axis = 15

x_start = 0

plot_dt = 30


# differential equations
def forward_euler(psi, v, gamma, delta_t=0.1):
   psi_dot = v * np.tan(delta) * (1.0/car_length)
   v_dot = 0.0
   gamma_dot = v * np.tan(psi-gamma) * (1.0/distance_car_trailer_axis)

   return [psi + delta_t*psi_dot,  v + delta_t*v_dot, gamma + delta_t*gamma_dot ]


def calc_pos(state, last_pos):
   psi = state[0]
   v = state[1]
   x = last_pos[0]
   y = last_pos[1]

   x_new = x + np.cos(psi) * v
   y_new = y + np.sin(psi) * v

   return [x_new, y_new]


def calc_trailer_pos(state, last_pos):
   gamma = state[2]
   x = last_pos[0]
   y = last_pos[1]

   return [x - np.cos(gamma) * distance_car_trailer_axis, y - np.sin(gamma) * distance_car_trailer_axis ]

state_traj = [
   # psi, v, gamma
   [0.0, 1.0, 0.0]
]

pos_traj = [
   # x, y
   [x_start, 0],
]

trailer_traj = [
   [x_start - distance_car_trailer_axis, 0]
]

for t in range(timesteps):
   state_traj.append(forward_euler(*state_traj[-1]))
   pos_traj.append(calc_pos(state_traj[-1], pos_traj[-1]))
   trailer_traj.append(calc_trailer_pos(state_traj[-1], pos_traj[-1]))


pos_traj = np.array(pos_traj)
trailer_traj = np.array(trailer_traj)

fig,ax = plt.subplots(1)

for i, row in enumerate(pos_traj):
   if i%plot_dt != 0: continue
   x_vec = [trailer_traj[i][0], pos_traj[i][0]]
   y_vec = [trailer_traj[i][1], pos_traj[i][1]]

   car_x = pos_traj[i, 0]
   car_y = pos_traj[i, 1]
   psi = state_traj[i][0]

   # deichsel
   ax.plot(x_vec, y_vec, 'k-')

   # car
   ax.plot([car_x, car_x + np.cos(psi) * car_length], [car_y, car_y + np.sin(psi) * car_length],  color="red", linewidth=10)

   # trailer
   ax.scatter(trailer_traj[i, 0], trailer_traj[i, 1], label="trailer", color="blue")

print(state_traj[-1])

plt.axis("equal")
# plt.legend()
plt.grid(True)

plt.show()

Generating Behavior for Vehicles was originally published by Daniel Pollithy at Notes on November 06, 2019.

Localization of mobile agents

2019-09-23T00:00:00+02:00

In this post I am going to try to explain the basic concepts of localization of mobile agents which I have learned in the lecture “Lokalisierung mobiler Agenten” at the KIT.

The main outline:

Wheel and vehicles (one lane kinematic model and differential model)
Systems and discretization
Euler method
Dead reckoning
Example: Error accumulating with constant velocity model
Static localization (measurements, objective function and least squares)
Angular measurements (atan2)
Distance measurements (linearizing)
Example: GPS
Recursive Least Squares
Bayesian Filtering Framework (Linear Gaussian Model)
Dynamic localization (fusioning principles: BLUE, naive fusion, recursive fusion)
Kalman Filter (derivation, optimality)
Example: Tracking Car position and velocity
Nonlinear Filtering (EKF, UKF, Particle Filter)
EKF
Wonham Filter
Bingham Quaternion Filter
SLAM
Online SLAM (EKF-SLAM)
Full SLAM (Graph-SLAM)
Example: EKF-SLAM for 3d pose
FAQ

Localization of Mobile Agents

Wheel and vehicles

As we don’t want to spend too much time on modeling forces and we assume that our mobile agents drive with moderate velocity on wheels we use the idealized wheel.

It has a radius r, a wheel velocity $\omega$,
no slip in lateral direction
no slip in longitudinal direction.
It has point contact to the ground and
only rolls into the direction of the steering angle
it has no roll resistance.

The Instantaneous Center of Rotation (ICR, “Momentanpool”) is the point where the wheel axis intersect. There are four different cases with our model:

The axes meet at a point inside our outside of the vehicle and move on a circular path around the ICR.
The wheel axes are on the same line. This is the case for the differential drive. The position of the ICR is controlled by the difference of the wheel velocities. If their difference is zero then ICR lays between the wheels.
If wheel axes are parallel then the ICR is located at infinite distance. The vehicle is driving forward without a curved path.
The wheel axes do not intersect at a single point. Within our model the vehicle is unable to move. Other models can handle this but they are a little bit more complex. For example: The dynamic one track model which models the slip angles of the wheels.

The magnitude form of the euler velocity equation (“Betragsform der eulerschen Geschwindigkeitsgleichung”) is used to set the angular velocity and the vehicle velocity into relation.

\[|\omega| = \frac{||v||}{||p||}\]

where $\omega$ is the angular velocity, v the vehicle velocity at a point at the vehicle and p the line from the ICR to the point.

For two different points p1 and p2 on the vehicle with different velocities v1 and v2, we can also use:

\[|\omega| = \frac{||v2 - v1 ||}{||p2 - p1||}\]

Kinematic one track model

Differential drive

Note: If $v_L + v_R = 0$ the ICR is at the center of the axis between the two wheels.

Systems

Let $x$ be the state of the system, $u$ the controllable inputs and $w$ the uncontrollable changes to the state (like noise). The system equation $a(x, \dot{x}, u, w) = 0$ holds. A simple specialization of this equation which we are going to use: $\dot{x} = a(x, u, w)$

If we manage to design $a(...)$ in form a linear function then we can write our system in the following form and call it a Linear System:

\[\dot{x}(k) = A_k x(k) + B_k u(k) + w_k\] \[y(k) = C_k x(k) + D_k u(k)\]

In the case that the system’s behavior does not change over time, we call it a Linear Time-invariant System (LTI):

\[\dot{x}(k) = A x(k) + B u(k) + w\] \[y(k) = C x(k) + D u(k)\]

In order to access the state of the system at a point in time $t$ we can use the the solution for the time-continuous linear time-invariant system:

\[x(t) = e^{A (t - t_0)} x_0 + \int_{t_0}^{t}{e^{A (t-\tau)} B u(\tau) d\tau }\]

Discretization

Computers generally do not know how to solve time-continuous differential equations. The main problem is “how to solve the integral”? It is also difficult for humans if the antiderivative cannot be calculated.

For the use in digital computers we have to discretize the with respect to time, only then can we use it together with the discrete state sampling rate. So we introduce discrete timesteps $t_k = k \cdot T_s, k \in \mathbf{Z}^+$.

Now we assume that the input signal is constant between two timesteps:

\[x(t_{k+1}) = e^{A T_s} x(t_k) + \int_{0}^{T_s}{e^{A (T_s - \tau)} B d\tau \cdot u(t_k) }\]

Because the system is time-invariant we see that everything expect the x(t) and u(t) is constant, which leads us to the following discrete form:

\[x(t_{k+1}) = A' x(t_k) \cdot B' u(t_k)\]

So we still have an integral here. How do we get A’ and B’?

With the use of the Laplace transformation if possible or numerical approximations.

Euler method

Given a differential equation which describes the system dynamics we want to calculate the state of the system after time t.

\[x(t) = x_0 + \int_0^t{\dot{x}(\tau) d\tau}\]

We use the Euler Method. It tries to fit small rectangles under the curve of which we want to calculate the integral. If we use the height of the function at timestep $t_k$ as the height of the rectangle we call it Forward/explicit euler. If the we use the height at $x_{k+1}$ then it’s called Backward euler.

The backward euler is a little bit extra effort because you have to solve an equation to get to the next state but it has the advantage of being stable (solution can be found with the Newton method).

Instead of using the quotient of differentials we have to use the difference quotient.

\[\frac{d x(t)}{dt}\approx \frac{x(t + T_s) - x(t)}{T_s} = \frac{x_{k+1} - x_k}{T_s}\]

In Matlab a continuous system can be transformed numerically with the command c2d: sys_disc = c2d(sys_cont,Ts)

Until rounding errors happen in the computer, a smaller $T_s$ leads to a better approximation.

Solution 3

Constant acceleration or constant velocity models.

Dead reckoning

Definition:

The task of dead reckoning is to get the pose of an object with respect to an external coordinate system.

To solve this task formulate a motion model in the object’s coordinate system.

Advantages:

High sampling rate
No infrastructure needed
low latency

Disadvantages:

Pose is only relative
Accumulated error which leads to drift

How it works: We measure increments from the last state (change of place, velocity or acceleration). And then we use this measured information as system input to our discrete system model in order to calculate where our current position should be next.

“Schritthaltend”: Abtastrate höher als Rechenzeit für neuen Zustand.

Error

The total error of a linear system depends on all noise vectors:

\[e_k = x_k - x_{k}^{\text{ground truth}} = e_k (w_{k-1}, ..., w_0)\]

For linear systems: The longer ago the error has been added, the more influence it has, because the exponentiated system matrix usually becomes larger over time:

\[e_k = A^{k-1}\cdot w_0 + A^{k-2}\cdot w_1 + ... + A\cdot w_{k-1}\]

Important Note: When talking about the error it is important to realize that the error is not the difference between the estimate and the measurement but it is the difference between the ground truth and the estimate!

(The ground truth could be manually annotated or retrieved with another system, for example with a VICON or GPS)

Static localization

Sensors measure absolute pose in world coordinates.

Advantage

No increasing error (“keine Fehlerfortpflanzung”)

Disadvantage

External infrastructure necessary

Measurement principles

Time-of-Flight
Radar
visual (cameras)
strength of a signal (WLAN, Bluetooth, GSM)

The measurement function z = h(x) generates a measurement from the true pose of the robot.

\[z \in R^M , x \in R^N, M >= N\]

Example 1: Trilateration

You want to localize the robot by measuring the distance to point landmarks.

The ith measurement is the distance to the point i (with coordinates p_x and p_y). The robot’s position is x_x, x_y.

\[z_i = \sqrt{ (x_x - p_x)^2 + (x_y - p_y)^2 } = || p - x ||\]

ToDo: Linearize!

The vector z contains the measurements to all the landmarks: $z = [z_1, ..., z_M]^T$ And we assume that the measurement function h(x) produced this vector $z = [h_1(x), ..., h_M(x)]$.

This leads us to a non-linear least squares (LSQ) problem with the objective function and the weighting matrix W (pos. definite):

\[arg_x min G(x) = ||z - h(x)||^2_W = (z - h(x))^T W^{-1} (z - h(x))\]

The weighting matrix allows us to express our knowledge about the measurements. Whether we want to trust the measurement of one sensor more than others.

Once we solve this optimization problem we will localize the robot. If we cannot write the measurement function as a linear equation then we could do this numerically.

If we can write the function as a linear equation then we can solve the problem analytically by deriving it, setting the derivation to zero and solving for x. (If we do not knew the orientation of the parabola then we would also have to check whether the optimum is a minimum or a maximum by checking the sign of the second derivation).

Example 2: Trilateration with walls

Let’s go back to the example and see whether we can reformulate the measurement function. Let’s assume that the landmarks are points on walls of a room.

In this case we can model the walls with Hesse’sche Normalform $n_i * x + d_i = 0$ with $||n_i||^2 = 1$ and $d_i$ the distance to the wall.

The distance to the ith wall is $s_i(x) = h_i * x + d_i$

\[s_i(x) - d_i = h_i * x\] \[z_i = h_i * x\] \[z_i = [h_{i_x}, h_{i_y}] * [x y]^T\]

Which is a linear equation.

In the easiest case where we have exactly as many observations as state variables we obtain a quadratic measurement matrix:

\[[z_0, z_1] = [h_{0_x}, h_{0_y}; h_{1_x}, h_{1_y}] * [x y]^T\] \[z = H x\]

=> $x = H^{-1} z$

We see that it would be convenient to always use a quadratic measurement matrix H but it has the downside that we would not use all of our measurements if we have more walls in the room and therefore we would be more error prone if one of the sensors returns a noisy value.

Least Squares

That’s why we come back to the LSQ which minimizes the sum of the squared errors between our measurements and the estimated state.

\[arg_x min G(x) = ||z - H x||^2_W = (z - Hx)^T W^{-1} (z - Hx)\]

Multiplying the terms:

\[G(x) = (z - Hx)^T W^{-1} (z - Hx) = z\] \[z^T W^{-1} z - x^T H^T W^{-1} z - z^T W^{-1}Hx + x^T H^T W^{-1} H x\]

Derive:

\[\frac{d }{d x} G(x) = - H^T W^{-1} z - z^T W^{-1}H + 2 H^T W^{-1} H x\]

Set to zero:

\[-2 H^T W^{-1} z + 2 H^T W^{-1} H x = 0\] \[+ 2 H^T W^{-1} H x = +2 H^T W^{-1} z\] \[H^T W^{-1} H x = H^T W^{-1} z\] \[x = (H^T W^{-1} H)^{-1} H^T W^{-1} z\]

Checking the second derivative:

\[\frac{d }{d x} -2 H^T W^{-1} z + 2 H^T W^{-1} H x z\] \[2 H^T W^{-1} H z\]

Check whether it is positive definite:

\[2 H^T W^{-1} H z > 0\]

Important: H has to have at least full column rank otherwise the first bracket could not be inverted. This means that the columns have to be linearly independent of each others. Full column rank is the same as saying that rank(H) = dim(x) which enough to make rank(H) >= dim(x) true.

Note: If the weight matrix W is only semi-definite then the matrix $(H^T W^{-1} H)$ then the inversion is not uniquely determined (infinite solutions). We can still choose one by picking the with the smallest norm ||x||^2.

How to choose W? Diagonal matrix, positive definite. A large entry on the diagonal like 999 becomes 1/999 after inverting the matrix. And the multiplication with such a small value nearly cancels its influence onto $\hat{x}$. Therefore: Large values in W result in mistrust for the related sensor value.

Example 3: Triangulation

If we can’t measure the distance to known landmark there might be the possibility to measure the angle.

The measurement equation:

\[\alpha_i = atan2(y_{LM}^i - y, x_{LM}^i - x) - \psi\]

is not linear.

A quick reminder. The tan(x) looks like this:

Which is undefined at 90 degree because it is sin(90) / cos(90) and cos(90) is zero.

The atan2 is defined as the angle between the x axis in the range between ]-pi, pi]:

It resolves the ambiguity that -90 degree is the same as 270 degree.

Recursive Least Squares

The normal least squares method has the downside of processing all measurements in one matrix. In reality the measurements are obtained incrementally and we do not want to repeat the processing of the measurement matrix again and again. Especially when it is becoming larger and larger.

The recursive least squares fusions the last result with the newly obtained measurement. The measurement matrix’s size stays constant and therefore the processing time.

We are now going to work with batch matrices. They are matrices whose entries are matrices themselves.

It is important that the block weight matrix W is a diagonal matrix. If that is not the case then we have to diagonalize it: $W = V^T D V$.

Note: $C_k$ can be calculated independently if H and W are independent of the measurements. With this in mind we see that with $k to \infty: C_k$ ~= 0

We obtain recursive equations which we can solve for every timestep in constant time. But how do we start the recursion?

There are two options:

Wait until we have N measurements where N = dim(x). Then H is quadratic so we easily invert it directly. And we calculate $C_k = (H_{1:K}^T W_{1:K}^{-1} H_{1:K})^{-1}$
We initialize x with the zero vector and set its uncertainty to infinite by defining $C_k = \lambda \cdot I$ where I is the identity matrix and $\lambda$ a large positive number.

Information form

With the relationship: $(I - K_k H_{k+1}) = C_{k+1} C_{k}^{-1}$

we can transform the prediction:

\[x_{k+1} = (I - K_k H_{k+1}) x_k + K_k z_{k+1}\]

… to:

\[x_{k+1} = C_{k+1} C_k^{-1} x_k + C_{k+1} H_{k+1}^T W_{k+1}^{-1} z_{k+1}\]

Wikipedia states: In the information filter, or inverse covariance filter, the estimated covariance and estimated state are replaced by the information matrix Y and information vector y respectively.

We multiply $C_{k+1}^{-1}$ on both sides left:

\[C_{k+1}^{-1} x_{k+1} = C_k^{-1} x_k + H_{k+1}^T W_{k+1}^{-1} z_{k+1}\] \[y_{k+1} = y_{k} + H_{k+1}^T W_{k+1}^{-1} z_{k+1}\]

And the Inverse Covariance matrix is calculated like this:

\[C_{k+1}^{-1} = C_{k} + H_{k+1}^T W_{k+1}^{-1} H_{k+1}\] \[Y_{k+1} = Y_{k} + H_{k+1}^T W_{k+1}^{-1} H_{k+1}\]

But why?

In this inverse formulation of the covariance matrix we can define infinite mistrust with a simple zero
As long as we do not need any steps in between, we can calculate the information matrix and information vector separated from each others
Biggest advantage according to Wikipedia: “The main advantage of the information filter is that N measurements can be filtered at each timestep simply by summing their information matrices and vectors.” (“Kovarianz wird additiv”)

Bayesian Filtering Framework

We are going to approach the localization from a probabilistic context. We treat the state $x$ as a probability density function over the state space with its expectation value as the most likely position and the covariance matrix of the pdf as the measure of uncertainty.

This is equivalent to hidden markov models.

Estimate the current state after all the measurements of the past:

\[P(x_k|z_{1:k}) =^{\text{Bayes}}= \frac{P(z_k|x_k, z_{1:k-1}) \cdot P(x_k|z_{1:k-1})}{P(z_k|z_{1:k-1})}\]

With the Markov assumption:

\[\frac{P(z_k|x_k) \cdot P(x_k|z_{1:k-1})}{P(z_k|z_{1:k-1})}\]

The partition function might not be tractable therefore we continue with a term which is proportional to the last one:

\[P(z_k|x_k) \cdot P(x_k|z_{1:k-1})\]

$P(z_k\|x_k)$ is a probabilistic form of the observation model.

The probability of a state given the past observations is the prediction problem: $P(x_k|z_{1:k-1}) = P(x_{k+1}|z_{1:k})$

We can marginalize this: $P(x_{k+1}|z_{1:k}) = \int_{x_k}{P(x_{k+1}|x_k, z_{1:k}) \cdot P(x_{k}|z_{1:k})}$

If we assume time invariance then we can write:

\[P(x_{k+1}|z_{1:k}) = \int_{x_k}{P(x_{k+1}|x_k) \cdot P(x_{k}|z_{1:k})} dx_k\]

$P(x_{k+1}\|x_k)$ is the probabilistic version of the system model.

And $P(x_{k}\|z_{1:k})$ is the question where we started but one timestep earlier. So we see that we can solve this problem with recursion.

The main problem with this is the integral after the marginalization. A restriction is necessary to solve it.

Linear Gaussian Model

“A linear-Gaussian model is a Bayes net where all the variables are Gaussian, and each variable’s mean is linear in the values of its parents. They are widely used because they support efficient inference. Linear dynamical systems are an important special case. “ - metacedemy

\[x_{k+1} = A_k x(k) + B_k u(k) + w_k\] \[z_k = H_k x(k) + v_k\]

Where:

$A_k$ is the system model
$B_k$ the input matrix
u(k) the controllable input vector
$w_k$ the normally distributed white process noise (-> mean=0, cov=$C_w$)
$H_k$ the measurement matrix
$v_k$ the normally distributed white measurement noise (cov=$C_v$)

As a result: x(k) and z(k) are also random variables with $N(\hat{x}, C_x)$ and measurement uncertainty $C_z$.

Summary: The state and the measurement are random variables. The Bayesian Framework is helpful to see how a filter should calculate the posterior distribution from the prior distribution, measurements, the measurement model and the system model. The mentioned restriction of the motion model leads to the fact that we have only one possible predecessor for a given state. As a result, we do not have to build the integral in the Bayesian Framework anymore.

Filter Step without priori knowledge

“Filterschritt ohne Vorwissen”

We start with the measurement equation of the linear system.

\[z_k = H_k x_k + v_k\]

Attention: dim(z_k) >= dim(x_k) and H full column rank.

A first non-formal way is to use the deterministic Least Squares for the random variable mapping with the unknown weight matrix W:

\[x_k^{LS} = (H_k^T W_k^{-1} H_k)^{-1} H_k^T W_k^{-1} z_k\]

And then determine the first and second moment:

Expectation value: $\hat{x}_k^{LS} = (H_k^T W_k^{-1} H_k)^{-1} H_k^T W_k^{-1} \hat{z}_k$

Covariance matrix: $C_k^{LS} = (H_k^T W_k^{-1} H_k)^{-1} H_k^T W_k^{-1} C_k^z W_k^{-1} H_k (H_k^T W_k^{-1} H_k)^{-1}$

This solution contains the unknown weighting matrix and has no proof of optimality in any way.

Therefore the next step is to derive a better solution with formal restrictions:

Affine estimator (“linear”)
Unbiased
Minimum covariance (“best”)

These requirements form the “Best Linear Unbiased Estimator” (BLUE)

We start with an affine estimator:

\[x^e_k = K_k z_k + c_k\]

We require (2) that the expectation value of the estimator is the same as of the real value (which is unknown) $E[x^e_k] = E[x_k]$

\[E[x^e] = K_k E[z_k] + c_k\]

Now we use the measurement model:

\[E[x^e] = K_k H_k E[x_k] + c_k\]

$c_k$ has to be zero in order to let the estimator be unbiased:

\[E[x^e] = K_k H_k E[x_k]\]

Now we add that the expectation values should be the same:

\[K_k H_k E[x_k] = I E[x_k]\]

This brings us to the following relationship which is not unique.

\[K_k H_k = I\]

This is where our third requirement comes into play: We want the Cov(x_k^e)=C_k^e to be minimal. We can formulate this demand with the inverted observation model and the least squares approach:

\[x_k^e = K_k^{opt} z_k\]

with

\[K_k^opt = (H_k^T (C_k^z)^{-1} H_k)^{-1} H_k^T (C_k^z)^{-1}\]

We calculate the covariance matrix like this:

\[C_k^e = (H_k^T (C_k^z)^{-1} H_k)^{-1}\]

Note: We get the same result for the formal approach and the initial least squares solution if we use the covariance matrix of the measurement as our weighting matrix.

Optimal fusion

Now we have the case that there are two unbiased but uncertain state estimations $x_1$ with covariance $C_1$ and $x_2$ with $C_2$.

Linear estimator

\[x^e = K_1 x_1 + K_2 x_2\]

Unbiased

\[E[x^e] = K_1 E[x_1] + K_2 E[x_2]\]

$x_1$ and $x_2$ are unbiased.

\[= K_1 E[x] + K_2 E[x]\] \[(K_1 + K_2) \cdot E[x] =^{!}= E[x]\] \[K_1 + K_2 = I\]

We can reduce the amount of work by only defining one weighting matrix K and its opposite:

\[K = K_1\] \[K_2 = I - K\]

And plug it into the estimator:

\[x^e = K x_1 + (I - K) x_2\]

The covariance matrix of the estimator becomes:

\[C^e(K) = K^T C_1 K + (I - K)^T C_2 (I - K)\]

As with the filter step without bias, we want the uncertainty of the combination to be minimal. We achieve this by projecting the covariance matrix onto a unit vector, deriving this projection, setting the derivate to zero and then solve for the optimal K.

Projection: $P(K) = e^T C^e(K) e$

Derivative:

\[\frac{d}{dK} P(K) = 2 e e^T (K C_1 + (I - K) C_2)\]

Set to zero:

\[2 e e^T (K C_1 + (I - K) C_2) = 0\] \[K C_1 - (I - K) C_2 = 0\] \[K C_1 - C2 + K C_2 = 0\] \[K C_1 + K C_2 = C_2\] \[K (C_1 + C_2) = C_2\] \[K = C_2 (C_1 - C_2)^{-1}\]

Finally the optimal fusion of two unbiased states:

\[x^e = C_2 (C_1 + C_2)^{-1} x_1 + C_1 (C_1 + C_2)^{-1} x_2\]

With the new Cov:

\[C^e = C_2 (C_1 + C_2)^{-1} C_1\]

Prediction

First of all we rewrite our linear model a little bit. We assume that the input random vector is a sum of a known input vector $\hat{u}_k$ and process noise $w_k$.

\[x_{k+1} = A_k x_k + B_k (\hat{u}_k + w_k)\]

The expectation value of the prediction becomes:

\[x_{k+1}^p = A_k \hat{x}_k + B_k \hat{u}_k\]

Under the assumption that the system noise is not correlated to the old state we obtain the new covariance matrix:

\[C_{k+1}^p = A_k C_k^x A_k^T + B_k C_k^w B_k^T\]

Dynamic Localization

Use the optimal fusion, dead reckoning and prediction to build the naive recursive estimator.

Important: Because we are going to calculate a full state from the given measurement we have to require the $dim(z_k) >= dim(x_k)$

It works as follows:

Get a measurement z
Deduce a state variable from z and C_z
Use the system dynamics to predict the state variable until we have a new measurement
Deduce a new state from the new measurement
Use a BLUE estimator which fusions the predicted state and the deduced state from the new measurement

Kalman filter step

In the Kalman filter step we assume that the dimension of the measurement vector is smaller than the dimension of the state vector and we have priori knowledge.

When a new measurement arrives, the predicted state and the measurement matrix are used to “generate” a predicted measurement that gets compared to the true measurement. The difference between both is called the innovation and it is used to correct the predicted state.

We are looking for a BLUE estimator again:

Linear estimator:

\[x^e = K_1 x^p + K_2 z_k\]

shall be unbiased (the predicted estimation and the z_k are also unbiased):

\[E[x^e] = K_1 E[x^p] + K_2 E[z_k]\] \[E[x] = K_1 E[x] + K_2 H_k E[x]\] \[I = K_1 + K_2 H_k\] \[K = K_1\] \[K_2 = I - K H_k\]

Third requirement: Minimum variance. Again with projection onto a unit vector and minimizing we get:

\[C_k^e = (I - K_k H_k) C_k^p\]

A popular interpretation of the Kalman filter step uses a different formulation of the filter:

\[\hat{x}_k^e = \hat{x}_k^p + K_k (z - H_k \hat{x}_k^p)\]

The difference in the brackets is called “innovation”. And the difference is multiplied by the Kalman Gain Matrix. In block diagrams this can be written as a controller (more specific a “regulator”) that regulates the difference between the measurement and the projected measurement to zero with a proportional factor of K.

Recursive Kalman estimator

The recursive Kalman estimator works the same as the recursive estimator but it uses the Kalman filter step. This means that a new measurement is not transferred into state space instead it gets compared directly to a predicted measurement of the predicted state. The difference is multiplied by the Kalman gain and added to the predicted to state. This way the Kalman filter only stores the mean and covariance of the state estimation and it also does not get more complex with the number of used measurements.

Approximation of general systems

When we are dealing with a nonlinear system and we can’t trick it into linearity then we have to linearize it.

If we don’t do this, then the main problem becomes that we are not mapping from gaussian to gaussian. The following image shows the blue gaussian curve projected by a linear function h(x) and by a nonlinear function h’(x).

We solve this by linearizing the system model $x_{k+1} = a(x, u)$ and the measurement model $z = h(x, v)$. Why? Because all of our equations until now only work with matrices A, B and H.

The first order Taylor polynomial is used:

\[x_{k+1} = a(x, u) \approx a(x, u) + J_{a, x}^x \cdot \Delta x + J_{a, x}^u \cdot \Delta u\]

$J_{a, x}^u$ is the jacobi matrix of function $a$ around the point $x$ containing the partial derivatives only after u.

\[x_{k+1} \approx a(x, u) + J_{A_k} * \Delta x + J_{B_k} * \Delta u\]

This is a linear equation but it still does not look like the system equation that we need for our Kalman filter.

But if we stop modelling the full state of the system and only track the difference between the states ($x_{k+1} \approx x_{k} + \Delta_{x_k}$) we get the form in which we are interested:

\[\Delta x_{k+1} \approx J_{A_k} * \Delta x_k + J_{B_k} * \Delta u_k\]

Note: If the system was time invariant or the development point around which we linearized was an equilibrium point, the Jacobi matrices are also time invariant.

We linearize the measurement model in the same way:

\[\Delta z_k \approx J_{H_k} * \Delta x_k\]

The Extended Kalman Filter (EKF) is the regular Kalman filter using these linearized equations.

The predicted state is the linearization point.

???

ToDo: Mitschrieb Vorlesung 12

???

UKF

The problem with EKF is that large uncertainties or large $\Delta$s are not transformed correctly when the nonlinear function is only linearized at one place.

Therefore the Unscented Kalman Filter uses so called sigma points to represent remarkable points of the input normal distribution, transforms them with the nonlinear function and reestimates a new normal distribution from the new sigma points.

Particle Filter

The procedure of the UKF is already close to the Particle filter. Here we represent the priori distribution with particles instead of a parametric function. Then the particles are transformed by the nonlinear function. The function might not be deterministic, it could add random noise.

The predicted particles are weighted according to a distance measure of a new measurement. For example: Particles which are close to the real measurement get a large weighting.

In order to avoid degeneration of the particles we resample the particles by sampling the same number of particles but not uniformly. The distribution from which we sample is the distribution of the samples.

As a result we obtain many particles near the location where a single particle with large weight had been. And random other particles.

The particle filter is also called Sequential Monte Carlo.

SLAM

Simultaneous Localization and Mapping is a “Chicken and Egg” problem. We don’t know the pose of the robot and neither the positions of the landmarks but we want to create a map and localize the robot within.

Challenges:

Nonlinearities
Data association: which measurement belongs to which landmark
Complexity of big maps
Non-static environment

Approaches for 2d SLAM:

Online SLAM -> Map and last pose. The state contains the last pose and the landmarks: $x^T = [x,y,\phi, LM_1^x, LM_^y, ..., LM_L^x, LM_L^y]$. It has a size of 3 + 2L. The Covariance matrix has the size squared entries.
Full SLAM -> Map and trajectory The state contains 3k + 2L entries.

EKF SLAM

Initialization:

Naive init: Estimate the number of landmarks the robot is going to see and initialize the values with zero and infinite uncertainty
Preferred init: Start with a small state vector and expand it every time a new landmark has been measured.

Whether it is a new landmark or a measurement belongs to an already known landmark is called the data association problem and an easy approach is to use 1-Nearest-Neighbour with the Mahalanobis distance that gets weighted according to the uncertainty of the landmarks. If this distance is above a threshold we see a new landmark.

Prediction:

The landmarks are static accordingly they do not move and no change in position has to be predicted. The robots moves as usual. The dead reckoning then adds uncertainty to the robot pose and consequently we also have to increase the uncertainty between the landmarks and the robot’s pose but the uncertainty between landmarks is not updated.

Attention: For the prediction of the mean we just apply the nonlinear motion model to the robot’s pose: $x_{k+1}^p = a(x_k^e, u_k)$. For the prediction of the covariance matrix we do have to linearize $a(.)$ with the Jacobi matrix J_{A_k}.

\[C_{k+1}^p = J_{A_k}^T C_k^e J_{A_k} + C_k^w\]

$C_k^w$ is an all zeros matrix except for the robots covariance which gets an additive noise term due to the system noise.

Only the first column and first row are updated by the prediction step:

\[C_{k+1}^p = \begin{bmatrix} C_{k+1}^{Robot} & C_{k+1, 1}^{Robot,LM} & \dots & \dots & C_{k+1, L}^{Robot,LM} \\ C_{k+1, 1}^{LM, Robot} & C_{1}^{LM} & C_{1, 2}^{LM, LM} & \dots & C_{1, L}^{LM,LM} \\ C_{k+1, 2}^{LM, Robot} & C_{2, 1}^{LM, LM} & C_{2}^{LM} & \dots & C_{2, L}^{LM,LM} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ C_{k+1, L}^{LM,Robot} & \vdots & \dots & \dots & C_{L}^{LM} \end{bmatrix}\]

Complexity: The prediction step is linear in the number of landmarks.

Filtering

The robot measures an angle $\alpha$ and the distance $d$ to a landmark. Now we have got two different cases:

Landmark has not been seen before
Landmark has already been measured

Unknown Landmark:

If the landmark is new we initialize the position as follows:

\[x_{LM} = g_x(x, z) = x_{Robot} + d \cdot cos(\phi_{Robot} + \alpha)\] \[y_{LM} = g_y(y, z) = y_{Robot} + d \cdot sin(\phi_{Robot} + \alpha)\]

where $\phi_{Robot}$ is the robots orientation. g(…) is the nonlinear function responsible for the mapping. L is the number of landmarks. These two new values are stacked into the state vector.

For the extension of the covariance matrix we have to linearize the function $g$ at the robot position and at the new landmark position with the Jacobian $G_R$ and $G_{L+1}$.

\[C_{k+1}^{LM} = G_R C_k^{Robot} G_R^T + G_{L+1} C_k^{v, L+1} G_{L+1}^T\]

$C_k^{v, L+1}$ is the measurement uncertainty.

With $C_{k+1}^{LM}$ we can initialize the diagonal entry of the extended covariance matrix of the state vector but we also have to add the cross covariance between this landmark and the robot $C_{k+1}^{Robot, LM}$ and between this landmark and the other landmarks i $C_{i, L+1}^{LM}$. We add these values in the row and column of the new landmarks covariance matrix.

The following image shows where the three new covariance matrices are added to the batch covariance matrix of the state:

2nd case: Known Landmark

We have a nonlinear measurement model which uses the estimated position of the landmark and the estimated position of the robot from the state in order to generate a measurement. The landmark and the robots position are extracted from the state using a selection matrix.

\[z_k^i = h(x_k^{Robot}, x_k^{LM i}) + v_k^i\]

This function h(..) gets linearized and the Jacobi matrix $H_k$ is used to calculate the Kalman gain $K_k$.

Complexity: The field of view is usually limited and the robot never observes all of the landmarks that is has in its map. The complexity of the matrix inversion in order to calculate the Kalman gain is cubic in the number of observed landmarks but the number of observed landmarks is « total landmarks in the map.

Update

Now we can update the state estimation and its covariance matrix in the previously described EKF way.

Complexity: The update changes all entries of the covariance matrix. Therefore this is quadratic in the number of landmarks. This operation dominates the EKF.

Note: The correlation between the landmarks and the robot is important and cannot be ignored. Otherwise the uncertainties decrease too fast and the robot would be overconfident about the positions of the landmarks and therefore fail.

The landmarks’ uncertainty decreases monotonically that means that they had the largest uncertainty at initialization.

The lower bound for the uncertainty is the robot’s pose uncertainty at the time it made its first landmark measurement. It can’t be lower that that unless other external information (like GPS) is integrated.

Graph SLAM

This is a full slam approach and I assume that the name comes from graphical model.

FAQ

How can we see that the Kalman filter minimizes the MSE? -> The Kalman Gain is chosen to minimize the Covariance matrix of the estimate. This matrix can be derived by minimizing the trace of of the covariance which is the same as the mean variance. Which is the same as the mean squared derivation of the correct value. Which is the mean squared error.

Localization of mobile agents was originally published by Daniel Pollithy at Notes on September 23, 2019.

Boltzmann Machines

2019-07-11T00:00:00+02:00

This article tries to explain Boltzmann machines, RBMs, DBN, a little bit about energy models and contains a python example which shows how a generative model can be trained with contrastive divergence.

Boltzmann Machines

I have explained Binary Hopfield Networks in my last post. They had three main problems:

Bad efficiency
Spurious states
No possibility of “escaping” the wrong attractor region

Hinton and Sejnowski worked with a probabilistic graphical model which they called Boltzmann machine (prev. Harmonium). The main differences:

Every unit can have a unique bias
The weighted sum that is calculated by each hidden unit has no threshold but a sigmoid activation function (the activation function is interpreted as the probability of the unit being active)
Due to the stochasticity of the process a particle on the energy function can leave a local minimum

Generally speaking: A Boltzmann machine is a stochastic recurrent neural network with visible and hidden units. It is an undirected graph without self-connections.

The structure of the net is generally not constrained and therefore sampling and learning become a big problem.

The hidden units are denoted with $h_i$ and the visible units with $v_i$. Both of these units are written as $s_i$.

As mentioned above, every unit can either be active (1) or deactivated (0). Additional to the Hopfield Update rule we add a bias $b_i$ and the sigmoid function:

$z_i = b_i + \sum_{j}{s_j \cdot w_{ij}}$, $p(s_i=1) = \sigma(z_i) = \frac{1}{1+e^{-z_i}}$

$w_ij$ is the weight of the connection between i and j and vice versa (for i!=j).

Energy

Therefore we can write the energy of the network as:

\[E = -\sum_{i<j}{s_i \cdot w_{ij} \cdot} - \sum_{i} b_i \cdot s_i\]

This is different from the Hopfield Networks because of the bias. Although the weights are undirected, two connected units do not have to have the same bias.

What is the probability p(v) of a given state v? We defer the single probability of state to the energy function and divide it by all possible values of the Energy function:

\[p(v) = \frac{e^{-E(v)}}{\sum_{u}{e^{-E(u)}}} = \frac{e^{-E(v)}}{Z}\]

(Z is the partition function)

Entropy and the Boltzmann distribution

If you measure the microscopic state of a thermodynamic system whose thermodynamic state is determined by a constant average energy, and only a constant average energy,

∑𝑖𝑝𝑖𝐸𝑖=⟨𝐸⟩

and you measure it many times, waiting between measurements long times in order to remove temporal correlations between them, then the set of measurements will follows the Boltzmann distribution

𝑝𝑖=𝑒−𝐸𝑖/𝑇∑𝑗𝑒−𝐸𝑗/𝑇

where 𝐸𝑖 is the energy of the i-th microscopic state, 𝑇 is the temperature and 𝑝𝑖

is the probability for each of the measurements to result in i-th microscopic state.

Notoriously, the Boltzmann distribution is the one you obtain if you maximize the entropy of the system

𝑆=−∑𝑖𝑝𝑖log2𝑝𝑖

as a function of the distribution 𝑝𝑖 subject to the set of thermodynamic constraints defining the thermodynamic state of the system, which in our case it is a constant average energy, only. This happens to be that way because, among all the microscopic states the system can possibly be, if you force the system to be in a specific thermodynamic state, then, you impose restrictions onto the set of microscopic states that the system can actually reach and with which probability (the 𝑝𝑖). This is the reason for which the thermodynamic state gives you some information about which are going to be your measurements about the microscopic states of the system. In other words, the entropy quantifies how much extra information you need, in average, to determine which is the microscopic state of the system at the time of your measurements, given that you only know its thermodynamic state.

source: quora

Heat

The more you know about the state of system, the more you can take advantage of it in order to extract energy from it. For example, if a rock is falling from the sky, you can extract energy from it if you know where it is going to fall. The more you know about how and where the rock is falling, the more energy you will be able to extract from the event. In a sort of analogy, knowing that a rock is falling from the sky is like knowing the macroscopic state. On the other hand, knowing where it is going to fall is like knowing the microscopic state.

Following the previous idea, you can only extract energy from a thermodynamic system from what you know of its microscopic state, which in the present context, it is the thermodynamic state. You can think of heat as energy contained in the system that you cannot extract from it because you ignore the precise microscopic state of the system at the moment where you want to extract the energy.

source: quora

Free energy

What is free energy? A thermodynamic quantity equivalent to the capacity of a system to do work.

The free-energy is then, in an average sense, the amount of energy contained in the system, i.e. it is total energy 𝑈=⟨𝐸⟩ minus the energy you cannot extract 𝑇𝑆 due to the remaining uncertainty or ignorance you have about which is its precise microscopic state the system is given that you only know about it its thermodynamic state.

This is free energy according to Helmholtz: Hemholtz free energy (U - TS)

There is also Gibbs free energy defined as

$ G = U + PV - TS = H - TS $

in which U is internal energy, P is pressure, V is volume, T is absolute temperature, S is entropy, and H is enthalpy.

It measures the usefulness or process-initiating work obtainable from a system at a constant temperature and pressure. If it is a chemical system, a reaction that lowers G occurs spontaneously.

source: quora

Where the sigmoid comes from

see wikipedia

The difference of energies of two states is

\[\Delta E_i = E_\text{i=off} - E_\text{i=on}\]

Substituting the energy of each state with its relative probability according to the Boltzmann factor (the property of a Boltzmann distribution that the energy of a state is proportional to the negative log probability of that state) gives:

\[\Delta E_i = -k_B\,T\ln(p_\text{i=off}) - (-k_B\,T\ln(p_\text{i=on}))\]

where $k_B$ is Boltzmann’s constant and is absorbed into the artificial notion of temperature $T$. We then rearrange terms and consider that the probabilities of the unit being on and off must sum to one:

$\frac{\Delta E_i}{T} = \ln(p_\text{i=on}) - \ln(p_\text{i=off})$ $\frac{\Delta E_i}{T} = \ln(p_\text{i=on}) - \ln(1 - p_\text{i=on})$ $\frac{\Delta E_i}{T} = \ln\left(\frac{p_\text{i=on}}{1 - p_\text{i=on}}\right)$ $-\frac{\Delta E_i}{T} = \ln\left(\frac{1 - p_\text{i=on}}{p_\text{i=on}}\right)$ $-\frac{\Delta E_i}{T} = \ln\left(\frac{1}{p_\text{i=on}} - 1\right)$ $\exp\left(-\frac{\Delta E_i}{T}\right) = \frac{1}{p_\text{i=on}} - 1$

Solving for $p_\text{i=on}$, the probability that the $i$-th unit is on gives:

\[p_\text{i=on} = \frac{1}{1+\exp(-\frac{\Delta E_i}{T})}\]

where $T$ is referred to as the temperature of the system. This relation is the source of the logistic function found in probability expressions in variants of the Boltzmann machine.

Altering the factor T makes the logistic function wider or steeper.

Convergence

As with the Hopfield Nets the network shall converge to a stable state but “with a little bit of random”. This randomness is expressed by the steepness of the sigmoid function:

This steepness can be thought of as the learning rate of sgd. So we initially choose it large and then we let it become smaller so it can converge into a small region.

This is related to a method called simulated annealing. The size of the learning rate is called temperature and its getting colder with every step. The temperature determines the steepness in our example. The following image is taken from wikipedia to illustrate this process:

If the temperature is fixed and after enough unit updates the system only depends on the energy we say that it reached its thermal equilibrium.

Restricted Boltzmann Machines

In order to cope with some of the problems the model had to be restricted to a bipartite graph called the Restricted Boltzmann Machine. The two cores of the bipartite graph are the visible and the hidden units.

It was proven in 2010 by Long and Servedio that Z is intractable for both machines.

Energy

\[E(h,v) = -\sum_{h_i}{\sum_{v_j}{w_{ij} h_i v_j}} - \sum_{h_i}{h_i b_i} - \sum_{v_j}{v_j b_j}\]

The probability of a visible unit v can be calculated easier:

\[p(v_j|H) = \sigma(b_j + \sum_{h_i}{w_{ij} h_i} )\]

And vice versa for a hidden unit:

\[p(h_i|V) = \sigma (b_i + \sum_{v_j}{w_{ij} v_j} )\]

The free energy in a RBM can be calculated and one might use it to tell something about the “potential energy” we still have when we converged to an equilibrium. It can be used to determine if a RBM is overfitting.

Learning with Contrastive Divergence

Source: Youtube

The idea is to pick a data vector and set it as visible units. Now because the graph is bipartite we can update all hidden units at the same time and vice versa with the visible units until the units of the RBM reached thermal equilibrium. This is a Markov Chain Monte Carlo method particularly Gibbs sampling.

The weight update becomes: $\Delta w_{ij} = \epsilon ( <v_i h_j>^0 - <v_i h_j>^{\infty} )$

This is a “fight” between what the memory wants (the state in the thermal equilibrium $<v_i h_j>^{\infty}$) and the data $<v_i h_j>^0$. In the beginning the memory is random therefore we don’t want to make too many steps into its direction. If we don’t wait until the thermal equilibrium is reached but rather use the “wrong” gradient the method is called Contrastive divergence.

The longer the training continues the more iterations of contrastive divergence learning are used. In the beginning it is one and then we increase it.

Example of CD

The following python snippet visualizes as training with CD on MNIST.

import matplotlib.pyplot as plt
import sklearn
import numpy as np

from scipy.stats import logistic
from sklearn.datasets import load_digits

digits = load_digits()

plt.gray()

<Figure size 432x288 with 0 Axes>

mask = digits.target == 3
threes = digits.images[mask]

# Init
h = np.random.normal(0, 1, 25)
# bias
b1 = np.random.normal(0, 1, 64)
b2 = np.random.normal(0, 1, 25)
# from x to h
W = np.random.normal(0, 1, [64, 25])

# parameter
lr = 0.01
iterations = 100
contrastive_divergence_steps = 50

# debug
print_every_nth = 100

# contrastive divergence CD-50

# train on the first 8 threes
for x in threes[:8]:
    for iteration in range(iterations):
        v = x.flatten() / np.max(x)

        p_of_h = logistic.cdf(W.T.dot(v) + b2)
        positive_phase = np.dot(v[:, np.newaxis], p_of_h[:, np.newaxis].T)

        for i in range(contrastive_divergence_steps):
            # plt.matshow(v.reshape([8,8]))

            # calculate p(h|v)
            p_of_h = logistic.cdf(W.T.dot(v) + b2)

            # sample p(h|v)
            h = p_of_h

            # calculate p(h|v)
            p_of_v = logistic.cdf(W.dot(h) + b1)

            # sample p(h|v)
            v = p_of_v

        if iteration % print_every_nth == 0:
            plt.matshow(v.reshape([8,8]))
            plt.show()
            print(iteration, " size of W: ", np.sqrt(np.sum(W**2)))

        negative_phase = np.dot(v[:, np.newaxis], p_of_h[:, np.newaxis].T)

        update = positive_phase - negative_phase

        W += lr * update

0  size of W:  39.924335666308195

0  size of W:  39.17467666748022

0  size of W:  38.97155882317313

0  size of W:  38.944982809345895

0  size of W:  38.80202589068636

0  size of W:  38.74259814116064

0  size of W:  38.6967212149249

0  size of W:  38.74791680409241

This model of a three could already be used for dichotomizing threes and not-threes.

In the code I wrote something about a positive and a negative “phase” (or gradient). In the case of a correct sample the positive gradient increases the likelihood of the data whereas the negative gradient increases the likelihood of all others (which means that it decreases the likelihood of the current point).

There are discussions whether the latter might be something like “unlearning” what humans do when they dream.

Deep Belief Networks

Imagine we stacked another RBM on top of our hidden layer. We would treat the hidden units as visible and do the same training layer by layer (Greedy Layer-wise training). The RBMs are not really connected for the training. Once the first layer is ready we don’t train it anymore. This yields a Deep Belief Network which is something that Geoffrey Hinton has been working on for quite some time.

In contrast to the Hopfield Net and the Boltzmann Machine is a Belief Net a directed probabilistic graphical model. For this reason are the first models Markov Random Fields (MRF) and the Belief Net a Bayesian Net. The MRFs are coupled with Energy-based models because symmetric weights are an important property to converge. Directed graphs have the intention to model causal relationships and not only correlations.

The advantage of this “stacking” is to build hierarchical features. Although neural networks with two hidden layers are in theory already a general function approximator depth introduces a valuable inductive bias especially when coupled with a convolution. What I mean with that is that there are hierarchical patterns which can be uncovered easily when we assume locality and increasing complexity.

A Belief Net is:

A directed acyclic graph
typically sparsely connected (this is a prerequisite for some smart algorithms to work)
every variable could be observed or hidden (“visible effects” and “hidden stochastic causes”)
First target: Infer the state of unobserved variables
Second target: “Learning”: Adapt the interactions between nodes in the graph in order to maximize the likelihood

Sampling the leaf nodes is easy but sampling the posterior is hard. Why is this interesting? In supervised learning we assume that all our training samples were sampled from the data manifold. Why can’t we just use the training data as samples from the posterior?

Unfortunately I have a technical answer only. For Variational Autoencoders (VAE). We assume that there are some hidden causes which we will never observe but we want to make them explicit and in the case of VAE even use them to control the generation of images or robotic hands.

An intuitively understandable reason for why sampling this posterior is difficult can be given by the concept of Explaining-Away. Explaining-Away is basically a pattern of reasoning where one cause of an effect explains that effect entirely thereby reducing (need not eliminate) the need to verify other alternative causes.

A quote from a quora answer:

Imagine an ‘observation’ ‘Ob’, say “you scored 100/100 in Math”. It could have had two (for the sake of simplicity) causes, viz ‘C1’ - ‘the question-paper was too easy’ and ‘C2’ - ‘you are too smart’. Now, ‘Ob’ could be because of either of those two causes (or both) i.e. you scored 100/100 because either the question-paper was way too easy or you’re just too smart or both. Confirmation of one cause, say ‘C1’ i.e. ‘paper was too easy’ doesn’t provide any evidence for the other alternate cause ‘C2’ (‘you are too smart’). But, in a probabilistic scheme of reasoning, confirmation of the first reasoning ‘C1’ (‘paper was easy’) SHOULD REDUCE the probability of the other cause (‘you are smart’) even though both can be true simultaneously. This is called “Explaining Away”. I mean to say if I know that you scored 100/100 and also know that the paper was way too easy, I tend to assume you’re not very very smart even if you actually are.

The variational approach that was kicked-off by the Wake-Sleep Algorithm just does not use an unbiased sample from the posterior but a more or less similar distribution. With this approximation we can make maximum likelihood. So it seems like we perform the same two steps as in Expectation Maximization.

There is a lot more to read and write about this topic but it feels like I am drifting to variational methods, therefore I am going end here and leave the rest for the next time.

A Practical Guide to Training Restricted Boltzmann Machines

Boltzmann Machines was originally published by Daniel Pollithy at Notes on July 11, 2019.

Hopfield Network

2019-07-05T00:00:00+02:00

This article explains Hopfield networks, simulates one and contains the relation to the Ising model.

Hopfield Nets

Let’s assume you have a classification task for images where all images are known. The optimal solution would be to store all images and when you are given an image you compare all memory images to this one and get an exact match. But then you realize that this “look up” (comparing one image after another) becomes costly the more images you have to memorize. Even if you try to arange the images in a tree structure you will never be as fast as the following solution: store all images in Content addressable memory (CAM). CAM is specialized hardware that is used in routers. Ternary-CAM additionally has a “Don’t-Care” state (“?”) with which parts of the pattern can be masked (this is useful for IP routing).

In German CAMs are referred to as “Assoziativspeicher” (associative memory). Intuitively you don’t use an address to access data but instead you use data to access the address of the data.

Now imagine we had many pictures of dogs in our memorized array. And then comes a new image of a dog. Expect for the masking option (“?”) we cannot perform any kind of fuzzy search on hardware CAM.

As we will see in the following section, a Hopfield Network is a form of a recurrent artificial neural network and it serves as some kind of associative memory. It is interesting to analyse because we can model how neural networks store patterns.

Binary Hopfield Networks

A network with N binary units which are interconnected symmetrically (weight $T_{ij}=T_{ji}$) and without self-loops ($T_{ii} = 0$). Every unit can either be positive (“+1”) or negative (“-1”). Therefore we can describe the state of the network with a vector U. For example U = (+,-,-,-,+…).

Intuitively we initially set a state and wait until the network relaxes into another stable state. That means it does not change anymore once it reached that state. The reached state is the output value. For the associative memory example with images, the initial state is the image (black and white) and the stable state can also be interpreted as an image.

Each of the units is a McCulloch-Pitts neuron with a step-function as non-linearity. That means if we only update a single unit (“neuron”) then we calculate the weighted sum of the neighbours and set the neuron to “+” if the sum is greater or equals a threshold.

Weighted sum of neuron $x_j$: $g(x_j) = \sum_{i}{x_i \cdot T_{ji}}$
Step function: $x_j = \begin{cases} +1, g(x_j) \ge 0\\ -1, else \end{cases}$

Update procedure:

Synchronous update: A global clock tells all neurons at the same time to update. As a consequence all new states only depend on the old states
Asynchronous update: One unit is updated after another. This seems to be more plausible for biological neurons or ferro-magnets because they have no common clock. The order is random.

Learning refers to finding appropriate weights. For the associative memory we want the connections between two neurons positive if they are often active together and we want the value to be negative if they are often different. This is Hebbian Learning Theory which is summed up as “Neurons that fire together, wire together. Neurons that fire out of sync, fail to link”.

Example

The following example simulates a Hopfield network for noise reduction. The training patterns are eight times “+”/”-“, six times “+”/”-“ and six times the result of “+”/”-“ AND “+”/”-“. The reason for the redundancy will be explained later.

The images of the simulations have the number of state at the x-axis and the time step as y-axis.

import numpy as np
import random
import matplotlib.pyplot as plt

patterns = [
#     Left variable                |    Right variable              =   Left AND Right    
    [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1],
    [-1, -1, -1, -1, -1, -1, -1, -1, +1, +1, +1, +1, +1, +1, +1, +1,   -1, -1, -1, -1, -1, -1],
    [+1, +1, +1, +1, +1, +1, +1, +1, -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1],
    [+1, +1, +1, +1, +1, +1, +1, +1, +1, +1, +1, +1, +1, +1, +1, +1,   +1, +1, +1, +1, +1, +1]
]

size = len(patterns[0])
weights = np.array([[0]*size]*size)

for pattern in patterns:
    for i in range(0, size):
        for j in range(0, size):
            if j==i:
                pass
            else:
                weights[i,j] += pattern[i] * pattern[j]

def simulate(x1, x2, eps=30, first_state=None, visualize=True):
    number_of_iteration_unchanged = 0

    if not first_state:
        first_state = [x1]*8 + [x2]*8 + [+1, -1, +1, -1, +1, -1]
    trace = [
        first_state
    ]

    while (number_of_iteration_unchanged < eps):    
        if len(trace) > 1 and trace[-1] == trace[-2]:
            number_of_iteration_unchanged += 1
        else:
            number_of_iteration_unchanged = 0

        random_unit = random.randint(0, size-1)

        all_new_states = (np.array(np.dot(weights, trace[-1]) >= 0.0, dtype=np.int)*2) - 1
        trace.append(trace[-1].copy())
        trace[-1][random_unit] = all_new_states[random_unit].copy()

    np_trace = np.array(trace)

    if visualize:
        plt.imshow(np_trace)
        plt.show()

    return trace[-1]

simulate(-1,-1, eps=100);

simulate(1,1, eps=100)

[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

simulate(-1,+1, eps=100);

simulate(+1,-1, eps=100);

We see that all of the simulations “relaxed” to a solution which did not change for a long time (epsilon). Running the simulations multiple times showed that the (0,0) and (1,1) cases always converged to the correct pattern whereas the distinct cases were torn between the the correct state and its inverse.

But why does the network converge at all?

Stability theory

In system theory there is the quest to prove the stability of different kind of systems (see here). A system is said to be stable if small perturbations of the input data lead to small variations of the output value.

Stability can be intuitively grasped by imagining the velocity and phase of a pendulum. See the image by Krishnavedala.

If the pendulum was damped the phase portrait would look like a spiral. The center to which it converges over time is a of point of attraction which is called equilibrium. The circle in the diagram is called “orbit”. An important question is for example: Will the trajectory of a given initial state converge to the point or will it diverge.

And what happens if the system gets “pushed” a little bit. A pendulum would oscilate a little bit and converge back to the equilibrium but a football on a hill would roll down.

For linear systems we can use the magnitude of the derivative of a point a. If it is less than one the system is said to be stable.

For more information see Stability theory on Wikipedia.

The Lyapunov Function can be used to show asymptotic stability (it gets really close) or lyapunov stability (all points which were at time t in an orbit are at t+1 in a small neighbourhood). It works by choosing a Lyapunov function which has an analogy to the potential function of classical dynamics. In real physical systems a precise Energy function can be used. You can imagine a state x of pendulum. If the system loses energy the system will reach a final resting point. The final state is the attractor. Lyapunov showed that it is not necessary to know the energy function but it is enough to find a Lyapunov function V(x) with the following criteria:

\[V(0) = 0\]
$V(x) > 0$ if $x \ne 0$
$\dot{V}(x) = \frac{d}{dt}V(x) = \nabla V \cdot f(x) \le 0$ if $x \ne 0$

Lyapunov Stability

Energy function

Let’s apply this to Binary Hopfield Networks. We saw in the simulation that the Hopfield nets converge. Will this always be the case?

We choose our Energy function: $E = -0.5 \sum_{j}{ \sum_{i \ne j}{u_i \cdot u_j \cdot T_{ji}} }$

Actually we cannot assume to have a single global optimum on the surface of the energy function. At least every pattern we want to store should a local optimum. So we just show that the energy at t+1 is less or equal to t and therefore we always find a local optimum of the energy function.

In our asynchronous, random update procedure we pick a unit u. It either has the state -1 and we change it to +1 or it has the state +1 and we change it to -1. In the first case the difference is “+2” and the the second one it is “-2”. => $\Delta u_j \in \{ -2, +2 \}$

How does the energie of unit j change?

\[\Delta E_j = E_{j_{new}} - E_{j_{old}} =\] \[-0.5 \sum_{i \ne j}{u_{i} u_{j_{new}} T_{ji}} -0.5 \sum_{i \ne j}{u_i u_j T_{ji}} =\] \[-0.5 \cdot (\sum_{i \ne j}{u_{i} u_{j_{new}} T_{ji}} -\sum_{i \ne j}{u_i u_j T_{ji}} ) =\] \[-0.5 \cdot (\sum_{i \ne j}{u_{i} u_{j_{new}} T_{ji} -{u_i u_j T_{ji}}} ) =\] \[-0.5 \cdot (\sum_{i \ne j}{u_{i} \cdot T_{ji} \cdot (u_{j_{new}} - u_j}) ) =\]

Define: $\Delta u_j = u_{j_{new}} - u_j$

\[-0.5 \cdot (\sum_{i \ne j}{u_{i} \cdot T_{ji} \cdot \Delta u_j }) =\] \[-0.5 \Delta u_j (\sum_{i \ne j}{u_{i} \cdot T_{ji} }) =\]

Two possible cases for the neuron j:

“+2” case: The weighted neighbours are greater equal zero ($\sum_{i \ne j}{u_{i} \cdot T_{ji} } >= 0$), $\Delta u_j=+2$, $\Delta E_j = -1 \cdot (\sum_{i \ne j}{u_{i} \cdot T_{ji} }) \le 0$

“-2” case: The weighted neighbours are less than zero ($\sum_{i \ne j}{u_{i} \cdot T_{ji} } < 0$), $\Delta u_j=-2$, $\Delta E_j = +1 \cdot (\sum_{i \ne j}{u_{i} \cdot T_{ji} }) < 0$

Conclusion: In both cases the energy either reduces or stays the same. Every minimum of the energy function is a stable state.

In the example of the associative memory we see that a pattern (state) can only converge to the correct solution (pattern) if it is in its attractor region. Imagine a 2d hill. On which side of the hill we roll down is determined by the random sampling process. This explains why the “AND-simulation” from above can converge into different states.

Note: Not all attractor regions have the same size. Therefore one state can attract more patterns than another. For the associative memory this leads to some kind of prior. Additionally mixture of an odd number of patterns cause mixture states which do not correspond to real patterns. Even numbers would sum up to zero.

For every pattern x the inverse of x is also a local minimum.

If we try to store too many patterns so called spin glass states are generated. They are no linear combination of the original states. A network of N units has the capacity to store ~ 0.15N uncorrelated patterns. All the minima that have no associated pattern are called spurious states.

If all weights are “+1” then the function calculated is something like k-Means/Hamming distance.

More math about Hopfield nets here.

Ising model

Hopfield nets are isomorph to the Ising model in statistical physics which is used to model magnetism at low temperatures. Every neuron equals an atom in a solid body. The state of the unit coincides with the spin (the magnetic moment). In the Heisenbergmodel the spin can be multivariate. In the Ising model the spin is either parallel or antiparallel to a given axis z). The latter can be written as the Binary Hopfield Network.

In physics the energy of the atoms is “measured” with the Hamilton operator H. It is the sum of the potential and kinetic energies in the system.

\[H = -0.5 \sum_{ij} T_{ij} \cdot s_i \cdot s_j - H_z \sum_{i=1}{s_i}\]

$s_i$ the spin of an atom
$T_{ij}$ the coupling constant between the spin of two atoms
$H$ is the strength of the magnetic field

The critical temperature was calculated with this model. A system with less than its critical temperature is dominated by quantum mechanical effects. If the majority of the couplings have a positive sign then the body is ferromagnetic. Which means you could use a permanent magnet to magnetize it like a refrigerator magnet.

Hopfield Network was originally published by Daniel Pollithy at Notes on July 05, 2019.

Simulating heat flow with Fourier

2019-07-01T00:00:00+02:00

I recently read the book “Infinite Powers” by Steven Strogatz. In chapter 10 he told the story of Joseph Fourier and why he initially cared about approximating functions with a mixture of sine and cosine waves.

His objective was to solve the “heat problem”. How does heat distribute in in a rod (or regarding circuits: in a wire). For some backgrounds see the section “background”.

The following section contains a demo of such a wire that can cool down. By clicking on the first wire you can draw heat patterns which are smoothed with a Fourier transformations (actually by setting the higher order coefficients to zero). In the case that the “simulate” check-box was active, then the simulation will start after clicking on the wire.

Demo

Status: Ready

Input (click on the left half of the bar):

Betragsspektrum:

Abgeschnittenes Betragsspektrum:

Output:

Heat flow simulation:

FFT coefficients: 1… … 400
Simulate cooling:

Background

Initially I wanted to write a little bit about this but I found that 3b1b just published a video about this topic:

Simulating heat flow with Fourier was originally published by Daniel Pollithy at Notes on July 01, 2019.

Car model datasets

2019-06-29T00:00:00+02:00

List of car datasets

This is a simple lookup page to answer questions like “In which dataset are images of the Renault Clio?”

The lists are extracted from the datasets either by reading the directory structure or the accompanying matlab files which contain the class labels.

Stanford-196

see /images/cars_stanford196.txt

CompCars

see /images/compcars.txt

BoxCars

see /images/boxcars.txt

Car model datasets was originally published by Daniel Pollithy at Notes on June 29, 2019.

Optimization

2019-06-23T00:00:00+02:00

Optimization

Search for the optimum

Assume we define a function f which takes a profession and returns the annual salary. It sounds naturally to search for the optimum. Maybe we even have some constraints like “the job should be in Europe” or “I want to work 30 hours per week”. This is a so called optimization problem.

We define:

Variables to optimize
A criteria function which depends on the variables
Constraints that have to be satisfied by the solution

There exists the general dichotomization into static and dynamic optimization. In static opt. the variables are parameters whereas in dynamic optimization the variables are functions themselves. The criteria function then is a function of a function which has the name functional. A general example for a functional is the integral from zero to one of a function f. It maps the function to a scalar. Control in robotics is mostly dynamic. Classifiying images or speech signals belongs to the static class.

There is the general possibility to find extrema by differentiating and finding the zeros of it. This is done in the Lagrangian method or in a probability context with the Maximum Likelihood estimation.

The term search refers to the attempt of finding a feasible point that satisfies all constraints and yields the lowest possible error (in the example of the beginning this would be the highest salary). Mathematically speaking this is expressed with $\operatorname*{argmin}_\theta f(x)$ plus some constraints.

Local search starts from a point (a solution) and searches its neighbours for a better solutions which can be optimized again.

Global search: The state space gets evaluated (not only locally) in order to find an optimum. Example: Dynamic Programming

Stochastic search: Finding a new solution involves a nondeterministic component. Like a randomized search in ML.

Uninformed search: All of the options and permutations of them have to be covered. We can’t discard a path until we have tried it out. Breadth-first or depth-first search are examples of this scheme in graphs.

Informed search: Contrary to the uninformed search we use some kind of heuristic to guide the process of obtaining a new solution. The heuristic might exclude parts of the state space so we don’t have to search there. An example is the A* algorithm or the dijkstra. This is related to the global search.

Convex optimization

Linear least squares is the method of finding a line that best fits given data points. “Best fit” means that the sum of the squared distances between a point and the line is minimal. The vertical distance between the point and the line gets called residual.

We assume the observations y are generated by a linear relationship (m) with the data X which is affected by white noise e.

\[y = m * x + t + noise\]

import numpy as np
import matplotlib.pyplot as plt

n = 100
X = np.random.uniform(0, 5, n)
y = 0.3 * X + 2 + np.random.normal(0, 0.1, n)

plt.scatter(X, y)

The process that generates the data has two unknown variables $m$ and $t$. If the error was not present we could solve for the perfect solution in with two points:

\[2.0 = m * 0 + t => t = 2.0\] \[2.5 = m * 2 + 2.0 => 2m = 0.5 => m = 0.25\]

In reality we cannot rely on the data nor can we rely on our model. Therefore we want to use as many points as possible. We just assume that the error has no bias, because we want to take the average of all estimates.

This idea is captured in the Linear least squares:

\[arg_W min \sum_{i}{ (y_i - m x_i + t)^2 }\]

This is called the quadratic error term. We “search” parameters m and t that minimize the error.

Let’s see how the error term behaves for different values:

ts = np.linspace(-100, 100, 100)
ms = np.linspace(-100, 100, 100)

cost_fn = lambda m, t: np.sum( np.square( y - m*X + t ) )

cost = []
for t in ts:
    cost.append( [cost_fn(m, t) for m in ms] )
    
cost = np.log(np.array(cost))

tt, mm = np.meshgrid(ts, ms)

plt.contourf(tt, mm, cost)
plt.xlabel("t")
plt.ylabel("m");

From the equation of the error and the plot, we see that this is a paraboloid. Its curve belongs to the class of convex curves. Finding the optimum is straightforward:

Before doing this we switch to the matrix vector notation because we can handle the sum better in that way.

\[y = (y_1, ..., y_n)^T\]

$X = (x_1, ..., x_n)^T$ where $x_i$ is a 2d vector because we add a “1” with the bias trick. For example $x_i = (1, 2.3)^T$

$w = (t, m)^T$ (“weight vector”)

\[y = X \cdot w^T\]

We want to solve for the parameters w:

\[w^T = X^{-1} y\]

For the “normal” case where X is a non-square matrix we can use the Moore-Penrose Pseudoinverse with alpha=0:

\[w^T = (X^T \cdot X)^{-1} \cdot X^T y\]

Note: The result can be proven by differentiating, setting it to zero. $X$ has to be of full column rank, or else the $X^T X$ would not be invertible.

Let’s apply this closed form solution to the data:

X_bias = np.vstack([np.ones(X.shape[0]), X]).T
X_bias[:5]

array([[1.        , 4.79006299],
       [1.        , 3.30198279],
       [1.        , 0.63063158],
       [1.        , 1.58619112],
       [1.        , 3.48292161]])

np.matmul(
    np.linalg.inv(
        np.matmul(
            X_bias.T, 
            X_bias)
    ), X_bias.T
).dot(y)

array([1.99713 , 0.302953])

This is really close to the original parameters 2.0 and 0.3.

The “goodness” of the ordinary least squares is proven in the Gauss-Markov Theorem under the assumptions that X has full column rank, the noise is white, the noise per dimensions all have the same variance and they are uncorrelated. The assumptions about the noise signify that the Covariance Matrix of the noise is a diagonal matrix which is scaled by one single factor.

It is the best linear unbiased estimator (BLUE) and this estimator is unique.

Under the assumtion that the noise is normally distributed with a expectation value of zero the method of linear least squares is the same as the linear regression.

Polynomial regression

The concept of the linear regression can be extended to non-linear curve fitting with a simple trick. We transform the data points $X$ into its higher polynomials, perform the usual least sqaures fitting and use the parameters in the higher order model.

X = np.random.uniform(-5, 5, n)
y = X**5 + np.random.normal(0, 300, n)

plt.scatter(X, y);

# We treat the polynomials as new dimensions in our data matrix X
X_poly = np.vstack([X**0, X, X**2, X**3, X**4, X**5, X**6, X**7, X**8]).T

# The linear regression from above
beta = np.matmul(
    np.linalg.inv(
        np.matmul(
            X_poly.T, 
            X_poly)
    ), X_poly.T
).dot(y)

beta

array([-1.42284625e+02, -5.03193657e+01,  1.79237377e+01,  3.62618409e+00,
       -2.23686147e+00,  1.87793816e+00,  2.53151667e-01, -4.53556005e-02,
       -8.67038500e-03])

y_2 = [sum([beta[i]*x**i for i in range(len(beta))]) for x in X]

plt.scatter(X, y_2)
plt.scatter(X, y);

We note that the original parameters were not restored. This is due to the fact that the polynomials are highly correlated and different polynomials can form similar curves.

There is a lot more to say about regression but this text is about optimization which is the reason why I continue with the next topic.

Optimization under constraints

Let’s go through an example: You want to throw a ball over a wall with as little effort as possible. The ball flies in a parabola. You stand at (x=-5, y=0). The wall is at x=0 with a height of 2.

plt.scatter(X, -0.08*X*X + 2.05)
plt.scatter(np.zeros(20), np.linspace(0, 2, 20))

The function we want to maximize is the inverse of the parabola because we want it to be as low as possible:

… (ToDo: Find better example)

These problems were all static optimization problems:

Linear opt: linear cost function, linear constraint
Quadratic optim.: quadratic cost function, linear constraint
Nonlinear optimization: either cost function or a constraint are non linear

Nonlinear optimization

As long as the criteria function f(x) is linear it can be written in linear algebra as a matrix vector product: $Ax$. Solving these problems comes down to inverting a matrix. There are different ways to do this. It depends on the size of the matrix and its conditioning. For most of the modules availabe these days LAPACK/BLAS takes care of this.

What can we do if we are facing a nonlinear function? Such a term appears for example in the loss function of neural networks.

Performing local search (which is always iterative): In this case we sample the value of the error function at the initial point. We could sample a neighbouring point assuming continuity of the function and calculate the difference quotient. But this does not work in practice because of numerical imprecisions.

What we do instead is to calculate the gradient with backpropagation in our sampled point.

Why should the gradient be of any interest? A short answer might be: We assumed continuity and therefore we can execute a step into the opposite direction (compare this to the Forward-Euler).

A longer answer: What we do by using the gradient is that we linearize the nonlinear error surface around our sampled point (we assume the neighbourhood to be a plane). This is a standard linearization which can be justified by the Taylor approximation. The higher order terms are omitted. So the gradient tells us if we make an infinitesimal step into its direction whether the error will be higher or lower.

Of course the local linearity is not always given. In theory we could for example determine whether a function is Lipschitz-continuous but what would this change? Maybe descrease the step size of gradient descent…

Another option is Newton’s method which involves the second order of the taylor series expansion. With this method we can make faster progress in flat areas (because we now assume that the error is quadratic) but in general it does not help with arbitrary functions. The Hessian can be used to get the curvature of the surface around the point (conditioning number).

Calculating the Hessian is resource intense. The Gauss Newton method approximates the Hessian with the square of the Jacobian. The rest of it is comparable to the Newton method.

The Gauss Newton Method can be damped. With this addition the method is called Levenberg-Marquardt algorithm (also called damped least sqaures). The damping factor

Optimization was originally published by Daniel Pollithy at Notes on June 23, 2019.

Dead Reckoning 2d

2019-05-19T00:00:00+02:00

Dead reckoning

The following notebook contains a simple implementation of dead reckoning with the Explicit and Implicit Euler Method for a bi-cycle given the steering angle and velocity at every timestep t.

import math
import numpy as np
import matplotlib.pyplot as plt

Explicit Euler Method

$X_{k+1} = X_{k} + T \cdot X_{k}'$ where T is the discretization step size.

The derivative can be approximated by a circular path around the instant center of rotation “ICR”). $X_{k}' = ( v \cdot cos(\theta_k) , v \cdot sin(\theta_k) , v \cdot \frac{tan(\alpha_t)}{l})^T$

def explicit_euler(length, t_end, T, X_start, v_func, alpha_func):
    # explicit euler
    explicits = [X_start]
    
    t_steps = int(math.ceil((t_end - 0.0) / T))
    
    for i in range(t_steps):
        # velocity and steer at time i
        v_k = v_func(i)
        alpha_k = alpha_func(i) if alpha_func(i) != 0 else 0.00001
        R = length / math.tan(alpha_k)

        # old X and derivative of old X
        x_k, y_k, theta_k = explicits[-1]
        x_dot = v_k * np.array([math.cos(theta_k), math.sin(theta_k), 1/R])

        # store values and norm angles only between 0 and 2 pi
        explicits += [ explicits[-1] + T * x_dot] 
        explicits[-1][2] %= 2*math.pi

    np_explicits = np.array(explicits)
    
    return np_explicits

Implicit Euler Method

$X_{k+1} = X_{k} + T \cdot X_{k+1}'$ yields system of linear equations.

\[x_{k+1} = x_{k} + T \cdot v \cdot cos(\theta_{k+1})\]
\[y_{k+1} = y_{k} + T \cdot v \cdot sin(\theta_{k+1})\]
\[\theta_{k+1} = \theta_{k} + T \cdot v \cdot 2 \cdot tan(\alpha_{k+1})\]

The third equation yields $ \theta_{k+1} $ which can be inserted into the first and second.

def implicit_euler(length, t_end, T, X_start, v_func, alpha_func):
    implicits = [X_start]
    t_steps = int(math.ceil((t_end - 0.0) / T))
    
    for i in range(t_steps):
        # old state
        x_k, y_k, theta_k = implicits[-1]
        
        # next control variables
        alpha_k_plus_1 = alpha_func(i+1)
        v_k_plus_1 = v_func(i+1)
        
        # X_k+1
        theta_k_plus_1 = theta_k + T * v_k_plus_1 * 2 * math.tan(alpha_k_plus_1)
        x_k_plus_1 = x_k + T * v_k_plus_1 * math.cos(theta_k_plus_1)
        y_k_plus_1 = y_k + T * v_k_plus_1 * math.sin(theta_k_plus_1)

        # store values and norm angles only between 0 and 2 pi
        implicits += [ np.array([x_k_plus_1, y_k_plus_1, theta_k_plus_1]) ] 
        implicits[-1][2] %= 2*math.pi

    np_implicits = np.array(implicits)
    
    return np_implicits

def visualize_state_sequence(Xs):
    plt.plot(range(Xs.shape[0]), Xs)
    plt.show()
    plt.scatter(Xs[:, [0]], Xs[:, [1]]) 
    plt.show()

Test the functions

Assume a constant velocity and a stair function for steering angles.

# command variables
def v(t):
    return 2

def alpha(t):
    if t == 0:
        return 0.001
    if 1 < t < 2:
        return 0.3 * math.pi / 180.0
    
    return 0.15 * math.pi / 180.0

x1 = explicit_euler(length=0.5, t_end=3.0, T=0.1, X_start=np.zeros(3), v_func=v, alpha_func=alpha)

visualize_state_sequence(x1)

x2 = implicit_euler(length=0.5, t_end=3.0, T=0.1, X_start=np.zeros(3), v_func=v, alpha_func=alpha)

visualize_state_sequence(x2)

x3 = explicit_euler(length=0.5, t_end=15.0, T=0.1, X_start=np.zeros(3), v_func=v, alpha_func=math.sin)

visualize_state_sequence(x3)

x4 = implicit_euler(length=0.5, t_end=15.0, T=0.1, X_start=np.zeros(3), v_func=v, alpha_func=math.sin)

visualize_state_sequence(x4)

Dead Reckoning 2d was originally published by Daniel Pollithy at Notes on May 19, 2019.

Harris Corner Detection

2019-05-12T00:00:00+02:00

This is an interactive demo for the Harris Corner Detector running in your browser with tensorflow.js.

The goal is to find points in an image which are corners. These points have Hessians with large, similar eigenvalues. If only one of the eigenvalues would be large, then the point might be part of an edge but not a corner.

Instead of calculating the eigenvalues (which is by the way not straight forward to implement in tensorflow.js) the authors suggested to calculate a score which is easier to obtain from the Hessian. They named it the Harris-Cornerness-function f.

\[f = det(H) - kappa * trace^2 (H)\]

det(H) is equal to the product of the eigenvalues. And trace(H) is equal to the sum of the eigenvalues.

Combined in the f value we get an indication about how different and how large the eigenvalues are.

See for more details: Wikipedia

In the case you are wondering what this is good for: I am attending multiple computer vision courses this semester (video analysis, automotive vision, deep learning for computer vision) which have some overlaps. In order to practice matrix operations and get some visuals I decided to build a collection of demos with tensorflow.js that implement the classic cv algorithms.

Note: Instead of the Hessian H the structure matrix M is used most of the times. (dy and dy are the gradient in x and y direction):

\[\begin{pmatrix} dx^2 & dx \cdot dy \\ dy \cdot dx & dy^2 \end{pmatrix}\]

Harris Corner Settings:

Kappa

Threshold

Harris Corner Detection was originally published by Daniel Pollithy at Notes on May 12, 2019.

Information theory

2019-04-27T00:00:00+02:00

Whether it is cryptography, automatic speech recognition or computer vision: Information theory seems to be everywhere in computer science. This post is a collection of intuitions and formulas.

Information theory reasons about channels and how to optimally encode messages on a channel with or without noise.

Self-Information

Imagine that a friend calls you every morning and tells you about the weather in his hometown. He lives in the desert, therefore 99% of the days he just tells you that the sun is shining (S). It is really rare that he calls you and it is raining (R).

We can estimate the probability of what your friend says by maximum likelihood estimation which essentially means in this case, that we are just counting the occurrences and divide by the total.

P(Rain) = 0.01 and P(Sun) = 0.99

The first thing to realize is that there is an inverse proportionality between the probability of an event and the self-information $I$ with $P \sim 1/I$

Intuitively we know that the information contained in the unlikely event of rain is much more interesting than the sun.
If an event has probability 100%, then our code should have length 0 because we don’t have to communicate anything
Improbable events should have longer codes.

The negative logarithm seems to be a natural choice for this relationship but we can justify the usage of the negative logarithm from a better perspective:

What is the ideal self-information for an event x. For example I(Rain)? Let’s divide the probability space into subspaces of the size of the smallest probability unit. So in our example we could draw a rectangle (which illustrates the probability space omege), divide it into 100 cells of the same size. Each cell shall be part of an event $x \in \Omega$. The more likely the event is, the more space will it cover. In our example: 99 cells belong to the event Sun and only one cell to the event Rain.

Imagine that we split the probability space recursively at halfs to form a binary tree. The root in this tree divides the space into two areas with probability split 50% to 50%. Stepping down into the left subtree has the code symbol “0” attached and to the right subtree a “1”. The sequence of “0”s and “1”s is what we call the code of the leaf node where we end up. We repeat the splitting until the granularity is fine enough to capture the smallest probability of our events (in our case 1%).

Obviously this depends on the smallest probability. For an Event x with probability P(x) the amount of cells in the probability space is number_cells = 1/P(x). For example for rain P(rain) = 0.01 => number_cells = 1/0.01 = 100

With the splitting from above we can form a binary tree that has 100 leaf nodes (rounding errors are omitted). To answer the question for the optimal encoding for the event “Rain” we only have to find the highest level at which the probability space is divided fine-granular enough to contain a cell with the correct probability.

For a binary tree we know that the amount of nodes k at a given depth d is $k = 2^d$. We start to count the depth at 0. We can transform this to answer the question about the depth: $d = log_{2}{k}$

With k=number_cells we get the formula for the self-information I:

\[I(x) = log_{2}{(\frac{1}{P(x)})}\]

This is equivalent to a more typical form: $I(x) = -log_{2}{(P(x))}$

Of course we could use a tree with a branching factor different to 2. The base=2 means that our discrete alphabet has two symbols. We know some of these alphabets:

bits/shannons: base=2
nats: base=e
trits: base=3
dits/hartleys: base=10

Self-information tells us the optimal length to encode the event x.

We can calculate the length of the code for a sequence with this. For example, we might receive the following sequence: “S,S,S,S,S,S,S,S,S,Rain”

We just have to sum over:

9 times Sun: 9 times the optimal length for the event sun => 9 * I(Sun)
1 times Rain: 1 * I(Rain)

Shannon entropy

But what if we don’t have a given message and we are just interested in the general case aka the mean case. So what would be the message length if we received infinitely many sequences from the channel.

Until now we only calculate the expected self-information of a single event x. For example for Rain or Sun. But we know that these two events are transmitted over the same channel. This is the reason why we might ask ourselves: What is the expected information of the whole channel?

We define the random variable $X$ from which we draw a sequence of small $x$. In our example: $Rain \in X$ and $Sun \in X$

We can rephrase the question from above: What is the expected information drawn from an event from X. The expectation value means that we infinitely often draw an event x from X and calculate the mean of the I(x).

Let’s say we draw n samples with $n \rightarrow \infty$

\[\mathbb{E}_{x \sim P}[I(x)] = 1/n * \sum_{i=1}^{n}{I(x_i)}\]

We can split the sum into |X| different sums (“distinct events of X”). For every event x one sum. And we know, that every event x should appear P(x) * n times in the sum. For example if drew 100 events from our friend’s weather sequences we “expect” that 100 * 0.01 events of rain should be in there. Which is 1.

\[\begin{equation} \begin{split} \mathbb{E}_{x \sim P}[I(x)] & = 1/n * \sum_{i=1}^{n}{I(x_i)} \\ & = 1/n * \sum_{x_i \in X}{P(x) * n * I(x)} \\ & = 1/n * n * \sum_{x_i \in X}{P(x) * I(x)} \\ & = \sum_{x_i \in X}{P(x) * I(x)} \\ & = H(x) \end{split} \end{equation}\]

That’s it! H(x) is called the Shannon Entropy H(x):

\[H(x) = \mathbb{E}_{x \sim P}[I(x)] = \sum_{x_i \in X}{P(x) * I(x)}\]

Because the probability distribution P is so central to this measure we denote the shannon entropy H(P).

In our example: H(P) = P(Sun) * I(Sun) + P(Rain) * I(Rain) = 0.99 * log_2(1/0.99) + 0.01 * log_2(1/0.01) ~= 0.08

So when your friend calls you the information transmitted is 0.08 bits ;)

We observe that a low entropy means that we know what to expect when we draw an event from a channel. Whereas a low entropy means that there is a high uncertainty.

This is the reason why entropy is called the measure of surprise of a channel.

If P is a uniform distribution for example P(rain) = P(sun) = 0.5 then H(P) is at its maximum which is H(P) = 0.5 * log(1/0.5) + 0.5 * log(1/0.5) = 2 * (0.5 * log(1/0.5)) = 2 * (0.5 * log_2(2)) = 2 * (0.5 * 1) = 2 * 0.5 = 1

Cross entropy

Let’s start with the example again: Imagine that you also have a second friend who lives in the same town as the first friend. The 2nd friend also calls you to also tell you the weather every morning.

Now you are wondering: How many errors happen here? Both sequences should be the same, therefore the probability distributions P and Q should be equal. P is estimated from the sequence of the first friend and Q from the 2nd friend.

We can use the concept of entropy to compare two probability distributions P and Q over the same random variable X. (It has to be the same random variable, don’t get this confused with the conditional entropy which I explain in a later part of this post.)

Main idea: What is the entropy of the sequence of the first friend if we use the probability distribution Q of the 2nd friend to calculate it.

\[\begin{equation} \begin{split} H(P,Q) = \mathbb{E}_{x \sim P}[I_{Q}(x)] & = \mathbb{E}_{x \sim P}[log(1/Q(x))] \\ & = \sum_{x \in X}{P(x) \cdot log(\frac{1}{Q(x)})} \end{split} \end{equation}\]

(formula of cross entropy)

Let’s have a look at three examples to get a feeling for the cross entropy:

First Example: Let’s say that the 2nd friend reports less rain -> Q(Sun) = 0.999, Q(Rain) = 0.001 Then H(P,Q) = P(Sun) * log(1/Q(Sun)) + P(Rain) * log(1/Q(Rain)) = 0.99 * log_2(1/0.999) + 0.01 * log_2(1/0.001) ~= 0.101
Second Example: The friends tell you the same things -> P=Q $H(P,P) = \sum_{x \in X}{P(x) \cdot log(\frac{1}{P(x)})} = H(P)$
Third example: the 2nd friend reports more rain -> Q(Sun) = 0.981, Q(Rain) = 0.019 Then H(P,Q) = P(Sun) * log(1/Q(Sun)) + P(Rain) * log(1/Q(Rain)) = 0.99 * log_2(1/0.981) + 0.01 * log_2(1/0.019) ~= 0.085

Observations:

We observe that the cross entropy is equal to the entropy if P==Q.
The function is not symmetric
The higher the distance to the entropy, the more are the distributions divergent

Kullback-Leibler divergence

We have seen that the cross entropy always contains the entropy: $H(P,Q) = H(P) + divergence_of_P_and_Q$

This “divergence_of_P_and_Q” is used a lot and has its own name: Kullback-Leibler divergence $D_{KL}(P \parallel Q)$.

\[H(P,Q) = H(P) + D_{KL}(P \parallel Q)\]

Follows:

\[\begin{equation} \begin{split} D_{KL}(P \parallel Q) &= H(P,Q) - H(P) \\ & = \mathbb{E}_{x \sim P}[I_{Q}(x)] - \mathbb{E}_{x \sim P}[I_{P}(x)] \\ & =\mathbb{E}_{x \sim P}[I_{Q}(x) - I_{P}(x)] = \\ & =\mathbb{E}_{x \sim P}[log (1/Q(x)) - log (1/P(x))] = \\ & =\mathbb{E}_{x \sim P}[log (\frac{1/Q(x)}{1/P(x)})] = \\ & =\mathbb{E}_{x \sim P}[log (\frac{\frac{1}{Q(x)}}{\frac{1}{P(x)}})] = \\ & = \mathbb{E}_{x \sim P}[log \frac{P(x)}{Q(x)}] = \\ & =\mathbb{E}_{x \sim P}[log(P(x)) - log(Q(x))] \end{split} \end{equation}\]

The last term show that when P and Q are equal, the $D_{KL}(P \parallel Q)$ becomes zero.

Observations:

From the observation about the cross entropy we can conclude that the KL-divergence is also not symmetrical -> therefore it is per definition not a distance measure => $D_{KL}(P \parallel Q) \ne D_{KL}(Q \parallel P)$
$D_{KL}(P \parallel Q)$ is always positive
The Kullback-Leibler divergence has a nice intuition for probability distributions. Namely how many additional bits do we need to encode a message with the true probability distribution P if we are using a coding scheme that was developed using the prob. dist. Q

Log loss

In machine learning we often use something called the “log loss”. For example when we train a neural network for classification. In this case we usually have training data X and a one-hot-encoded label y. The network gets fed the data X. The data is feed forward through the network. As a last layer we often find the softmax function which outputs a vector that sums up to one and every value lies between 0 and 1. This is enough to count as a probability distribution (although we have not checked the sigma-addition).

The output of the network after the softmax is denoted ŷ as it is the estimator for the true label y. To train the neural network we would like to minimize the difference between y and ŷ.

We could just compare the two and say that the error is 1 if they are not the same and 0 if they are. But this would yield a step function which has the disadvantage of having no gradient at important places. That is bad because we need the gradient to perform gradient descent…

Therefore we treat y and ŷ as probability distributions and calculate the cross entropy between them.

\[H(Y, Ŷ) = \sum_{class ~ c}{Y(c) \cdot log( \frac{1}{Ŷ(x)} )}\]

Because for classification there is only one correct class, we know that:

For all classes which are not the true class c’: $Y(c)=0$
And for the true class c’: $Y(c') = 1$

\[\begin{equation} \begin{split} H(Y, Ŷ) &= Y(c^{'}) * log( \frac{1}{Ŷ(c^{'} )}) \\ &= 1 * log( \frac{1}{Ŷ(c^{'} )}) \\ &= log( \frac{1}{Ŷ(c^{'} )}) \\ &= -log(Ŷ(c^{'})) \end{split} \end{equation}\]

The last line is what we call log loss. It can be found everywhere. In tensorflow tf.losses.log_loss or in sckit sklearn.metrics.log_loss…

It has the advantage that we only have to calculate the log of a single value.

Conditional entropy

With the two mentioned friends you can be pretty sure to know the weather in the town. We will call the weather in the town “random variable X”.

Then there is this third friend of you :D She is selling ice cream in the same town. On a daily basis she informs you whether she sold ice cream today (“Ice”) or not (“No ice”). We call this random variable Y.

Now you are wondering: How much additional information is necessary to tell whether she sold Ice Cream if we have the weather information (X) from the other two friends?

This can be calculated using the conditional entropy H(Y|X). Speak: Y conditioned on X. Intuitively we can say, if the two events are independent of each others we have to store the entire sequence of Y as if X did not exist. If they are the same we don’t have to store anything additionally.

The formula:

\[\begin{equation} \begin{split} H(Y|X) &= \sum_{x \in X} {( P(x) \cdot \sum_{y \in Y}{P(y|x) \cdot log( \frac{1}{P(y|x)} )} )}\\ &= - \sum_{x \in X} ( P(x) \cdot \sum_{y \in Y}{P(y|x) \cdot log(P(y|x)} ) = \\ &= - \sum_{x \in X} (\sum_{y \in Y}{ P(x) P(y|x) \cdot log(P(y| x)} ) = \\ &= - \sum_{x \in X} (\sum_{y \in Y}{ P(x,y) \cdot log(P(y|x)} ) = \\ &= - \sum_{x \in X, y \in Y} P(x,y) \cdot logP(y|x) = \\ \end{split} \end{equation}\]

(conditional entropy)

observations:

$H(Y\vert X) = 0$, if H(Y)=H(X) -> “no additional information”
$H(Y\vert X) = H(Y)$, iff X and Y are independent random variables

Joint entropy

We can define a random variable Z which is the union of X and Y. P(Z) = P(X,Y)

According to this we can “extend” the entropy definition to:

\[H(Z) = H(X,Y) = \sum_{x \in X, y \in Y}{P(x,y) \cdot I(x,y)}\]

Mutual Information

The mutual information H(X;Y) is the entropy of the intersection of two random variables X and Y. It captures the dependence of two variables.

$I(X;Y) = MI(X;Y) = H(X) - H(X|Y)$ (Mutual Infor)

Mutual information maximization estimation (MMIE) can be used to train speech recognition systems.

Perplexity

$2^{H(P)}$ is called perplexity in speech recognition. It can be interpreted as the average branching factor in language models (for example a context-sensitive grammar). The lower the perplexity the less ambiguities have to be resolved by the acoustic model of the speech recognition system.

Entropy distance

In ASR we got introduced to the entropy distance and the weighted entropy distance. The context was the Kai-Fu Lee Clustering for subtriphones. For every subtriphone you have to train a semi-continuous HMM on the pool of Gaussians. Then we can use the mixture weights of the Gaussians as a probability distribution. Now we can calculate the entropy distance between two subtriphones to get a distance measure. The closest subtriphones are merged together. The big advantage was that we don’t have to train a new HMM for the merged cluster but we can just calculate the weighted sum of the two probability distributions.

The entropy distance: $d(P, Q) = H(P+Q) - 0.5 H(P) - 0.5 H(Q)$

The subtriphones differed in the amount of samples in the training data. We want to trust the clusters with fet data points less than the clusters with many data points. This is expressed in the weighted entropy distance:

\[d(P, Q) = (n_{P} + n_{Q})H(P+Q) - n_{P} H(P) - n_{Q} H(Q)\]

Information theory was originally published by Daniel Pollithy at Notes on April 27, 2019.