<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="http://blog.sharaal.de/feed.xml" rel="self" type="application/atom+xml" /><link href="http://blog.sharaal.de/" rel="alternate" type="text/html" /><updated>2026-04-21T07:25:43+00:00</updated><id>http://blog.sharaal.de/feed.xml</id><title type="html">Sharaal’s Blog</title><subtitle>Yet another Blog by and for AI Enthusiasts and Developers</subtitle><author><name>Sharaal</name></author><entry><title type="html">DragonTD: Performance Logging</title><link href="http://blog.sharaal.de/2026/04/21/dragontd-performance-logging.html" rel="alternate" type="text/html" title="DragonTD: Performance Logging" /><published>2026-04-21T00:00:00+00:00</published><updated>2026-04-21T00:00:00+00:00</updated><id>http://blog.sharaal.de/2026/04/21/dragontd-performance-logging</id><content type="html" xml:base="http://blog.sharaal.de/2026/04/21/dragontd-performance-logging.html"><![CDATA[<p>Um herauszufinden, ob es in DragonTD aktuell Performance-Probleme gibt, habe ich dafür das Logging erweitert. Es werden die wichtigsten Performance-Metriken während eines Spieldurchlaufs zusammen mit den bereits vorhandenen Playthrough-Daten geloggt.</p>

<!--more-->

<p>Link zum Spiel: <a href="https://dragontd.de">DragonTD (https://dragontd.de)</a></p>

<h2 id="vorgaben-an-das-rendering">Vorgaben an das Rendering</h2>

<p>Das Spiel hat rendertechnisch grundlegend die Vorgaben: 60 FPS zu erreichen, die Größe des verfügbaren Browserfensters zu nutzen, Fullscreen optional anzubieten für die Browserversion und immer im Format 16:9 zu sein.<br />
Für die Performance ist davon vor allem zunächst die Zielgröße von 60 FPS der wichtigste Parameter. Daher darf jeder Frame nicht länger als 16,67 ms dauern.</p>

<h2 id="metriken-für-die-performance">Metriken für die Performance</h2>

<p>Es gibt grundlegend folgende Metriken für jeden Spieldurchlauf:</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">frames</code>: Wie viele Frames wurden berechnet</li>
  <li><code class="language-plaintext highlighter-rouge">avgMs</code>: Wie lange hat ein Frame im Durchschnitt gedauert</li>
  <li><code class="language-plaintext highlighter-rouge">p95Ms</code> / <code class="language-plaintext highlighter-rouge">p99Ms</code>: Wie lange haben die schnellsten 95 % / 99 % der Frames jeweils im Durchschnitt gedauert</li>
  <li><code class="language-plaintext highlighter-rouge">maxMs</code>: Wie lange hat der langsamste Frame gedauert</li>
</ul>

<p>Diese Metriken sind zusammengenommen zunächst nur nützlich für die Entscheidung, wie gut die Performance ist.<br />
Wenn diese nicht gut ist, braucht es eine Aufteilung der Metriken auf verschiedene Bereiche und Zeitpunkte, um herauszufinden, wo das Problem liegen könnte.</p>

<h2 id="unterteilung-in-spielbereiche-und--phasen">Unterteilung in Spielbereiche und -phasen</h2>

<p>Denn jeder Frame wird in zwei Bereiche unterteilt:</p>
<ul>
  <li>Die Simulation berechnet alle Parameter der Spiellogik</li>
  <li>Das Rendering kümmert sich um die Anzeige</li>
</ul>

<p>Daher werden alle Metriken zweimal erhoben, jeweils unterteilt in Simulation und Rendering.</p>

<p>Des Weiteren gibt es verschiedene Spielphasen, nämlich die verschiedenen Maps und Waves, die großen Einfluss auf die Performance haben können. 
Zum Beispiel kann die Performance grundsätzlich gut sein, aber immer dann einbrechen, wenn ein bestimmter Tower mit einem bestimmten Effekt gebaut wird. Oder sie ist nur in einer bestimmten Welle schlecht, weil es dort einen Gegner mit einem bestimmten Effekt gibt.
Welche Tower gebaut werden, habe ich bereits in den Playthrough-Daten geloggt.<br />
Welche Wellen und Gegner es gibt und welche Effekte Tower und Gegner haben, ist in den Mod-Daten vorhanden.<br />
Was also noch fehlt, sind die Metriken, aufgesplittet nach den verschiedenen Maps und Waves.</p>

<h2 id="ergebnis">Ergebnis</h2>

<p>Und genau das ist nun das Performance-Logging:<br />
Fünf Metriken, jeweils für Simulation und Rendering, für den gesamten Durchlauf sowie unterteilt nach jeder Map und Wave.</p>

<p>Dazu gibt es auch ein kleines, schickes UI, das mir alle Spieldurchläufe auflistet und alle notwendigen Daten zur Analyse in die Zwischenablage kopiert, um anschließend zusammen mit Codex eine Analyse durchzuführen:</p>

<p><img src="/assets/2026-04-21-dragontd-performance-logging/images/dragontd-analytics.png" alt="DragonTD Analytics" /></p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Um herauszufinden, ob es in DragonTD aktuell Performance-Probleme gibt, habe ich dafür das Logging erweitert. Es werden die wichtigsten Performance-Metriken während eines Spieldurchlaufs zusammen mit den bereits vorhandenen Playthrough-Daten geloggt.]]></summary></entry><entry><title type="html">Neustart für den Blog mit DragonTD</title><link href="http://blog.sharaal.de/2026/04/20/restarting-the-blog-with-dragontd.html" rel="alternate" type="text/html" title="Neustart für den Blog mit DragonTD" /><published>2026-04-20T00:00:00+00:00</published><updated>2026-04-20T00:00:00+00:00</updated><id>http://blog.sharaal.de/2026/04/20/restarting-the-blog-with-dragontd</id><content type="html" xml:base="http://blog.sharaal.de/2026/04/20/restarting-the-blog-with-dragontd.html"><![CDATA[<p>Ich hatte mich letztes Jahr bewusst dazu entschieden, beim AI-Slop rund um News nicht weiter mitzumachen. Deshalb habe ich alle KI-generierten Artikel dieses Blogs ins Archiv verschoben und die automatische Erstellung neuer Beiträge abgeschaltet.</p>

<p>Dadurch ist der Blog insgesamt allerdings auch wieder eingeschlafen.</p>

<p>Seit ein paar Wochen arbeite ich nun an einem neuen privaten Projekt: einem Tower-Defense-Spiel namens DragonTD. Mit diesem Projekt, meinem vierten Spiel, möchte ich auch meinen Blog wiederbeleben und künftig vor allem über die Entwicklung und die technischen Erkenntnisse daraus schreiben.</p>

<!--more-->

<p>DragonTD begann als kleine Tech-Demo an einem Wochenende, ausgelöst durch den Release der Codex App und einige Runden Bloons TD 6, die zusammengenommen bei mir direkt Lust geweckt haben, so etwas selbst auszuprobieren. Die aktuelle Live Version kann man sich hier anschauen: <a href="https://dragontd.de">DragonTD (https://dragontd.de)</a>.</p>

<p>Danach war das Ziel, innerhalb der ersten Woche eine spielbare Version zu bauen. Diese Version kann man sich hier anschauen: <a href="https://releases.dragontd.de/v1.0/">DragonTD v1.0 (https://releases.dragontd.de/v1.0/)</a>.</p>

<p>Inzwischen entwickelt sich das Ganze zunehmend zu einem vollwertigen Spiel. Es ist auch längst kein Solo-Projekt mehr, hat das Ziel, auf Steam, im Play Store und im Apple App Store veröffentlicht zu werden, und es gibt auch schon konkrete Pläne für größere Features, mit denen es sich klar von der Konkurrenz abheben soll.</p>

<p>Für dieses Spiel habe ich am Wochenende einige sehr intensive Exkurse in Themen wie Performance-Analyse, Rendering in Phaser und UI-Layout-Systeme gemacht. Dabei sind für mich viele neue Erkenntnisse entstanden.</p>

<p>Genau das möchte ich nun zum Anlass nehmen, diesen Blog wiederzubeleben und diese Erkenntnisse zu teilen - auch innerhalb meiner Firma. Die Themen könnten für einige interessant sein, sind für klassische Tech Talks aber vermutlich zu speziell und zu weit von meinem direkten Firmenkontext entfernt. Für meinen Blog hier ist das aber genau der richtige Rahmen.</p>

<p>Außerdem werde ich den Blog hoffentlich zeitnah auch technisch wieder auf einen aktuellen Stand bringen. Der Neustart betrifft also nicht nur neue Inhalte, sondern auch die technische Basis.</p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Ich hatte mich letztes Jahr bewusst dazu entschieden, beim AI-Slop rund um News nicht weiter mitzumachen. Deshalb habe ich alle KI-generierten Artikel dieses Blogs ins Archiv verschoben und die automatische Erstellung neuer Beiträge abgeschaltet. Dadurch ist der Blog insgesamt allerdings auch wieder eingeschlafen. Seit ein paar Wochen arbeite ich nun an einem neuen privaten Projekt: einem Tower-Defense-Spiel namens DragonTD. Mit diesem Projekt, meinem vierten Spiel, möchte ich auch meinen Blog wiederbeleben und künftig vor allem über die Entwicklung und die technischen Erkenntnisse daraus schreiben.]]></summary></entry><entry><title type="html">Die Zukunft der Arbeitswelt: Chancen und Herausforderungen durch KI</title><link href="http://blog.sharaal.de/2025/06/20/die-zukunft-der-arbeitswelt-chancen-und-herausforderungen-durch-ki.html" rel="alternate" type="text/html" title="Die Zukunft der Arbeitswelt: Chancen und Herausforderungen durch KI" /><published>2025-06-20T00:00:00+00:00</published><updated>2025-06-20T00:00:00+00:00</updated><id>http://blog.sharaal.de/2025/06/20/die-zukunft-der-arbeitswelt-chancen-und-herausforderungen-durch-ki</id><content type="html" xml:base="http://blog.sharaal.de/2025/06/20/die-zukunft-der-arbeitswelt-chancen-und-herausforderungen-durch-ki.html"><![CDATA[<p>Die Arbeitswelt wandelt sich durch KI: Prozesse werden automatisiert, Arbeitsplätze ersetzt und neue Chancen entstehen. Dieser Artikel beleuchtet, wie KI die Arbeitswelt verändert und welche Chancen und Herausforderungen sich daraus ergeben.</p>

<!--more-->

<h2 id="die-realität-aktuell-ki-ersetzt-bereits-arbeitsplätze">Die Realität aktuell: KI ersetzt bereits Arbeitsplätze</h2>

<p>KI verändert bereits die Arbeitswelt:</p>
<ul>
  <li>Klarna setzt KI-Chatbots ein, um Kundenanfragen zu bearbeiten, was bereits zur Einsparung von 700 Arbeitsplätzen geführt hat. [1]</li>
  <li>Shopify hat im April 2025 eine neue Richtlinie eingeführt, die KI-Kompetenz als Grundvoraussetzung für alle Neueinstellungen festlegt und neue Stellen nur besetzt werden, wenn erklärt wurde, warum KI dafür nicht ausreicht. [2]</li>
</ul>

<h2 id="ist-dieser-trend-eigentlich-schlimm">Ist dieser Trend eigentlich schlimm?</h2>

<p>Der Wandel hat auch positive Seiten: KI kann monotone Arbeit ersetzen und Raum für erfüllende Tätigkeiten schaffen. Beispiele wie AlphaFold [3] zeigen zudem gesellschaftliche Vorteile, etwa in der Medizin.</p>

<h2 id="auswirkungen-auf-den-geldfluss">Auswirkungen auf den Geldfluss</h2>

<p>Der traditionelle Geldfluss basiert auf einem Kreislauf: Unternehmen erwirtschaften Gewinne, zahlen Gehälter an ihre Mitarbeiter, die dieses Geld für Konsum ausgeben. Dieser Konsum wiederum stärkt die Unternehmen.</p>

<p><img src="/assets/2025-06-20-die-zukunft-der-arbeitswelt-chancen-und-herausforderungen-durch-ki/images/aktueller-geldfluss.png" alt="Aktueller Geldfluss" /></p>

<p>Wenn jedoch Arbeitsplätze durch KI ersetzt werden, droht dieser Kreislauf ins Stocken zu geraten.</p>

<h2 id="mögliche-lösung-bedingungsloses-grundeinkommen">Mögliche Lösung: Bedingungsloses Grundeinkommen</h2>

<p>Ein bedingungsloses Grundeinkommen könnte dies durch Umverteilung von Unternehmensgewinnen stabilisieren. Unternehmen zahlen ein Teil der Gewinne an den Staat. Der Staat könnte dieses Geld dann an die Bürger umverteilen, um deren Kaufkraft zu sichern.</p>

<p><img src="/assets/2025-06-20-die-zukunft-der-arbeitswelt-chancen-und-herausforderungen-durch-ki/images/bedingungsloses-grundeinkommen.png" alt="Bedingungsloses Grundeinkommen" /></p>

<p>Die Konzentration von Technologie- und KI-Unternehmen in wenigen Ländern wie den USA und China verschärft dabei Ungleichheiten durch die Abwanderung der Gewinne.</p>

<h2 id="die-zeit-des-wandels-als-chance-für-arbeitnehmer">Die Zeit des Wandels als Chance für Arbeitnehmer</h2>

<p>Der KI-Wandel bietet aber auch Chancen: Neue Berufe wie KI-Automation Manager oder Prompt Engineer entstehen. Aktuelle Berufe können durch KI effizienter und weniger repetativ werden. Wer KI versteht und integriert, wird zum unverzichtbaren Wegbereiter für die Zukunft des eigenen Unternehmens. Es lohnt sich daher, die Möglichkeiten von KI aktiv zu erkunden und für die eigene Karriere zu nutzen.</p>

<h2 id="fazit">Fazit</h2>

<p>KI wird die Arbeitswelt grundlegend verändern. Während Arbeitsplätze wegfallen, entstehen neue Chancen für diejenigen, die sich auf den Wandel einlassen. Gleichzeitig sind Lösungen wie das Grundeinkommen und internationale Abkommen nötig, um soziale und wirtschaftliche Stabilität zu gewährleisten. Mit den richtigen Maßnahmen können wir die Chancen von KI nutzen, um eine innovative und gerechte Arbeitswelt zu schaffen.</p>

<p>Quellen:</p>
<ul>
  <li>Inspiriert durch: <a href="https://www.youtube.com/watch?v=Et_zWHih7zo">Analyse: KI ersetzt also DOCH Menschen? - The Morpheus</a></li>
  <li>[1] <a href="https://www.zdfheute.de/wirtschaft/kuenstliche-intelligenz-wirtschaft-klarna-arbeitsplaetze-kundenservice-100.html">Wo KI schon heute Menschen im Job ersetzt - zdfheute</a></li>
  <li>[2] <a href="https://www.heise.de/news/Shopify-CEO-setzt-auf-KI-statt-Neueinstellungen-und-mehr-Ressourcen-10343727.html">Shopify-CEO hat “fundamentale Erwartung”, dass Angestellte KI benutzen - Heise</a></li>
  <li>[3] <a href="https://www.heise.de/news/AlphaFold-3-Googles-KI-sagt-Struktur-aller-Molekuele-voraus-9713361.html">AlphaFold 3: Googles KI sagt Struktur aller Moleküle voraus - Heise</a></li>
</ul>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Die Arbeitswelt wandelt sich durch KI: Prozesse werden automatisiert, Arbeitsplätze ersetzt und neue Chancen entstehen. Dieser Artikel beleuchtet, wie KI die Arbeitswelt verändert und welche Chancen und Herausforderungen sich daraus ergeben.]]></summary></entry><entry><title type="html">Daily AI News GitHub Action With Langdock API</title><link href="http://blog.sharaal.de/2025/06/14/daily-ai-news-github-action-with-langdock-api.html" rel="alternate" type="text/html" title="Daily AI News GitHub Action With Langdock API" /><published>2025-06-14T00:00:00+00:00</published><updated>2025-06-14T00:00:00+00:00</updated><id>http://blog.sharaal.de/2025/06/14/daily-ai-news-github-action-with-langdock-api</id><content type="html" xml:base="http://blog.sharaal.de/2025/06/14/daily-ai-news-github-action-with-langdock-api.html"><![CDATA[<p>Ich wollte etwas Produktives mit der Langdock-API in einem GitHub-Action-Workflow machen. Da ich mich für Neuigkeiten rund um KI interessiere, habe ich beides einfach verbunden, um KI-generierte Artikel für meinen Blog erstellen zu lassen.</p>

<!--more-->

<p>Anfangs habe ich einen Assistenten in Langdock erstellt, der für einen bestimmten Zeitraum (z.B. heute) und ein Thema die aktuellen News zusammensucht. Dieser war in einer GitHub Action eingebunden, die die Ausgabe dann direkt als neuen Artikel veröffentlicht hat. Das Prompt des Assistenten habe ich mir ebenfalls von der KI generieren lassen. Es war sehr umfangreich und vielversprechend.
Die daraus entstandenen News-Artikel waren aber dennoch relativ schlecht. Zum einen waren die Themen nicht nur von heute, haben sich also über mehrere Tage wiederholt, und zum anderen war der gemischte Artikel über mehrere Themen nicht sehr ausführlich.</p>

<p>Also habe ich den Workflow wie folgt geändert:</p>
<ul>
  <li>Es werden alle Titel der bereits existierenden Posts und Drafts eingelesen.</li>
  <li>Es wird ein Request an die Langdock-API geschickt, der nur die Titel der drei wichtigsten heutigen Neuigkeiten zurückgibt und im Prompt auch alle existierenden Titel samt der Anweisung, diese zu ignorieren, mitgibt.</li>
  <li>Für jeden dieser Titel wird ein eigener Request geschickt, um einen umfassenden News-Artikel zu schreiben und die Quellenangaben am Ende hinzuzufügen.</li>
  <li>Statt diese direkt als Posts zu veröffentlichen, werden sie als Drafts gespeichert.</li>
  <li>Das Überprüfen, Bereinigen und Veröffentlichen (durch das Verschieben aus dem _drafts- ins _posts-Verzeichnis) wird dann von Hand gemacht.</li>
</ul>

<p>Dabei gab es auch einige neue Herausforderungen:</p>
<ul>
  <li>Da ich die Websuche für aktuelle News benötige, funktioniert die Chat-API nicht und es muss ein temporärer Assistent verwendet werden.</li>
  <li>Da die Ausgabe immer KI-generierter Text ist, ist es nicht möglich, wirklich strukturierte Daten (wie ein valides JSON-Objekt) zurückzubekommen. Man kann dem Prompt zwar mitgeben, “gib mir ein valides JSON-Objekt zurück”, aber die KI hat in der Ausgabe dann dennoch Einleitungstext oder Backtick-Blöcke hinzugefügt.</li>
  <li>Dadurch musste die Ausgabe umfangreicher geparst werden. Also habe ich kein JSON zurückgeben lassen, sondern eine Liste der Titel, bei der jeder Titel mit “- “ startet.</li>
  <li>Dieses Verhalten ist ebenfalls bei den Artikeln wiederzufinden, wo die KI einleitende oder abschließende Worte hinzufügt, die manuell entfernt werden müssen.</li>
</ul>

<p>Die fehlende Möglichkeit, strukturierte JSON-Objekte zurückgeben zu lassen, ist schade für eine API. Dennoch sind der Workflow und die daraus resultierenden Artikel nun deutlich sinnvoller, und der manuelle Schritt des Reviews und der Veröffentlichung fügt dem Ganzen noch ein “Quality Gate” hinzu.</p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Ich wollte etwas Produktives mit der Langdock-API in einem GitHub-Action-Workflow machen. Da ich mich für Neuigkeiten rund um KI interessiere, habe ich beides einfach verbunden, um KI-generierte Artikel für meinen Blog erstellen zu lassen.]]></summary></entry><entry><title type="html">SQL Workshop</title><link href="http://blog.sharaal.de/2019/06/06/sql-workshop.html" rel="alternate" type="text/html" title="SQL Workshop" /><published>2019-06-06T00:00:00+00:00</published><updated>2019-06-06T00:00:00+00:00</updated><id>http://blog.sharaal.de/2019/06/06/sql-workshop</id><content type="html" xml:base="http://blog.sharaal.de/2019/06/06/sql-workshop.html"><![CDATA[<p>Because of security issues I did a very small SQL workshop in my company focused on SQL Injections.</p>

<p>So I explain them what they are, what the issues are and how to ensure not having any issues with them. Also some more topics in a rough walkthrough like proper database design and some best practices.</p>

<!--more-->

<h2 id="sql-injection">SQL Injection</h2>

<p>SQL Injection means to change the expected behaviour of a SQL query by sending parts of the SQL query instead of only values and can always be done if the values are not right escaped. It’s the number one reason of all security problems out in the world.</p>

<p>e.g. Update email of an user:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">app</span><span class="p">.</span><span class="nx">post</span><span class="p">(</span><span class="dl">'</span><span class="s1">/users/email</span><span class="dl">'</span><span class="p">,</span> <span class="k">async</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">email</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">email</span>
  <span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">user</span><span class="p">.</span><span class="nx">id</span>
  <span class="k">await</span> <span class="nx">db</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span><span class="s2">`UPDATE users SET email = '</span><span class="p">${</span><span class="nx">email</span><span class="p">}</span><span class="s2">' WHERE id = </span><span class="p">${</span><span class="nx">id</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
  <span class="nx">res</span><span class="p">.</span><span class="nx">sendStatus</span><span class="p">(</span><span class="mi">204</span><span class="p">)</span>
<span class="p">})</span>
</code></pre></div></div>

<p>The expected behaviour if there is a post which contains:</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"email"</span><span class="p">:</span><span class="w"> </span><span class="s2">"sharaal@example.com"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>works fine.</p>

<p>But what happens if there is an attacker trying some injection by sending:</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"email"</span><span class="p">:</span><span class="w"> </span><span class="s2">"attacker@example.com' WHERE id = 1 --"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>In this case the SQL which is executed looks like:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">UPDATE</span> <span class="n">users</span> <span class="k">SET</span> <span class="n">email</span> <span class="o">=</span> <span class="s1">'attacker@example.com'</span> <span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="mi">1</span> <span class="c1">--' WHERE id = 5</span>
</code></pre></div></div>

<p>And if the user with the ID 1 is an admin, the attacker can to a password forget with his own email address and took successfully over these account.</p>

<h3 id="parameters">Parameters</h3>

<p>To solve that kind of security issues all values which are not hardcoded in the SQL query needs to be escaped. DB drivers should provide own solution for that, so it’s recommended to use that and never building an own solution.</p>

<p>For PostgreSQL parameters are the way to go: using placeholders in the SQL query, send the values separately and the database will handle them securely.</p>

<p>e.g. Update email of an user:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">app</span><span class="p">.</span><span class="nx">post</span><span class="p">(</span><span class="dl">'</span><span class="s1">/users/email</span><span class="dl">'</span><span class="p">,</span> <span class="k">async</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">email</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">email</span>
  <span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">user</span><span class="p">.</span><span class="nx">id</span>
  <span class="k">await</span> <span class="nx">db</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
    <span class="s2">`UPDATE users SET email = $1 WHERE id = $2`</span><span class="p">,</span>
    <span class="p">[</span><span class="nx">email</span><span class="p">,</span> <span class="nx">id</span><span class="p">]</span>
  <span class="p">)</span>
  <span class="nx">res</span><span class="p">.</span><span class="nx">sendStatus</span><span class="p">(</span><span class="mi">204</span><span class="p">)</span>
<span class="p">})</span>
</code></pre></div></div>

<p>That’s all. Follow these and never have security issues because of SQL Injections.</p>

<h3 id="identifiers-tablecolumn-names">Identifiers (table/column names)</h3>

<p>Because of technical reasons (changed query paths), placeholders we use for values are not allowed for identifiers. Also here different databases has different solutions to solve these, some just don’t have any.</p>

<p>PostgreSQL is one of the database don’t have a solution, but a well documented format of identifiers, so we can easily write an own escape function:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">escapeIdentifier</span> <span class="p">(</span><span class="nx">identifier</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">return</span> <span class="s2">`"</span><span class="p">${</span><span class="nx">identifier</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/"/g</span><span class="p">,</span> <span class="dl">'</span><span class="s1">""</span><span class="dl">'</span><span class="p">)}</span><span class="s2">"`</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Every <code class="language-plaintext highlighter-rouge">"</code> will be replaced by <code class="language-plaintext highlighter-rouge">""</code> to be escaped an the identifier is surrounded by <code class="language-plaintext highlighter-rouge">"</code>.</p>

<p>So if the table name should be dynamically in the user email example, it will looks like:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">tableName</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">users</span><span class="dl">'</span>

<span class="nx">app</span><span class="p">.</span><span class="nx">post</span><span class="p">(</span><span class="dl">'</span><span class="s1">/users/email</span><span class="dl">'</span><span class="p">,</span> <span class="k">async</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">email</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">email</span>
  <span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">user</span><span class="p">.</span><span class="nx">id</span>
  <span class="k">await</span> <span class="nx">db</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
    <span class="s2">`UPDATE </span><span class="p">${</span><span class="nx">escapeIdentifier</span><span class="p">(</span><span class="nx">tableName</span><span class="p">)}</span><span class="s2"> SET email = $1 WHERE id = $2`</span><span class="p">,</span>
    <span class="p">[</span><span class="nx">email</span><span class="p">,</span> <span class="nx">id</span><span class="p">]</span>
  <span class="p">)</span>
  <span class="nx">res</span><span class="p">.</span><span class="nx">sendStatus</span><span class="p">(</span><span class="mi">204</span><span class="p">)</span>
<span class="p">})</span>
</code></pre></div></div>

<h2 id="additional-solution-sql-tagged-template-literal">Additional Solution: SQL Tagged Template Literal</h2>

<p>With ES6 tagged template literals was introduced and can be used to build sql queries. The user email example using them would look like:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">tableName</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">users</span><span class="dl">'</span>

<span class="nx">app</span><span class="p">.</span><span class="nx">post</span><span class="p">(</span><span class="dl">'</span><span class="s1">/users/email</span><span class="dl">'</span><span class="p">,</span> <span class="k">async</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">email</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">email</span>
  <span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="nx">req</span><span class="p">.</span><span class="nx">user</span><span class="p">.</span><span class="nx">id</span>
  <span class="k">await</span> <span class="nx">sql</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
    <span class="nx">sql</span><span class="s2">`UPDATE </span><span class="p">${</span><span class="nx">sql</span><span class="p">.</span><span class="nx">key</span><span class="p">(</span><span class="nx">tableName</span><span class="p">)}</span><span class="s2"> SET email = </span><span class="p">${</span><span class="nx">email</span><span class="p">}</span><span class="s2"> WHERE id = </span><span class="p">${</span><span class="nx">id</span><span class="p">}</span><span class="s2">`</span>
  <span class="p">)</span>
  <span class="nx">res</span><span class="p">.</span><span class="nx">sendStatus</span><span class="p">(</span><span class="mi">204</span><span class="p">)</span>
<span class="p">})</span>
</code></pre></div></div>

<p>The sql tag handles the splitting of the query and the values automatically and provide also a method to escape identifiers together with a lot more convenience methods. Using this kind of solution does it makes ensures SQL Injections are not possible to have.</p>

<h2 id="database-design">Database Design</h2>

<h3 id="id">ID</h3>

<ul>
  <li>Always have a serial ID column named <code class="language-plaintext highlighter-rouge">id</code></li>
</ul>

<h3 id="timestamps">Timestamps</h3>

<ul>
  <li>Always have a <code class="language-plaintext highlighter-rouge">created_at</code> column automatically filled with the current timestamp if inserting new rows and a <code class="language-plaintext highlighter-rouge">updated_at</code> column automatically filled with the current timestamp if inserting new rows or updating existing ones</li>
</ul>

<p><code class="language-plaintext highlighter-rouge">updated_at</code> needs a special solution for PostgreSQL:</p>

<p>Custom function:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">OR</span> <span class="k">REPLACE</span> <span class="k">FUNCTION</span> <span class="n">on_update_timestamp</span><span class="p">()</span>
  <span class="k">RETURNS</span> <span class="k">trigger</span> <span class="k">AS</span> <span class="err">$$</span>
  <span class="k">BEGIN</span>
    <span class="k">NEW</span><span class="p">.</span><span class="n">updated_at</span> <span class="o">=</span> <span class="n">now</span><span class="p">();</span>
    <span class="k">RETURN</span> <span class="k">NEW</span><span class="p">;</span>
  <span class="k">END</span><span class="p">;</span>
  <span class="err">$$</span> <span class="k">language</span> <span class="s1">'plpgsql'</span>
</code></pre></div></div>

<p>Trigger for every table, e.g. users:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TRIGGER</span> <span class="n">users_updated_at</span>
  <span class="k">BEFORE</span> <span class="k">UPDATE</span> <span class="k">ON</span> <span class="n">users</span>
  <span class="k">FOR</span> <span class="k">EACH</span> <span class="k">ROW</span>
  <span class="k">EXECUTE</span> <span class="k">PROCEDURE</span> <span class="n">on_update_timestamp</span><span class="p">()</span>
</code></pre></div></div>

<h3 id="indices">Indices</h3>

<ul>
  <li>All columns used in joins and where conditions should be indexed</li>
  <li>Combined indexes needs care about the order of the including columns</li>
</ul>

<h3 id="relations">Relations</h3>

<ul>
  <li>Always use INNER JOIN if OUTER JOIN is not explicit needed</li>
  <li>Always use the ON of the JOINs for the foreign keys instead of compare them in the WHERE</li>
</ul>

<p>Different relations:</p>
<ul>
  <li>1:1: related table has an unique foreign key to the host table</li>
  <li>1:n: related table has a foreign key to the host table</li>
  <li>m:n: relation table with both IDs of the related tables</li>
</ul>

<h2 id="best-practices">Best Practices</h2>

<h3 id="uppercased-keywords--lowercased-identifier">Uppercased Keywords / Lowercased Identifier</h3>

<p>Easier to read if used consequent independently if syntax highlighting is available (e.g. in logs). e.g.:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- bad</span>
<span class="k">select</span> <span class="o">*</span> <span class="k">from</span> <span class="n">users</span>

<span class="c1">-- good</span>
<span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">users</span>
</code></pre></div></div>

<h3 id="always-use-lowercased-identifier">Always use lowercased identifier</h3>

<p>Even if it’s possible, it’s hard to work with identifiers having uppercased characters, e.g.:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- bad</span>
<span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="nv">"Users"</span>

<span class="c1">-- good</span>
<span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">users</span>
</code></pre></div></div>

<h3 id="never-insert-into-without-columns">Never INSERT INTO without columns</h3>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- bad</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">users</span> <span class="k">VALUES</span>
  <span class="p">(</span><span class="k">null</span><span class="p">,</span> <span class="s1">'sharaal@example.com'</span><span class="p">,</span> <span class="k">CURRENT_TIMESTAMP</span><span class="p">,</span> <span class="k">CURRENT_TIMESTAMP</span><span class="p">)</span>

<span class="c1">-- Not possible to omit columns with default values</span>
<span class="c1">-- It will break if there are changes in the table schema</span>

<span class="c1">-- good</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">users</span> <span class="p">(</span><span class="n">email</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="s1">'sharaal@example.com'</span><span class="p">)</span>
</code></pre></div></div>

<h3 id="split-migrations-and-service-technologies">Split Migrations and Service Technologies</h3>

<ul>
  <li>Evaluate and use solutions for migrations and querying in services separately</li>
  <li>e.g. <code class="language-plaintext highlighter-rouge">knex</code> for migrations, <code class="language-plaintext highlighter-rouge">pg</code> for querying in services</li>
  <li>Think about query builder if the queries become more complex with the native <code class="language-plaintext highlighter-rouge">pg</code> use</li>
</ul>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Because of security issues I did a very small SQL workshop in my company focused on SQL Injections. So I explain them what they are, what the issues are and how to ensure not having any issues with them. Also some more topics in a rough walkthrough like proper database design and some best practices.]]></summary></entry><entry><title type="html">Semicolons in JavaScript are optional?</title><link href="http://blog.sharaal.de/2019/03/28/semicolons-in-javascript-are-optional.html" rel="alternate" type="text/html" title="Semicolons in JavaScript are optional?" /><published>2019-03-28T00:00:00+00:00</published><updated>2019-03-28T00:00:00+00:00</updated><id>http://blog.sharaal.de/2019/03/28/semicolons-in-javascript-are-optional</id><content type="html" xml:base="http://blog.sharaal.de/2019/03/28/semicolons-in-javascript-are-optional.html"><![CDATA[<p>I won’t make a suggestion for this nearly religion discussion using or omitting semicolons in JavaScript. But I experienced they are not always optional and omitting them can result in errors. So I will describe these scenarios and the workarounds.</p>

<!--more-->

<h2 id="error-scenarios">Error scenarios</h2>

<h3 id="immediately-executing-anonymous-function">Immediately executing anonymous function</h3>

<p>First the example:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">example</span><span class="dl">'</span><span class="p">)</span>
<span class="p">(</span><span class="k">async</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{})()</span>
</code></pre></div></div>

<p>This results in the error <code class="language-plaintext highlighter-rouge">TypeError: console.log(...) is not a function</code>.</p>

<p>This happens because no semicolon will be automatically inserted and the interpreter try to run the code as it would be written like:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">example</span><span class="dl">'</span><span class="p">)(</span><span class="k">async</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{})();</span>
</code></pre></div></div>

<p>And the return value of <code class="language-plaintext highlighter-rouge">console.log</code> isn’t a function.</p>

<h3 id="destructioning">Destructioning</h3>

<p>First the example:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">a</span><span class="p">,</span> <span class="nx">b</span>
<span class="kd">const</span> <span class="nx">c</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="p">[</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">]</span> <span class="o">=</span> <span class="nx">c</span>
</code></pre></div></div>

<p>This results in the error <code class="language-plaintext highlighter-rouge">ReferenceError: c is not defined</code>.</p>

<p>This happens because no semicolon will be automatically inserted and the interpreter try to run the code as it would be written like:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">c</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">][</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">]</span> <span class="o">=</span> <span class="nx">c</span>
</code></pre></div></div>

<p>And <code class="language-plaintext highlighter-rouge">c</code> is not defined at this point.</p>

<h2 id="workarounds">Workarounds</h2>

<p>In both error scenarios the <a href="http://inimino.org/~inimino/blog/javascript_semicolons">automatically insertion of the semicolons</a> doesn’t work, because the line isn’t a completed sequence. So we need to add the semicolons manually between the two lines, at the end of the first line (which collide with the rule to omit semicolons in the linters) or at the start of the second line.</p>

<p>So let’s have a look at the two error scenarios which works with a manually added semicolon.</p>

<p>Immediately executing anonymous function:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">example</span><span class="dl">'</span><span class="p">)</span>
<span class="p">;(</span><span class="k">async</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{})()</span>
</code></pre></div></div>

<p>Destructioning:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">a</span><span class="p">,</span> <span class="nx">b</span>
<span class="kd">const</span> <span class="nx">c</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="p">;[</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">]</span> <span class="o">=</span> <span class="nx">c</span>
</code></pre></div></div>

<h2 id="summary">Summary</h2>

<p>Semicolons are not always optional, but with the rule of thumb <code class="language-plaintext highlighter-rouge">start lines with ; when they start with [ or (</code> it’s easy to avoid the error scenarios and be able to add the rule to omitting semicolons at the end of the lines.</p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[I won’t make a suggestion for this nearly religion discussion using or omitting semicolons in JavaScript. But I experienced they are not always optional and omitting them can result in errors. So I will describe these scenarios and the workarounds.]]></summary></entry><entry><title type="html">Knex vs alternatives</title><link href="http://blog.sharaal.de/2019/03/12/knex-vs-alternatives.html" rel="alternate" type="text/html" title="Knex vs alternatives" /><published>2019-03-12T00:00:00+00:00</published><updated>2019-03-12T00:00:00+00:00</updated><id>http://blog.sharaal.de/2019/03/12/knex-vs-alternatives</id><content type="html" xml:base="http://blog.sharaal.de/2019/03/12/knex-vs-alternatives.html"><![CDATA[<p>Working in code with a relational database can be done in different abstraction forms from writing SQL queries over using a query builder to completely abstract the type of the database by using an ORM.
But it’s not only querying a database, also migration scripts needs to be defined and managed.</p>

<p>So let’s have a critical look at <a href="https://knexjs.org/">knex</a>, where is it worth to use, where not and what are the alternatives.</p>

<!--more-->

<h2 id="migrations">Migrations</h2>

<p>Here <code class="language-plaintext highlighter-rouge">knex</code> really shines. The setup with the <code class="language-plaintext highlighter-rouge">knexfile.js</code> and the commands are easy and the migrations with the definition of the schema table and their changes are readable and works smoothly. So there is no reason not to use it.</p>

<p>An example <code class="language-plaintext highlighter-rouge">knexfile.js</code>:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
  <span class="na">client</span><span class="p">:</span> <span class="dl">'</span><span class="s1">pg</span><span class="dl">'</span><span class="p">,</span>
  <span class="na">connection</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">DATABASE_URL</span>
<span class="p">}</span>
</code></pre></div></div>

<p>The commands used via <code class="language-plaintext highlighter-rouge">npx</code>:</p>
<ul>
  <li>Create a new migration file: <code class="language-plaintext highlighter-rouge">npx knex migrate:make migration_name</code>
Example migration (for more schema documentation see <a href="https://knexjs.org/#Schema">https://knexjs.org/#Schema</a>):
    <div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span><span class="p">.</span><span class="nx">up</span> <span class="o">=</span> <span class="nx">knex</span> <span class="o">=&gt;</span> <span class="nx">knex</span><span class="p">.</span><span class="nx">schema</span><span class="p">.</span><span class="nx">createTable</span><span class="p">(</span><span class="dl">'</span><span class="s1">users</span><span class="dl">'</span><span class="p">,</span> <span class="nx">table</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">table</span><span class="p">.</span><span class="nx">increments</span><span class="p">()</span>

  <span class="nx">table</span><span class="p">.</span><span class="nx">string</span><span class="p">(</span><span class="dl">'</span><span class="s1">email</span><span class="dl">'</span><span class="p">).</span><span class="nx">notNullable</span><span class="p">()</span>
  <span class="nx">table</span><span class="p">.</span><span class="nx">unique</span><span class="p">(</span><span class="dl">'</span><span class="s1">email</span><span class="dl">'</span><span class="p">)</span>

  <span class="nx">table</span><span class="p">.</span><span class="nx">string</span><span class="p">(</span><span class="dl">'</span><span class="s1">passwordhash</span><span class="dl">'</span><span class="p">).</span><span class="nx">notNullable</span><span class="p">()</span>
<span class="p">})</span>

<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span><span class="p">.</span><span class="nx">down</span> <span class="o">=</span> <span class="nx">knex</span> <span class="o">=&gt;</span> <span class="nx">knex</span><span class="p">.</span><span class="nx">schema</span><span class="p">.</span><span class="nx">dropTable</span><span class="p">(</span><span class="dl">'</span><span class="s1">users</span><span class="dl">'</span><span class="p">)</span>
</code></pre></div>    </div>
  </li>
  <li>Migrate to the latest schema: <code class="language-plaintext highlighter-rouge">npx knex migrate:latest</code></li>
</ul>

<p>For more commands and their descriptions see: <a href="https://knexjs.org/#Migrations-CLI">https://knexjs.org/#Migrations-CLI</a></p>

<h2 id="query-builder">Query builder</h2>

<p>That’s the part where the discussions and the cons of <code class="language-plaintext highlighter-rouge">knex</code> start. As base for discussions it’s good to have a look at this article <a href="https://medium.com/@gajus/stop-using-knex-js-and-earn-30-bf410349856c">https://medium.com/@gajus/stop-using-knex-js-and-earn-30-bf410349856c</a> (thx for <a href="https://github.com/gajus">gajus</a> for the article and his library <a href="https://www.npmjs.com/package/slonik">slonik</a> which inspired me to spend time into this topic).</p>

<p>The cons which are well described:</p>
<ul>
  <li>SQL is a well known language, the syntax of <code class="language-plaintext highlighter-rouge">knex</code> needs to be learned on top of that. That makes it hard to think / prototype in plain SQL and then switch to the <code class="language-plaintext highlighter-rouge">knex</code> query building syntax. Also it makes it hard to read later on</li>
  <li><code class="language-plaintext highlighter-rouge">knex</code> is NOT an abstraction layer. It’s not easily possible to move from e.g. <code class="language-plaintext highlighter-rouge">sqlite</code> to <code class="language-plaintext highlighter-rouge">pg</code> because the API of <code class="language-plaintext highlighter-rouge">knex</code> is different, e.g.:
    <ul>
      <li>Insert statement returning the incremental via <code class="language-plaintext highlighter-rouge">.returning('id')</code> in <code class="language-plaintext highlighter-rouge">pg</code></li>
      <li>The result object structure is different because it’s the native structure of the database driver</li>
    </ul>
  </li>
</ul>

<p>On top of that the heavy use of the fluent interface makes it really hard to write mocks for automatic tests.</p>

<p>An example use case inserting a row and give the insert ID back:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">url</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="p">(</span><span class="k">await</span> <span class="nx">knex</span><span class="p">(</span><span class="dl">'</span><span class="s1">urls</span><span class="dl">'</span><span class="p">).</span><span class="nx">insert</span><span class="p">({</span> <span class="nx">url</span> <span class="p">}).</span><span class="nx">returning</span><span class="p">(</span><span class="dl">'</span><span class="s1">id</span><span class="dl">'</span><span class="p">))[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span>
</code></pre></div></div>

<p>And the unit test for it with the mocked knex object:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">knex</span> <span class="o">=</span> <span class="nx">sinon</span><span class="p">.</span><span class="nx">fake</span><span class="p">.</span><span class="nx">returns</span><span class="p">({</span>
  <span class="na">insert</span><span class="p">:</span> <span class="nx">sinon</span><span class="p">.</span><span class="nx">fake</span><span class="p">.</span><span class="nx">returns</span><span class="p">({</span>
    <span class="na">returning</span><span class="p">:</span> <span class="nx">sinon</span><span class="p">.</span><span class="nx">fake</span><span class="p">.</span><span class="nx">returns</span><span class="p">(</span><span class="nb">Promise</span><span class="p">.</span><span class="nx">resolve</span><span class="p">([{</span> <span class="na">id</span><span class="p">:</span> <span class="dl">'</span><span class="s1">5</span><span class="dl">'</span> <span class="p">}]))</span>
  <span class="p">}),</span>
  <span class="na">select</span><span class="p">:</span> <span class="nx">sinon</span><span class="p">.</span><span class="nx">fake</span><span class="p">()</span>
<span class="p">})</span>
<span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">mocked url</span><span class="dl">'</span>

<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="p">(</span><span class="k">await</span> <span class="nx">knex</span><span class="p">(</span><span class="dl">'</span><span class="s1">urls</span><span class="dl">'</span><span class="p">).</span><span class="nx">insert</span><span class="p">({</span> <span class="nx">url</span> <span class="p">}).</span><span class="nx">returning</span><span class="p">(</span><span class="dl">'</span><span class="s1">id</span><span class="dl">'</span><span class="p">))[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span>

<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">.</span><span class="nx">calledOnce</span><span class="p">)</span>
<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">.</span><span class="nx">calledWith</span><span class="p">(</span><span class="dl">'</span><span class="s1">urls</span><span class="dl">'</span><span class="p">))</span>

<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">().</span><span class="nx">insert</span><span class="p">.</span><span class="nx">calledOnce</span><span class="p">)</span>
<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">().</span><span class="nx">insert</span><span class="p">.</span><span class="nx">calledWith</span><span class="p">({</span> <span class="nx">url</span> <span class="p">}))</span>

<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">().</span><span class="nx">insert</span><span class="p">().</span><span class="nx">returning</span><span class="p">.</span><span class="nx">calledOnce</span><span class="p">)</span>
<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">().</span><span class="nx">insert</span><span class="p">().</span><span class="nx">returning</span><span class="p">.</span><span class="nx">calledWith</span><span class="p">(</span><span class="dl">'</span><span class="s1">id</span><span class="dl">'</span><span class="p">))</span>

<span class="nx">assert</span><span class="p">(</span><span class="nx">knex</span><span class="p">().</span><span class="nx">select</span><span class="p">.</span><span class="nx">notCalled</span><span class="p">)</span>

<span class="nx">assert</span><span class="p">(</span><span class="nx">id</span> <span class="o">===</span> <span class="mi">5</span><span class="p">)</span>
</code></pre></div></div>

<h2 id="alternative-sql-queries-with-pg">Alternative: SQL queries with <code class="language-plaintext highlighter-rouge">pg</code></h2>

<p>In the code SQL is directly written:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">url</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="p">(</span><span class="k">await</span> <span class="nx">client</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
  <span class="s2">`INSERT INTO urls (url) VALUES ($1) RETURNING ('id')`</span><span class="p">,</span>
  <span class="p">[</span><span class="nx">url</span><span class="p">]</span>
<span class="p">)).</span><span class="nx">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span>
</code></pre></div></div>

<p>And the test for it:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">client</span> <span class="o">=</span> <span class="p">{</span>
  <span class="na">query</span><span class="p">:</span> <span class="nx">sinon</span><span class="p">.</span><span class="nx">fake</span><span class="p">.</span><span class="nx">returns</span><span class="p">(</span><span class="nb">Promise</span><span class="p">.</span><span class="nx">resolve</span><span class="p">([{</span> <span class="na">id</span><span class="p">:</span> <span class="dl">'</span><span class="s1">5</span><span class="dl">'</span> <span class="p">}]))</span>
<span class="p">}</span>   

<span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">url</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="p">(</span><span class="k">await</span> <span class="nx">client</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
  <span class="s2">`INSERT INTO urls (url) VALUES ($1) RETURNING ('id')`</span><span class="p">,</span>
  <span class="p">[</span><span class="nx">url</span><span class="p">]</span>
<span class="p">)).</span><span class="nx">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span>

<span class="nx">assert</span><span class="p">(</span><span class="nx">client</span><span class="p">.</span><span class="nx">query</span><span class="p">.</span><span class="nx">calledOnce</span><span class="p">)</span>
<span class="nx">assert</span><span class="p">(</span><span class="nx">client</span><span class="p">.</span><span class="nx">query</span><span class="p">.</span><span class="nx">calledWith</span><span class="p">([</span>
  <span class="s2">`INSERT INTO urls (url) VALUES ($1) RETURNING ('id')`</span><span class="p">,</span>
  <span class="p">[</span><span class="nx">url</span><span class="p">]</span>
<span class="p">]))</span>
<span class="nx">assert</span><span class="p">.</span><span class="nx">equal</span><span class="p">(</span><span class="nx">id</span><span class="p">,</span> <span class="dl">'</span><span class="s1">5</span><span class="dl">'</span><span class="p">)</span>
</code></pre></div></div>

<h2 id="alternative-sql-tagged-template-literal">Alternative: SQL tagged template literal</h2>

<p>Coming with ES6 the tagged template literals are a powerful new feature to define strings with values needs to be processed which perfectly fits into the requirements of a query builder.</p>

<p>The example by using SQL tag:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">url</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="p">(</span><span class="k">await</span> <span class="nx">client</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
  <span class="nx">sql</span><span class="s2">`INSERT INTO urls (url) VALUES (</span><span class="p">${</span><span class="nx">url</span><span class="p">}</span><span class="s2">) RETURNING ('id')`</span>
<span class="p">)).</span><span class="nx">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span>
</code></pre></div></div>

<p>The pros of this approach:</p>
<ul>
  <li>The automatic tests are as simple as using directly SQL queries</li>
  <li>It’s not needed to care about the positioning of the values in the statements</li>
  <li>Syntax highlighting for SQL inside the string is possible</li>
  <li>With additional tags it’s really powerful also for complex value types and nested queries</li>
</ul>

<p>But wait, is it not highly vulnerable for SQL injection if forgetting the tag?
Yes. That’s the reason a SQL tag should provide an own <code class="language-plaintext highlighter-rouge">.query()</code> function to check these before give <code class="language-plaintext highlighter-rouge">pg</code>’s native <code class="language-plaintext highlighter-rouge">client.query</code> the query:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">url</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="p">(</span><span class="k">await</span> <span class="nx">sql</span><span class="p">.</span><span class="nx">query</span><span class="p">(</span>
  <span class="nx">sql</span><span class="s2">`INSERT INTO urls (url) VALUES (</span><span class="p">${</span><span class="nx">url</span><span class="p">}</span><span class="s2">) RETURNING ('id')`</span>
<span class="p">)).</span><span class="nx">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">id</span>
</code></pre></div></div>

<p>By using the combination of SQL tag and own <code class="language-plaintext highlighter-rouge">.query()</code> function it’s basically not possible to have SQL injections.</p>

<p>It’s also very easy to build smart helpers on top of the SQL tag:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">url</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">id</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">client</span><span class="p">.</span><span class="nx">insert</span><span class="p">(</span><span class="dl">'</span><span class="s1">urls</span><span class="dl">'</span><span class="p">,</span> <span class="p">{</span> <span class="nx">url</span> <span class="p">})</span>
</code></pre></div></div>

<p>So this really helps if a service use complex or often SQL queries without loosing the full control over the database driver (it’s still <code class="language-plaintext highlighter-rouge">pg</code>’s native <code class="language-plaintext highlighter-rouge">client.query</code>) and without an oversized library.</p>

<h2 id="recommendation">Recommendation</h2>

<ul>
  <li><code class="language-plaintext highlighter-rouge">knex</code> for migration scripts</li>
  <li><code class="language-plaintext highlighter-rouge">pg</code> for writing SQL and testing the <code class="language-plaintext highlighter-rouge">client.query</code> calls</li>
  <li>If there are complex / often use of SQL queries in a service, have a look at SQL tagged template literals, e.g. <a href="https://www.npmjs.com/package/sql-pg">sql-pg</a> or <a href="https://www.npmjs.com/package/slonik">slonik</a></li>
</ul>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Working in code with a relational database can be done in different abstraction forms from writing SQL queries over using a query builder to completely abstract the type of the database by using an ORM. But it’s not only querying a database, also migration scripts needs to be defined and managed. So let’s have a critical look at knex, where is it worth to use, where not and what are the alternatives.]]></summary></entry><entry><title type="html">Durations - Avoid numbers without the format</title><link href="http://blog.sharaal.de/2018/02/06/durations-avoid-numbers-without-the-format.html" rel="alternate" type="text/html" title="Durations - Avoid numbers without the format" /><published>2018-02-06T00:00:00+00:00</published><updated>2018-02-06T00:00:00+00:00</updated><id>http://blog.sharaal.de/2018/02/06/durations-avoid-numbers-without-the-format</id><content type="html" xml:base="http://blog.sharaal.de/2018/02/06/durations-avoid-numbers-without-the-format.html"><![CDATA[<p>Durations are easy to understand. They are the timespan between two timestamps. But they are hard to manage, because different APIs need different formats: milliseconds? seconds? Also they should be human readable.</p>

<!--more-->

<h2 id="formats">Formats</h2>

<p>JavaScript handles timestamps and durations in milliseconds. That can be error-prone if used together with other APIs like Redis, which handles them in seconds (and an expire of 3000 days instead of 3 days can be a problem, someday).</p>

<p>Because of this I avoid to use variables of <code class="language-plaintext highlighter-rouge">number</code>, hand over durations always as <code class="language-plaintext highlighter-rouge">object</code> and the consumer which knows his needed format use his format:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">expire</span><span class="p">(</span><span class="nx">expire</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// JavaScript needs milliseconds</span>
  <span class="nx">setTimeout</span><span class="p">(</span>
    <span class="p">()</span> <span class="o">=&gt;</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">expired</span><span class="dl">'</span><span class="p">),</span>
    <span class="nx">expire</span><span class="p">.</span><span class="nx">asMilliseconds</span><span class="p">()</span>
  <span class="p">);</span>

  <span class="c1">// Redis needs seconds</span>
  <span class="nx">redis</span><span class="p">.</span><span class="nx">expire</span><span class="p">(</span><span class="dl">'</span><span class="s1">key</span><span class="dl">'</span><span class="p">,</span> <span class="nx">expire</span><span class="p">.</span><span class="nx">asSeconds</span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div></div>

<h2 id="readability">Readability</h2>

<p>The duration should be define by a string (to be defined by a environment variable) and easy to read by a human. So I came up with this:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// instead of</span>
<span class="kd">const</span> <span class="nx">expire</span> <span class="o">=</span> <span class="mi">259200000</span><span class="p">;</span>

<span class="c1">// or</span>
<span class="kd">const</span> <span class="nx">expire</span> <span class="o">=</span> <span class="mi">3</span> <span class="o">*</span> <span class="mi">24</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">1000</span><span class="p">;</span>

<span class="c1">// and never with a comment which can be wrong someday</span>
<span class="kd">const</span> <span class="nx">expire</span> <span class="o">=</span> <span class="mi">3</span> <span class="o">*</span> <span class="mi">24</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">1000</span><span class="p">;</span> <span class="c1">// 1 day</span>

<span class="c1">// define it with the format</span>
<span class="kd">const</span> <span class="nx">expire</span> <span class="o">=</span> <span class="nx">duration</span><span class="p">(</span><span class="dl">'</span><span class="s1">3 days</span><span class="dl">'</span><span class="p">);</span>
</code></pre></div></div>

<h2 id="summary">Summary</h2>

<p>Handle numbers without a definition for what they stand for can be error-prone, not only for durations. So always define the format to avoid confusions and convert it into the right format depending on the context.</p>

<p>For durations feel free to have a look at <a href="https://www.npmjs.com/package/@dnode/duration">@dnode/duration</a> or <a href="http://momentjs.com/docs/#/durations/">moment.duration</a>.</p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Durations are easy to understand. They are the timespan between two timestamps. But they are hard to manage, because different APIs need different formats: milliseconds? seconds? Also they should be human readable.]]></summary></entry><entry><title type="html">async behavior in JavaScript</title><link href="http://blog.sharaal.de/2018/01/25/async-behavior-in-javascript.html" rel="alternate" type="text/html" title="async behavior in JavaScript" /><published>2018-01-25T00:00:00+00:00</published><updated>2018-01-25T00:00:00+00:00</updated><id>http://blog.sharaal.de/2018/01/25/async-behavior-in-javascript</id><content type="html" xml:base="http://blog.sharaal.de/2018/01/25/async-behavior-in-javascript.html"><![CDATA[<p>Understanding the async behavior is one of the largest milestone in learning JavaScript. I will give you a very first overview what it is.</p>

<!--more-->

<h2 id="procedural-code">Procedural code</h2>

<p>JavaScript is a language which supports procedural code. What does this mean? It means just the code runs line by line:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">variable</span><span class="p">;</span>
<span class="nx">variable</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">example</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">variable</span><span class="p">);</span>
</code></pre></div></div>

<p>The order what happens is easy to understand:</p>

<ul>
  <li>The variable gets defined</li>
  <li>It gets assigned with <code class="language-plaintext highlighter-rouge">example</code></li>
  <li>It gets logged out to the console</li>
</ul>

<h2 id="event-loop-and-processnexttick">Event loop and process.nextTick()</h2>

<p>But procedural is not all, JavaScript have an event loop. It runs all the code line by line, have a look if there is something next in the event loop, runs this code line by line and so on:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">1</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">process</span><span class="p">.</span><span class="nx">nextTick</span><span class="p">(()</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">3</span><span class="dl">'</span><span class="p">);</span>
<span class="p">});</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">2</span><span class="dl">'</span><span class="p">);</span>
</code></pre></div></div>

<p>The order the code runs now is not anymore the same order we wrote the lines:</p>

<ul>
  <li>First <code class="language-plaintext highlighter-rouge">1</code> gets logged</li>
  <li>Then we add a function to the end of the event loop</li>
  <li><code class="language-plaintext highlighter-rouge">2</code> gets logged</li>
  <li>The current loop is over, JavaScript look at the event loop, find and run our
function we added, <code class="language-plaintext highlighter-rouge">3</code> gets logged</li>
</ul>

<p>This is what single threaded and non blocking means. The code run in a single thread, only one part of the code can run at the same time. But its possible to add something to the end of the event loop without blocking the followed code.</p>

<h2 id="async-behavior">Async behavior</h2>

<p>With this in mind we now better understand how async behaves:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">user</span><span class="p">;</span>
<span class="nx">db</span><span class="p">.</span><span class="nx">findUserById</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">record</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">user</span> <span class="o">=</span> <span class="nx">record</span><span class="p">;</span>
<span class="p">});</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">user</span><span class="p">);</span>
</code></pre></div></div>

<p>We get an <code class="language-plaintext highlighter-rouge">undefined</code>, because <code class="language-plaintext highlighter-rouge">db.findUserById()</code> add the callback at the end of the event loop. So the <code class="language-plaintext highlighter-rouge">console.log(user)</code> runs before the callback which assign user to the record.</p>

<h3 id="event-driven">Event driven</h3>

<p>In the last example we got in touch with another concept named event driven. The <code class="language-plaintext highlighter-rouge">db.findUserById()</code> will not run immediately at the end of the event loop, because the database need some time to find the user. It register an event which will be triggered after the user is found. But that doesn’t change the async behavior, because also if the database is fast enough to find the user, the callback can’t run before the current code in the event loop is finished.</p>

<h2 id="summary">Summary</h2>

<p>It’s a very important step to understand async and start to think about which functions runs async and which code runs in which order.</p>

<p>Read more about how to <a href="/2018/02/02/write-async-code-in-javascript.html">write async code in JavaScript</a>.</p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Understanding the async behavior is one of the largest milestone in learning JavaScript. I will give you a very first overview what it is.]]></summary></entry><entry><title type="html">AI first @ maxdome</title><link href="http://blog.sharaal.de/2017/04/28/ai-first-at-maxdome.html" rel="alternate" type="text/html" title="AI first @ maxdome" /><published>2017-04-28T00:00:00+00:00</published><updated>2017-04-28T00:00:00+00:00</updated><id>http://blog.sharaal.de/2017/04/28/ai-first-at-maxdome</id><content type="html" xml:base="http://blog.sharaal.de/2017/04/28/ai-first-at-maxdome.html"><![CDATA[<p>Google ushered a new era with AI first last year. So we started preparing us for this future with a lot of PoC’s (= proof of concepts). I will describe a few of them to give you an overview about AI in general, Alexa and Google Assistent in particular.</p>

<!--more-->

<h2 id="whats-ai">What’s AI?</h2>

<p>Because AI is a very wide topic let’s specify first a bit. AI here means <strong>the way an user interact with a service</strong>: Not with a mobile app or a website, but <strong>with its own natural written or spoken language</strong>. Understanding natural language is an own and very complex topic, so it’s good Alexa and Google Assistent do this for us and we focus on linking our business logic to that platforms.</p>

<h2 id="general-workflow">General Workflow</h2>

<p><img src="/assets/2017-04-28-ai-first-at-maxdome/images/general-workflow.png" alt="General Workflow" /></p>

<ol>
  <li>An user write/say something</li>
  <li>The platform (Alexa or Google Assistent) interpret it with the help of some sample utterances to an intent</li>
  <li>Our business service gets a request with the intent and response with an answer</li>
</ol>

<p>So what’s our part in this workflow? a) Providing the names of the intents together with sample utterances we want to support and b) an API for the request response.</p>

<h2 id="build-an-action">Build an action</h2>

<h3 id="a-prepare-the-platform">a) Prepare the platform</h3>

<p>I don’t give this part to much focus in this post, because its already well documented for both platforms <a href="https://developer.amazon.com/blogs/post/TxDJWS16KUPVKO/New-Alexa-Skills-Kit-Template:-Build-a-Trivia-Skill-in-under-an-Hour">Alexa (ignore Step#2)</a> and <a href="https://console.api.ai/api-client/#/getStarted">Google Assistent</a>.</p>

<h3 id="b-api-for-the-business-service">b) API for the business service</h3>

<h4 id="step-1-poc-for-alexa">Step 1: PoC for Alexa</h4>

<p>The first thoughts was “keep it simple, just start with a PoC for Alexa”. So we prepared the alexa platform and built a RestAPI with our business logic understanding and responding in the <a href="https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/alexa-skills-kit-interface-reference">Alexa JSON format</a>. We made a few skills (see <a href="https://github.com/dragonprojects/dalexa-maxdome">dalexa-maxdome</a>) to try out the different topics: a very basic one, a skill with slots, sessions, pagination and at the end account linking with OAuth 2.0.</p>

<p>Our learnings:</p>

<ul>
  <li>It worked!</li>
  <li>It’s really easy and streight forward because its just getting and responding JSON</li>
  <li>OAuth 2.0 is cool</li>
  <li>A “Show a comedy” and the TV starts would be nice, but need a lot more effort than we wanted to spend for a PoC</li>
  <li>All of the things worked without changing existing services, new microservice(s) for new features rocks</li>
  <li>The most difficult part was “what’s the minimum information does an user need for an asset to know if its interesting”. Only the title is not enough, if it’s not a well known blockbuster and the description is most times to long. Later in the week, this becomes an own NPM package (see <a href="https://github.com/dragonprojects/ai-renderer-maxdome">ai-renderer-maxdome</a>)</li>
</ul>

<h4 id="step-2-think-about-abstraction">Step 2: Think about abstraction</h4>

<p>A bit later a Google Home stand on our desktop and of course we wanted to play with it and try out our PoC on that device too.
We thought about support it streight forward by bulding a new microservice with our business logic from scratch understanding and responding in the <a href="https://developers.google.com/actions/reference/webhook-format">Google Assistant JSON format</a>.
But we decided to spend a bit more time and build an abstraction layer to make the business logic independ of the supported platform.</p>

<p><img src="/assets/2017-04-28-ai-first-at-maxdome/images/abstracted-workflow.png" alt="Abstracted Workflow" /></p>

<p>Like you can see in the diagram, we built “connectors” (see <a href="https://github.com/dragonprojects/ai-connector-alexa">ai-connector-alexa</a> and <a href="https://github.com/dragonprojects/ai-connector-apiai">ai-connector-apiai</a>) which transforms the platform specific JSON formats to an internal used format for the business logic named “webhook” (see <a href="https://github.com/dragonprojects/ai-webhook-maxdome">ai-webhook-maxdome</a>). And also the OAuth 2.0 integration got an own service named “oauth” (see <a href="https://github.com/dragonprojects/ai-oauth-maxdome">ai-oauth-maxdome</a>), luckily both platforms supporting the same account linking concept.</p>

<p>Our learnings:</p>

<ul>
  <li>It worked!</li>
  <li>New features can independently built in the “webhook” and later supported by the “connectors”</li>
  <li>New “connectors” allow to adopt new platforms very fast and easy</li>
  <li>All the services are very small and easy to test</li>
</ul>

<h2 id="our-next-steps">Our next steps?</h2>

<ul>
  <li>Let us care about our core business: Streaming, start playing movies on a TV by our voice</li>
  <li>Adopt other platforms with new connectors like Cortana, Siri and bixby. But also non voice platforms like Facebook Messenger and Skype</li>
</ul>

<h2 id="bonus-alexas-tägliche-zusammenfassung---flash-briefing-skill-api">Bonus: Alexa’s “Tägliche Zusammenfassung” - Flash Briefing Skill API</h2>

<p>In Alexa it’s possible to choose services which you think are interesting for a daily summary. Then you can just say “Alexa, was gibt es neues” and get for all choosen services the news.
This feature is not like a custom skill where we get a JSON request, it’s simply a RSS or JSON feed.
Also for this we made a PoC for maxdome (see <a href="https://github.com/dragonprojects/ai-flashbriefing-maxdome">ai-flashbriefing-maxdome</a>).</p>

<p>Later in the week: The PoC grew up and <a href="https://www.amazon.de/maxdome-GmbH-Tipp-des-Tages/dp/B06ZYMH963/">“maxdome - Tipp des Tages”</a> is the first published skill in the “Film &amp; Fernsehen” section with the “Tägliche Zusammenfassung” feature, yeah. :)</p>

<p>Also <a href="https://github.com/maxdome/ai-flashbriefing-maxdome">ai-flashbriefing-maxdome</a> and the <a href="http://ai-flashbriefing-maxdome-prod.a4z2vg6thb.eu-central-1.elasticbeanstalk.com/">deployed service</a> moved to the company.</p>

<h2 id="summary">Summary</h2>

<p>AI first is a very wide topic. But with a small managable effort we got a lot of insights to decide if, when and with which scope we want to jump deeper. And in addition the first AI product of us can go live: The “Tägliche Zusammenfassung” for Alexa.</p>

<h2 id="faq">FAQ</h2>

<h3 id="why-an-abstraction-layer-apiai-already-have-alexa-integration">Why an abstraction layer, API.ai already have Alexa integration?</h3>
<p>Yes, but the Alexa integration is only an export of the intent schema and the sample utterances. The webhook which gets the request must still works for both JSON formats and responding in the platform specific format.</p>

<h3 id="why-spend-time-into-this-topic-it-doesnt-have-business-value">Why spend time into this topic, it doesn’t have business value?</h3>
<p>Yes, currently we wouldn’t sell something over this platforms or generate many new subs. But if Google’s vision become true, AI first will be as important as todays “Mobile first” and for our product a must have part. So it’s worth to start getting familiar with it.</p>

<p>Later in the week, Amazon and Google push this topic very hard:</p>

<ul>
  <li><a href="https://www.heise.de/newsticker/meldung/Amazons-neuer-Fire-TV-Stick-in-Deutschland-Quadcore-Prozessor-und-Sprachassistentin-Alexa-3631507.html">The new FireTV Stick gets Alexa</a></li>
  <li><a href="https://www.heise.de/newsticker/meldung/Google-liefert-seinen-Assistenten-fuer-die-Allgemeinheit-aus-3635423.html">Many Android smartphones gets Google Assistant</a></li>
</ul>

<p>So maybe this topic becomes faster as assumed important for all companies.</p>]]></content><author><name>Sharaal</name></author><summary type="html"><![CDATA[Google ushered a new era with AI first last year. So we started preparing us for this future with a lot of PoC’s (= proof of concepts). I will describe a few of them to give you an overview about AI in general, Alexa and Google Assistent in particular.]]></summary></entry></feed>