DEV Community: Wuzhong

Getting Started with Qlik Core

Wuzhong — Mon, 29 Oct 2018 11:56:59 +0000

A few months back, Qlik announced Qlik Core, a Dockerized version of Qlik’s associative data engine. After playing with Core for a few weeks (the developer version is free…plus, I work for Qlik), I’ve come to realize how awesome the product is for visualizing and analyzing data. So I’m writing this blog to explain, in my humble opinion, who will benefit from Core, and how to start using it.

First of all, why Qlik Core?

Nowadays, there are several visualization libraries, D3 probably being the most widely used library of the bunch. Click here to see a fully baked example of an interactive visualization built on top of D3. For an application like this, static data is packed and loaded together with your application. But there are a few limitations with this approach, especially when the size data is large (for example, greater than 200 MB)…

The app takes a lifetime to load.
The browser occupies a large amount of memory, which can even cause the browser to crash if the size of data exceeds the limit.
It’s not easy to handle filtering and association.
It can be difficult to manage state as the data is explored and filtered.
It’s difficult to handle dynamic data.

One alternative is to query a database, requesting and receiving data as we go, but this also has limitations. The main drawback is the query latency that the user may experience. Usually, a query based system takes 0.5–3 seconds to respond to user action, which diminishes the user experience and purpose of an interactive dashboard. 😟

Qlik Core to the rescue — but what is Qlik Core?

Irrelevant Paw Patrol picture (I’m a new parent), from here.

Qlik Core gives you direct access to the first data engine of its kind. And “data engine” is neither a data warehouse nor a database. It is a mechanism to handle data association, analytics and state, especially in large volumes of data.

And when you have this data engine at the 🧡 of an application, you don’t need to send a query to receive the data. Even pulling from different sources, Qlik Core can load the data into memory, process and index it, then power web applications reactively. The data becomes an in-memory instance and your web app now connects to Qlik Core via a WebSocket— all calculations and associations are handled remotely.

So, as mentioned, there are many advantages to using Qlik Core, especially if the data volume is large…

The application data resides in a server instead of being passed to the frontend. This not only increases usability from the viewer’s perspective, but it also reduces the load of the server, causing both frontend and backend performance to improve significantly.
The developer doesn’t need to worry about wiring up filtering and association through their code. All of this, as well as state management, is handled by Qlik Core now and the app’s visualizations and interactivity can simply hook into the engine.
On top of all this, Qlik Core is great at loading data from different data sources and meshing them into one.

So, we’re killing several birds with one stone as they say. And now you know why I think this product is amazing (😍 → that’s me building with Qlik Core and I swear it’s not just because I work at Qlik).

Ready to get started?

Hopefully, you're nodding like this.

Qlik Core is pretty new so getting started can be somewhat tricky (we’re working on improving this but in the meantime, please give us feedback). The official documentation for Qlik Core is at https://core.qlik.com/. There you’ll find this tutorial which I find pretty straightforward. Following the instructions will get your Qlik Core instance running quickly, especially if you’re familiar with Docker. There is also a tutorial video created by Qlik Branch (more to come but this our home for all that is developer and Qlik) which goes through the basics.

But what is after this?

Let me show you with some simple examples…

I have built some examples to illustrate how Qlik Core is used. Basically, to use Qlik Core, you need enigma.js (one of Qlik’s open source libraries) to easily communicate with the engine using JavaScript. The only way to talk to Qlik Core is via our Engine API which is a language-agnostic, WebSocket-based API. Enigma.js is simply a wrapper library for this.

So, here are some examples so you can better understand how Qlik Core works…

*️⃣ Hello engine example

*️⃣ Hello data example

*️⃣ Check if a Core instance is healthy

*️⃣ View data model in an app

*️⃣ Upload a pre-built app to Qlik Core

Hopefully, this is enough to get you started learning about and using Qlik Core. Over time, we’ll be adding more tutorials, tips and tricks to help developers not only get started with Qlik Core but also with the APIs themselves.

So keep an eye for that on Knowledge section of Qlik Branch. In the meantime, check out the tutorials there explaining more about our associative data engine, how it works and how to work with it. And if you are really interested in Qlik Core or need help, let me know. Just leave a comment below or join our Qlik Branch Slack (see the #qlik-core channel or DM me [at]wu_zh).

Cheers, happy coding and happy Halloween everyone.

— Wu

System Integration with Qlik’s APIs — Why and How

Wuzhong — Thu, 30 Aug 2018 03:35:57 +0000

First of all, this article is not a “how to” tutorial but rather a white paper. It is NOT MEANT FOR TECHNICAL USERS. If you understand the value of APIs and how to use Qlik’s APIs, this is not for you.

____________________Qlik Pro’s Stop Here______________________

“What are Qlik’s APIs?” 🤔

“Why would someone want to use an API?” 🤔

“What can you build using Qlik’s APIs.” 🤔

If you have these questions, you may find this article helpful. This blog is about various types of system integration using Qlik Sense APIs.

So let’s dive in. With APIs, you can do amazing things. But wait, what is an API?

Behind every click, various events are triggered on the back-end.

The APIs are the bridge between the front and the back.

Usually, user interactions are restricted to front-end elements: buttons, charts, filters, etc.

So when a piece of software implements APIs, it means interactions with the back-end are no longer limited to the software’s native front-end, which opens many possibilities.

Now that you know what an API is…what can you build with Qlik Sense APIs?

APIs have always been an important and unique part of Qlik. They are the building blocks of mashups and extensions. If you don’t know about those, below is a brief explanation:

But since this article is about system integration, we will focus on mashups.

Here is a random fact: you can build a mashup with extensions. That way, technically you put the visualization you put into Qlik system into your own system.

Mashup Lv.1 — build a webpage of iframes

This is probably the simplest kind of system integration. Every chart or table in Qlik Sense has an ID and an URL, which can be found in Qlik Sense DevHub -> single configurator portal.

If you copy the URL into your browser, you can see the object on the page. A single line of code extracted from the DevHub can, therefore, be added into an existing website to embed a Qlik chart to a web page. Put multiple iframes onto an html file and you have a dashboard.

So in less than 1 minute, you can build a web page with four lines of code copied from DevHub. 👍

Required skill level: ★☆☆☆☆

Mashup Lv.2 — mashup with APIs and libraries

See how it is implemented in the [Walkers Dashboard](http://playground.qlik.com/liveshowcase/playground-zombie/main.html)

The Walkers Dashboard is an example on Qlik Playground created by Francis Kabinoff which uses Qlik charts within a web page showing stats on the tv show The Walking Dead. It is a typical mashup built with the Qlik Sense capability APIs. Different from an iframe, charts are rendered using a Qlik javascript library to control the user interaction and object retrieval. This means faster and cleaner execution, customizable elements and other features.

There is an easy way to create and host your own mashup too. In Qlik Sense DevHub, you can find a tool called Mashup Editor. It allows users to design their own web application by dragging and dropping Qlik objects onto predefined templates. But if you want to build something different, like The Walker Dashboard, you need to learn javascript and html.

Required skill level: ★★☆☆☆

Mashup Lv.3 — building a Qlik dashboard which looks completely non-related to Qlik

See how it is implemented in theMaple Leafs Dashboard

Qlik Sense is business intelligence software, but not only business intelligence software. By using APIs, users may create their own visualizations, on top of a Qlik application, without using charts from Qlik. In this use case, Qlik is no longer a visualization server. It is a data warehouse plus data engine (Qlik’s Associative Engine) behind your dashboard. It handles the data processing, storage and association part but lets third-party technologies take care of the presentation part.

If you are doing this, you are entitled to call yourself a Qlik Pro. 😁 The Qlik Sense DevHub Mashup Editor won’t help you at this level. You have to write every line of front-end code by yourself.

Required skill level: ★★★☆☆

Mashup Lv.4 — using enigma.js and engine API

Most Qlik developers do not use enigma.js and the engine API. For many, the capability API is enough for most use cases.

Enigma.js is a wrapper library on top of the engine API. While the capability APIs are run from a user’s browser, enigma.js can be run on an application server to interact with Qlik before serving a web page to a user. In fact, the capability API also implements enigma.js inside of an Angular application packed using require.js.

The engine API is the communication protocol between the Qlik Sense server and the Qlik Sense client. The Engine API Explorer is a tool in DevHub where you can learn how the front-end talks to the back-end. Unless you are building your own support library, Enigma.js should cover most of your integration needs.

Required skill level: ★★★★☆

Mashup(?) Lv.MAX — leveraging on the engine itself

When you know the technology inside out, you may want to use the data engine (aka Qlik’s Associative Engine) outside of the business intelligence space. Examples speak for themselves, check them out...

See how it is implemented in theBoston Building Emissions By Axis Group

C40 cities building energy by Slalom

Qlik Core

Then there are some use cases that are difficult to define — they are certainly not mashups, some of them are not even web applications.

For example, a business intelligence chatbot…

Qlik BotNet

And a mirror…
// Detect dark theme var iframe = document.getElementById('tweet-981633788244647937-571'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=981633788244647937&theme=dark" }

And many people don’t know this —there is a Qlik application behind branch.qlik.com. The application handles search and content management specifically. Qlik Branch is an open source project, you can find the code here.

Qlik Branch

Required skill level: ★★★★★

So, how should you get started?

Just start experimenting! There are resources and examples on Qlik Playground, open source examples on Qlik Branch and plenty of community members on our Qlik Branch Slack. You can also find blogs and Youtube videos about Qlik’s APIs almost effortlessly online.
Let me know your thoughts.
Happy coding. And shout out to all of the amazing developers out there.
Cheers.
— Wu