The latest articles on DEV Community by Soney Mathew (@soneymathew). https://dev.to/soneymathew https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F217460%2F6f786d79-77fe-4447-bce7-cb767490dc06.jpeg https://dev.to/soneymathew en Soney Mathew Mon, 03 Aug 2020 15:41:13 +0000 https://dev.to/soneymathew/graphql-lies-aka-lists-2lc2 https://dev.to/soneymathew/graphql-lies-aka-lists-2lc2 <p>Home page of <a href="https://graphql.org/">https://graphql.org/</a> states this fact</p> <p>Describe your data<br> </p> <div class="highlight"><pre class="highlight plaintext"><code>type Query { company(name: String): Company } type User { name: String } type Company { name: String tagline: String # List of board members boardMembers: [User] } </code></pre></div> <p>Ask for what you want<br> </p> <div class="highlight"><pre class="highlight plaintext"><code>{ company(name: "Acme") { name tagline boardMembers { name } } } </code></pre></div> <p>Get predictable results<br> </p> <div class="highlight"><pre class="highlight plaintext"><code>{ "data": { "company": { "name": "Acme", "tagline": "Acme is all you need!", "boardMembers": [ { "name": "Tom" }, { "name": "Jerry" } ] } } } </code></pre></div> <p>Are you sure? Are you sure you are getting predictable results?</p> <p>This API is breaking a fundamental promise given by GraphQL! You simply are not getting predictable results!</p> <p>As you scale the number of items returned by the list might become unpredictably large. See the image below for example, there is no predictability in the size of items returned in the list</p> <p><a href="https://res.cloudinary.com/practicaldev/image/fetch/s--iE6rCwA9--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/ddxrqs0aksjhfj7wrmdv.gif" class="article-body-image-wrapper"><img src="https://res.cloudinary.com/practicaldev/image/fetch/s--iE6rCwA9--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/i/ddxrqs0aksjhfj7wrmdv.gif" alt="Unpredictable size of the list"></a></p> <p>Give the control back to the client, always set limits. Let the client decide how much they want to fetch.<br> </p> <div class="highlight"><pre class="highlight plaintext"><code>type Company { name: String tagline: String # List of board members boardMembers(first:Int) : [User] } </code></pre></div> <p>Now client can limit the query results to exactly how much they need<br> </p> <div class="highlight"><pre class="highlight plaintext"><code>{ company(name: "Acme") { name tagline boardMembers(first:3) { name } } } </code></pre></div> <p>Refer: <a href="https://graphql.org/learn/pagination/#slicing">https://graphql.org/learn/pagination/#slicing</a></p> <h1> 🙏 </h1> graphql connection limit pagination Soney Mathew Mon, 18 May 2020 14:51:50 +0000 https://dev.to/soneymathew/compiled-graphql-best-thing-since-sliced-bread-5hkh https://dev.to/soneymathew/compiled-graphql-best-thing-since-sliced-bread-5hkh <p>OT: <code>Daddy! It's the Best thing since sliced bread!</code> is a phrase my 5 year old son keeps repeating to me these days.</p> <blockquote> <p>This post is to explain the concept and merits of compiled GraphQL from a web developer's perspective. </p> </blockquote> <p>Once upon a time there was a Client that never changed, it always made the same request to the server. Server in turn optimised the Query to the DataStore to fetch it efficiently and return it.<br> </p> <div class="highlight"><pre class="highlight plaintext"><code> ┌──────┐ ┌──────┐ ┌─────────┐ │Client│ │Server│ │Datastore│ └──┬───┘ └──┬───┘ └────┬────┘ │ Request1 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─&gt;| │ │ │ │ │ │ Query1 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ &gt;│ │ │ │ │ │ Record(s) │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─│ │ │ │ │ Response1 │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─| │ ┌──┴───┐ ┌──┴───┐ ┌────┴────┐ │Client│ │Server│ │Datastore│ └──────┘ └──────┘ └─────────┘ </code></pre></div> <p>Customers got the best performance possible. Then the world changed, new requirements and designs came at startling pace. Then this happened... again and again...<br> </p> <div class="highlight"><pre class="highlight plaintext"><code> ┌──────┐ ┌──────┐ ┌─────────┐ │Client│ │Server│ │Datastore│ └──┬───┘ └──┬───┘ └────┬────┘ │ Request1 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─&gt;| │ │ │ │ │ │ Query1 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt;| │ │ │ │ │ Record(s) for Query1│ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─| │ │ │ │ Response1 │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─| │ │ │ │ │ Request2 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─&gt;| │ │ │ │ │ │ Query2 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt;| │ │ │ │ │ Record(s) for Query2│ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ Response2 │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ Request3 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ Query3 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ Record(s) for Query3│ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ Response3 │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ Request4 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ Query4 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ Record(s) for Query4│ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ Response4 │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ │ ┌──┴───┐ ┌──┴───┐ ┌────┴────┐ │Client│ │Server│ │Datastore│ └──────┘ └──────┘ └─────────┘ </code></pre></div> <p>We always delivered changes, but the customers suffered due to the request waterfalls that cause sluggish experience, Server got more stressed. Datastore suffered with all these extra queries.</p> <p>Then clients discovered GraphQL. My first introduction to GraphQL was as a great solution to reduce the API waterfall. Some libraries just made it super easy to glue together the backend APIs and abstract it behind a GraphQL schema. Clients rejoiced for it became this!<br> </p> <div class="highlight"><pre class="highlight plaintext"><code> ┌──────┐ ┌──────────────┐ ┌──────┐ ┌─────────┐ │Client│ │GraphQL_Server│ │Server│ │Datastore│ └──┬───┘ └──────┬───────┘ └──┬───┘ └────┬────┘ │ GraphQL Request │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ │ Request1 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ │ Query1 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ Record(s) for Query1│ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ Response1 │ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ │ Request2 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ │ Query2 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ Record(s) for Query2│ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ Response2 │ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ │ Request3 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ │ Query3 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ Record(s) for Query3│ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ Response3 │ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ │ Request4 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ │ Query4 │ │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │ │ Record(s) for Query4│ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ Response4 │ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ │ │ GraphQL Response │ │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ┌──┴───┐ ┌──────┴───────┐ ┌──┴───┐ ┌────┴────┐ │Client│ │GraphQL_Server│ │Server│ │Datastore│ └──────┘ └──────────────┘ └──────┘ └─────────┘ </code></pre></div> <p>The High speed network between GraphQL_Server and Server improved things quite a bit. Clients moved faster when changes came their way. Some Datastores made it cheaper to fetch all-the-things in one Query. Then I saw this strategy happen.<br> </p> <div class="highlight"><pre class="highlight plaintext"><code> ┌──────┐ ┌──────────────┐ ┌─────────┐ │Client│ │GraphQL_Server│ │Datastore│ └──┬───┘ └──────┬───────┘ └────┬────┘ │ GraphQL Request │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │─ ─ ┐ │ │ | Convert GraphQL Query to Datastore query │ │&lt; ─ ┘ │ │ │ │ │ Optimal Query1 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ Record(s) for Query1 │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ GraphQL Response │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ ┌──┴───┐ ┌──────┴───────┐ ┌────┴────┐ │Client│ │GraphQL_Server│ │Datastore│ └──────┘ └──────────────┘ └─────────┘ </code></pre></div> <p>This was very efficient when possible. But there are a few things that kept repeating every request. </p> <ul> <li>The client kept building the GraphQL query for every request. </li> <li>The server kept translating the GraphQL Query to Datastore Query every request.</li> </ul> <p>Compiled GraphQL is just one more step ahead in this journey to avoid this two runtime overheads that are predictable at build time.</p> <p>At build time, we persist the queries that get's repeated every request.<br> </p> <div class="highlight"><pre class="highlight plaintext"><code> ┌────────────┐ ┌───────────┐ ┌────────────┐ │Client_Build│ │Query_Store│ │Server_Build│ └─────┬──────┘ └─────┬─────┘ └─────┬──────┘ │────┐ │ │ │ Generate persisted │ │&lt;───┘ Query PQ1 │ │ │ │ │ │ │ Persist Query PQ1 │ │ │ ──────────────────────&gt; │ │ │ │ │ │ Get Persisted Queries│ │ │ &lt;───────────────────── │ │ │ │ │ │────┐ │ │ │ │ Generate Datastore query │ │ │&lt;───┘ DQ1 from PQ1 │ │ │ │ │ │ │ │ Persist Query │ │ │ DQ1 Mapped to PQ1 │ │ │ &lt;───────────────────── ┌─────┴──────┐ ┌─────┴─────┐ ┌─────┴──────┐ │Client_Build│ │Query_Store│ │Server_Build│ └────────────┘ └───────────┘ └────────────┘ </code></pre></div> <p>At runtime, thus we avoid the overhead.<br> </p> <div class="highlight"><pre class="highlight plaintext"><code> ┌──────┐ ┌──────────────┐ ┌─────────┐ │Client│ │GraphQL_Server│ │Datastore│ └──┬───┘ └──────┬───────┘ └────┬────┘ │ GraphQL Request with PQ1│ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ │─ ─ ┐ │ │ | Lookup corresponding Datastore query DQ1 │ │&lt; ─ ┘ │ │ │ │ │ Datastore query with DQ1 │ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─&gt; │ │ │ │ │ Record(s) │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ GraphQL Response │ │ │ &lt;─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ ┌──┴───┐ ┌──────┴───────┐ ┌────┴────┐ │Client│ │GraphQL_Server│ │Datastore│ └──────┘ └──────────────┘ └─────────┘ </code></pre></div> <p>If you look closely, this is the same old world we started in. The key difference is we are ever more powerful and flexible to adapt to the changing needs of the customers.</p> <h2> Summary </h2> <p>This concept can be realised using multiple implementations. I will be keen to learn what building blocks you will use to realise this? I will reserve my comments and biases for a follow up post where I will illustrate this with one or more concrete example(s).</p> <h2> References </h2> <ul> <li><a href="https://blog.kensho.com/compiled-graphql-as-a-database-query-language-72e106844282">Compiled GraphQL as a database query language</a></li> <li><a href="https://blog.grandstack.io/graphql-resolveinfo-deep-dive-1b3144075866">Building Efficient GraphQL Resolvers By Generating Database Queries</a></li> <li><a href="https://relay.dev/docs/en/compiler-architecture">Relay Compiler Architecture</a></li> <li><a href="https://github.com/hasura/graphql-engine#architecture">Hasura GraphQL Engine</a></li> </ul> nplus1nomore graphql compiled compiledgraphql