<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Kirill Birger</title>
    <description>The latest articles on DEV Community by Kirill Birger (@kbirgerdev).</description>
    <link>https://dev.to/kbirgerdev</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1027393%2F9afe62aa-69b4-428a-9baf-b8275fc872a6.jpeg</url>
      <title>DEV Community: Kirill Birger</title>
      <link>https://dev.to/kbirgerdev</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/kbirgerdev"/>
    <language>en</language>
    <item>
      <title>How to Track Down Any Bug</title>
      <dc:creator>Kirill Birger</dc:creator>
      <pubDate>Fri, 22 Sep 2023 16:55:09 +0000</pubDate>
      <link>https://dev.to/kbirgerdev/how-to-track-down-any-bug-2l37</link>
      <guid>https://dev.to/kbirgerdev/how-to-track-down-any-bug-2l37</guid>
      <description>&lt;h2&gt;
  
  
  Intro
&lt;/h2&gt;

&lt;p&gt;If you're wary and skeptical of this title, then it is rightfully so. But fear not, this is not another fluffy article like "Five secret troubleshooting techniques that senior devs don't want you to know", but rather a general framework for how to troubleshoot issues in software. The approach described here can be applied to most types of software, whether it be a single executable, or an application composed of multiple services that are distributed across multiple machines.&lt;/p&gt;

&lt;h2&gt;
  
  
  An example issue
&lt;/h2&gt;

&lt;p&gt;Because this guide is intended to be conceptual, and not focused on specific types of software, we will use a simple non-software example. Let's start with a simple example of a light switch. The product specification says that you have a light switch, and a light.&lt;/p&gt;

&lt;p&gt;Electricity can also be a complicated domain, so there are certain details which we will omit for simplicity.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--03vOkb6P--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fyyiuvr0q3jnbrqtgj5x.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--03vOkb6P--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fyyiuvr0q3jnbrqtgj5x.png" alt="Image description" width="716" height="274"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;You flip the switch, and somehow the light turns on&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--3c6zp4YH--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ootdq4nif78ry6gxnt8m.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--3c6zp4YH--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ootdq4nif78ry6gxnt8m.png" alt="Image description" width="800" height="235"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Easy enough.&lt;/p&gt;

&lt;p&gt;You receive a bug report:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Today I flipped the switch and nothing happened&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--IA3qGNAu--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ig70xzhogzhb8s46sgkh.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--IA3qGNAu--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ig70xzhogzhb8s46sgkh.png" alt="Image description" width="800" height="233"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How to think about the issue
&lt;/h2&gt;

&lt;p&gt;The first thing to do is to shine some light on the black box (pun intended). The black box in our example doesn't literally represent a black box, but is rather a metaphor for  the components of a system that we either do not know about, or do not fully understand.&lt;/p&gt;

&lt;p&gt;Whenever I read a bug report or listen to an issue being described, I automatically run through this process in my head: "What are all of the components involved?"&lt;/p&gt;

&lt;p&gt;Sometimes a diagram helps. It may be the case that such a diagram is available, but oftentimes it is not. When one is not available, sometimes I draw one out myself to help me wrap my head around the issue. If I'm not using paper, I generally use a free tool called &lt;a href="https://www.diagrams.net"&gt;diagrams.net&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Let's take an example of a very simple circuit that can power a light bulb.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--DH058LQb--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/apnlm9l20h0wvfimymax.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--DH058LQb--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/apnlm9l20h0wvfimymax.png" alt="Image description" width="775" height="838"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We have some power source, a light switch, a light, and some wires. While this is still very simple, we have shed some light on the situation, and now have a good idea of what components are actually involved. There could be an issue with any of these components, or with the wires connecting them! &lt;/p&gt;

&lt;p&gt;The red question marks in the image above illustrate each potential failure point in our product.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Our light is broken&lt;/li&gt;
&lt;li&gt;Our switch is broken&lt;/li&gt;
&lt;li&gt;Our power source is broken&lt;/li&gt;
&lt;li&gt;The wire between our power source and switch is damaged&lt;/li&gt;
&lt;li&gt;The wire between the switch and light is damaged&lt;/li&gt;
&lt;li&gt;The wire between the light and battery is damaged&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Our switch is a regular toggle switch. The bulb is a standard incandescent bulb, and the power source is a battery that is switched periodically by a maintenance person.&lt;/p&gt;

&lt;h2&gt;
  
  
  The process
&lt;/h2&gt;

&lt;p&gt;We will want to break down the issue, collect information, then attempt to isolate the issue. When isolating the issue, we want to first use information available to us to get as much intuition about the problem as we can, before starting the potentially time-consuming process of troubleshooting. In many cases, it is possible to narrow down the source of the problem substantially, without ever looking at code.&lt;/p&gt;

&lt;h2&gt;
  
  
  Breaking down the issue
&lt;/h2&gt;

&lt;p&gt;It is helpful to consider each component (power source, switch, light) and each pair of components (each wire) in turn and ask yourself a few questions.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What kinds of issues can the component experience? &lt;/li&gt;
&lt;li&gt;What would be true if the problem is X?&lt;/li&gt;
&lt;li&gt;What tools are available to verify retroactively whether the problem observed in the bug report corresponds with the problem being X? *&lt;/li&gt;
&lt;li&gt;What tools are available to verify this as I try to reproduce this? **&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The results from considerations like this come in two forms. The first is a set of to-dos for yourself when you troubleshoot the issue. The second is a set of questions for the reporter of the issue.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;In a software scenario, these might be logs, alerts, or monitoring
** In a software scenario, these might be a debugger&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Light
&lt;/h3&gt;

&lt;h4&gt;
  
  
  What kinds of issues can the component experience?
&lt;/h4&gt;

&lt;p&gt;For the sake of our simple example, we will say that our light bulb is such that it either works, or the filament (the piece of metal in an incandescent bulb that heats up) has burnt out, and the light will never light again.&lt;/p&gt;

&lt;h4&gt;
  
  
  What would be true if the light is broken?
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;The issue would be 100% reproducible. As of a certain time, the light never turns on when the switch is toggled.&lt;/li&gt;
&lt;li&gt;Other lights on the same circuit would continue to work.&lt;/li&gt;
&lt;li&gt;If connected directly to another power source, the light would not turn on.&lt;/li&gt;
&lt;li&gt;Even though there is a signal running to the light (electrical current), the light does not turn on.&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify retroactively?
&lt;/h4&gt;

&lt;p&gt;There might be other corroborating reports within the same time frame that indicate that the issue has occurred. There would not be a single case where after this issue was reported, where someone has experienced the light turning on.&lt;/p&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify while troubleshooting?
&lt;/h4&gt;

&lt;p&gt;A multimeter could be used to check if we can detect current running to and away from the light bulb. If there is current, then it is likely that the light bulb is the source of the issue. However, if there is no current, we cannot confirm that the light bulb is not part of the problem! Sometimes, we are unfortunate, and there are multiple problems all contributing to the reported behavior. &lt;/p&gt;

&lt;p&gt;I can also investigate other lights on the same circuit to observe their behavior. If these lights are behaving correctly, then the wiring to this circuit is likely to be intact all the way through. This does not provide us a smoking gun, but it does increase the likelihood that this light bulb is the problem.&lt;/p&gt;

&lt;p&gt;Finally, if I wish to isolate the light, I can remove it from the circuit and attach it directly to another power source, with wires that are known to be healthy, and see if the bulb turns on. Sometimes, in software, the analogous approach to this is not possible, but when it is, it can oftentimes be the most direct and definitive way to test.&lt;/p&gt;

&lt;h3&gt;
  
  
  Switch
&lt;/h3&gt;

&lt;h4&gt;
  
  
  What kinds of issues can the component experience?
&lt;/h4&gt;

&lt;p&gt;Similar to the light, we will say that the switch either works or it doesn't.&lt;/p&gt;

&lt;h4&gt;
  
  
  What would be true?
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;Issue would be 100% reproducible.&lt;/li&gt;
&lt;li&gt;If the switch were removed from the circuit, the light would turn on and remain lit as long as the power source is active.&lt;/li&gt;
&lt;li&gt;Other lights in the system that are not connected through a switch would continue to work.&lt;/li&gt;
&lt;li&gt;Any other lights connected to the switch would not work.&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify retroactively?
&lt;/h4&gt;

&lt;p&gt;Same as the light.&lt;/p&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify while troubleshooting?
&lt;/h4&gt;

&lt;p&gt;We can use the multimeter to check whether there is current running through the switch when it is in the ON position. We can also try to attach different bulbs to this wiring. The deduction process is the same as it was for the light source.&lt;/p&gt;

&lt;p&gt;See if you can think through the possible deductions we can make by implementing these techniques.&lt;/p&gt;

&lt;h3&gt;
  
  
  Power source
&lt;/h3&gt;

&lt;h4&gt;
  
  
  What kinds of issues can the component experience?
&lt;/h4&gt;

&lt;p&gt;Our power source is a battery, so it can run out of stored energy. When it is low on energy, it can provide inconsistent levels of power.&lt;/p&gt;

&lt;h4&gt;
  
  
  What would be true?
&lt;/h4&gt;

&lt;p&gt;Since we have two potential problems here with different traits, we should make sure to consider both of them.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Issue would not be 100% reproducible, because the battery may have died and been replaced, or it may be low.&lt;/li&gt;
&lt;li&gt;If the battery is currently dead and has not been replaced, the issue would be 100% reproducible.&lt;/li&gt;
&lt;li&gt;Other lights on the circuit would experience the same issues at the same time.&lt;/li&gt;
&lt;li&gt;If the battery is currently low, we may see the light dimming periodically. This is the only case where we have observed dimming behavior.&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify retroactively?
&lt;/h4&gt;

&lt;p&gt;In addition to corroborating reports, I can check the maintenance logs to see if the battery has been changed recently. If the battery has been changed after the issue was reported, the issue may be resolved now, but I will still need to make sure.&lt;/p&gt;

&lt;h4&gt;
  
  
  What tools are available to verify while troubleshooting?
&lt;/h4&gt;

&lt;p&gt;If I do not find maintenance logs, I can test the battery directly, using a multimeter. If it is just as easy for me to verify directly, I will prefer to do this even before checking the logs. It is of course possible that even a new battery may be either defective, or may have been drained prematurely due to a short, so simply knowing that it was replaced, does not definitively prove the issue is resolved.&lt;/p&gt;

&lt;h3&gt;
  
  
  Wiring
&lt;/h3&gt;

&lt;p&gt;So far, we have identified 3 wires.  Based on what we have outlined in the system so far, the behavior would be the same for each of them.&lt;/p&gt;

&lt;h4&gt;
  
  
  What kinds of problems can the wires have?
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;The connection between the wire and either of the components could have come loose completely.&lt;/li&gt;
&lt;li&gt;The wire itself could be frayed, causing a faulty connection, or the connection could be poor, but not completely loose.&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  What would be true?
&lt;/h4&gt;

&lt;p&gt;Since we have two potential problems here with different traits, we should make sure to consider both of them.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;If the connection has come loose, the issue would be 100% reproducible. &lt;/li&gt;
&lt;li&gt;If the connection has come loose, we would observe no current in the circuit.&lt;/li&gt;
&lt;li&gt;If the wire is frayed, or the connection is poor, we would observe the light sometimes turning on when the switch is turned on.&lt;/li&gt;
&lt;li&gt;If the wire is frayed, or the connection is poor, we would observe flickering.&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify retroactively?
&lt;/h4&gt;

&lt;p&gt;Corroborating reports.&lt;/p&gt;

&lt;h4&gt;
  
  
  What tools do I have to verify while troubleshooting?
&lt;/h4&gt;

&lt;p&gt;We are able to visually inspect each connection. Some of the wires may be hidden, so we may or may not be able to inspect them. The multimeter, placed at the ends of each wire, can help to observe the flow of current and check for voltage fluctuations.&lt;/p&gt;

&lt;h2&gt;
  
  
  Gathering information
&lt;/h2&gt;

&lt;p&gt;Although the initial bug report was very vague, we were able to develop some intuition around the issue by breaking it down and formulating possible questions to ask, either of the reporter of the bug, or of the support team. Of course, some of what we were able to learn so far might lead us towards having to do some investigation. &lt;/p&gt;

&lt;p&gt;In the case of our example, the investigation might be simply trying to reproduce the issue and observing the behavior. Do &lt;em&gt;we&lt;/em&gt; see the light misbehaving? Is it flickering when it does so? &lt;/p&gt;

&lt;p&gt;In the case of software, the investigation might involve looking at some system logs.&lt;/p&gt;

&lt;p&gt;This step is important in saving our time down the line, so that we pursue only the most relevant leads. For instance, if the light turns on at least some of the time, then we know it's not the light. If there is another light bulb powered by the same switch, and it works correctly 100% of the time, then it's not the switch, and likely not the power source.&lt;/p&gt;

&lt;p&gt;Perhaps we are able to look at maintenance logs and see that the battery has been changed once a month, every month for a year, but then has not been changed for the last 3 months. It may not be necessary to spend a lot of time digging into the issue, as it is unlikely that the new battery is able to last substantially longer.&lt;/p&gt;

&lt;p&gt;We continue this process and exclude possible causes as we go, to narrow our search. Sometimes, only one possibility remains.&lt;/p&gt;

&lt;h2&gt;
  
  
  Troubleshooting
&lt;/h2&gt;

&lt;p&gt;On the other hand, if we have gathered all of the information we could, and made all of the deductions we can make, but still do not have a clear cause, then we must troubleshoot. &lt;/p&gt;

&lt;p&gt;This is the part where we revisit the information we have noted down during the phase of breaking down the problem and start to evaluate it. We can go through each of the "must be true" statements above, and apply our tools to see if we can eliminate further. &lt;/p&gt;

&lt;p&gt;For instance, we have received no information about any other lights. Do any even exist? Perhaps the entire circuit actually resembles the diagram below&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--eqC6icBT--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vwe5fzggi5i1wwep1k07.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--eqC6icBT--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vwe5fzggi5i1wwep1k07.png" alt="Image description" width="569" height="567"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Whether or not some of these lights function correctly may tell us where the problem is. For instance, if the light labeled "1" turns on, then we know the issue is not with our switch, and not with the wiring, because a faulty wire would cause all lights to malfunction.&lt;/p&gt;

&lt;h2&gt;
  
  
  Additional tips and thoughts
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Reproducibility
&lt;/h3&gt;

&lt;p&gt;In most cases, it is worth establishing right away whether the issue is reproducible 100% of the time, or only sometimes. If it is only reproducible sometimes, it is also good to establish any characteristics of when it is reproducible. For example: "Each morning", "Once every hour or so", "Exactly every third time".&lt;/p&gt;

&lt;p&gt;This information is a great heuristic to keep in mind even as you build out your mental model of the problem and the system, and can be a good way to start brainstorming.&lt;/p&gt;

&lt;h3&gt;
  
  
  Deductive Reasoning
&lt;/h3&gt;

&lt;p&gt;The process outlined here is effectively a combination of system knowledge, intuition, and deductive reasoning. As you practice, you will strengthen all three of these skills. System knowledge and intuition will be strengthened naturally through practice, but you may need to pay extra attention to your deductive reasoning skills.&lt;/p&gt;

&lt;p&gt;Most Computer Science educations cover some form of &lt;a href="https://en.wikipedia.org/wiki/First-order_logic"&gt;First-Order Logic&lt;/a&gt;. If you haven't been exposed to this, or if it has been a long time, then it may be worthwhile to get acquainted with the concepts. It is not usually necessary to practice this formally, but keeping it in your mind as you go throug the process will help to make sure you do not miss things.&lt;/p&gt;

&lt;p&gt;This is a large topic, and I will not attempt to teach it in this article, but will rather leave you, dear reader, with this riddle:&lt;/p&gt;

&lt;p&gt;If some doctors are men, and some men are tall, does it follow that some doctors are tall? &lt;/p&gt;

&lt;p&gt;I'm looking forward to hearing your answers in the comments, as well as your reasoning! Please let me know if you like this content.&lt;/p&gt;

&lt;p&gt;Happy troubleshooting!&lt;/p&gt;

</description>
      <category>troubleshooting</category>
      <category>learning</category>
    </item>
    <item>
      <title>AI is making our jobs harder, not easier</title>
      <dc:creator>Kirill Birger</dc:creator>
      <pubDate>Tue, 18 Jul 2023 23:38:59 +0000</pubDate>
      <link>https://dev.to/kbirgerdev/ai-is-making-our-jobs-harder-not-easier-4031</link>
      <guid>https://dev.to/kbirgerdev/ai-is-making-our-jobs-harder-not-easier-4031</guid>
      <description>&lt;p&gt;Or it will be, when we learn more about the implications, both ethical and societal.&lt;/p&gt;

&lt;p&gt;Already, we find ourselves getting suspicious all the time, of content we see. We question whether content was generated by man or machine.&lt;/p&gt;

&lt;p&gt;How will this affect the way that we, those who were around for this paradigm-shifting moment in human history, think and see the world, compared to generations before and after us? Only time will tell.&lt;/p&gt;

&lt;p&gt;However, upon reading &lt;a href="https://code.visualstudio.com/blogs/2023/03/30/vscode-copilot#_responsible-ai"&gt;this article&lt;/a&gt;, I began to think. Chris Diaz is great for calling out the issue of responsibility directly, and it's good to see industry leaders like Microsoft show a strong example of values.&lt;/p&gt;

&lt;p&gt;While we can, and will debate the issues of ethical responsibility on the side of Microsoft, and other organizations providing similar products and services, we must also accept responsibility ourselves.&lt;/p&gt;

&lt;p&gt;We are now the adults. As Diaz says, this technology is still new. This child is growing, and learning. We are now parents!&lt;/p&gt;

&lt;p&gt;We are still very far away from creating an AGI, or anything close, and I have personal doubts that we will ever even get there. However, this technology is already starting to shape the way we act. &lt;/p&gt;

&lt;p&gt;We can use the tools to be more productive, but we still have to make sure we research and understand the code we generate.&lt;/p&gt;

&lt;p&gt;The tool learns based on other code on GitHub. So, if there is a common coding error present in a lot of code on GitHub, and the LLM trains on it, the same error will be duplicated, mutated and suggested to authors of other code.&lt;/p&gt;

&lt;p&gt;This is where work gets hard. When we come up with a solution to a problem, as engineers we break it up, and understand each piece. We must continue to remember to do so with code generated by a machine.&lt;/p&gt;

&lt;p&gt;Confidently wrong. This is after all the same technology as ChatGPT. The suggestions have a certain compelling quality to them. As humans we will tend to build repetition bias. The more often that we get good results out of AI generated code, the more we will tend to trust it, and that's dangerous. It is not logical to conclude that a result it right simply because the source has been reliable recently. &lt;/p&gt;

&lt;p&gt;At least for the foreseeable future, we are the ones responsible for our code, not the AI, and not the vendor. Even as it speeds up certain tasks, we must remember to slow down and maintain our engineering rigor to make sure that the results are correct.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>productivity</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>Vertical patterns for organizing backend code</title>
      <dc:creator>Kirill Birger</dc:creator>
      <pubDate>Tue, 30 May 2023 12:59:15 +0000</pubDate>
      <link>https://dev.to/kbirgerdev/vertical-patterns-for-organizing-backend-code-1lm2</link>
      <guid>https://dev.to/kbirgerdev/vertical-patterns-for-organizing-backend-code-1lm2</guid>
      <description>&lt;h2&gt;
  
  
  Synopsis
&lt;/h2&gt;

&lt;p&gt;There is no shortage of articles prescribing and explaining the use of specific design patterns. Many of them do a great job. Rather than focusing on a specific design pattern, the the purpose of this article is to present a helpful way to organize your webservice or microservice code to maximize maintainability and testability. By having a structured way to think about code design, we can achieve much more with &lt;/p&gt;

&lt;p&gt;I generally like to think of design patterns as belonging to one of two axi: horizontal and vertical.&lt;/p&gt;

&lt;p&gt;All code can be organized in layers of concern. This is equally true for desktop or command line applications as well as web applications, although some layers can be different.&lt;/p&gt;

&lt;p&gt;Horizontal patterns are those which generalize or abstract complexity across a single layer. Many web frameworks provide the concept of a controller. Controller is a pattern that is horizontal. That is, you may have several controllers, that perform the same sort of functionality across different areas of your application that are all concerned with HTTP. &lt;/p&gt;

&lt;p&gt;Vertical design patterns abstract complexity across different layers of code, but for a single, specific area of concern. This may involve the way that a specific web service framework structures its stack of components to handle a single request from start to finish. For example, a request to your web service will usually cross the HTTP, Business logic, and data layers of your application. Most frameworks lay this out for the developer, but it is important to understand how to use what is provided effectively.&lt;/p&gt;

&lt;p&gt;The reason I find this metaphor effective is because it works on a geometric level as well. In geometry, a vertical line is perpendicular to a vertical line, and thus has an intersection. Similarly, vertical patterns will intersect with horizontal patterns. There are many controllers, which form a horizontal "line", but each controller is also part of a vertical "line" for a set of given requests. When leveraged properly, you have well organized, maintainable, and testable code.&lt;/p&gt;

&lt;p&gt;The following diagram lays out some horizontal and vertical patterns that are common in many application frameworks. It may be helpful to refer back to the diagram as parts of this article will cover some of the ideas that it lays out.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbn20ivrwpqv81bklmoyz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbn20ivrwpqv81bklmoyz.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Lifecycle of a web request
&lt;/h2&gt;

&lt;p&gt;This is a generalized view of a web request&lt;br&gt;
&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvi2mqlb6301eoqr40yf7.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvi2mqlb6301eoqr40yf7.png" alt="Web Request Life Cycle"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;HTTP is a text-based protocol. When your web service receives a request, it is actually receiving a series of bytes that is decided into text. Different frameworks expose this in different ways, but for most use cases, we do not need to concern ourselves with how this happens. Nonetheless, for completeness, I will include it.&lt;/p&gt;

&lt;p&gt;This leaves us with the following concerns, which happen at different stages:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Deserialization: the bytes are deserialized into data structures that we deal with&lt;/li&gt;
&lt;li&gt;Cross-cutting handling of requests: usually implemented with something resembling a middleware.&lt;/li&gt;
&lt;li&gt;Request routing&lt;/li&gt;
&lt;li&gt;Business logic&lt;/li&gt;
&lt;li&gt;Error handling&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Key Concepts
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Single Responsibility
&lt;/h3&gt;

&lt;p&gt;We will focus primarily on one of the SOLID principles. Namely, the Single Responsibility principle. I will not go into depth on this subject as it has been covered extensively. Briefly, as the name suggests, it is the idea that a class should have a single responsibility. In our case, we will be looking at common web service constructs and their responsibilities.&lt;/p&gt;

&lt;h3&gt;
  
  
  Vertical layers of a request and responsibilities
&lt;/h3&gt;

&lt;h4&gt;
  
  
  Web Layer / API Layer
&lt;/h4&gt;

&lt;p&gt;This is the first layer activated by a request. It is responsible for handling HTTP complexity. It is the only layer which should be directly exposed to the representation of the request and response. This layer deals with request headers, query strings, encoding, response generation, etc.&lt;/p&gt;

&lt;p&gt;It is the application boundary on the incoming side.&lt;/p&gt;

&lt;p&gt;This layer must perform no business logic.&lt;/p&gt;

&lt;h4&gt;
  
  
  Application layer
&lt;/h4&gt;

&lt;p&gt;This is the layer which contains all of the business logic for your application, and the associated complexity. You may structure code in this layer in the way that is most appropriate for the functionality you build, but it must not be exposed to any HTTP internals, and must not directly influence HTTP response construction.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;A good rule of thumb is to ask yourself: could I put this code behind a different transport mechanism (such as a message bus, queue, or command line) without changing it?&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;It is important to have good error handling in this layer. Good error handling will explain the cause of the error, and usually provide some sort of machine readable mechanism that can be interpreted to distinguish classes of errors.&lt;/p&gt;

&lt;h4&gt;
  
  
  IO Layer
&lt;/h4&gt;

&lt;p&gt;Most web services do not only perform computations, but also perform IO. For the purposes of this topic, it does not matter whether that IO takes the for  of file system activity, HTTP, or database.&lt;/p&gt;

&lt;p&gt;The IO layer should be completely unaware of the context of a web request, and should have no business logic in it. It serves as another application boundary.&lt;/p&gt;

&lt;h3&gt;
  
  
  Common Framework constructs
&lt;/h3&gt;

&lt;p&gt;Different frameworks name these differently, and not all frameworks provide each of these. What is important is to understand the concepts and decide how to apply them.&lt;/p&gt;

&lt;h4&gt;
  
  
  Web Layer / API Layer
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;Middleware and interceptors: cross-cutting constructs which can define operations to be applied in the Web layer for any number of endpoints. There are typically used for concerns such as logging, auth, caching, serialization, content type negotiation, and certain kinds of validation&lt;/li&gt;
&lt;li&gt;Error Filters or Error Handlers: these are a class of construct that are responsible for converting errors that bubble up from your code into HTTP responses. Sometimes they may be used for error logging, but it is preferable to do that in Middleware.&lt;/li&gt;
&lt;li&gt;Controllers and Route Handlers: these are responsible for mapping a request to an entry point to some business logic for handling the operation. If your framework does not provide other, more specialized constructs, they may be used to format a response, or convert errors to HTTP responses (if error filters are not available). Many frameworks provide an automatic way to tokenize the request route into parameters, parse query strings, and deserialize the body, so that this layer can receive these as data structures. If your framework does not do this for you, you should do so in the controller. Controllers should be kept as light as possible, to minimize testing effort. It is generally a good idea to let the controller call a single method or function and handle the HTTP complexity associated with its result.&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  Application business logic layer
&lt;/h4&gt;

&lt;p&gt;Typically, frameworks do not provide anything for this layer. Many frameworks do provide a dependency injection framework to help orchestrate your business code.&lt;/p&gt;

&lt;p&gt;As such, we will simply say that you should provide a "service" for your controller to call. In the context of DI a service is an implementation of business logic that can be instantiated by some reference to it, such as an injection token. This class can orchestrate any work necessary to perform a required operation.&lt;/p&gt;

&lt;h4&gt;
  
  
  IO Layer
&lt;/h4&gt;

&lt;p&gt;Similarly to above, the only common touch point between your framework and the IO layer is the DI framework.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;By splitting our code up in this fashion, and by creating well-defined interfaces between these layers, we are able to write code that is maintainable and easy to understand. It also becomes clear which patterns make sense to use in which parts of the code. To help in this task, we can also try to think of each layer as its own product, with its own contracts and supported features. By doing so, it forces us think more carefully about what we expose, and how.&lt;/p&gt;

&lt;p&gt;Do you have other ways to split your code? Do think there are other patterns that are valuable to understand in this context? Please write in the comments below.&lt;/p&gt;

</description>
      <category>backend</category>
      <category>designpatterns</category>
      <category>microservices</category>
    </item>
    <item>
      <title>Abstracting remote pagination through TypeScript generators</title>
      <dc:creator>Kirill Birger</dc:creator>
      <pubDate>Sat, 18 Feb 2023 02:40:00 +0000</pubDate>
      <link>https://dev.to/kbirgerdev/abstracting-remote-pagination-through-typescript-generators-3phk</link>
      <guid>https://dev.to/kbirgerdev/abstracting-remote-pagination-through-typescript-generators-3phk</guid>
      <description>&lt;h2&gt;
  
  
  Background
&lt;/h2&gt;

&lt;p&gt;One of my passions is to write code which is clean and maintainable. While there are several things that go into writing code that is easy to understand and update, one major one that I coach people on is complexity.&lt;/p&gt;

&lt;p&gt;As engineers, we face complex tasks on a daily basis, and it is easy to embrace the chaos. However, I think we've all been in a situation where we are asked to add a feature to some code only to find functions that span several screens, with a dozen variables, and types like &lt;code&gt;PersonObject | boolean&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;One thing that can help us with such code is to encapsulate complexity using any of a number of patterns that we have been exposed to. However, some things are easier to abstract than others.&lt;/p&gt;

&lt;h2&gt;
  
  
  Overview
&lt;/h2&gt;

&lt;p&gt;In this post, I'd like to propose to you a nice technique for dealing with a paginated API when you have to iterate through an indeterminate number of results. &lt;/p&gt;

&lt;p&gt;While pagination is a great solution to retrieving variable amounts of data from an api. Dealing with pagination logic is not the most challenging task many readers will face, but it is also an unnecessary distraction in most cases. &lt;/p&gt;

&lt;p&gt;Therefore, we will discuss a technique which lets you conceal that logic from downstream code using a JavaScript/TypeScript concept known as generators. The code in this post is in TypeScript, but the idea is equally applicable to JavaScript projects.&lt;/p&gt;

&lt;h2&gt;
  
  
  Generators
&lt;/h2&gt;

&lt;p&gt;This topic is widely covered on the web and in other blog posts, so I will be brief about the overall concept and focus more on a specific set of use cases. If you want a deep dive into Generators, I recommend reading the following post first:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://dev.to/gsarciotto/generators-in-typescript-4b37"&gt;dev.to: Generators in Typescript&lt;/a&gt;&lt;br&gt;
This is a great post explaining the details &lt;/p&gt;

&lt;p&gt;Additional resources are found at the end.&lt;/p&gt;
&lt;h2&gt;
  
  
  Brief overview
&lt;/h2&gt;

&lt;p&gt;We are all likely familiar with code resembling the following:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;people&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;Person&lt;/span&gt;&lt;span class="p"&gt;[]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;fetchPeople&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;person&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;people&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
   &lt;span class="nx"&gt;printPerson&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;person&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because &lt;code&gt;people&lt;/code&gt; is an array, it is an &lt;code&gt;Iterable&lt;/code&gt;, and we can use the &lt;code&gt;for-of&lt;/code&gt; syntax to iterate over it. However, arrays are not the only iterable type in JS/TS.&lt;/p&gt;

&lt;p&gt;You can implement a generator function, which will execute normally until it hits a &lt;code&gt;yield&lt;/code&gt; directive. At this point, function execution will pause, and return (yield) a value. Using language functionality, the function execution can be resumed to yield further values. Consider the following&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;fetchPeople&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Alice&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Bob&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;person&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;people&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;person&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// Prints:&lt;/span&gt;
&lt;span class="c1"&gt;// Alice&lt;/span&gt;
&lt;span class="c1"&gt;// Bob&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;*&lt;/code&gt; operator following the &lt;code&gt;function&lt;/code&gt; keyword tells our parser that this function returns a generator. TypeScript would define the return type of this function as &lt;code&gt;Generator&amp;lt;string, void, unknown&amp;gt;&lt;/code&gt;. &lt;/p&gt;

&lt;p&gt;Recent versions of TypeScript also support &lt;code&gt;AsyncGenerator&lt;/code&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;fetchPeopleAsync&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Alice&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Bob&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;person&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;people&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;person&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// Prints:&lt;/span&gt;
&lt;span class="c1"&gt;// Alice&lt;/span&gt;
&lt;span class="c1"&gt;// Bob&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I will provide links at the bottom of this post that go into this topic in more depth.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conventional pagination code
&lt;/h2&gt;

&lt;p&gt;This is the definition of a &lt;code&gt;Widget&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kr"&gt;interface&lt;/span&gt; &lt;span class="nx"&gt;Widget&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;price&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Let's say we have a repository class for retrieving Widgets from a remote data source. There are tens of thousands of widgets, so it is impractical to return all of them at once. You have been assigned a task to retrieve and print every widget that contains the word "cat" (this is the Internet, after all, and we love cats).&lt;/p&gt;

&lt;p&gt;You search the API documentation, and much to your chagrin, there is no filtering capability on the remote API. :sadkitty:&lt;/p&gt;

&lt;p&gt;You may end up writing code resembling this&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="cm"&gt;/** 
 * Retrieves an array of `Widget`s from a remote source
 * @param nameFilter string indicating how to filter widget names
 */&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="nx"&gt;findWidgets&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// we fetch widgets 3 at a time&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;take&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;Widget&lt;/span&gt;&lt;span class="p"&gt;[]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;

    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;index&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;total&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nx"&gt;take&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;total&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nx"&gt;total&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="c1"&gt;// take one "page" of widgets&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;repository&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;getSome&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;index&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;take&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="c1"&gt;// track our total, so we do not make extra requests&lt;/span&gt;
      &lt;span class="nx"&gt;total&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;total&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

      &lt;span class="c1"&gt;// filter the page&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;filteredPage&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
      &lt;span class="nx"&gt;index&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="c1"&gt;// most succint, though not the most efficient way to add several items to an array&lt;/span&gt;
      &lt;span class="nx"&gt;widgets&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[...&lt;/span&gt;&lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;&lt;span class="nx"&gt;filteredPage&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;// return an ARRAY of widgets&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's a lot of code. We have to manage the state of iterating over our data source, we have to make sure our conditions are correct for doing so, and we have to use a while loop. If we needed to do something like limit the number of widgets we return, or paginate back to the caller of &lt;code&gt;findWidgets&lt;/code&gt;, it gets even more complicated.&lt;/p&gt;

&lt;p&gt;We also collect everything into an array and return it, which can be memory intensive.&lt;/p&gt;

&lt;h2&gt;
  
  
  Adding a generator
&lt;/h2&gt;

&lt;p&gt;As opposed to arrays, which are eagerly stored in memory in their entirety, Generators give the caller of your function or method the ability to control the flow of data. That is, in the above example, if the caller of &lt;code&gt;findWidgets&lt;/code&gt; needs to look at the results, evaluate them in some way, and may decide that only the first 3 are needed, then we have wasted the time in retrieving all of the remaining widgets, and we have wasted the memory required to store them all in the array.&lt;/p&gt;

&lt;p&gt;What if we used a generator for exposing a different API for retrieving widgets?&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="cm"&gt;/**
 * Returns an `AsyncGenerator` for widgets
 * @param bufferSize number of widgets to fetch at a time from remote
 */&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;getAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;bufferSize&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;index&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;total&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nx"&gt;take&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;total&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nx"&gt;total&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="c1"&gt;// take one "page" of widgets&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;repository&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;getSome&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;index&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;bufferSize&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="c1"&gt;// track our total, so we do not make extra requests&lt;/span&gt;
      &lt;span class="nx"&gt;total&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;total&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;

      &lt;span class="nx"&gt;index&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is not our filter function yet, although it resembles it very much. Here we are separating our fetching logic out from  our filter. There are a few important things to note:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;We have removed our &lt;code&gt;return&lt;/code&gt; statement&lt;/li&gt;
&lt;li&gt;We have removed our array, so we no longer store a big buffer in memory&lt;/li&gt;
&lt;li&gt;We now fetch a page of Widgets and &lt;code&gt;yield&lt;/code&gt; them one at a time. This is known as buffering.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We can now write any number of functions that operate on some subset of our widgets iteratively and never repeat paging logic or have to store extra data in memory.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nx"&gt;findWidgets&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;matchedWidgets&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;getAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nx"&gt;matchedWidgets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;matchedWidgets&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This version of the function will fetch widgets in the background from the generator and gives us a nice iterable interface to scan widgets. It shows that anyone who uses our &lt;code&gt;getAll&lt;/code&gt; function has the ability to fetch widgets at their own rate, and only the amount that they want, without having to specify additional parameters to &lt;code&gt;getAll&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Unfortunately, in this version we are still collecting the widgets into an array, which can ge large in memory. But using the techniques discussed so far, we can improve this further.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;findWidgets&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;getAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;matchedWidgets&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now, we can even compose and chain these functions the same as we do with arrays.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;findWidgetsByName&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;AsyncGenerator&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;Widget&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;void&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;unknown&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
   &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
     &lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;nameFilter&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
       &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
     &lt;span class="p"&gt;}&lt;/span&gt;
   &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;findWidgetsByPrice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;maxPrice&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;AsyncGenerator&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;Widget&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;void&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;unknown&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
   &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;widgets&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
     &lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;price&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nx"&gt;maxPrice&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
       &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
     &lt;span class="p"&gt;}&lt;/span&gt;
   &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nx"&gt;catWidgetApp&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// For feline lovers on a budget&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;cheapCatWidgets&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; 
    &lt;span class="nx"&gt;findWidgetsByPrice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
      &lt;span class="mf"&gt;25.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nx"&gt;findWidgetsByName&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;cat&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;getAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;widget&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;cheapCatWidgets&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// imagine this function uses a stdin library like inquirer to present an interactive prompt to the user asking if they want to buy the widget&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;shouldBuy&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;promptUser&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="c1"&gt;// if the user elects to buy this widget, we do so, and exit. Otherwise, we continue to iterate, possibly fetching more widgets&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;shouldBuy&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;buyWidget&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;widget&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;This example shows the elegance and simplicity of using generators for cases where we would like to abstract away the complexity of extracting data from a data source from the code consuming that data. In the above function, we compose three async generators to produce a final one that we iterate over. I have also illustrated our ability to control the rate of flow arbitrarily from calling code. That is, we do not fetch all of our widgets at once. We may fetch the first set of 3 widgets, and then pause for several minutes, while the user makes a decision. If the user gets through the first 3 widgets and does not want to buy them, only then do we fetch anything else. Best of all, this is completely transparent to the calling code.&lt;/p&gt;

&lt;p&gt;Did you enjoy this article? Do you have any other ideas for the usage of Generators? Are there any other subjects you'd like to see me write about? Please comment below!&lt;/p&gt;

&lt;h2&gt;
  
  
  Further resources on generators
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncGenerator"&gt;https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncGenerator&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-6.html"&gt;https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-6.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://basarat.gitbook.io/typescript/future-javascript/generators"&gt;https://basarat.gitbook.io/typescript/future-javascript/generators&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-3.html"&gt;https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-3.html&lt;/a&gt;&lt;/p&gt;

</description>
      <category>typescript</category>
      <category>webdev</category>
      <category>maintainability</category>
      <category>javascript</category>
    </item>
  </channel>
</rss>
