DEV Community: boroskoyo

Using Grafana to visualize CI Workflow Stats

boroskoyo — Thu, 02 Mar 2023 08:43:53 +0000

Grafana is an excellent tool for creating dashboards to gather data from various sources and serve them in a single place. According to Newstack, Grafana is the second most popular observability tool, counting 800,000 active installs and over 10 million users. These numbers also include some of our customers, and we have collected much feedback about bringing Foresight’s data & charts to Grafana Dashboards.

Using a dashboard like Grafana helps teams have a whole view of what the team is doing. Improving visibility and having a streamlined analysis for faster problem-solving. When technical issues arise, having all the data in a single place can help you solve them more quickly. Instead of spending time gathering data from multiple sources, you can access everything you need in one location. This can reduce downtime and minimize the impact of technical issues on productivity.

Makers define Grafana as “ the open source analytics & monitoring solution for every database” and we wanted to make Foresight one of those. Making it easier for our users to observe their CI workflow health & stats.

Introducing Foresight Plugin for Grafana!

Foresight is a powerful tool for monitoring and managing your continuous integration (CI) and testing workflows within GitHub Actions. With Foresight, you can easily track key metrics related to your workflows, such as build times, failure rates, etc. Now, with the Foresight plugin for Grafana, you can bring those insights directly into your Grafana dashboards. This plugin enables you to visualize Foresight’s workflow stats, and workflow failure heatmap features directly within Grafana, making it easier than ever to monitor the performance of your workflows and take action when issues arise. Whether you’re a developer, an operations team member, or a project manager, the Foresight plugin for Grafana can help you stay on top of your workflows and improve your team’s productivity.

This panel plugin comes with two tabs,

Stats of the workflows:

Total execution count
Fail & Success Rate
Fail count
Average Duration
P95

Workflow failure heatmap

number of errors per day & workflow

Many more features are coming, and your feedback is highly appreciated.

In conclusion, Grafana is a powerful data visualization tool that can help you make sense of your technical data and increase productivity. By bringing all your technical data to a central location and visualizing it in meaningful ways, you can gain insights into your operations, identify areas for improvement, and make data-driven decisions that lead to greater efficiency. With the Foresight plugin for Grafana, you can take your data analysis to the next level by visualizing key CI and testing metrics directly within your Grafana dashboards. To learn more about Grafana and the Foresight plugin, check out the documentation today and explore the power of data-driven decision-making.

https://app.runforesight.com/signup

Originally published at https://www.runforesight.com.

Collecting error stacks from backend as exceptions occur in E2E tests — A tutorial on Cypress

boroskoyo — Mon, 28 Nov 2022 06:59:01 +0000

Discover the missing piece of E2E test results

This article is the first chapter of the Sidekick Stack-based Testing Series. Tools mentioned in this article (Cypress & Sidekick) are both open-source, and after reading it you will learn:

How you can include backend exceptions in your Cypress E2E test results

Exceptions occur in both backend & frontend, and many of them are handled to provide users with trouble-free sessions. Having these error-handling mechanisms is healthy for users’ experience, but they can prevent us, developers, from observing the exceptions and backend-related anomalies in E2E tests.

E2E tests ensure that the application performs as expected by running the test cases based on the users’ workflows. Tools such as Cypress, Playwright & Selenium enable us to easily define and apply those tests. In this article, I’ll explain backend exceptions, a missing piece of E2E test results, and how we can collect them in a test run in Cypress.

Well-known E2E testing tools mentioned above allow us to define tests for each action, and there are countless add-ons to improve their abilities. Since those tests are based on users’ behaviors they are mainly focused on frontend experience, and there are barely any significant resources for backend-related test definitions. Most methods/plugins we have are limited to making requests to backend and defining tests based on responses. Although those might be more than enough for mocking users’ perspective, these tests lack depth about backend’s state. Primarily when exceptions are handled, and there is no trace left of anomalies. In the end, even passing all E2E tests might not mean a system with zero issues.

A way to collect exceptions & stacks from the backend can help us understand the missing pieces in E2E tests. Sidekick’s Test Module comes in handy for this use case. Sidekick’s Test Module and it’s upcoming test framework plugins enable developers to bring Sidekick actions (logpoints, tracepoints & error stack collection) to E2E tests, eventually giving developers ability to collect errors as they happen and define stack based tests to ensure reliability of backend’s state. This article focuses on the first one, collecting backend errors as they occur in a test run.

Sidekick’s Error Stack Collection gathers real-time stack data from backends whenever they have an exception. You can read more about it here:

Capturing Exception Call Stacks from Node.js Applications

boroskoyo ・ Sep 23 '22

#javascript #node #observability #debug

https://medium.com/runsidekick/capturing-exception-call-stacks-in-python-applications-with-sidekick-d4474374659

Error Stack Collection is already helpful for problems in production. With Sidekick’s new Test Module it will also be a literal sidekick to developers for discovering root causes of errors in tests. Such data is nifty for finding neglected issues & root causes of some flaky tests.

Setting up Error Stack Collection within our E2E tests in Cypress :

Our setup will be consisted of a simple Node.js To-Do application powered by Express framework. We will define tests in Cypress as we normally do and observe stacks of exceptions thrown in the backend as a result of those tests.

You can find To-do app’s repo here:

boroskoyo / todos-express-password-sidekick

Todo app using Express and Passport for sign in with username and password.

todos-express-password-sidekick

This example is derived from https://github.com/passport/todos-express-password

This app illustrates how to setup Sidekick Node.js Agent with Express to live debug your applications.

Use this example as a starting point for your own web applications.

Quick Start

To run this app, fork & clone the repository and install dependencies:

$ git clone https://github.com/<your-user-name>/todos-express-password-sidekick.git
$ cd todos-express-password-sidekick
$ npm install

Get your Sidekick API Key: https://docs.runsidekick.com/installation/get-api-key-and-token

Change the < api-key > field at 25th line of app.js with your API Key.

Then start the server.

$ npm start

Navigate to http://localhost:3000.

Next Steps

Connect your source if you are using Web IDE

Check docs
Continue with your IDE
- Visual Studio Code Extension
- WebStorm Plugin
Start debugging!

License

The Unlicense

Credit

Original version Created by Jared Hanson

View on GitHub

Requirements:

Cypress (this can be applied in any other test framework)
Having a Sidekick SaaS account or a Sidekick Open-Source Broker up and running: https://www.runsidekick.com/
A Node.js backend configured to work with Sidekick agent (Sidekick Error Stack Collection in supported in both Node.js & Python runtimes)
- Check the example setup here: https://www.youtube.com/watch?v=tc9TupvV-TM
- Related docs: https://docs.runsidekick.com/installation/agents/node.js/error-stack-collection
Sidekick Test Module up and running:
- An open-source project that can be found here: https://github.com/runsidekick/sidekick-test-mode

Setup:

Before starting our test, we need to setup the Sidekick agent & enable error stack collection.

1- Installing Sidekick agent to our backend

You can find extra info here: https://docs.runsidekick.com/installation/agents/node.js/installation

//add the code below block to top of your project

const SidekickDebugger = require('@runsidekick/sidekick-agent-nodejs');

2- Enable Error Stack Collection mode

//after importing Sidekick Agent, initialize it with the given configuration  
SidekickDebugger.start({   
  applicationName: 'Todo - Express',  
  applicationStage: 'prod',  
  applicationVersion: '1.0.0',  
  apiKey: '<Sidekick-api-key>',  
  errorCollectionEnable: true,  
  errorCollectionEnableCaptureFrame: true  
});

Now your backend is ready for our tests.

3- Configure & Start Sidekick Test Module (check its repo for more info)

Now we are good to go!

Test definition:

The user will login to the system before each test.

First, it will add a new item “Feed the cat” and check if it is added.

Secondly, it will try to add a new item with an empty string and expect nothing to change. Submitting an empty string results in an error in our backend, but we are catching those errors so that user is not inconvenienced.

UI test results in Cypress

Finally, in the last test, we will make a request to Error Stack endpoint of Sidekick Test Module to get collected errors within the error. The expected result is an empty array, and anything else means that we had an issue in our backend.

You can find the example definition below:

  it("Check backend errors", () => {  
    const getErrorStacks = {  
      url: "http://localhost:8081/api/events/errorstack",  
      headers: {  
        Authorization: "Bearer apikey",  
      },  
    };  
    cy.request(getErrorStacks)  
          .its("body")  
          .should("be.an", "array")  
          .and("have.length", 0);  
      });

Results:

As our test completes, we can see that although our frontend-related tests pass, our E2E test fails as Sidekick had collected errors when it tried to submit an empty string in the test run.

Overall test results

If there are errors occur in the backend, Sidekick Test Module should return a list of collected errors and their stacks, as seen below.

Failing test case and collected error information

Now we can click on the related line to get info about the collected error or check the error from any Sidekick client (Web IDE, VSCode, or JetBrains IDEs). Soon we will bring a better view of those errors with our upcoming Cypress plugin.

Detailed information about collected error’s stack

In conclusion, we have used Sidekick agent & Test Module to bring backend errors to our E2E test, and we were able to capture the snapshot of the state of our backend as it threw an exception.

Where to go from here:

With our current setup, we got in-depth error information from our backend apps. In further chapters, we will be bringing Sidekick’s tracepoint (non-breaking breakpoint) feature to Cypress that will allow us to define tests based on state and stack information of our backend. Right now, we are also working on a Cypress plugin that will make using Sidekick Actions within Cypress as easy as it gets, and your support and feedback will guide us through our road. After Cypress, supporting other test frameworks is on our roadmap. You can find us at Discord or contact us via our website https://runsidekick.com if you want to participate or become a part of our community

Live Debugging Python Applications

boroskoyo — Thu, 24 Nov 2022 13:49:16 +0000

Speed up finding the root cause and minimize the impact of such incidents

A few months back, I read an interesting story where astronauts in the International Space Station had to abandon a mission and seek shelter due to a cloud of orbital debris. According to an article by National Geographic, it is a result of Russian anti-satellite missile testing (Nudal ASAT), performed live for the first time on an actual object in orbit.

This incident made me think of how things can act differently in production compared to development and testing environments. As developers, we often face these unexpected scenarios. And sometimes it is impossible to recreate them locally to find solutions.

However, having a standard live debugging mechanism in production will significantly speed up finding the root cause and minimize the impact of such incidents. So, in this article, I will discuss how we can use live debugging in Python applications to give you a better understanding.

Importance of Debugging in Production Environments

As we all know, testing is an essential part of the development process before releasing into the production environment. Especially modern applications go through extensive manual and automatic testing processes to identify potential bugs.

However, its hardly possible detect all issues prior to production since these environments have differences from one another. For example, there are errors, caused by data, access patterns, load in production is difficult to replicate identically in development and testing environments. Therefore, we have to debug such errors in the production.

As developers, we must be aware of different tools and approaches we can use for remote debugging to improve efficiency. Sidekick, is one such tool that provides a web IDE & IDE plugins for real-time application debugging. So, let’s see how we can use Sidekick with Python.

How to Integrate Sidekick with Python Applications

In this article, I will be creating a simple Python application with distributed microservices using Flask. Then, I will connect it to the Sidekick web IDE and discuss how to use it for live debugging for Python applications.

Step 1 — Create a Sidekick account

As the first step, you need to create a Sidekick account. After that, you can easily signup with Sidekick using your Google or GitHub account.

Step 2 — Install sidekick-agent-python to your application

After creating your Python project, you need to install sidekick-agent-python to connect the application to Sidekick.

pip install sidekick-agent-python

Then, you need to configure the environment variables. Sidekick requires the following as environment variables, and you can use a .env file to include them.

SIDEKICK_APIKEY= <API-KEY>  
SIDEKICK_APPLICATION_NAME= POST_APP  
SIDEKICK_APPLICATION_STAGE= PROD  
SIDEKICK_APPLICATION_VERSION= 1.0.0

You can find SIDEKICK_APIKEY on the home page of Sidekick IDE.

Finally, you can add Sidekick to your application and enable tracing as follows:


from pathlib import Path  
from dotenv import load_dotenv  

env_path = Path(<.env file path>).resolve()  
load_dotenv(dotenv_path=env_path)  

try:  
    import tracepointdebug  
    tracepointdebug.start()  
except ImportError as e:  
    pass

Step 3 — Connect the application to Sidekick web IDE

Sidekick allows you to connect GitHub, GitLab, and Bitbucket repositories directly to the web IDE. You can use the source tab in the IDE to select your repository.

Most importantly, you don’t need to worry about the security of your repository since Sidekick will only have read access to your source code, and it will never fetch the code into their servers.

Step 4 — Enabling Tracepoints

After connecting the Git repository, you need to connect your application to Sidekick IDE and enable tracepoints. At the Applications tab you will see a list of applications that use your Sidekick API key.

Please note that your application must be running to get listed in the Applications tab.

Then, you can create tracepoints (non-breaking breakpoints) in the Sidekick editor, and all the tracepoints will be listed in the Tracepoints tab.

That’s it. You have successfully integrated Sidekick for your Python application. Now, you can start monitoring the application using tracepoints.

Debugging with Sidekick

As mentioned, Sidekick tracepoints are a type of nonbreaking remote breakpoints. Tracepoints do not pause your codes as normal breakpoints, and they can also work with Thundra’s distributed technology to enable connections between several tracepoints in a transaction.

After enabling tracepoints, Sidekick will automatically start collecting snapshots of the local variables when the application execution passes that tracepoint. You can find those events in the Sidekick Events tab and filter them by any text or distributed trace IDs.

The above image shows the tracepoint events of the posts service in the example application.

However, taking snapshots every time, even without any issues, might not be the ideal approach. Sidekick provides conditional tracepoints to overcome this issue, and you can enable a tracepoint based on a specific condition you define.

Using Conditional Tracepoints in Sidekick

Although the posts service of the example application worked as expected, the user service didn’t return the expected response. Instead, it returned an empty response when the get users endpoint is invoked. So, let’s place a conditional tracepoint in Sidekick IDE to trigger to find the reason for this behavior.

You have to pick the tracepoint you need to edit from the tracepoint list and click the edit icon to enable conditional tracepoints.

It will open a new window like below, where you can define the condition. In this example, the following tracepoint will only take snapshots if the response status code is not equal to 200.

You need to rerun your application and explore the tracepoint snapshot to get more details.

As you can see, the response code for the API request is 404, and the reason is Not Found. It means that the application can not find the API we are calling. So, there must be an error in the API path. Likewise, you can easily use Sidekick IDE to monitor and debug issues with your production-level applications easily.

Besides, Thundra provides a trace map feature where you can visualize the high-level components in your application and trace its behavior in real-time. It helps to identify issues that span across components and identify what’s causing them. For that, you need to integrate the Thundra APM agent. You can find more details about that in Thundra documentation.

Tracemap of sample ToDo app configured with Sidekick & Thundra

Live Stack Tracing using Sidekick Sidekick & Thundra

The images below show a trace map of a sample ToDo app configured with Thundra & Sidekick and a live stack trace provided by Thundra.

Conclusion

In this article, I have discussed the importance of remote debugging production environments and how to integrate Sidekick with a Python project. Sidekick resolves a burning problem in monitoring and debugging live environments. It has introduced non-breaking breakpoints (tracepoints), and developers can monitor application status without pausing the application workflow.

I hope this article will help you improve the remote debugging of your applications and improve the productivity of your development work. Thank you for reading!

Originally published at https://www.runsidekick.com.

Bringing the simplicity of print() to production debugging using PyCharm

boroskoyo — Thu, 20 Oct 2022 08:45:51 +0000

The way we develop software evolves, way we debug can’t stay the same!

Bringing the simplicity of print() to production debugging using PyCharm

Writing clean and readable code, adding tests and everything is a must for today’s software development teams but it is never enough to avoid bugs from appearing.

As it is explained in Fireship’s latest video, the best way to avoid writing bugs is to avoid writing code.

You might think having a QA team or constantly testing your software is enough but the tweet below is a great example of what you will experience in the real life:

Even though the way we structure our code or develop architectures changes, anomalies, and bugs are here to stay. So we should start by accepting that fact and find ways to better debug experiences.

Encyclopedia.com defines debugging as

“The identification and removal of localized implementation errors — or bugs — from a program or system.”

So the first step is to identify the errors. This is why we have debuggers and thankfully Python ecosystem has a vast number of debugger alternatives. In this article I’ll explain you fundamental debugging tools for Python and new ways to debug your Python applications in production with the comfort of your PyCharm instance.

A Sea Full of Debugging Solutions

There are plenty of solutions for debugging Python applications. Below I’ve listed a few that I’ve found remarkable.

Good ol’ print()

It wouldn’t be fair to publish this article without paying homage to our old friend, print(). It has been a puppy love for all of us and it is by far the easiest one to start with.

Sadly, when code gets complicated over time, print() falls short as we need a more sophisticated arsenal to find the root cause of production bugs.

Check out the article below to learn more about its drawbacks:

https://medium.com/analytics-vidhya/are-you-writing-print-statements-to-debug-your-python-code-690e6ba098e9

pdb

Python comes with its own debugger called pdb that allows you to start a debugging session with an interactive console to inspect your code while it runs. Furthermore, it also allows you to add a breakpoint to your code and inspect stack frames on the go.

In addition to the points above, pdb is easily extendable as is implemented as a class in the standard Python library.

In short, pdb is powerful and easy to use, especially for small to medium-sized projects. One major drawback of using it in production is that you need to edit your code every time you want to debug your application, which in the end will require a redeploy and restart of your application.

Below you can find extra information about pdb:

https://medium.com/pragmatic-programmers/investigating-with-pdb-breakpoints-d75b4127223b

Python Debugging With Pdb – Real Python

In this hands-on tutorial, you'll learn the basics of using pdb, Python's interactive source code debugger. Pdb is a great tool for tracking down hard-to-find bugs and allows you to fix faulty code more quickly.

realpython.com

IDE Debuggers

Many IDEs like Visual Studio Code & PyCharm offer ways to visually debug your Python applications. They provide an easier and more efficient way of debugging

Debugging using IDE’s default debugger is usually the go-to way at the early stage of projects. They are more than enough for daily debugging with the only drawback of lacking real-time data.

Since traditional debuggers work with the code in your local, they require you to reproduce the buggy scenarios locally. For such scenarios where you can’t reproduce, need real-time data, and debug a remote application remote debugging is the way to go.

PyCharm Remote Debugging

Remote debugging is one of PyCharm’s most powerful features that enable IDE to connect running instances on remote servers. This lets you put breakpoints to your running applications just like you do in a local system.

Remote Debugging with PyCharm | PyCharm

jetbrains.com

PyCharm Remote Debugging and other IDE-provided remote debugging alternatives are great to collect data from remote applications but they require additional setup and can be a major source of lag when used in production.

runsidekick.com

For cases when you want to collect stack and exception data and add new logs on-demand to your production & pre-production applications live application debuggers are here to help.

Live application debuggers offer a smoother and more lightweight debugging experience that enables you to collect data from running applications without compromising performance.

One of those live application debuggers is Sidekick. It is open-source and in addition to having traditional features like adding logpoints and putting breakpoints it is packed with extra features like stack collection when an exception occurs and integrations to ingest collected data to targets you desire like Elastic and Loki.

PyCharm Sidekick Plugin

Sidekick PyCharm plugin brings live debugging features to PyCharm making remote debugging hassle-free and lightweight. Enabling developers to debug their Python applications remotely from the comfort of their IDE. Unlike PyCharm remote debugging it is easier to setup, brings near zero overhead, and comes with extra features.

Sidekick brings simplicity of print() to production debugging using PyCharm

Sidekick’s features include:

Tracepoints

A tracepoint is basically a non-breaking remote breakpoint. Tracepoints differ from ordinary breakpoints in two ways:

First, it doesn’t pause the code execution but just takes a screenshot of the variables when the code hits that line.
Second, it’s integrated with Thundra’s distributed technology enabling the connection between several tracepoints in the same transaction.

Logpoints

Logpoints are points that you can add to your running application, just like Tracepoints, for adding logs dynamically.

Since you can add and remove your logs at any time, getting the log wherever and whenever you need it becomes a walk in the park!

Exception Stack Collection

With Sidekick Python agent’s latest update, developers can now activate automatic error stack collection and start collecting exception call stacks from their running applications without stopping them.

Simply we can describe this new feature as an always-on tracepoint that is looking out for uncaught exceptions.

Read on:

https://medium.com/runsidekick/capturing-exception-call-stacks-from-running-node-js-applications-d9cd81407593

Data Redaction

Sidekick agents let you define custom redaction functions that run every time before sending any data to Sidekick. This way you get full control over your data.

You can find the Sidekick PyCharm plugin here:

Sidekick - IntelliJ IDEs Plugin | Marketplace

plugins.jetbrains.com

Sidekick does everything above and more while adding near-zero overhead to your applications. No redeploy and restart needed to start collecting new data. This makes it a perfect tool for a production environment, a true sidekick that you can trust.

One more thing…

It is open-source!

Sidekick and its agents bring a lot to the table and promise to improve the developer experience by far. Furthermore, they are completely open-source so you can both learn how they work, expand them, and self-host your own instances.

You can check out the repository here:

runsidekick / sidekick

Free and open-source live application debugger. Like chrome dev tools but for your backend

Explore Docs » Quick Start Tutorial »

Table of Contents

What is Sidekick?

Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running.

Add dynamic logs and put non-breaking breakpoints in your running application without the need of stopping & redeploying.

Sidekick Open Source is here to allow self-hosting and make live debugging more accessible. Built for everyone who needs extra information from their running applications.

Sidekick Actions:

Sidekick has two major actions; Tracepoints & Logpoints.

A tracepoint is a non-breaking remote breakpoint. In short, it takes a snapshot of the variables when the code hits that line.
Logpoints open the…

View on GitHub

Worth mentioning:

Visual Studio Code

Visual Studio code also has great modern debugging tools and we will further discuss it in another article. In addition to its tools Visual Studio Code also has a Sidekick plugin itself, which you can find here: https://marketplace.visualstudio.com/items?itemName=Sidekick.sidekick-debugger

tldr;

Whether your apps are running on a single EC2, docker or a large k8s cluster, Sidekick allows you to collect realtime stacks, logs and errors from your running applications. All without leaving your favourite IDE.

Visit our site to learn more.

You can get in touch with us through Twitter, and join our Discord community.

Capturing Exception Call Stacks from Node.js Applications

boroskoyo — Fri, 23 Sep 2022 12:41:32 +0000

Sidekick allows you to collect stacks as errors happen, without needing for putting a tracepoint beforehand

Sidekick is an open-source live application debugger that lets you troubleshoot your applications while they keep on running. It is like Chrome DevTools for your and built for everyone who needs extra information from their applications.

runsidekick / sidekick

Free and open-source live application debugger. Like chrome dev tools but for your backend

Explore Docs » Quick Start Tutorial »

Table of Contents

What is Sidekick?

Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running.

Add dynamic logs and put non-breaking breakpoints in your running application without the need of stopping & redeploying.

Sidekick Open Source is here to allow self-hosting and make live debugging more accessible. Built for everyone who needs extra information from their running applications.

Sidekick Actions:

Sidekick has two major actions; Tracepoints & Logpoints.

A tracepoint is a non-breaking remote breakpoint. In short, it takes a snapshot of the variables when the code hits that line.
Logpoints open the…

View on GitHub

Sidekick Actions ( logpoint and tracepoint ) are giving developers dynamic logging and live debugging abilities. They are most powerful when you know exactly where you should put them.

Our recent user research shows that when uncaught exceptions occur, proactively putting tracepoints for stack collection might not be enough as in some rare cases, putting a tracepoint after spotting a problem can’t help until the error happens again. That’s why we have developed;

Automatic error stack collection

With our Node agent’s latest update, developers can now activate automatic error stack collection and start collecting exception call stacks from their running applications without stopping them.

Simply we can describe this new feature as an always on tracepoint that is looking out for uncaught exceptions. Of course this can bring an overhead but we see a great potential to lower MTTR’s for exceptions.

See it in action:

We will be using our quickstart repository from a previous tutorial.

boroskoyo / sidekick-nodejs-quickstart

Quick start example for Sidekick.

Sidekick Quickstart

Simple project to show how you can start using Sidekick Open Source in minutes
Explore Sidekick » Related Article »

About Sidekick

Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running.

Add dynamic logs and put non-breaking breakpoints in your running application without the need of stopping & redeploying.

Sidekick Open Source is here to allow self-hosting and make live debugging more accessible. Built for everyone who needs extra information from their running applications.

Check out Sidekick here

About the project

This project aims to show how you can start live debugging using a self-hosted Sidekick instance.

(back to top)

Built With

Express.js Socket.io

(back to top)

Getting Started

This is an end-to-end guide to show you how you can start using Sidekick Open Source.

Prerequisites

Docker

  Make sure docker daemon is running.

npm
```
npm
```
…

View on GitHub

The only difference will the 2 new parameters we are adding at the initialization. errorCollectionEnabled is set true to enable the basic error collection and the latest parameter is set true to enable detailed frame capturing in case of errors.

app/app.js

To show an example of uncaught exception we will add

throw new Error(‘yet another error’);

at the end of the function.

routes/speakers.js

Now then we run our application and visit the speakers page it will throw an exception. It is time to collect information about this exception. For this tutorial we will be using our Sidekick Node.js Client and visualisation of collected errors will soon come to our Web IDE and extensions. You can get our client from:

@runsidekick/sidekick-client - npm

Sidekick Node.js Client. Latest version: 0.0.4, last published: 19 hours ago. Start using @runsidekick/sidekick-client in your project by running `npm i @runsidekick/sidekick-client`. There are no other projects in the npm registry using @runsidekick/sidekick-client.

npmjs.com

Below you can see a simple script to collect error snapshots and print them to console. Sidekick client allows you to customize your ingest functions, check out our recipes page for examples.

client script

We have started both our main application and client script. After visiting the speakers page our app throw an exception and the agent sent every single detail about it to our broker.

To keep the tutorial simple we have printed everyting to console but to make thing a little bit prettier we will use JSON visualization tool called JSON Crack to understand our collected data. In your own cases sending the collected data to your elastic instances or any other alternative would be much better way to use this data.

Simple visualization tool for your JSON data. No forced structure, paste your JSON and view it instantly.

jsoncrack.com

Below is the initial node of our collected data. You can see when and where it happened and other details like the error message, file name, line no etc.

Since we have enabled frame capture we were able to collect every information in our scope. So as you can see below in case of an error, rather than trying to reproduce it or looking out for clues we are now able get everything we need automatically.

All thanks to Sidekick error stack collection

If you would like to ingest this collected data to your desired target you can use Sidekick’s integrations (Logz.io, Slack, Sumo Logic etc.) or see the examples below to learn how you can build your own.

boroskoyo / sidekick-elastic

send your collected tracepoint & logpoint events to Elasticsearch

+

Sidekick Recipes: Elasticsearch Ingest

Send your logs and traces to Elasticsearch in seconds
Explore the docs »

Sidekick Home · Report Bug & Request Feature

Table of Contents

About The Recipe

boroskoyo / sidekick-loki

Ingest your Sidekick logs & snapshots to Loki. Thanks to winston & winston-loki

+

Sidekick Recipes: Send logs to Loki -Add missing logs to your running microservices

Send your collected tracepoint & logpoint events to your Loki instances
Explore the docs »

Sidekick Home · Report Bug & Request Feature

Table of Contents

About The Recipe

TLDR;

With Sidekick Node agent’s latest update, developers can now activate automatic error stack collection and start collecting exception call stacks from their running applications without stopping them. A great addition to Sidekick Actions that will help you boost your developer productivity even more.

Sidekick is open-source and brings almost no overhead to your applications. Visit runsidekick.com to meet your new live application debugger.

runsidekick.com

Please let us know your invaluable ideas because they will guide us on our path to discovering the future of application observability. You can get in touch with us through Twitter, and join our Discord community.

Achieving Rule-based observability using Sidekick and Camunda

boroskoyo — Mon, 12 Sep 2022 13:15:29 +0000

Collect what you need, only when you need it

Sidekick is an open source live application debugger that comes with many built-in features to make data collection both easier and more efficient. Collecting snapshots and generating logs from running applications are powerful for learning how your code works as long as you collect the data you need.

Logging everything is not a smart move as you can find yourself searching in a log pile full of mostly unnecessary logs in a very short time. Similar case applies for the snapshots & traces. That is why we have included conditional data generation in Sidekick. Combined with hit and time limits, they allow you to collect only what you need, leaving a graveyard of useless data behind.

In addition to our out-of-the-box conditional limitation features, Sidekick’s API’s and clients let you to bring your own rules which leads to a relatively new approach to observability, a rule based one. With this approach we will be able to put tracepoints when rules met and start collecting data automatically based on rule engine’s output.

Rule Engines

There are plenty of different rule engines & workflow automation tools that we can use for such a scenario. A few can be listed as:

The Universal Process Orchestrator | Camunda

Camunda's process automation platform allows developers to design, automate and improve processes. Start your free trial today.

camunda.com

Drools - Drools - Business Rules Management System (Java™, Open Source)

Drools introduces the Business Logic integration Platform which provides a unified and integrated platform for Rules, Workflow and Event Processing.

drools.org

json-rules-engine - npm

Rules Engine expressed in simple json. Latest version: 6.1.2, last published: a year ago. Start using json-rules-engine in your project by running `npm i json-rules-engine`. There are 102 other projects in the npm registry using json-rules-engine.

npmjs.com

node-rules - npm

Business Rules Engine for JavaScript. Latest version: 7.2.0, last published: 24 days ago. Start using node-rules in your project by running `npm i node-rules`. There are 15 other projects in the npm registry using node-rules.

npmjs.com

We will be using Camunda for this example for broader use cases as Camunda is more than a rule engine it is a universal process orchestrator. For a usage example of Camunda you can check out:

Use Camunda as an easy-to-use REST-based orchestration and workflow engine (without touching Java)

Using Sidekick with Camunda

Camunda provides external task clients that allow developers to implement tasks to run when a workflow hits to the related step. We will be using Node.js client but cases below can also be achieved with different clients of Camunda and Sidekick’s upcoming Python & Java clients.

camunda-external-task-client-js - npm

Implement your [BPMN Service Task](https://docs.camunda.org/manual/latest/user-guide/process-engine/external-tasks/) in NodeJS.. Latest version: 2.2.0, last published: 5 months ago. Start using camunda-external-task-client-js in your project by running `npm i camunda-external-task-client-js`. There are 15 other projects in the npm registry using camunda-external-task-client-js.

npmjs.com

Example task method:


// Subscribe to the topic: 'topicName'  
client.subscribe("topicName", async function({ task, taskService }) {  
  // Put your business logic  
  // Complete the task  
  await taskService.complete(task);  
});

@runsidekick/sidekick-client - npm

Sidekick Node.js Client. Latest version: 0.0.2, last published: a month ago. Start using @runsidekick/sidekick-client in your project by running `npm i @runsidekick/sidekick-client`. There are no other projects in the npm registry using @runsidekick/sidekick-client.

npmjs.com

Task method with Sidekick Put tracepoint event:

// Subscribe to the topic: 'topicName'  
client.subscribe("topicName", async function({ task, taskService }) {    const params= {  
      applicationFilters: [  
          {  
            name: "Demo application",  
            version: "v1.0",  
            stage: "prod"  
          }  
        ],  
      fileName: "gitlab.com/repos/...",  
      lineNo: 23,  
      expireSecs: -1,  
      expireCount: -1,  
      enableTracing: true,  
      persist: true  
  }  
  apiClient.putTracepoints(params);  
  await taskService.complete(task);  
});

Example use cases

Case 1 — Activating Tracepoints based on error scenarios

Camunda’s DMN engine provides us ability to define rules in an excel like interface.

Sidekick DMN

As you can observe at the DMN above we can define where we put tracepoints when a certain type of error occurs.

Triggering workflows with error types makes a great use case as it enables us to automatically start collecting extra information based on different cases. Enabling different tracepoints at once is especially useful for spotting state changes.

Note: In addition to this use case Sidekick Node.js agent has ability to automatically capture error stacks, without any tracepoint. check out: https://www.npmjs.com/package/@runsidekick/sidekick-agent-nodejs

Example Sidekick BPMN

Case 2 —Putting Tracepoints on workflow errors

image from: https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e

Camunda comes with an error handling mechanism that lets you run actions in cases of errors. In addition to Sidekick client, we can also attach Sidekick agents to task services to observe their state changes and debug them effectively.

Case 3 — Activating Tracepoints on certain events

Last but not least, Sidekick Actions bring more than debugging to the table. They are also great tools to understand how the system works. Especially for the cases like understanding why and how edge cases occur or understanding when does a certain step is reached.

We have briefly explained how Sidekick can be used with Camunda to achieve rule based observability, for better understanding systems, having a smoother debug experience and lowering monitoring costs with conditional data collection. The 3 cases we have explained are just the tip of the iceberg and there is a lot more to discover here.

Sidekick Open Source Live Debugger : Embed Sidekick features to your applications

boroskoyo — Wed, 31 Aug 2022 10:32:21 +0000

Control Sidekick agents from a React & Express application

What is Sidekick?

Sidekick is an open-source live application debugger that lets you troubleshoot your applications while they keep on running.

Check out Sidekick Open-source repo to learn more:

runsidekick / sidekick

Free and open-source live application debugger. Like chrome dev tools but for your backend

Explore Docs » Quick Start Tutorial »

Table of Contents

What is Sidekick?

Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running.

Add dynamic logs and put non-breaking breakpoints in your running application without the need of stopping & redeploying.

Sidekick Open Source is here to allow self-hosting and make live debugging more accessible. Built for everyone who needs extra information from their running applications.

Sidekick Actions:

Sidekick has two major actions; Tracepoints & Logpoints.

A tracepoint is a non-breaking remote breakpoint. In short, it takes a snapshot of the variables when the code hits that line.
Logpoints open the…

View on GitHub

Sidekick agents can be controlled both via a UI from Sidekick Web IDE, Visual Studio Code & IntelliJ IDEA and via a headless approach from REST API & Node.js Client. Sidekick REST API and Node.js client allows developers to control Sidekick actions from their own solutions.

About the example

We have prepared an example project to show how you can develop a custom application to control your Sidekick agents using Sidekick Node.js Client.

This example consists of 2 parts. First part is a standard web application that we will live debug and the second one is our custom Sidekick control app. Both parts are connected to a self hosted version of Sidekick.

Sidekick Control App that we have build has 2 main features.

1 - Putting tracepoints

In the React applicaion you can find 2 buttons that let you put tracepoints to specified locations.

Tracepoints are added via making a call to express server that is running behind. The express server uses the Sidekick client’s putTracepoints function to make an api call to Sidekick REST API.

apiClient.putTracepoints(params);

2 - Listening events

Sidekick client’s onTrigger feature is used to listed activities from Sidekick broker.

onTrigger(clientInfo);

We have implemented a custom ingest function for tracepoints to send them to the react application using socket.io.

function ingestFunc(index) {   
    return async function (data) {   
        console.log(data);   
        io.to(“stack-room”).emit(“stack”, data);   
    };  
}

get extra info about Sidekick Node.js client here: Sidekick Node.js Client

Built With

Getting Started

Prerequisites

Sidekick

Make sure Sidekick broker is running. (check main Sidekick repo)

If you have changed the default settings, update token and port values in this tutorial accordingly.

Check out our Quick Start Guide.

Part-1 | Employee System (Main App)

# Clone Employee System repository  
git clone [https://github.com/boroskoyo/sidekick-example-employee-management-system](https://github.com/boroskoyo/sidekick-example-employee-management-system)  
# Go into the repository  
cd sidekick-example-employee-management-system  
# Install dependencies  
npm i --legacy-peer-deps  
# Run the app  
npm run start

UI will start running at http://localhost:3000 and backend will be using the port 8081

server/index.js - This is where we include our Sidekick agent. You can edit the agent settings according to your setup and needs.

Agent settings can be seen below:

Agent Settings

Part-2 | Control Application

This is the application we have built to control our Sidekick agent.

# Clone control app repository  
git clone https://github.com/boroskoyo/sidekick-electron-quick-start  
# Go into the repository  
cd sidekick-electron-quick-start  
# Install dependencies  
npm i --legacy-peer-deps  
# Run the app  
npm run dev-app

You can also run the browser version — http://localhost:5173

npm run dev

Sidekick Explorer UI

server.js - Contains the main functions for putting Tracepoint.
params - This is where we prepare our params for Sidekick API requests for putting tracepoints.

parameters for tracepoint

filename - Filname paramters consists of git address of the file and the commit hash.

Usage

After running all applications. Go to http://localhost:3000 and check out the employee app. Then use the put tracepoint buttons on the Electron app to put tracepoints on the running application.

Tracepoint Locations:

Get Employees (Line 62)

Delete Employees (Line 202)

Visit the home page and capture stack related to employee list endpoint.
Delete an employee to collect related data.
Check out Electron app to see your collected events.

You can than further edit those apps to explore Sidekick Client’s features.

Conclusion

As you can see above, Sidekick clients allow developers to easily use collected data and control Sidekick Actions from third party applications. We are planning to implement Sidekick control plugins for different dashboards and platforms, share your desired targets and feature requests via issues and discussions.

Please let us know your invaluable ideas because they will guide us on our path to discovering the future of application observability. You can get in touch with us through Github, Twitter, and join our Discord community.

If you liked this article and learn more about Sidekick you can check out our docs and start using it right away!

Repo related to this article:

boroskoyo / sidekick-electron-quick-start

an example app to show how you can control your agents from a React application:

Sidekick Example: Embed Sidekick features to your applications

Example to show how you can control your agents from a React application
Explore Sidekick »

About the project

This project aims to show how you can develop a custom application to control your Sidekick agents using Sidekick Node.js Client.

Built With

React
Express
Socket.io
Electron
Sidekick Node.js Client

Getting Started

This tutorial consists of 2 parts. First part is a standard web application that we will live debug and the second one is our custom Sidekick control app.

Prerequisites

Sidekick

 Make sure Sidekick broker is running. (check main Sidekick repo)
 If you have changed the default settings, update token and port values in this tutorial accordingly

Check out our Quick Start Guide.

Part-1 | Employee System (Main App)

# Clone Employee System repository
git clone https://github.com/boroskoyo/sidekick-example-employee-management-system
# Go into the repository
cd sidekick-example-employee-management-system
# Install dependencies
npm i --legacy-peer-deps
#

…

View on GitHub

State of Sidekick Open Source Live Application Debugger

boroskoyo — Thu, 25 Aug 2022 15:30:07 +0000

It has been around 6 months since we released Sidekick as a standalone solution. It is time for our biggest update ever.

You can see where we were at day zero here: https://www.runsidekick.com/blog/announcing-sidekick-instant-live-debugging-to-fix-software-bugs

At the beginning we were describing Sidekick as;

“a modern live debugging tool that helps development teams to understand their code instantly by getting snapshots of their running applications wherever they run.”

Initially, improving developers’ day-to-day debugging routines with tracepoints aka non-breaking breakpoints was our main goal. As we got to know our customers and the market more, we discovered that Sidekick has a lot more to bring to the table. First, we added log points and give our users the ability to collect traces and generate logs on-demand without stopping & redeploying their applications. Then we started working on discovering the extra benefits of Sidekick-generated data. As an initial step, we developed an MVP for Sidekick Node.js Client that lets you use your generated data programmatically.

A subset of what we achieved in this relatively short period of time;

Node.js support
Team support
New Sidekick Action : Logpoints!
Webhooks support
Visual Studio Code extension
Improved IntelliJ IDEA plugin
Collaboration support
Improved On-premise solution with Web IDE support
Newer & better Web IDE with improved user experience
Sandbox page for frictionless onboarding
OpenTelemetry support for Sidekick Agents
Improved data reduction at the edge
REST API → allowing programmatically managing Tracepoints & Logpoints
Sidekick Node.js Client
Sidekick recipes → Ingest your Sidekick-generated data to your observability platform : https://docs.runsidekick.com/sidekick-recipes

Team Sidekick after the latest sprint

One thing led to another and we started to discover new surface areas for Sidekick. Now it is a headless addition to your observability stack. With new OpenTelemetry support and REST API, it came to a stage where it allows you to put your logpoints / tracepoints programmatically and use the generated data again using your programming skills. A true developer-first tool that makes the developers a part of the core loop.

Today;

In addition to its live debugger superpowers, Sidekick is a plus one for your observability stack.

Our journey continues and there are many more steps to come, but the big news is future of Sidekick is bright and open! Open as in Open-Source!

🥁🥁🥁Drum rolls!🥁🥁🥁

Sidekick is now open-source!

With this release anybody will be able to have thier self hosted Sidekick instances, this will open up new opportunites for community to meet live debugging and contribute to Sidekick.

Sidekick Open Source Github Repository: https://github.com/runsidekick/sidekick (don't forget to give us a ⭐)

We still have a lot to do and we are constantly looking for feedbacks and contributors. Your support is important for improving Sidekick's live debugging experience more.

Sidekick Open Source Live Debugger : Get started in 5 mins

boroskoyo — Wed, 24 Aug 2022 13:22:00 +0000

Sidekick Open Source is finally ready! Here is how you can start using it in 5 minutes.

Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running.

Sidekick Open Source is here to allow self-hosting and make live debugging more accessible. Built for everyone who needs extra information from their running applications.

Today is the big day for Sidekick Open Source as we are launching on Product Hunt! Your upvotes and comments will help us a lot and we are looking forward you hear your feedback.

Our Product Hunt page: https://www.producthunt.com/posts/sidekick-12

Sidekick Open Source Github repo: (don’t forget to star it ⭐)

https://github.com/runsidekick/sidekick

Quickstart

We have prepared a simple project to show how you can start live debugging using a self-hosted Sidekick instance in just 5 minutes.

Prerequisites

Docker

Make sure docker daemon is running.

npm install npm@latest -g

Visual Studio Code

( Note: Sidekick has many other clients. Check out our headless options if you don’t want to install a new IDE : https://docs.runsidekick.com/headless/)

Installation

Running Sidekick with default settings

1.Clone the Sidekick repo

git clone https://github.com/runsidekick/sidekick.git

2.Go to the docker folder under the project

3.Run command docker-compose up -d and wait for a minute

Your Sidekick instance is ready

Running Quickstart Project

1.Clone the Quicstart application repo

git clone https://github.com/boroskoyo/sidekick-nodejs-quickstart.git

2.Install NPM packages

npm install

3.Run!

npm start

Now you have your self-hosted Sidekick instance and Quickstart application up and running.

It is time to use a client to put tracepoints and collect data from Quickstart application.

And live debugging for all!

Install Sidekick Extension form the VSCode marketplace:

Open the folder of Quickstart application at VSCode

.vscode folder in this repo should set the config for you. If you have any issues just check the instructions here: https://docs.runsidekick.com/plugins/visual-studio-code-extension-python-and-node.js/self-hosted-settings

Now login using Sidekick plugin

Head to the Sidekick: Trace Events panel to enable it

Right click on 10th line on app/routes/index.js and put your first tracepoint. Check docs for extra info: https://docs.runsidekick.com/plugins/visual-studio-code-extension-python-and-node.js/tracepoints-and-logpoints

Visit http://localhost:3000/ to trigger your tracepoint.

Observe your collected trace events at VSCode Sidekick: Trace Events panel!

That’s it!

That is all you need to start using Sidekick Open Source.

Happy debugging y’all!

repo containing this project : https://github.com/boroskoyo/sidekick-nodejs-quickstart

How to add logs to your running applications on-demand without redeploying, restarting, or stopping the execution

boroskoyo — Mon, 15 Aug 2022 10:23:48 +0000

Logs have become a fundamental part of traditional software development as they help us understand what is going on with our systems while they are running.

Photo by Thierry Chabot on Unsplash

Regardless of your runtime you can add logs anywhere you want and start collecting information but there is always a catch:

As your system becomes more complex and/or the number of logs you are generating increases it is getting easier to get lost in log piles. On the flip side, logs only help you if they are placed in the right place and any replacement etc. requires a new deployment which does not end to be the most chill scenario if you are trying to find a bug in production.

These two points create a dilemma:

You either log everything for your comfort and get lost in gigabytes or maybe a larger of logs you have created or you end up with inadequate logs that you need to populate by adding more logs into your code and redeploying it.

Solution: Dynamic logging

Our latest update brings logpoints along with many improvements.

Logpoints are points that you can add to your running application, just like Tracepoints, for adding logs dynamically.

Logpoints solve the dilemma by letting you add logs to your running applications without stopping and redeploying. Since you can add and remove your logs at any time, getting the log wherever and whenever you need it becomes a walk at the park!

Not only solving the issues of traditional logging, but Sidekick Logpoint feature also comes with extras. In addition to adding and removing logs on demand, with Sidekick you can:

Define conditions for your logpoints
Define time to live and maximum hit counts
Change log levels dynamically
Enrich your log content with variable information from the context
Configure agent to print logs directly to stdout
Send generated logs to any target you desire!

Our integrations with Slack and some of the well-known logging tools are at the corner and we also added custom webhook support for you to extend the possibilities. Don’t forget to join our Discord channel for news and to share your integration requests.

TL;DR

Sidekick’s Logpoints feature, in the end, opens the way of dynamic logging to its users. It replaces traditional logging with dynamic logging. This means you easily have the potential to lower stage sizes, costs, and time for log searching. Additionally, logpoints feature brings the ability to add new logpoints without editing the source code, redeploying, or restarting the application.

Collect Realtime Stack Traces from NodeJS Applications

boroskoyo — Fri, 12 Aug 2022 12:16:46 +0000

Whether you are working on a dev branch or prod, you can now remotely debug your running Node.js applications using Sidekick’s web IDE.

In this post, I’ll show you how you can quickstart collecting stack traces from your Node.js applications using Sidekick. All without restarting & redeploying.

To keep everything nice and short I’ve started with the Hello World example from Express.js’ website. (http://expressjs.com/en/starter/hello-world.html)

Then I plugged in body-parser and added a new endpoint to demonstrate what you can achieve in a few seconds.

const express = require('express')  
const app = express()  
const port = 3000  
const bodyParser = require('body-parser')  

app.use(bodyParser.json())  

app.get('/', (req, res) => {  
  res.send('Hello World!')  
})  

app.post('/:id', (req, res) => {  
    let body = req.body  
    res.json(req.body.foo)  
  })  

app.listen(port, () => {  
  console.log(\`Example app listening on port ${port}\`)  
})

So as you can see, my new endpoint is returning a part of the request body. Now we will use Sidekick to observe more and see what other information was included in the request body.

First, we will install our Sidekick Node.js agent.
npm install @runsidekick/sidekick-agent-nodejs
Then we will sign in to our Sidekick account and copy the API Key of our workspace.

Sidekick comes with a 7-day free trial so you can use this link to sign up and follow along with this article.

‍https://app.runsidekick.com/

Then we will add the Sidekick agent on top of our project. Final code will look like this.

const SidekickDebugger = require('@runsidekick/sidekick-agent-nodejs');  

SidekickDebugger.start({   
    apiKey: 'your-sidekick-api-key'  
});  

const express = require('express')  
…

And we are good to go. Now we can run our application and start observing its state on the go.

To put your first tracepoint, head to app.runsidekick.com

Select your running application.

Add your tracepoint and call your endpoint to collect related data.

‍

There you go, now that you have collected your data you can observe what is going on behind the curtains.

(checkout the initial image for the zoomed in version)

As you can see starting with Sidekick is a breeze.

If you liked this article and learn more about Sidekick you can check out our docs and start using it right away!

Production Debuggers — 2022 Benchmark Results

boroskoyo — Thu, 11 Aug 2022 06:29:57 +0000

Production Debuggers — 2022 Benchmark Results — Part-1

Passive impact of agents under high load

Case1: Passive impact under high load

Story Behind the Research

We develop Sidekick to give developers new abilities for collecting data from their running applications. On our road to making live debugging & observability easier for developers, our performance impact is among the most questioned topics. To answer this question, we decided to make a research to observe how much overhead Sidekick and its competitors Lightrun and Rookout bring to the applications.

This benchmarking research consists of 3 parts.

Case1: Passive impact under high load
Case2: Comparing Sidekick & Lightrun & Rookout load test performance with active tracepoints
Case3: Comparing Sidekick and Rookout performances with active tracepoints with 1000 hit limits under under high load

All the cases will be consisted of load tests and we will perform these tests using JMeter. You check it out here: https://jmeter.apache.org/

To test out the performance impact we have decided to go with the pet clinic app example served by the Spring community itself for a fair comparison. Below you can see the system’s overall design.

You can find our testing repo below if you want to repeat this test yourself. Both pet clinic example and our testing repo including JMeter test JMX files can be found in this repo:
https://github.com/runsidekick/sidekick-load-test

JMeter & Throughput Shaping Timer plugin link:
(https://jmeter.apache.org/)
(https://jmeter-plugins.org/wiki/ThroughputShapingTimer/)

Hardware info:

We will be using AWS EC2 instances with default settings to run. You can find image details below.


"Images": \[   
{  
  "Architecture": "x86_64",  
  "CreationDate": "2022-04-28T21:51:14.000Z",  
  "ImageId": "ami-0ca285d4c2cda3300",  
  "ImageLocation": "amazon/amzn2-ami-kernel-5.10-hvm-2.0.20220426.0-x86_64-gp2",  
  "ImageType": "machine",  
  "Public": true,  
  "OwnerId": "137112412989",  
  "PlatformDetails": "Linux/UNIX",  
  "UsageOperation": "RunInstances",  
  "State": "available",  
  "BlockDeviceMappings": \[  
    {  
      "DeviceName": "/dev/xvda",  
      "Ebs": {  
        "DeleteOnTermination": true,  
        "SnapshotId": "snap-0ce66497b69246e3d",  
        "VolumeSize": 8,  
        "VolumeType": "gp2",  
        "Encrypted": false  
      }  
    }  
  \],  
  "Description": "Amazon Linux 2 Kernel 5.10 AMI 2.0.20220426.0 x86_64 HVM gp2",  
  "EnaSupport": true,  
  "Hypervisor": "xen",  
  "ImageOwnerAlias": "amazon",  
  "Name": "amzn2-ami-kernel-5.10-hvm-2.0.20220426.0-x86_64-gp2",  
  "RootDeviceName": "/dev/xvda",  
  "RootDeviceType": "ebs",  
  "SriovNetSupport": "simple",  
  "VirtualizationType": "hvm"  
}  
\]

There are multiple EC2 instance types in AWS environment. Amazon EC2 C5 instances deliver cost-effective high performance at a low price per compute ratio for running advanced compute-intensive workloads. We have used c5.4xlarge with default configurations for all our cases. You can find the details for this instance below. If you would prefer to learn more about EC2 C5 instances, you can check out the official AWS website.


{  
  "InstanceTypes": \[  
    {  
      "InstanceType": "c5.4xlarge",  
      "CurrentGeneration": true,  
      "FreeTierEligible": false,  
      "SupportedUsageClasses": \[  
        "on-demand",  
        "spot"  
      \],  
      "SupportedRootDeviceTypes": \[  
        "ebs"  
      \],  
      "SupportedVirtualizationTypes": \[  
        "hvm"  
      \],  
      "BareMetal": false,  
      "Hypervisor": "nitro",  
      "ProcessorInfo": {  
        "SupportedArchitectures": \[  
          "x86_64"  
        \],  
        "SustainedClockSpeedInGhz": 3.4  
      },  
      "VCpuInfo": {  
        "DefaultVCpus": 16,  
        "DefaultCores": 8,  
        "DefaultThreadsPerCore": 2,  
        "ValidCores": \[  
          2,  
          4,  
          6,  
          8  
        \],  
        "ValidThreadsPerCore": \[  
          1,  
          2  
        \]  
      },  
      "MemoryInfo": {  
        "SizeInMiB": 32768  
      },  
      "InstanceStorageSupported": false,  
      "EbsInfo": {  
        "EbsOptimizedSupport": "default",  
        "EncryptionSupport": "supported",  
        "EbsOptimizedInfo": {  
          "BaselineBandwidthInMbps": 4750,  
          "BaselineThroughputInMBps": 593.75,  
          "BaselineIops": 20000,  
          "MaximumBandwidthInMbps": 4750,  
          "MaximumThroughputInMBps": 593.75,  
          "MaximumIops": 20000  
        },  
        "NvmeSupport": "required"  
      },  
      "NetworkInfo": {  
        "NetworkPerformance": "Up to 10 Gigabit",  
        "MaximumNetworkInterfaces": 8,  
        "MaximumNetworkCards": 1,  
        "DefaultNetworkCardIndex": 0,  
        "NetworkCards": \[  
          {  
            "NetworkCardIndex": 0,  
            "NetworkPerformance": "Up to 10 Gigabit",  
            "MaximumNetworkInterfaces": 8  
          }  
        \],  
        "Ipv4AddressesPerInterface": 30,  
        "Ipv6AddressesPerInterface": 30,  
        "Ipv6Supported": true,  
        "EnaSupport": "required",  
        "EfaSupported": false  
      },  
      "PlacementGroupInfo": {  
        "SupportedStrategies": \[  
          "cluster",  
          "partition",  
          "spread"  
        \]  
      },  
      "HibernationSupported": true,  
      "BurstablePerformanceSupported": false,  
      "DedicatedHostsSupported": true,  
      "AutoRecoverySupported": true  
    }  
  \]  
}

For the databases we will be using db.t2.micro MySQLinstances. T2 instances are burstable general-purpose performance instances that provide a baseline level of CPU performance with the ability to burst above the baseline. T2 instances are a good choice for a variety of database workloads including micro-services and test and staging databases. General details for t2.micro instance hardware info is shared below. For getting more information, you can visit the official AWS website.


{  
  "InstanceTypes": \[  
    {  
      "InstanceType": "t2.micro",  
      "CurrentGeneration": true,  
      "FreeTierEligible": true,  
      "SupportedUsageClasses": \[  
        "on-demand",  
        "spot"  
      \],  
      "SupportedRootDeviceTypes": \[  
        "ebs"  
      \],  
      "SupportedVirtualizationTypes": \[  
        "hvm"  
      \],  
      "BareMetal": false,  
      "Hypervisor": "xen",  
      "ProcessorInfo": {  
        "SupportedArchitectures": \[  
          "i386",  
          "x86_64"  
        \],  
        "SustainedClockSpeedInGhz": 2.5  
      },  
      "VCpuInfo": {  
        "DefaultVCpus": 1,  
        "DefaultCores": 1,  
        "DefaultThreadsPerCore": 1  
      },  
      "MemoryInfo": {  
        "SizeInMiB": 1024  
      },  
      "InstanceStorageSupported": false,  
      "EbsInfo": {  
        "EbsOptimizedSupport": "unsupported",  
        "EncryptionSupport": "supported",  
        "NvmeSupport": "unsupported"  
      },  
      "NetworkInfo": {  
        "NetworkPerformance": "Low to Moderate",  
        "MaximumNetworkInterfaces": 2,  
        "MaximumNetworkCards": 1,  
        "DefaultNetworkCardIndex": 0,  
        "NetworkCards": \[  
          {  
            "NetworkCardIndex": 0,  
            "NetworkPerformance": "Low to Moderate",  
            "MaximumNetworkInterfaces": 2  
          }  
        \],  
        "Ipv4AddressesPerInterface": 2,  
        "Ipv6AddressesPerInterface": 2,  
        "Ipv6Supported": true,  
        "EnaSupport": "unsupported",  
        "EfaSupported": false  
      },  
      "PlacementGroupInfo": {  
        "SupportedStrategies": \[  
          "partition",  
          "spread"  
        \]  
      },  
      "HibernationSupported": true,  
      "BurstablePerformanceSupported": true,  
      "DedicatedHostsSupported": false,  
      "AutoRecoverySupported": true  
    }  
  \]  
}

The case:

In this case, we are investigating the passive impact of Java Agents of production debuggers which we took in our scope.

Agents in this case:

Agentless: Agentless version of the pet clinic app which we will use as a reference index for our comparison.
Sidekick: https://www.runsidekick.com
Rookout: https://www.rookout.com
Lightrun: https://www.lightrun.com

For all agents we will have separate EC2 instances and we will be using JMeter for all our tests. For each case only dependent variable will be the agent.

We will make same amount of requests to the same endpoints for each setup and observe the impact of each agent via comparing their latencies and throughputs.

Below you can find our JMeter setup for the load test:

The pattern above(shaping-timer-load) will be repeated for each setup. You can download the .jmx file from here: https://github.com/runsidekick/sidekick-load-test/blob/master/petclinic-app/src/test/jmeter/petclinic_test_plan.jmx

Passive Impact Benchmark Results

Agentless (Ref):

Reference Statistics

Lightrun:

Lightrun Statistics

Rookout:

Rookout Statistics

Sidekick:

Sidekick Statistics

Results Summary:

Statistics comparison table

Observing that the least amount of Transactions/s is 982.30 and it is only 1.14% lower than the reference value we can see how low impact is done when agents are in their passive state. Furthermore, Min & Median latency values are almost identical and we can use graphs to observe Max values are ususally one time occurrences that does not represent a negative outcome.

Conclusion:

All 3 alternatives and the agentless instance performed almost identically. This means you can ship your applications with any live debugger agent and don’t worry about performance losses.

Bearing in mind that this performance research is done under the constraints of idle positions. To be more specific, we plugged in the agents and did not use the products yet. We did not put any tracepoints and sent any requests from the agents.

In this research, we came to the conclusion that neither the Sidekick agent nor the Rookout, and Lightrun agents bring very little negligible overhear compared to the application without any agent plugged.

This means no performance loss is gained using these 3 Java agents in the idle situation. In another perspective, production debuggers differ from APMs or Error/bug tracking tool in means of performance.

Sidekick is going open source to allow self-hosting and make live debugging more accessible.

Subscribe and get the latest news from Sidekick Open-Source here:

https://www.producthunt.com/upcoming/sidekick-open-source-collect-traces-and-generate-logs-on-demand-without-stopping-redeploying-your-applications

DEV Community: boroskoyo

Using Grafana to visualize CI Workflow Stats

Introducing Foresight Plugin for Grafana!

Collecting error stacks from backend as exceptions occur in E2E tests — A tutorial on Cypress

Discover the missing piece of E2E test results

Capturing Exception Call Stacks from Node.js Applications

boroskoyo ・ Sep 23 '22

Setting up Error Stack Collection within our E2E tests in Cypress :

boroskoyo / todos-express-password-sidekick

Todo app using Express and Passport for sign in with username and password.

todos-express-password-sidekick

Quick Start

Next Steps

License

Credit

Requirements:

Setup:

Results:

Where to go from here:

Live Debugging Python Applications

Speed up finding the root cause and minimize the impact of such incidents

Importance of Debugging in Production Environments

How to Integrate Sidekick with Python Applications

Step 1 — Create a Sidekick account

Step 2 — Install sidekick-agent-python to your application

Step 3 — Connect the application to Sidekick web IDE

Step 4 — Enabling Tracepoints

Debugging with Sidekick

Using Conditional Tracepoints in Sidekick

Conclusion

Bringing the simplicity of print() to production debugging using PyCharm

The way we develop software evolves, way we debug can’t stay the same!

Bringing the simplicity of print() to production debugging using PyCharm

A Sea Full of Debugging Solutions

Good ol’ print()

pdb

IDE Debuggers

PyCharm Remote Debugging

PyCharm Sidekick Plugin

runsidekick / sidekick

Free and open-source live application debugger. Like chrome dev tools but for your backend

What is Sidekick?

Sidekick Actions:

Worth mentioning:

Capturing Exception Call Stacks from Node.js Applications

Sidekick allows you to collect stacks as errors happen, without needing for putting a tracepoint beforehand

runsidekick / sidekick

Free and open-source live application debugger. Like chrome dev tools but for your backend

What is Sidekick?

Sidekick Actions:

See it in action:

boroskoyo / sidekick-nodejs-quickstart

Quick start example for Sidekick.

Sidekick Quickstart

About Sidekick

About the project

Built With

Getting Started

Prerequisites

boroskoyo / sidekick-elastic

send your collected tracepoint & logpoint events to Elasticsearch

+

Sidekick Recipes: Elasticsearch Ingest

About The Recipe

Related blog post: https://medium.com/runsidekick/sidekick-recipes-1-elasticsearch-ingest-561d0970c030

boroskoyo / sidekick-loki

Ingest your Sidekick logs & snapshots to Loki. Thanks to winston & winston-loki

+

Sidekick Recipes: Send logs to Loki -Add missing logs to your running microservices

About The Recipe

Related blog post: https://medium.com/runsidekick/sidekick-recipes-2-add-missing-logs-to-your-running-microservices-and-send-them-to-loki-1f5a3449343c

TLDR;

Achieving Rule-based observability using Sidekick and Camunda

Collect what you need, only when you need it

Rule Engines

Using Sidekick with Camunda

Example use cases

Case 1 — Activating Tracepoints based on error scenarios

Case 2 —Putting Tracepoints on workflow errors

Case 3 — Activating Tracepoints on certain events