DEV Community: CharlyAutomatiza

From One Skill to a Plugin: My Story Behind the Initial Release of grafana-k6-plugin

CharlyAutomatiza — Tue, 31 Mar 2026 01:46:01 +0000

It started with a feeling I could not ignore

Some ideas do not arrive as a polished product vision. They arrive as discomfort.

I was spending more and more time working with LLMs inside developer workflows, and while the speed was impressive, I kept running into the same problem: the output often looked better than it actually was. The code was fluent, but not always trustworthy. The suggestions were fast, but not always current. The answers sounded confident, but that confidence was not always backed by real engineering judgment.

If you are new to the concept, skills are folders of instructions, scripts, and resources that Claude and other agents can load dynamically to perform better on specialized tasks.

In the k6 world, that gap matters a lot.

For anyone new to it, k6 is Grafana's open-source performance testing tool for load, stress, and reliability testing of APIs and services.

I was seeing models suggest deprecated code, outdated references, and quality patterns that might pass a superficial review but would never feel solid if you actually cared about long-term maintainability or performance realism. Models could generate code very easily - that did not mean they were generating the right code for the real problem.

As a Grafana k6 Champion with a strong performance engineering background, I knew the tool had enormous potential. I also knew a lot of that potential was being left on the table.

The real value is not just generating a test script. The real value is being able to ask: is this scenario realistic? Are these thresholds meaningful? Is this auth flow being handled safely? Is this a recommendation I would actually trust in front of a team?

I did not want to build something that simply made LLMs produce more code.
I wanted to build something that helped them produce better decisions.

At first, I thought this would be a single skill

The original idea was smaller. I was not thinking about a plugin. I was thinking about a skill.

I wanted to condense my practical experience into something reusable - something that could guide an agentic IDE toward stronger k6 outputs, better defaults, and more reliable performance testing behavior.

That first idea made sense for a while. A single skill felt elegant. Compact. Easier to reason about.

But as I kept building, I ran into a reality that changed the whole direction of the project. Once the idea started growing, putting too much responsibility into one skill would reduce precision instead of increasing it.

That is where a very old lesson came back to me.

In my first year at university, one of the ideas that stayed with me was the classic divide and conquer mindset: break complexity into clearer, specialized parts, make each part excellent at its own job, reduce ambiguity by reducing responsibility overlap.

That is exactly what this project needed.

So the idea stopped being "one smart skill" and became something much better: a plugin built around three dedicated skills, each one focused, specialized, and expert in its own job.

Instead of asking one giant skill to plan, build, validate, clarify, and reason about every edge case equally well, I could design a system where each part had a clearer mission and a tighter behavior contract.

That was the moment the project became more than an experiment.

I was not trying to outgenerate LLMs, I was trying to outguide them

This is probably the most important thing to understand about why I built grafana-k6-plugin.

I was never trying to compete with LLMs at generation speed. Almost any model can produce code with a decent prompt today - that is no longer the hard part. What interested me was something deeper: how to inject more judgment into the generation process. How to make the output aligned with current platform realities, aware of performance engineering tradeoffs, and resistant to fragile assumptions.

And especially: guided by expert questions when critical information is still missing.

That last part became one of the foundations of the plugin. I did not want skills that would guess when they should ask. I wanted skills that would know when not to guess.

In performance engineering, there are doubts that are negotiable and doubts that are not. If I do not know the target, the workload shape, the auth constraints, or the SLA expectations, I do not want a confident hallucination. I want the right question asked at the right time.

When uncertainty is critical, the skill should ask like an expert. And the human should remain the one making the decision.

That is not a weakness in the workflow. That is the workflow.

Learning about AskUserQuestion changed how I thought about skill design

One of the moments that pushed this idea forward came from something very practical.

My colleague Alan showed me the AskUserQuestion capability in Claude Code during a skills hackathon we had at work. That may sound like a small detail, but for me it clicked immediately.

Because suddenly the interaction model was not just "generate the answer." It became: detect what is missing, stop at the right moment, and ask the right thing.

That opened a much more powerful path. It meant I could build skills that behaved less like autocomplete with extra steps and more like a real specialist working alongside me.

The hackathon itself was part of the spark too. Seeing what people were building, how quickly a good idea could evolve into something tangible, and the energy of people discovering new ways to package judgment and domain expertise into agentic workflows - that pushed me further. It made me feel that this plugin was worth building for the wider ecosystem, not just for myself.

When I stopped relying on intuition and started building evidence

At some point, every ambitious build hits the same wall.

You can feel that you are improving things. You can believe the structure is getting stronger. But eventually, feeling is not enough.

I needed evidence. And this is where Anthropic's skill-creator workflow became fundamental for me - not just as a creation aid, but as part of an evaluation discipline.

References:

That meant designing evals, running them in structured iterations, comparing with-skill and without-skill behavior, using multiple agents where useful, and gathering empirical evidence instead of relying on taste.

The difference in quality of decisions was significant.

Before that, improvements sounded like: "this feels cleaner," "this looks more robust," "this answer reads better." After I started working through evals and comparisons, the conversation became sharper: "this iteration actually improved ambiguity handling," "this change is attractive but does not improve outcomes," "this needs a correction plan, not another vague rewrite."

I also wanted to pressure-test the skills under real conditions: HTTP cases, gRPC scenarios, browser-oriented flows, script validation issues, safety and quality edge cases, and guidance quality under incomplete inputs. Not just ideal prompts. Multi-agent execution helped simplify comparison and made it easier to see whether a behavior was genuinely robust or just accidentally tuned to one narrow interaction pattern.

When you are deep inside a project, doubt never disappears. But evidence gives doubt a productive direction. It turns anxiety into diagnosis and makes solid correction plans possible - the kind that say: this is the weak spot, this is the reason, this is the adjustment, and this is how we will know if it worked.

That kind of loop creates trust. Not only in the artifact, but in the process itself.

Review was another turning point, and it was not always comfortable

Before each PR, I kept iterating my PR review prompts. I did not want review to be a last-minute checkbox. I wanted it to be a designed stage with real value.

GitHub Copilot review was genuinely important in that stage. It surfaced useful issues, accelerated feedback loops, and gave me another layer of signal before merging.

But it also taught me something very important: good review support is not the same as full understanding.

One example captured the problem perfectly. Inside the skill references, I had DO and DONT examples. The DONT examples were intentional - educational anti-patterns that existed so the skills could recognize poor practices and steer people toward better ones.

At times, Copilot review identified those DONT examples as mistakes that should be corrected. From a pattern-matching point of view, that reaction made sense. From a human intent point of view, it was wrong.

Without professional judgment, it becomes easy to "fix" the wrong thing. You can clean up the signal that was supposed to teach the model. You can make the final system more fragile while convincing yourself you improved it.

That is not a theoretical risk. I saw versions of it happen.

This is one of the strongest lessons from the project: our role as professionals is not optional. I do not see human-in-the-loop as a fallback. I see it as part of the architecture.

Ship before you disappear into endless polishing

By the time I reached the initial release, the feeling was not "this is finished forever." It was something more grounded: this is now solid enough to put in front of the community and learn in public.

That is an important distinction.

I do not believe in waiting for some mythical state of perfection before sharing useful work. If I had waited for the moment when every possible improvement was already solved, this plugin would probably still be sitting in an endless spiral of iteration. There is always one more refactor, one more rule, one more benchmark, one more prompt tweak.

At some point, you have to stop polishing in private and step onto the field.

You have to ship.
You have to ask for feedback.
You have to let the real world respond.

I fully expect there to be opportunities for improvement - I actually want that. I hope people open PRs. I hope the project keeps evolving. But I believe deeply in finishing stages: close one chapter, learn from it, and continue.

That is healthier than living forever inside an unfinished draft.

Try it yourself

My repo:
https://github.com/charlyautomatiza/grafana-k6-plugin

Installation

Add this marketplace to Claude

/plugin marketplace add charlyautomatiza/charlyautomatiza-marketplace

Install a plugin from the marketplace

/plugin install grafana-k6

Or install directly from the plugin repository

npx skills add charlyautomatiza/grafana-k6-plugin

Once installed, you have three specialized skills available, each one with a clear role:

k6-plan: turns your objective into a concrete test strategy before code generation.
- "Plan a load test for my users API."
k6-builder: creates runnable k6 artifacts from your requirements or from a plan.
- "Generate a runnable k6 script for my checkout API."
k6-validate: reviews an existing script and prioritizes what to fix.
- "Review this k6 script and flag the top issues that could make results unreliable."

Across all three skills, if required context is missing, the agent will pause and ask the right questions before continuing.

Each one knows when to ask before it acts.

If you are curious about how I built and evaluated these skills, the Anthropic skill-creator workflow is the same workflow I used.

Build something, test it honestly, and put it out into the world

If there is one thing I hope people take from this story, it is this: do not underestimate what happens when you combine domain expertise, agentic tooling, and a serious feedback loop.

Code generation is no longer the differentiator. What matters is whether the code reflects judgment, context, and good engineering instincts. AI review can accelerate a lot, but it can also misunderstand why something exists - and that is where professional criteria becomes decisive. Specialization matters: turning one skill into a plugin with three dedicated expert skills was one of the best decisions in the entire project. And evidence beats vibes: if you want to improve a skill seriously, you need evals, comparison, and a repeatable process.

If you are working with agentic IDEs today, I really think this is the moment to build.

Create your own skills. Teach them to ask the right questions. Test them with real evals. Compare with and without your guidance. Review them hard. Ship them before you disappear into endless polishing.

That is how we get better tooling. That is how we stop treating AI assistance like magic and start treating it like engineering.

And if this plugin helps you do that, even a little, then this journey was worth it.

If you try it and see ways to improve it, send a PR. I mean that.

Big hug, Charly

🤖 Your Backlog Speaks, Your IDE listens: How to Build and Test Apps with MCP in Command

CharlyAutomatiza — Fri, 15 Aug 2025 13:57:42 +0000

"This isn't the future, it's the present. The tools are within everyone's reach." - Buenos Aires Meetup, July 2025

🎯 The Reality That Changed Everything

Picture this scene: It's 3 PM on a Friday, you have 20 tasks in the backlog, 3 critical bugs, and your PM just arrived with "a small modification" that needs to be ready by Monday. Sound familiar?

For 19 years in tech, I've lived this reality countless times. But something changed radically in recent months, and no, it's not another JavaScript framework 😄.

The revolution didn't come from where we expected. It came from being able to directly converse with our backlog.

🚫 The Problem We All Know

Traditional Development Limitations

In the traditional development cycle, regardless of whether you use Scrum, Kanban, or your preferred methodology, we face the same challenges:

Fragmented Communication: Between product, UX, development, and QA, there's always "broken telephone"
Infinite Manual Processes: From idea to code, there are 47 manual steps (I literally counted them once)
Gap Between Specification and Result: What we ask for vs. what we build vs. what we actually needed

Result: We spend 70% of our time on processes and only 30% solving real problems.

💡 The Silent Revolution: MCP Servers

MCP: My Code's Perfect (Finally!) 🎯

MCP (Model Context Protocol) is the framework that Anthropic launched to solve a problem we all faced: How do you give specific context to LLMs without going crazy?

Before MCP:

Each model needed its own implementation
Your project context was lost
You had to choose between Claude, GPT, Gemini... and start from scratch each time

With MCP:

One framework, any model
Your context + LLM power = magic
Tools that speak directly to your stack

The "Aha!" Moment

The turning point came when I realized something: We no longer need to be "10x engineers", we need to be engineers who leverage 10x tools.

🏗️ The Experiment: Wallet Sandbox

Setting Up the Laboratory

To demonstrate MCP's power, I created a virtual wallet application with:

Next.js 15 + App Router
TypeScript + Tailwind CSS
Redux Toolkit for state
Playwright for testing

The Crucial Part: All context was defined in structured documentation:

BACKLOG.md with detailed user stories
BUILD_STANDARDS.md with defined processes
PLAYWRIGHT_STANDARDS.md with testing standards

The Protagonists: GitHub MCP and Playwright MCP

1. GitHub MCP: Your AI Project Manager

Available tools:

Complete issues and PRs management
Automatic branch creation
Reviewer handling
Project tracking

2. Playwright MCP: Your Automated QA

Capabilities:

Automatic application exploration
Context-based test generation
Intelligent element detection
Automatic bug reporting

🎬 The Live Show: From Idea to Production

Act 1: "What Tasks Do I Have Assigned?"

👤 User: "What tasks do I have assigned in the backlog?"

🤖 AI: [Executes tools automatically]
- ✅ Reads BACKLOG.md
- ✅ Queries GitHub Issues
- ✅ Analyzes open PRs
- ✅ Classifies by priority

Response: "You have 3 pending critical bugs and 1 high-priority US. 
I also detected 2 PRs under review that might need your attention."

🔥 Plot Twist: I didn't tell it to go to GitHub. I didn't tell it to read the backlog. It simply... did it.

Act 2: "Implement Scheduled Transfers"

The user story [US-301] was defined in the backlog:

As a wallet user

I want to schedule recurring transfers

So that I can automate periodic payments

The Magic of Defined Context

Before executing any code, the system:

Verified the Mandatory Workflow:

   git checkout main
   git pull origin main
   git checkout -b feature/US-301-transferencias-programadas-v2

Analyzed Existing Architecture:
- Next.js App Router structure
- Established Redux patterns
- Available UI components
Followed Defined Standards:
- Naming conventions
- Strict TypeScript
- Server Actions patterns

The Result: 15 Minutes, Complete Functionality

In real-time, during the presentation, it built:

✅ Form to schedule transfers
✅ Frequency selector (one-time, daily, weekly, monthly)
✅ Future date validations
✅ Scheduled transfers management page
✅ Complete integration with existing flow

Visual Journey: From Concept to Reality

What would traditionally take 2-3 days of development, MCP delivered in just 15 minutes. Here's the actual implementation created during the live presentation:

📝 Step 1: Intuitive Transfer Scheduling

> Clean, user-friendly form that guides users through scheduling recurring transfers with frequency options, date validation, and clear input fields - all generated automatically following established design patterns.

✅ Step 2: Immediate Feedback & Confirmation

> Professional success screen providing users with complete transfer details and clear next steps. The confirmation experience feels polished and trustworthy - exactly what you'd expect from a production app.

📊 Step 3: Professional Management Dashboard

> Comprehensive dashboard where users can view, edit, and manage all their scheduled transfers. Each entry shows essential details like amount, frequency, and next execution date, with convenient action buttons for full control.

The Traditional vs. MCP Reality:

Traditional approach: Multiple wireframes, design iterations, component building, state management setup, validation logic, testing cycles
MCP approach: One conversation, instant implementation, production-ready result

Generated code: 347 lines across 5 files
Traditional development time estimate: 2-3 days
Real time with MCP: 15 minutes

Act 3: Automatic Testing with Playwright MCP

👤 User: "What tests do you suggest I perform?"

🤖 AI with Playwright MCP:
- ✅ Analyzes built functionality
- ✅ Identifies critical flows
- ✅ Generates and executes tests automatically
- ✅ Detects console errors
- ✅ Validates behavior across different browsers

The Surprising Part: It found bugs we didn't even know existed.

Technical Note: During the demo, I had to manage the tool limit in VS Code, strategically switching between GitHub MCP and Playwright MCP to optimize performance for different development phases.

Act 4: The Virtuous Circle

When Playwright MCP detected minor console errors:

👤 User: "Create a bug ticket with the detected error"

🤖 GitHub MCP:
- ✅ Analyzes the found error
- ✅ Gathers context and evidence
- ✅ Creates structured issue in GitHub
- ✅ Assigns appropriate labels and severity
- ✅ Includes reproduction steps

Result: Complete issue with all necessary information, ready to be worked on.

Act 5: Complete Automation Cycle - The Live Demo Finale

This is where the audience witnessed something unprecedented. After being satisfied with the implementation, I demonstrated the complete automated publication cycle:

The Automated Publication Process

👤 User: "I'm satisfied with the implementation. Let's publish this."

🤖 GitHub MCP (following predefined rules):
- ✅ Validates all code changes
- ✅ Executes automatic commit with proper message format
- ✅ Pushes branch to remote repository
- ✅ Creates pull request with detailed description
- ✅ Assigns configured reviewers automatically
- ✅ Links PR to original user story issue

What the audience saw: A complete Git workflow executed through natural language commands, all because the rules were predefined in the Copilot instructions.

The AI-Powered Code Review Cycle

Within minutes of the PR creation, the real magic happened:

🤖 GitHub Copilot (automatically):
- ✅ Reviews the generated code
- ✅ Analyzes implementation patterns
- ✅ Posts detailed review comments with suggestions
- ✅ Identifies potential improvements
- ✅ Suggests best practices

Accessing Review Feedback

During the demo, I showed how to interact with the review process:

👤 User: "Analyze all the comments on my PR"

🤖 GitHub MCP:
- ✅ Retrieves all review comments automatically
- ✅ Analyzes and summarizes feedback and suggestions
- ✅ Categorizes comments by importance
- ✅ Provides context for each suggestion
- ✅ Suggests possible action plan

Real review comments analyzed included:

Direct state mutation issues
Navigation in render problems
Hook usage improvements
Performance optimization opportunities

The Complete Cycle Metrics

Real demo timeline (~20 minutes total):

Feature implementation: 15 minutes
Automated testing: 2 minutes
Publication workflow: 1 minute
Copilot review: Automatic (2 minutes)
Issue creation: 30 seconds

Traditional workflow estimate:

Feature implementation: 2-3 days
Testing and debugging: 4-6 hours
Code review process: 1-2 days
Bug reporting and tracking: 1-2 hours
Total: 4-6 days

Productivity improvement: ~96% time reduction

🔧 The Ingredients for Success

1. Copilot Instructions (The Project's DNA)

The secret lies in the instructions you define. My instructions file included:

⚠️ **CRITICAL WORKFLOW REQUIREMENT** ⚠️
ALWAYS START by verifying backlog task and Git workflow steps BEFORE suggesting any technical solution.

1. **Enhanced Task Discovery Process**:
   - Read BACKLOG.md first
   - Search GitHub Issues if task not found
   - Confirm task ID (US-XXX, TT-XXX, BG-XXX)

2. **Git Workflow**:
   - Start from updated main
   - Create feature branch following naming convention
   - Follow commit message standards

3. **Development Standards**:
   - Next.js 15 with App Router
   - TypeScript strict mode
   - Component structure patterns

4. **Publication Rules**:
   - Auto-commit with proper message format
   - Auto-push to remote branch
   - Auto-create PR with detailed description
   - Auto-assign reviewers including Copilot
   - Auto-apply labels and issue linking

2. Structured Documentation

Every aspect of the project was documented:

BACKLOG.md: 50+ user stories with acceptance criteria
BUILD_STANDARDS.md: Development processes and Git workflows
PLAYWRIGHT_STANDARDS.md: Testing standards and Page Object Model

3. Context, Not Commands

Instead of saying "do X", I defined why and how:

## Test Automation Standards

### Pre-Publication Validation
- ALWAYS validate tests execute successfully before publishing changes
- Distinguish between test failures and application bugs
- Create GitHub Issues for application defects with comprehensive evidence

4. Technical Constraints and Solutions

Tool Limitations: VS Code has a tool limit for MCP servers, requiring strategic tool selection during different phases of development.

Context Management: Longer conversations can impact performance, sometimes requiring fresh contexts for complex workflows.

📊 The Numbers That Matter

Before vs. After

Process	Traditional Method	With MCP	Reduction
Backlog analysis	30 min	2 min	93%
Feature implementation	2-3 days	15 min	99%
Test creation	4-6 hours	2 min	99%
Bug reporting	20 min	30 seconds	97%
Code review setup	5 min	Automatic	100%
Complete cycle	4-6 days	~20 min	~96%

Output Quality

Standards adherence: Consistent (predefined rules)
Automatically detected bugs: Comprehensive test coverage
Generated documentation: Complete and up-to-date
Code review coverage: Automatic and thorough

🚀 Practical Implementation: Your Roadmap

Phase 1: Preparation (1 week)

Document Your Context:

   - Create BACKLOG.md with your current tasks
   - Define BUILD_STANDARDS.md with your processes
   - Structure TESTING_STANDARDS.md

Configure Your IDE:
- VS Code with native MCP support
- Or Cursor with cursor rules
- Install necessary MCP Servers
Define Your Copilot Instructions:
- Git workflows
- Code standards
- Testing processes
- Publication automation rules

Phase 2: First Experiments (1 week)

Start Simple:

   "What pending tasks do I have in my backlog?"
   "Analyze this code and suggest improvements"
   "Create a test for this functionality"

Iterate the Instructions:
- Observe what works and what doesn't
- Adjust context based on results
- Refine standards

Phase 3: Complete Integration (2 weeks)

Complete Workflows:

   Backlog → Implementation → Testing → PR → Deploy

Advanced Automation:
- Automatic bug detection
- Performance testing
- Documentation generation
- Automated code review cycles

Recommended Tools

IDEs with MCP Support

VS Code: Native support
Cursor: Excellent integration, no tool limits
Claude Desktop: For quick experimentation

Essential MCP Servers

GitHub MCP: Project and code management
Playwright MCP: Automated testing
Supabase MCP: Database and backend services - Getting Started

🚀 Your Next Move: The Revolution Starts Now

For Developers

The competitive advantage is real, and the window is open now.

This Week:
- Download VS Code or Cursor
- Configure GitHub MCP
- Have your first conversation with the backlog
This Month:
- Implement your first complete feature with MCP
- Automate your testing suite
- Measure the time saved
- Set up automated publication workflows
This Quarter:
- Make MCP your main workflow
- Train your team
- Document your processes
- Become the productivity multiplier for your organization

For Product Managers

It's time to get closer to technical implementation.

MCP democratizes development. Now you can:

Converse directly with the backlog
Validate implementations in real-time
Iterate ideas without waiting for complete sprints
See changes in production within hours, not weeks

For CTOs and Tech Leads

The competitive advantage lies in early adoption.

Teams that adopt MCP now will have:

Significantly faster development cycles
Consistent quality through predefined standards
Lower learning curve for new developers
Documentation that stays current automatically
Complete traceability from idea to production

🌟 Final Reflections

The New Reality

After 19 years in technology, I can affirm that we've never been so close to democratizing software development.

MCP doesn't replace technical knowledge, it amplifies it. You still need to:

Understand the problem you're solving
Know the technologies you use
Define clear processes and standards

But now, instead of spending time on repetitive tasks, you can converse with your backlog and build solutions.

The Live Demo Impact

During the Buenos Aires Meetup, someone asked: "How do you see the future of these tools?"

My answer was clear: Let's not talk about the future. Let's talk about the present.

The tools are here. MCP Servers are available. IDEs have native support.

What we demonstrated live wasn't a vision—it was reality. In 20 minutes, we went from user story to production-ready code with automated testing, code review, and bug tracking.

The only question left is: When are you going to start?

📚 Resources and Links

Project Code

Official MCP Servers

Connect With Me

Presentation Video

🎥 Buenos Aires Meetup - July 2025 (Video in Spanish)

Was this article useful to you? Share it with your team and let's start the MCP revolution together. Your backlog is waiting for you to talk to it.

The revolution has begun. The only question is: Are you part of it?

#MCP #Development #Automation #AI #Productivity #GitHub #Playwright #VSCode

Grafana K6 cheat sheet: everything a performance engineer should know

CharlyAutomatiza — Fri, 25 Oct 2024 01:48:35 +0000

Grafana K6 Cheat Sheet: Everything a Performance Engineer Should Know (with Examples and Best Practices)

1. Introduction to Grafana K6

Grafana K6 is an open-source tool designed for performance testing. It's great for testing APIs, microservices, and websites at scale, providing developers and testers insights into system performance. This cheat sheet will cover the key aspects every performance engineer should know to get started with Grafana K6.

What is Grafana K6?

Grafana K6 is a modern load testing tool for developers and testers that makes performance testing simple, scalable, and easy to integrate into your CI pipeline.

When to use it?

Load testing
Stress testing
Spike testing
Performance bottleneck detection
API testing
Browser testing
Chaos engineering

2. Grafana K6 Cheat Sheet: Essential Aspects

2.1. Installation

Install Grafana K6 via Homebrew or Docker:

brew install k6
# Or with Docker
docker run -i grafana/k6 run - <script.js

2.2. Basic Test with a Public REST API

Here's how to run a simple test using a public REST API.

import http from "k6/http";
import { check, sleep } from "k6";

// Define the API endpoint and expected response
export default function () {
  const res = http.get("https://jsonplaceholder.typicode.com/posts/1");

  // Define the expected response
  const expectedResponse = {
    userId: 1,
    id: 1,
    title:
      "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    body: "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto",
  };

  // Assert the response is as expected
  check(res, {
    "status is 200": (r) => r.status === 200,
    "response is correct": (r) =>
      JSON.stringify(JSON.parse(r.body)) === JSON.stringify(expectedResponse),
  });

  sleep(1);
}

2.2.1 Running the test and utilization of web dashboard

To run the test and view the results in a web dashboard, we can use the following command:

K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT=html-report.html k6 run ./src/rest/jsonplaceholder-api-rest.js

This will generate a report in the reports folder with the name html-report.html.

But we also can see the results in the web dashboard by accessing the following URL:

http://127.0.0.1:5665/

Once we access the URL, we can see the results on real time of the test in the web dashboard.

2.3. Test with a Public GraphQL API

Example using a public GraphQL API.

If you don't know what is a GraphQL API, you can visit the following URL: What is GraphQL?.

For more information about the GraphQL API we are going to use, you can visit the documentation of the following URL: GraphQL Pokémon.

For more information about how to test GraphQL APIs, you can visit the following URL: GraphQL Testing.

This is a simple test to get a pokemon by name and check if the response is successful.

import http from "k6/http";
import { check } from "k6";

// Define the query and variables
const query = `
  query getPokemon($name: String!) {
    pokemon(name: $name) {
      id
      name
      types
    }
  }`;

const variables = {
  name: "pikachu",
};

// Define the test function
export default function () {
  const url = "https://graphql-pokemon2.vercel.app/";
  const payload = JSON.stringify({
    query: query,
    variables: variables,
  });

  // Define the headers
  const headers = {
    "Content-Type": "application/json",
  };

  // Make the request
  const res = http.post(url, payload, { headers: headers });

  // Define the expected response
  const expectedResponse = {
    data: {
      pokemon: {
        id: "UG9rZW1vbjowMjU=",
        name: "Pikachu",
        types: ["Electric"],
      },
    },
  };

  // Assert the response is as expected
  check(res, {
    "status is 200": (r) => r.status === 200,
    "response is correct": (r) =>
      JSON.stringify(JSON.parse(r.body)) === JSON.stringify(expectedResponse),
  });
}

3. Best Practices for Structuring Performance Projects

3.1. Centralized Configuration

Define global configurations options such as performance thresholds, the number of virtual users (VU), and durations in one place for easy modification.

// ./src/config/options.js
export const options = {
  stages: [
    { duration: '1m', target: 100 }, // ramp up to 100 VUs
    { duration: '5m', target: 100 }, // stay at 100 VUs for 5 mins
    { duration: '1m', target: 0 },   // ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'], // 95% of requests should complete in under 500ms
  },
};

3.2. Code Modularity

3.2.1. Constants and Requests for the REST API

Separate code into reusable modules, for example, separating constants and requests from test logic.

For our REST API example, we can create a constants.js file to store the base URL of the API and a requests-jsonplaceholder.js file to store the functions to interact with the API.

// ./src/utils/constants.js
export const BASE_URLS = {
  REST_API: 'https://jsonplaceholder.typicode.com',
};

Now we can create the requests-jsonplaceholder.js file to store the functions to interact with the API.

// ./src/utils/requests-jsonplaceholder.js
import { BASE_URLS } from './constants.js';
import http from 'k6/http';

export function getPosts() {
    return http.get(`${BASE_URLS.REST_API}/posts`);
}

export function getPost(id) {
    return http.get(`${BASE_URLS.REST_API}/posts/${id}`);
}

export function createPost(post) {
    return http.post(`${BASE_URLS.REST_API}/posts`, post);
}

export function updatePost(id, post) {
    return http.put(`${BASE_URLS.REST_API}/posts/${id}`, post);
}

export function deletePost(id) {
    return http.del(`${BASE_URLS.REST_API}/posts/${id}`);
}

3.2.2. Integration of Requests in the Test Script of the REST API

Finally, we can create our test script jsonplaceholder-api-rest.js to use the functions we created in the requests-jsonplaceholder.js file.

// ./src/rest/jsonplaceholder-api-rest.js
import { check, sleep } from "k6";
import { getPost, createPost, updatePost, deletePost } from "../utils/requests-jsonplaceholder.js";
import { options } from "../config/options.js";

export default function () {
  // GET request
  let res = getPost(1);
  check(res, {
    "GET status is 200": (r) => r.status === 200,
    "GET response has correct id": (r) => JSON.parse(r.body).id === 1,
  });

  // POST request
  const newPost = {
    title: "foo",
    body: "bar",
    userId: 1,
  };
  res = createPost(newPost);
  check(res, {
    "POST status is 201": (r) => r.status === 201,
    "POST response has id": (r) => JSON.parse(r.body).id !== undefined,
  });

  // PUT request
  const updatedPost = {
    id: 1,
    title:
      "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    body: "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto",
  };
  res = updatePost(1, updatedPost);
  check(res, {
    "PUT status is 200": (r) => r.status === 200,
    "PUT response has updated title": (r) =>
      JSON.parse(r.body).title === "foo updated",
  });

  // DELETE request
  res = deletePost(1);
  check(res, {
    "DELETE status is 200": (r) => r.status === 200,
  });

  sleep(1);
}

Our script code is now much simpler to understand, and if something changes in the URL, parameters or if a new method needs to be added, the place where the changes need to be made is centralised, making our solution simpler to extend over time.

We could further improve our scripts by creating more atomic functions that we can reuse to create more complex scenarios in the future if necessary, it is getting simpler to understand what our test script does. For example if we wanted to test the existence of a post, we could create a function that gets a post and returns the response, then we could use this function in our test script jsonplaceholder-api-rest.js.

// ./src/rest/jsonplaceholder-api-rest.js
import { check, sleep } from "k6";
import { getPost, createPost, updatePost, deletePost } from "../utils/requests-jsonplaceholder.js";
import { options } from "../config/options.js";

// Function to test GET request
function testGetPost(id) {
  let res = getPost(id);
  check(res, {
    "GET status is 200": (r) => r.status === 200,
    "GET response has correct id": (r) => JSON.parse(r.body).id === id,
  });
}

// Function to test POST request
function testCreatePost(post) {
  let res = createPost(post);
  check(res, {
    "POST status is 201": (r) => r.status === 201,
    "POST response has id": (r) => JSON.parse(r.body).id !== undefined,
  });
}

// Function to test PUT request
function testUpdatePost(id, post) {
  let res = updatePost(id, post);
  check(res, {
    "PUT status is 200": (r) => r.status === 200,
    "PUT response has updated title": (r) => JSON.parse(r.body).title === post.title,
  });
}

// Function to test DELETE request
function testDeletePost(id) {
  let res = deletePost(id);
  check(res, {
    "DELETE status is 200": (r) => r.status === 200,
  });
}

// Main function
export default function () {
  testGetPost(1);

  const newPost = {
    title: "foo",
    body: "bar",
    userId: 1,
  };
  testCreatePost(newPost);

  const updatedPost = {
    id: 1,
    title: "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    body: "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto",
  };
  testUpdatePost(1, updatedPost);

  testDeletePost(1);

  sleep(1);
}

3.2.3. Constants and Requests for the GraphQL API

We can modify the constants.js file to add the base URL of the GraphQL API and the headers we need to use.

// ./src/utils/constants.js
export const BASE_URLS = {
  REST_API: 'https://jsonplaceholder.typicode.com',
  GRAPHQL_POKEMON: 'https://graphql-pokemon2.vercel.app',
};

export const HEADERS = {
  'Content-Type': 'application/json',
};

Now we can create the requests-graphql-pokemon.js file to store the functions to interact with the GraphQL API.

// ./src/utils/requests-graphql-pokemon.js
import { BASE_URLS, HEADERS } from "./constants.js";
import http from "k6/http";

// Get a pokemon by name and return the fields specified in the fields array
export function getPokemon(name, fields = ["id", "name", "types"]) {
  const fieldsString = fields.join(" ");
  const query = `
    query getPokemon($name: String!) {
      pokemon(name: $name) {
        ${fieldsString}
      }
    }
  `;
  const pokemonVariables = { name: name };
  const payload = JSON.stringify({ query: query, variables: pokemonVariables });

  return http.post(BASE_URLS.GRAPHQL_POKEMON, payload, { headers: HEADERS });
}

3.2.4. Integration of Requests in the Test Script of the GraphQL API

In this moment we can create our test script to use the functions we created in the requests-graphql-pokemon.js file. We will create a simple test script that will get the data of a pokemon and check if the response is successful.

// ./src/pokemon-graphql-test.js
import { check } from "k6";
import { getPokemon } from "../utils/requests-graphql-pokemon.js";

// Define the test function
export default function () {
  const res = getPokemon("pikachu");

  // Define the expected response
  const expectedResponse = {
    data: {
      pokemon: {
        id: "UG9rZW1vbjowMjU=",
        name: "Pikachu",
        types: ["Electric"],
      },
    },
  };

  // Assert the response is as expected
  check(res, {
    "status is 200": (r) => r.status === 200,
    "response is correct": (r) =>
      JSON.stringify(JSON.parse(r.body)) === JSON.stringify(expectedResponse),
  });
}

In the same way as for the example of api rest, we can improve our script by creating more atomic functions that we can reuse to create more complex scenarios in the future if necessary, it is getting simpler to understand what our test script does.

There is still a better way to optimise and have a better parameterisation of the response and request results, what do you imagine we could do?

3.3. Dynamic Data and Parameterization

Use dynamic data to simulate more realistic scenarios and load different data sets. K6 allows us to use shared arrays to load data from a file. Shared arrays are a way to store data that can be accessed by all VUs.

We can create a users-config.js file to load the users data from a JSON file users.json.

[
    { "id": 1 },
    { "id": 2 },
    { "id": 3 },
    { "id": 4 },
    { "id": 5 },
    { "id": 6 },
    { "id": 7 },
    { "id": 8 },
    { "id": 9 },
    { "id": 10 }
]

// ./src/config/users-config.js
import { SharedArray } from 'k6/data';

export const users = new SharedArray('User data', function () {
  return JSON.parse(open('../data/users.json')); // Load from a file
});

And then we can use it in our test script jsonplaceholder-api-rest.js.

// ./src/rest/jsonplaceholder-api-rest.js
import { check, sleep } from "k6";
import { getPost } from "../utils/requests-jsonplaceholder.js";
import { options } from "../config/options.js";
import { users } from "../config/users-config.js";
// Function to test GET request
function testGetPost(id) {
  let res = getPost(id);
  check(res, {
    "GET status is 200": (r) => r.status === 200,
    "GET response has correct id": (r) => JSON.parse(r.body).id === id,
  });
}

// Main function
export default function () {
  const user = users[Math.floor(Math.random() * users.length)];

  testGetPost(user.id);
  sleep(1);
}

4. Project Structure

A well-organized project structure helps in maintaining and scaling your tests. Here's a suggested folder structure:

/project-root
│
├── /src
│   ├── /graphql
│   │   ├── pokemon-graphql-test.js      # Test for GraphQL Pokémon API
│   │   ├── other-graphql-test.js        # Other GraphQL tests
│   │
│   ├── /rest
│   │   ├── jsonplaceholder-api-rest.js # REST API test for JSONPlaceholder
│   │   ├── other-rest-test.js          # Another REST API test
│   │
│   └── performance-scenarios.js  # Script combining multiple performance tests
│
├── /utils
│   ├── requests-graphql-pokemon.js # Reusable functions for GraphQL requests
│   ├── requests-jsonplaceholder.js # Reusable functions for REST requests
│   ├── checks.js                   # Reusable validation functions
│   ├── constants.js             # Global constants, like URLs or headers
│
├── /config
│   ├── options.js                # Global configuration options
│   ├── users-config.js           # Configuration for users data
│
├── /reports
│   └── results.html             # Output file for results (generated after running tests)
│
├── /data
│   └── users.json             # Users data
│
├── README.md                    # Project documentation
└── .gitignore                   # Files and folders ignored by Git

This structure helps in keeping your project organized, scalable, and easy to maintain, avoiding clutter in the project root.

Another option would be to group test scripts into folders by functionality, you can test and compare what makes the most sense for your context. For example, if your project about a wallet that makes transactions, you could have a folder for each type of transaction (deposit, withdrawal, transfer, etc.) and inside each folder you could have the test scripts for that specific transaction.

/project-root
│
├── /src
│   ├── /deposit
│   │   ├── deposit-test-1.js      # Test for deposit
│   │   ├── deposit-test-2.js      # Another deposit test
│   │
│   ├── /withdrawal
│   │   ├── withdrawal-test-1.js      # Test for withdrawal
│   │   ├── withdrawal-test-2.js      # Another withdrawal test
│   │
│   ├── /transfer
│   │   ├── transfer-test-1.js      # Test for transfer
│   │   ├── transfer-test-2.js      # Another transfer test
│   │
│   └── performance-scenarios.js  # Script combining multiple performance tests
│
├── /utils
│   ├── requests-deposit.js # Reusable functions for deposit
│   ├── requests-withdrawal.js # Reusable functions for withdrawal
│   ├── requests-transfer.js # Reusable functions for transfer
│   ├── checks.js             # Reusable validation functions
│   ├── constants.js          # Global constants, like URLs or headers
│
├── /config
│   ├── options.js                # Global configuration options
│   ├── users-config.js           # Configuration for users data
│   ├── accounts-config.js        # Configuration for accounts data
│
├── /reports
│   └── results.html             # Output file for results (generated after running tests)
│
├── /data
│   └── users.json             # Users data
│   └── accounts.json          # Accounts data
│
├── README.md                    # Project documentation
└── .gitignore                   # Files and folders ignored by Git

On this second example, we have a more complex data structure, but we can still reuse the same requests functions that we created for the first example.

Conclusion

Performance testing with K6 is critical for identifying bottlenecks and ensuring application scalability. By following best practices such as modularizing code, centralizing configurations, and using dynamic data, engineers can create maintainable and scalable performance testing scripts.

Big hug.

Charly Automatiza

Book review: Enhanced Test Automation with WebdriverIO: Unlock the superpowers of hybrid testing framework

CharlyAutomatiza — Fri, 26 Jan 2024 16:55:59 +0000

My motivation

This year I set myself the personal goal of reading at least one book a month that is not in my native language. In this case, I also have the goal of reading in English, but also to start reading books in Portuguese, as I am slowly learning this language.

Having said that, I am going to give you my personal opinion about this book.

About the book

In my opinion, WebDriverIO (Next-gen browser and mobile automation test framework for Node.js) is one of the most complete and versatile modern frameworks.
Just by seeing the wide variety of topics that were going to be covered just by reading the table of contents, I already had very high expectations and honestly it exceeded them.

If you are someone who is just starting out in the field of Test Automation, you will be able to see the basics little by little, although if you are an experienced SDET, you will see more than one tip in the first few pages that you certainly didn't know about.

We are gradually getting into more and more complex topics in a progressive and very enjoyable way. Something that I found very successful is that from the 3rd chapter we are taught how to debug our tests, remember that test automation is part of software development and as in any construction cycle, debugging errors is a fundamental part.

Complex topics such as callbacks, promises and async/await in JavaScript/TypeScript are addressed in a practical and simple way, the unfamiliar reader will quickly understand.

Chapters 5 to 8 dives right into the implementation of helpers, I really enjoyed this, showing these implementations to those who are just starting out is a great accelerator.

Chapter 9 will explain all about Page Object Model A.k.A POM, one of the most used design patterns in test automation, you will also see an alternative to implement POM/BDD using Klassi-js, a framework that I didn't know and that has several very interesting features.

Chapters 10 and 12 seem to me key chapters as they explain how to write robust selectors, I found a great success to show all the potential that the editor of selectors SelectorsHub gives us to once again, simplify our task and minimize future maintenance.

The authors also present very creative solutions to make our selectors "self-healing" and very tolerant to changes.

Additional note: It is always a good time to recommend the use of this tool, not because I am an SelectorsHub Ambassador, as I have always recommended this tool long before, which I consider a great time saver.

In the last chapters we will see how to integrate our solution to Allure Report, in my opinion one of the best Automation Test Reporting tool, and also how to run our tests in a scalable way in multiple browsers both in locally mounted environments using Selenium Grid and also using the powerful cloud solution such as LambaTest.

To close and complement the basic circuit of automation, we will see and learn how to use Jenkins, one of the most used tools for continuous integration, do not worry if it is your first time with Jenkins, the authors will accompany you to learn how to install and even show you how to have constant feedback through the integration of Allure Reports and also with Slack messages.

The appendix is super rich, reading it reminded me of the times I had those errors that are described there and the time I had to invest to be able to give solution, will be your almost constant consultation section to each error and surely you will find the solution and save a lot of time, I assure you.

Conclusion

I must say that I really enjoyed reading Enhanced Test Automation with WebdriverIO: Unlock the superpowers of hybrid testing frameworks, I felt identified having implemented, in real projects, similar solutions to those exposed by the authors and I also took a lot of knowledge that I did not have before.

It is recommended both for beginners and for those who may have extensive experience with test automation, shows the full potential of WebDriverIO profiling it on par with the most robust modern frameworks of today and prepares the ground to implement robust solutions from zero to CI/CD, building solid selectors and using cloud solutions.

Many thanks to the authors Paul M. Grossman and Larry C. Goddard for delivering a great work.

I also want to thank Shrinidh and Packt for sending me this book, I have learned a lot.

Charly