DEV Community: 🇷|🇺|🇧|🇪|🇳

AI Agents with MCP: Practical Takeaways from n8n and GitHub Copilot

🇷|🇺|🇧|🇪|🇳 — Mon, 07 Apr 2025 07:59:22 +0000

Key Takeaways for Developers

✅ Successful Approaches:

Breaking down tasks into focused, single operations
Maintaining control over the workflow instead of letting AI decide
Using AI for specific steps rather than complex control flows
Version controlling AI-assisted content

❌ Common Pitfalls:

Letting AI handle long-running operations (leads to timeouts and context loss)
Assuming consistent behavior between different AI models and platforms
Letting AI figure out control flow independently
Making AI think about too many things at once

AI Agents - Insights and Observations

The more I work with AI, the more I discover what is generic and predictable versus what constitutes actual original insights. There is also a third category: technical challenges. The expectation is that as technology develops, technical issues will decrease, and AI will handle more of the generic work. This means the only work that will remain is original thinking and insights. This is the optimistic outlook. Note this also means that you will actually have to think a lot harder during your knowledge work.

If you let an AI agent take a lot of actions, like reordering a filesystem, it takes a lot of slow repetitions and eventually runs into timeouts. You also notice that it starts to forget what it is doing while controlling a browser. For example, Playwright tries to click on links from the previous page which are stale/unclickable, whereas earlier in the execution it was able to remember to navigate back to click on the links.

Let it do one thing at a time to maintain reliability.

If you know in which order actions should be performed, you don't need non-deterministic control flow to handle the task.
Agentic AI and Copilot Chat edits in VS Code are removing the copy-paste steps out of the development process.

Speed is an advantage that might be especially beneficial for Gemini, as specialized hardware can make it a lot faster than other models.

If you know the control flow (i.e., what the agent should do), don't let the AI figure this out on its own. Instead, program the flow yourself and use AI function calling or something like Instructor to do the language processing. Letting the AI figure out the control flow is slow and only makes sense if the flow depends on intelligent decisions. It feels like you lose some intelligence by letting the AI think about too many things at once. The more focused you can make your task, the better output you can expect.

Examples where you know the control flow:

You are scraping LinkedIn profiles for gathering leads
You do a code review where you know you want to first look at the code then add comments
For file organization, you first scan files, then categorize, then move the files

The key point is that in these cases, you already know the sequence of actions. The AI's role is to handle individual steps (like understanding file content or formatting text) rather than figuring out what to do next.

Nice cases where you don't know what to do:

Programming with AI where it's unclear which files need to be edited and how
Search engines like Perplexity where the query path isn't predetermined
Data analysis where patterns aren't known beforehand
Customer support where each query requires a unique investigation path

The common factor in all these cases is that the input is a user query or problem where there is no clear algorithm that could be hardcoded. The AI needs to make decisions based on context and intermediate findings rather than following a predetermined path.

Understanding n8n and GitHub Copilot Agents

What is n8n?

n8n is an open-source workflow automation tool that allows users to connect various applications, APIs, and data sources to automate tasks. It provides a visual interface where you can build workflows by connecting nodes that represent different services or actions. Think of it as an alternative to tools like Zapier or Make (formerly Integromat), but with more flexibility and the ability to self-host.

Key features of n8n include:

Visual workflow builder
200+ pre-built integrations
Ability to run custom JavaScript code
Self-hosting option for complete control over your data
Community-contributed nodes to extend functionality

What are GitHub Copilot Agents?

GitHub Copilot agents are an extension of GitHub Copilot that goes beyond code completion. These agents can:

Have conversations with developers about code
Execute tools and perform actions on behalf of users
Interact with external services through MCP
Help with complex development tasks

To access Copilot agents:

Sign up for GitHub Copilot (requires a subscription)
Join the GitHub Copilot Insider program
Install the latest version of VS Code and the GitHub Copilot extension
Enable Copilot agent features in settings

What is Model Context Protocol (MCP)?

The Model Context Protocol (MCP) is a standardized way for AI models to interact with external tools and services. It acts as a "universal translator" that enables seamless communication between different systems, allowing AI agents to perform actions in the real world.

n8n with MCP Integration

The n8n-nodes-mcp community node allows users to connect MCP servers within their workflows. This integration supports:

Listing available tools on connected MCP servers
Executing tools through AI agents
Sending prompts and retrieving structured responses
Accessing resources from connected servers

With this integration, you can create AI-powered workflows that automate complex tasks like data analysis, web scraping, or file management.

Installing MCP Tools

Installing MCP tools is remarkably straightforward, typically requiring just a simple npx command to download and run a server. Most tools can be installed and started with a single command line instruction. While you can add tools directly through VS Code's command line interface, there are other installation methods available, such as using configuration files or installing the servers globally via npm. Each tool may need some basic configuration, like specifying allowed directories for filesystem access or setting browser preferences, but the overall process remains simple and makes it easy to enhance your AI agents with new capabilities.


sh
code-insiders --add-mcp '{"name":"playwright","command":"npx","args":["@playwright/mcp@latest"]}'

code-insiders --add-mcp '{"name":"filesystem","command":"npx","args":["-y","@modelcontextprotocol/server-filesystem","~/Downloads"]}'

Comparison: n8n vs GitHub Copilot Agents

Feature	n8n with MCP	GitHub Copilot Agents
Ease of use	More complex setup, steeper learning curve	Easier to work with, integrated into VS Code
Workflow complexity	Supports complex, multi-step workflows	Better for one-off tasks and development assistance
Persistence	Workflows can be saved and run repeatedly	Sessions are temporary, better for interactive use
MCP Playwright performance	Less stable, context issues between actions	Better performance, maintains browser context between actions
Debugging	Error messages can be cryptic and difficult to debug	More transparent, easier to see what's happening
Prompt engineering	Requires more explicit prompting	Handles vague instructions better
Automation frequency	Better for repeated automations	Better for ad-hoc automations
Tool behavior	More predictable tool usage patterns	Behavior can vary between models (e.g., Claude 3.5 vs 3.7)
Learning curve	Requires learning workflow concepts	Feels more like conversation with occasional "programming in a prompt"

The choice between these platforms depends on your specific needs:

Choose n8n for recurring automations, complex workflows, and when you need a visual representation of your process
Choose GitHub Copilot agents for development assistance, ad-hoc automations, and when you prefer a conversational interface

My Insights and Findings

MCP is useful in quickly adding some tools. I tried out MCP Playwright and MCP FileSystem. When comparing the platforms, Playwright works better in GitHub Copilot than in n8n. In n8n it ran into errors due to the context not being there. I suspect n8n just reruns the Playwright MCP server every action, while Copilot keeps it open so it can keep connecting to the existing tab and browser context.

In n8n you can make more complex workflows and define more steps, which is beneficial for certain use cases. It does work, but you get error messages that are difficult to debug, many due to data being different than expected.

Overall GitHub Copilot agents was easier to work with, but n8n is more powerful when you want to run automations many times.

File Organization Experiment

I experimented with using AI agents to organize files in a filesystem, which revealed interesting differences between AI models. When tasked with file organization, Claude 3.7 took a more cautious approach by writing a shell script to move files. In contrast, Claude 3.5 was more proactive, directly executing the script in the terminal. This experiment highlighted that VS Code provides terminal access to agents by default, allowing for direct interaction with the filesystem. Through multiple iterations, the directory organization process showed continuous improvement.

Testing different MCP tools revealed consistent patterns in AI behavior. Initially, agents would attempt to access directories without checking permissions. After encountering errors, they learned to first inquire about which directories they could even access. This adaptive behavior demonstrates how agents can learn from system responses and adjust their approach accordingly. The process became more efficient when agents were explicitly instructed to check available directories before attempting operations.

LinkedIn Lead Generation Experiment

I conducted an experiment using AI to find decision-makers on LinkedIn:

Set up a SQLite database to store contact information and post summaries
The AI showed unexpected intelligence by focusing on director-level positions automatically
Without being explicitly told, it knew to filter out consultants and engineers
Used MCP Playwright for browser automation and data collection

Key observations:

Different AI models were not equally reliable at storing data
ChatGPT 4.0 was better at remembering to store data while browsing
We learned that storing data immediately works better than saving it all at once

Writing this Blogpost Using an AI Agent

AI agents are helpful when writing. You can focus on your ideas instead of worrying about perfect sentences, grammar, and spelling. The AI helps make your writing flow better.

What matters most is including your own insights. If you let the AI write everything from scratch, you'll end up with generic content that misses the important points. That's why you should regularly save your blogpost to version control - this way, you can always go back if text gets lost or changed too much.

One useful approach is using tools like Perplexity to find documentation or copying READMEs from the internet yourself. Add these to your workspace, and then you can ask the AI agent to write about MCP using this documentation as context.

Some examples of challenges I ran into while writing this post:

The AI often rewrote whole sections when I just wanted small improvements
When writing about MCP setup, it gave generic, copy-pasted information instead of useful details
While AI agents seem like they'll save time, you actually spend extra time learning how to work with them effectively

The AI particularly excelled at:

Improving sentence structure and readability
Adding transition sentences between paragraphs
Catching grammar and spelling issues
Suggesting better ways to organize information
Maintaining consistent tone throughout the post

Conclusion

AI Agents take away the copy-paste steps out of your AI workflow. However, they are not a golden hammer yet. Just because AI Agents could theoretically take certain actions because they have the tools doesn't make them the best approach. Right now, the models often don't behave as expected and execute slowly. If you have to do a lot of prompt engineering to control the agent, you want tighter controls and asking smaller questions to better control the outcome.

I'm optimistic about the future because of the saying you're hearing more and more: what you see now is the worst that it's going to be.

Taking Frontend Architecture Serious with dependency-cruiser

🇷|🇺|🇧|🇪|🇳 — Mon, 25 Sep 2023 09:10:00 +0000

With dependency-cruiser, you can enforce which imports are allowed. This enables you to create an architecture fitness function that ensures your code continues to adhere to the initial design. You can also visualize your dependencies to gain a clearer understanding of your code's actual structure, allowing you to compare it with your mental model and make improvements where necessary.

An application architecture design defines a folder structure and specifies which files can import from other files. On the backend, you have design patterns like layered architecture, hexagonal (or ports and adapters) architecture. On the frontend, there's the classical Model-View-Controller architecture. Modern component-based frameworks also offer their own approaches like having page, feature, or technical folders. Often there is also a shared or common folder.

Applying any architecture pattern or folder structure has an impact on how code should be imported by the rest of the codebase. Verifying the correctness of imports manually can be challenging and is often overlooked during code reviews. An automated check can benefit not only the current team but also future developers. Even when you design the application architecture and implement the code, it's easy to unintentionally deviate from your own guidelines. One auto-import from an editor can compromise your design. Collaborating with others amplifies this challenge.

Keep Page-Specific Folders Isolated

You don't want modules from one page-specific folder importing modules from another page-specific folder. If there's a cross-page dependency, it likely means that some modules are actually shared modules placed in a page-specific folder by accident. The solution is to move the module to a shared or common folder.

In our codebase, we faced this issue because we built one page and later constructed a similar page, realizing we could reuse some components. We forgot to move the components to the shared folder, and this oversight wasn't caught during code review. With dependency-cruiser, the issue was easily discovered and will be avoided in the future.
See Isolating peer folders from each other

Finding Orphaned Code

If unused code is not removed as soon as it's no longer needed, detecting it later can be difficult. Especially for components that are imported by unit tests or Storybook stories, they may seem to be in use but are not actually included anywhere in the real application.

Dependency-cruiser can identify whether code is used by actual production code or not. Of course, exceptions can be made for configuration files that aren't supposed to be part of the application in the first place.

We found an unused component, complete with tests and Storybook stories. By removing it, we no longer need to maintain the code, and it eliminates confusion about why the code was there in the first place.

See Is a module actually used?

Shared Code

It's helpful to distinguish code that is genuinely shared between different parts from code that could be shared but isn't currently. Often, the way code is written makes assumptions about its usage.

By counting how many files depend on a module, you can determine whether a module should be considered shared. You can use this information to enforce that shared modules are placed in a common/ or shared/ folder. Conversely, you can ensure that all code in the shared folders is actually shared. When you remove an import to a shared module, you'll receive an error, allowing you to move the module to a page-specific folder.

Here, we also found code that could be, or had been, shared but, in reality, was specific to a single page. So, we moved this code from the shared folder to the page folder. This shift made the page code more cohesive and decreased its coupling to the shared code—both important architectural qualities.

See Is a utility module shared?

Visualization

Dependency-cruiser has advanced visualization capabilities built-in.

When you visualize the dependencies between different folders, you can see:

Dependencies that shouldn’t be there
Whether a dependency is a dynamic import (for lazy loading) or not
Circular dependencies
Which folders have shared code
If 3rd party code is kept separate or spread everywhere

This is also handy as documentation so everyone has the same understanding.

For the best visualization, you need to tweak the configuration to get the right amount of detail.

In the visualization below, we want to show that pages don’t depend on other pages but only on the shared and service layers. The service layer itself doesn’t rely on any view-specific folders. What isn’t shown, but is enforced by dependency-cruiser, is that the services folder doesn’t have any React-specific dependencies.

Comparison to eslint-plugin-imports

When comparing dependency-cruiser to eslint-plugin-import, a noticeable overlap between the two becomes evident. Both tools can identify circular dependencies, impose limits on certain imports, and assist in keeping test code separate from production code. However, dependency-cruiser offers additional features. It can identify orphaned files even if they are imported by tests. It facilitates the separation of sibling folders to maintain organized code. Additionally, dependency-cruiser displays the usage frequency of a component, enabling teams to discern shared parts of the codebase.

A distinct advantage of eslint is its superior editor integration. Unlike dependency-cruiser, eslint displays red lines beneath invalid imports, providing immediate feedback. If in-editor feedback is valuable, eslint, with its rule no-restricted-imports, serves well in ensuring proper encapsulation of third parties. While dependency-cruiser can perform similarly, it usually provides feedback after a failed CI build, whereas eslint aids developers instantly.

The visualization provided by dependency-cruiser is undeniably a unique feature absent in a typical linter.

I advise utilizing a combination of both tools to attain a developer experience that aligns with your team’s needs.

Real World Experience

Our nearly three-month experience with dependency-cruiser was overwhelmingly positive. Upon activating the rules, we uncovered numerous incorrect imports and misplaced files, even in our high-quality, relatively new React codebase. These revelations prompted enhancements in architectural qualities like separation of concerns, encapsulation, and modularity. The fixes primarily involved relocating files and shifting code between modules. Not enforcing all rules from the outset allowed for incremental improvements, especially beneficial for larger codebases.

Once we began enforcing the rules, dependency errors were scarce, thanks to our consistent code structure and established patterns. Encountered errors were accurate, with straightforward solutions, reinforcing that our design remains intact, averting the risk of incremental degradation, or, more starkly, death by a thousand cuts.

The visualization is invaluable, reflecting current realities over idealized mental models. It aids in onboarding new members and, more importantly, allows existing team members to discover architectural improvements.

Conclusion

Vigilance in software architecture is crucial throughout the development cycle to counter incremental degradation; this is applicable to both frontend and backend codebases. Dependency-cruiser simplifies the design review process and ensures design enforcement during development.

Unfortunately, the production codebase where I applied dependency-cruiser is not public. However, I did conduct some experiments during a Xebia Innovation Day. That code can be found on GitHub.

Monitoring consumer lag in Azure Event Hub

🇷|🇺|🇧|🇪|🇳 — Thu, 30 Sep 2021 15:25:37 +0000

Why

Consumer lag is the most important metric to monitor when working with event streams. However, it is not available as a default metric in Azure Insights. Want to have this metric available as part of your monitoring solution? You can set it up with some custom code. In this blog we show you how.

What

Consumer lag refers to the number of events that still need to be processed by the consumers of a stream. Consumer lag will be 0 most of the time, as every event is consumed immediately. However, there are a few events that can cause that number to rise. When a consumer runs into errors, like a functional issue caused by an event or a technical issue like network connectivity, it'll stop consuming events, increasing the consumer lag.

The lag will also increase if events are published faster than the consumer can process them. In that case, the problem will resolve itself when events are published at a lower rate, and the consumer catches up again.

You can trigger an alert when the consumer lag exceeds 0 for an extended period, like 10 minutes. What the best alert trigger configuration is for you depends on your situation. Before we continue to the solution, let's clarify some terms:

Definitions

Consumer groups enable multiple consumers to subscribe to the same event stream. Typically, a consumer group consists of multiple instances of the same application, that can be used for high availability and horizontal scaling.
Partitions enable events to be processed in parallel. All events within a partition have a fixed order. Events in different partitions can be received out of order because they are processed in parallel. A consuming application can have multiple instances that can each read from multiple partitions.
Namespace is a collection of event hubs/topics that can be managed together.
Checkpoints records the sequence number of the last consumed event. This value is used to ensure that, in the event of a restart, only the events that have not been consumed yet are resent. Typically, checkpoints are stored as a file in BlobStorage.

How

The Azure SDK can retrieve the sequence number of the last enqueued event of a partition. With the CheckpointStore you can retrieve the sequence number of the checkpoint. Since both are simple counters you can calculate the difference and publish this as a custom metric in azure insights. In order to make it a metric you can monitor, you will have to collect the metric periodically, let’s say, every minute.

There are two ways to collect the consumer lag metric:

Using the consumer application provides you with the Event Hub credentials, namespace, and consumer group. However, if something goes wrong and the consuming application shuts down, you'll no longer see if consumer lag rises because this information is not collected anymore. Use a separate process for monitoring to prevent this from happening.
Alternatively, alert the application failing its health check or the consumer lag metric being missing. The code examples below are in Typescript for conciseness. But the same approach can be used with the other Event Hub SDKs, like for C#, Java, Python, Go.

Collecting the consumer lag

// initialize checkpointStore and eventHubClient
const consumerGroup = 'my consumer group';
const checkpointStore = ...
const eventHubClient = ...

// Send the consumer lag every minute
setInterval(async () => {
  try {
    await measureConsumerLag(consumerGroup, eventHubClient, checkpointStore);
  } catch (error) {
    logger.error(error, 'The Event Hub Consumer Lag could not be sent to Application Insights');
  }
}, 60000);

export async function measureConsumerLag(
  consumerGroup: string,
  eventHubClient: EventHubConsumerClient,
  checkpointStore: BlobCheckpointStore
): Promise<void> {
  const partitionIds = await eventHubClient.getPartitionIds();

  // Should return either 0 or 1 checkpoint per partition
  const checkpoints = await checkpointStore.listCheckpoints(
    eventHubClient.fullyQualifiedNamespace,
    eventHubClient.eventHubName,
    consumerGroup
  );

  const checkpointSequenceNumberByPartitionId = Object.fromEntries(
    checkpoints.map(({ partitionId, sequenceNumber }): [string, number] => [partitionId, sequenceNumber])
  );

  await Promise.all(
    partitionIds.map(async partitionId => {
      const lastKnownSequenceNumber = checkpointSequenceNumberByPartitionId[partitionId] ?? 0;
      const { lastEnqueuedSequenceNumber } = await eventHubClient.getPartitionProperties(partitionId);
      const consumerLageMetric = {
        eventHubName: eventHubClient.eventHubName,
        consumerGroup,
        partitionId,
        namespace: eventHubClient.fullyQualifiedNamespace.split('.')[0],
        // The consumerLag calculation
        consumerLag: lastEnqueuedSequenceNumber - lastKnownSequenceNumber,
      };

      await trackEventHubConsumerLag(consumerLageMetric);
    })
  );
}

Sending the custom metric

import { defaultClient as appInsightsClient } from 'applicationinsights';

export interface ConsumerLagMetric {
  eventHubName: string;
  consumerGroup: string;
  partitionId: string;
  namespace: string;
  consumerLag: number;
}

export async function trackEventHubConsumerLag({
  eventHubName,
  namespace,
  consumerGroup,
  partitionId,
  consumerLag,
}: ConsumerLagMetric): Promise<void> {
  await trackMetric({
    name: 'Event Hub Consumer Lag',
    value: consumerLag,
    // Format property keys with a space, for readability in the Application Insights metrics dashboard
    properties: {
      'Event Hub': eventHubName,
      'Partition Id': partitionId,
      'Consumer Group': consumerGroup,
      Namespace: namespace,
    },
  });
}

Viewing the custom metric

In the Application Insights console, you’ll find your custom metric, split the chart by "Consumer Group", which represents an application. Depending on the zoom level the chart will show multiple measurements per datapoint. Usethe aggregation "Max" to get the best line.

This chart shows 3 microservices where 1 service is stuck processing an event. Whenever new events are published the consumer lag will increase. The events are published in bursts, so the consumer lag will increase in distinct increments.

When an issue is solved, the consumer lag will drop quickly.

Conclusion

Consumer lag will quickly show any functional or technical issue with your event stream. By using the code examples from this blogpost, you can avoid having to dive into the SDKs yourself. Of course, you can adjust the metric collection to send the metric to the logs or to another metrics system like prometheus, datadog, or open telemetry.

After collecting the metric, the next step is to create metric based alerts to ensure you detect the issues before your customer does!