<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: ImagineX</title>
    <description>The latest articles on DEV Community by ImagineX (imaginex).</description>
    <link>https://dev.to/imaginex</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Forganization%2Fprofile_image%2F9634%2F035402bf-0d6a-4de6-84e6-8ef75703c8f9.png</url>
      <title>DEV Community: ImagineX</title>
      <link>https://dev.to/imaginex</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/imaginex"/>
    <language>en</language>
    <item>
      <title>Optimizing Agentic Task Predictability</title>
      <dc:creator>Pablo Orozco</dc:creator>
      <pubDate>Wed, 13 Aug 2025 16:24:04 +0000</pubDate>
      <link>https://dev.to/imaginex/optimizing-agentic-task-predictability-3kg</link>
      <guid>https://dev.to/imaginex/optimizing-agentic-task-predictability-3kg</guid>
      <description>&lt;p&gt;One of the most important shifts in how we use AI is happening quietly, and it’s easy to miss. It’s not about making the model smarter. It’s about making its behavior predictable.&lt;/p&gt;

&lt;p&gt;Few days ago, OpenAI released the GPT-5 Prompting Guide in the Cookbook:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://cookbook.openai.com/examples/gpt-5/gpt-5_prompting_guide" rel="noopener noreferrer"&gt;GPT-5 Prompting Guide&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;On the surface, it looks like another collection of tips. It’s not. It’s a blueprint for transforming the model from a reactive answer machine into a reliable, &lt;strong&gt;collaborative agent&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The most valuable idea inside it is a pattern I think every builder should adopt: &lt;strong&gt;Optimizing Agentic Task Predictability&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The pattern is deceptively simple
&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;You are a methodical AI assistant with expertise in Next.js, TypeScript, TailwindCSS, and shadcn/ui. Here’s the development task: [insert task]. First, outline a detailed, step-by-step technical plan covering file structure, components, and integration details. Wait for my approval before writing any code. After approval, implement only the first step. Continue step-by-step after each confirmation.&lt;/p&gt;

&lt;p&gt;You are a methodical AI assistant with expertise in agile product management and Jira workflows. Here’s the product planning task: [insert task]. First, present a clear, step-by-step breakdown into user stories, acceptance criteria, and priorities. Wait for my approval before creating any Jira issues. After approval, create only the first item. Continue step-by-step after each confirmation.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ffemickkwp3fksvc8kxyq.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ffemickkwp3fksvc8kxyq.png" alt=" " width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Why this matters
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Control&lt;/strong&gt;: Multi-step tasks no longer run away from you. You decide what’s next.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Transparency&lt;/strong&gt;: The plan is explicit before execution begins.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Iterative collaboration&lt;/strong&gt;: You can adjust the approach before resources are spent.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  A real example
&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;You are a senior software architect with expertise in Vercel AI SDK v5 and Next.js App Router. Outline the detailed steps to build a streaming chat endpoint that validates incoming user messages before processing, using API Routes and Vercel AI SDK helpers. Include setup, route structure, and integration points in your plan. Wait for my approval before coding. After approval, implement only the validation logic for the first message type. Continue step-by-step after each confirmation.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Three things are happening here:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Role definition — The model takes on relevant expertise.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Precise task framing — You define exactly what needs to happen.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Phase separation — The plan and execution are split, giving you checkpoints.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Compared to older prompting styles, this is a mental shift. The old approach was: "&lt;strong&gt;Give me the answer&lt;/strong&gt;", the new approach is: "&lt;strong&gt;Let’s work this out together, step by step&lt;/strong&gt;".&lt;/p&gt;

&lt;p&gt;Agentic workflows are not just a feature of GPT-5, they’re the future of how we’ll work with AI. &lt;strong&gt;Predictability&lt;/strong&gt; is what makes them viable at scale. Without it, your "agent" is just a stochastic suggestion engine. With it, you have something closer to a dependable &lt;strong&gt;teammate&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;If you want to get more out of GPT-5, stop chasing clever prompts and start designing for predictable collaboration.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Empower Your Team with Databricks: Harness the Power of Data &amp; AI</title>
      <dc:creator>Jean Arias</dc:creator>
      <pubDate>Fri, 30 May 2025 16:09:16 +0000</pubDate>
      <link>https://dev.to/imaginex/empower-your-team-with-databricks-harness-the-power-of-data-ai-4p6h</link>
      <guid>https://dev.to/imaginex/empower-your-team-with-databricks-harness-the-power-of-data-ai-4p6h</guid>
      <description>&lt;p&gt;In today's landscape, generating value from data is increasingly essential for companies looking to stay ahead. However, this requires more than simply accumulating information. The key lies in effective collaboration from a highly skilled multidisciplinary team that leverages unified platforms like Databricks to turn raw data into intelligent products, strategic insights, and advanced AI solutions. This approach allows teams to focus on the data itself rather than the infrastructure, benefiting from a single, integrated, and collaboration oriented solution.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Ideal Team: A Story of Collaboration
&lt;/h3&gt;

&lt;p&gt;Imagine a real world scenario: a leading retail company wants to revolutionize its digital strategy through an advanced personalized recommendation system combined with a generative chatbot, aimed at optimizing customer experience and significantly increasing online sales and loyalty.&lt;/p&gt;

&lt;p&gt;To tackle this challenge, the company has relied on Databricks, a cloud platform leader in data, ML &amp;amp; AI solutions. And with it, four certified key roles come up:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Databricks Certified Data Analyst&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fi9obciq3plq9li1kmc78.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fi9obciq3plq9li1kmc78.png" alt="Databricks badge" width="571" height="792"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Databricks Certified Data Engineer&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftlndkqr9iouxaoirrgxk.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftlndkqr9iouxaoirrgxk.png" alt="Databricks badge" width="571" height="792"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Databricks Certified Machine Learning Engineer&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmi35pwg0yp2rks4n9llw.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmi35pwg0yp2rks4n9llw.png" alt="Databricks badge" width="570" height="792"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Databricks Certified Generative AI Engineer&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F9kbv1q8pbyr4en73jlrn.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F9kbv1q8pbyr4en73jlrn.png" alt="Databricks badge" width="571" height="792"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;p&gt;Let's walk through them and discover how each role will help with this initiative.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Certified Data Engineer: Building a Scalable Infrastructure
&lt;/h3&gt;

&lt;p&gt;Our journey begins with Carl, the Certified Data Engineer. Carl designs a robust architecture using Databricks Lakehouse, integrating heterogeneous real time sources, such as blob storages, event hubs, IoT hubs, db engines or ERPs. He also leverages Unity Catalog to provide centralized governance and secure data sharing between teams while maintaining data quality and compliance. With Apache Spark and Delta Live Tables, he builds reliable pipelines that deliver high quality data ready for real time analysis.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcohbcy3t306c5xhqlxnn.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcohbcy3t306c5xhqlxnn.png" alt="DE diagram" width="800" height="533"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  2. Certified Data Analyst: Identifying Strategic Opportunities
&lt;/h3&gt;

&lt;p&gt;Next is Laura, the Certified Data Analyst. Laura performs advanced exploratory analysis on large datasets from historical sales, digital interactions, social media, and customer surveys. She uses SQL, Python and interactive dashboards in Databricks notebooks to uncover patterns, correlations, and key segments. For example, she finds that specific customer groups are highly responsive to promotions and seasonal trends so she created a visualization to show this behavior to the business people, so they can make decisions in focusing efforts to those groups based on its characteristics.  &lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F525in2lvorcml7bswuy8.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F525in2lvorcml7bswuy8.png" alt="DA diagram" width="800" height="398"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  3. Certified Machine Learning Engineer: Building Smarter Models
&lt;/h3&gt;

&lt;p&gt;Ana, the Certified Machine Learning Engineer, designs and improves recommendation systems using Databricks. She combines different methods to suggest products or content based on what users like and their past behavior. Ana uses Databricks AutoML and Python to build the models, ML Runtime to run her models and MLflow to keep track of her experiments and versions. This helps her ensure the recommendations are always relevant and adapt to changes in user preferences.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fkldcy0ucr5nml0kctf5w.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fkldcy0ucr5nml0kctf5w.png" alt="ML diagram" width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  4. Certified Generative AI Engineer: Creating Natural, Personalized Interactions
&lt;/h3&gt;

&lt;p&gt;David, the Certified Generative AI Engineer, implements an advanced chatbot powered by GPT-4 using Databricks Model Serving and Dolly (LLM). This chatbot understands context and tailors responses in real time, integrating seamlessly with Ana's recommendations to deliver a smooth and engaging user experience boosting satisfaction and loyalty.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Frfdx773mmszs1xd902jp.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Frfdx773mmszs1xd902jp.png" alt="IA diagram" width="800" height="526"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;&lt;em&gt;Other use cases&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Let's explore some other examples of use cases in different industries:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Retail:&lt;/em&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Fraud Detection with real time analytics to identify unusual transaction patterns.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Demand Forecasting by integrating historical sales data, seasonal trends, and external factors like weather and economic conditions to predict demand with greater accuracy.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;em&gt;eCommerce:&lt;/em&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Customer Segmentation &amp;amp; Targeting with AI powered clustering techniques to segment customers based on demographics, purchase history, consumption habits and browsing behavior.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Real Time Pricing Optimization by analyzing competitor pricing, demand fluctuations, and customer buying patterns to dynamically adjust product prices.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;em&gt;Manufacturing:&lt;/em&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Early deffect detection by custom vision models.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Predictive Maintenance by detecting early signs of equipment failure.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Supply Chain Resilience by integrating data from IoT devices, historical disruptions, and external factors like weather and market demand that can helps manufacturers anticipate supply chain risks.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;All this work can be accomplished within a single platform, what ensures a unique source of truth, ensuring data consistency and availability to all team members. The collaboration is seamless, with analysts, engineers, and data scientists working together in a cohesive environment. This unified approach not only enhances productivity but also accelerates the delivery of insights and solutions, driving innovation and strategic decision making across the organization.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fxkybszvhd595y66t0eac.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fxkybszvhd595y66t0eac.png" alt="databricks ecosystem" width="800" height="645"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  Who Can Benefit from a Databricks Certification?
&lt;/h3&gt;

&lt;p&gt;Various roles can significantly enhance their impact by obtaining a relevant Databricks certification:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Backend Developers:&lt;/strong&gt; By certifying in Machine Learning, GenAI or Data Engineering, they can efficiently integrate predictive models or genAI solutions into existing applications via REST APIs or exported models. This enables them to quickly add advanced features like personalized recommendations and fraud detection, streamline deployment, and ensure scalable, consistent performance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Product Managers:&lt;/strong&gt; With a certification in Data Analysis, they can define, monitor, and adjust KPIs in real time using Databricks SQL and Genie (natural language query engine), enabling informed, strategic decisions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;QA and DevOps Engineers:&lt;/strong&gt; A certification empowers them to automate and optimize testing by creating automation pipelines using Databricks Jobs, enhancing efficiency and quality in data pipelines or machine learning models.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mobile Developers:&lt;/strong&gt; With a certification in Data Engineering, they can integrate real time analytics and machine learning models into mobile apps, enhancing user experience and app functionality.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Front End Developers:&lt;/strong&gt; By certifying in Data Analysis, they can create dynamic, datadriven user interfaces that provide realtime insights and visualizations to users.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Database Administrators (DBAs):&lt;/strong&gt; A certification in Data Engineering allows them to optimize data storage, management, and retrieval processes, ensuring efficient and secure data handling.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;DevOps / SREs:&lt;/strong&gt; Manage secure, continuous deployments using integrations with Azure DevOps, GitHub Actions, and MLflow to streamlining operational management.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Who Else Can Integrate Databricks into Their Workflow?
&lt;/h3&gt;

&lt;p&gt;Other profiles, even without certification, can benefit from Databricks by integrating it into their workflows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Product Designers:&lt;/strong&gt; Access precise insights on usage patterns, interactions, and user preferences to drive design improvements centered in the customer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Business Users:&lt;/strong&gt; Through interactive dashboards or using natural language queries with &lt;strong&gt;Genie&lt;/strong&gt;, they can generate deep, actionable insights. &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fimages.squarespace-cdn.com%2Fcontent%2Fv1%2F65a7c5c379a13972f7c20d75%2Fc20732cd-0ecc-4079-8a67-936ebd3af712%2FInstructions.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fimages.squarespace-cdn.com%2Fcontent%2Fv1%2F65a7c5c379a13972f7c20d75%2Fc20732cd-0ecc-4079-8a67-936ebd3af712%2FInstructions.gif" alt="Instrucciones" width="760" height="452"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  Conclusion: Databricks as a Strategic Catalyst
&lt;/h3&gt;

&lt;p&gt;Databricks goes beyond being a technical tool. It becomes a strategic enabler for cross-functional collaboration and innovation driven by data. From product design to business decision-making, passing through engineering and data science, Databricks helps organizations stay at the forefront of the digital era.&lt;/p&gt;

&lt;p&gt;Databricks truly transforms how teams and businesses operate by making collaborative, data-driven strategies a core part of their success.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ff0p3vowmy77nyu0vuj0o.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ff0p3vowmy77nyu0vuj0o.png" alt="databricks logo" width="800" height="418"&gt;&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>data</category>
      <category>datascience</category>
      <category>programming</category>
    </item>
    <item>
      <title>Model Context Protocol (MCP): An Open Standard for Connecting LLMs to Business Context</title>
      <dc:creator>Diego Pérez</dc:creator>
      <pubDate>Wed, 28 May 2025 01:08:54 +0000</pubDate>
      <link>https://dev.to/imaginex/model-context-protocol-mcp-an-open-standard-for-connecting-llms-to-business-context-419m</link>
      <guid>https://dev.to/imaginex/model-context-protocol-mcp-an-open-standard-for-connecting-llms-to-business-context-419m</guid>
      <description>&lt;h3&gt;
  
  
  What is this about?
&lt;/h3&gt;

&lt;p&gt;MCP (Model Configuration Protocol) has been gaining attention lately, and it is something more people should be aware of. It has been on the radar for the past few months and is becoming a key part of how systems are built and integrated with AI, both now and in the future.&lt;/p&gt;

&lt;p&gt;This post walks through the core ideas behind MCP and shows an implementation built using Spring IA and Java.&lt;/p&gt;

&lt;h2&gt;
  
  
  MCP and the Rise of a New AI Standard
&lt;/h2&gt;

&lt;p&gt;Nowadays, there’s a wide variety of very powerful LLMs out there. However, a key component has long been missing: a standardized way to provide these models with access to data and functionality from our own systems. This capability is essential for fully leveraging the potential of these models and achieving truly meaningful things within specific business domains.&lt;/p&gt;

&lt;p&gt;Because of this need, &lt;a href="https://www.anthropic.com/" rel="noopener noreferrer"&gt;Anthropic&lt;/a&gt;, a very important emerging company in the field of AI development, has proposed a new open standard called the &lt;a href="https://modelcontextprotocol.io/introduction" rel="noopener noreferrer"&gt;Model Context Protocol (MCP)&lt;/a&gt;. You can think of it like other ways we share data and functionality between systems, such as REST APIs for exchanging resources and business logic, but in this case, it’s for sharing context with an LLM.&lt;/p&gt;

&lt;p&gt;MCP makes it possible for LLMs to actually do useful things with real systems. Instead of giving surface-level responses, they can work with live business data, read files, follow instructions, and take real action. What used to be a basic chat becomes something much more interactive, like asking for a product and having it ordered for you, or triggering a business process with just a sentence and even get meaningful insights from your own data. The more systems you connect, the more creative and practical the use cases become.&lt;/p&gt;

&lt;h2&gt;
  
  
  MCP General Architecture
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fie0ys0donv19qc0r7y7d.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fie0ys0donv19qc0r7y7d.png" alt="MCP Architecture" width="582" height="426"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;MCP Server&lt;/strong&gt;: A backend program that exposes a variety of capabilities and data through the MCP protocol. It can connect to databases or other services to bring in useful features or information the model might need.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;MCP Client&lt;/strong&gt;: In a nutshell, it handles 1:1 communication with MCP Servers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;MCP Host&lt;/strong&gt;: This is essentially an app used to interact with an LLM. Through that interaction, you can be able to access and use the different tools available behind MCP Servers. Some examples of MCP Hosts are &lt;a href="https://claude.ai/download" rel="noopener noreferrer"&gt;Claude Desktop,&lt;/a&gt; &lt;a href="https://www.cursor.com/" rel="noopener noreferrer"&gt;Cursor&lt;/a&gt;, and &lt;a href="https://openai.github.io/openai-agents-python/" rel="noopener noreferrer"&gt;OpenAI Agents SDK&lt;/a&gt;, and the list keeps growing.&lt;/p&gt;

&lt;h2&gt;
  
  
  MCP Transports
&lt;/h2&gt;

&lt;p&gt;MCP is the protocol used for communication between the client and the server. At a high level, it relies on &lt;a href="https://www.jsonrpc.org/specification" rel="noopener noreferrer"&gt;JSON-RPC 2.0&lt;/a&gt; as the data exchange format. Basically, it’s responsible for translating MCP messages into JSON-RPC for transmission, and then converting the responses back into MCP messages on the way in. It’s the layer that makes everything flow between both sides.&lt;/p&gt;

&lt;p&gt;At a high level, you can imagine this communication as illustrated below:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fyroug3iu475xuc5kcp7u.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fyroug3iu475xuc5kcp7u.png" alt="MCP Client-Server communication" width="800" height="216"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In practice there are two implementations of MCP standard transport protocol:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Standard Input/Output (STDIO)&lt;/strong&gt;: In this case communication happens through the standard input and output streams. It’s perfect for local setups, fast, efficient, and easy to wire together with low latency and minimal overhead.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Server-Sent Events (SSE)&lt;/strong&gt;: On the other hand, with SSE the server pushes responses using HTTP streaming, and the client sends requests via HTTP POST. SSE works great for remote servers, supports multiple users, and fits naturally into HTTP-based architectures.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One big difference worth pointing out is that when using STDIO, the client is the one that starts and stops the MCP server. That doesn’t happen with SSE, where the server just runs on its own and listens for requests.&lt;/p&gt;

&lt;h2&gt;
  
  
  What can we expose through an MCP server?
&lt;/h2&gt;

&lt;p&gt;With MCP servers we have three primitives that we can expose:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Resources&lt;/strong&gt;: Basically read-only data such as file images, logs, database info, or API responses. In summary, any kind of data you want to make available to the MCP client.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Prompts&lt;/strong&gt;: These are reusable message templates that can be used to guide interactions. This is useful when you want to structure the way users interact and shape the workflows for those interactions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tools&lt;/strong&gt;: This is where we really want to focus our attention, because it’s the most important and powerful primitive. Tools let us expose executable actions that can interact with our systems, perform tasks, and actually change the state of things. Basically, we can expose any functionality we want to our MCP client.&lt;/p&gt;

&lt;p&gt;For further details please check: &lt;a href="https://modelcontextprotocol.io/docs/concepts/transports" rel="noopener noreferrer"&gt;https://modelcontextprotocol.io/docs/concepts/transports&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Building a MCP server with Spring IA
&lt;/h2&gt;

&lt;p&gt;To put all this theory into practice the idea is to create an MCP server for a real use case.&lt;/p&gt;

&lt;p&gt;There’s an official SDK for popular languages like Python, Java, and TypeScript, and plenty of community implementations for others. In this case, the &lt;a href="https://docs.spring.io/spring-ai/reference/api/mcp/mcp-overview.html" rel="noopener noreferrer"&gt;Spring AI MCP implementation&lt;/a&gt; is the one being used. It works especially well for those already familiar with the Spring ecosystem.&lt;/p&gt;

&lt;h3&gt;
  
  
  Building a Tool to Interact with AWS S3
&lt;/h3&gt;

&lt;p&gt;As an idea for an MCP server implementation, an interesting use case is to build a tool that allows users to interact with the AWS S3 service using natural language through an LLM. The goal is to enable the model to perform basic actions on this service, such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;List available S3 buckets&lt;/li&gt;
&lt;li&gt;List objects&lt;/li&gt;
&lt;li&gt;Get object metadata&lt;/li&gt;
&lt;li&gt;Upload an object&lt;/li&gt;
&lt;li&gt;Download an object&lt;/li&gt;
&lt;li&gt;Delete an object&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The purpose is to expose these tools so an LLM can interact with the AWS service and perform actions on behalf of the user. To make this work, it is only necessary to provide some basic configuration, such as the access key, secret key, and the region of the AWS account to be used.&lt;/p&gt;

&lt;p&gt;The idea is to connect this MCP server with tools like Cursor, enabling interaction with S3 buckets during development, all in one place. This avoids the need to switch to the AWS Console or use the AWS CLI for common tasks. Instead, everything can be done through prompts, in a much simpler and more integrated way.&lt;/p&gt;

&lt;p&gt;For this use case, the STDIO transport is a great fit, and it is the one selected here. Since the only requirement is a process running locally that brings this functionality to the models, nothing more than that is needed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Creating the Project&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The first step is to generate the project using &lt;a href="https://start.spring.io/" rel="noopener noreferrer"&gt;Spring Initializr&lt;/a&gt;. For this example, the configuration is as follows:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8re0himvm4ygrsvalmvd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8re0himvm4ygrsvalmvd.png" alt="Project Setup" width="800" height="407"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Configuring Project Dependencies&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;After generating the project, the necessary MCP dependencies will be included, assuming the MCP Server dependency was selected in Spring Initializr:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;lt;properties&amp;gt;
     ...
     &amp;lt;spring-ai.version&amp;gt;1.0.0&amp;lt;/spring-ai.version&amp;gt;
&amp;lt;/properties&amp;gt;`
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;    &amp;lt;dependencyManagement&amp;gt;

        &amp;lt;dependencies&amp;gt;
            &amp;lt;dependency&amp;gt;
                &amp;lt;groupId&amp;gt;org.springframework.ai&amp;lt;/groupId&amp;gt;
                &amp;lt;artifactId&amp;gt;spring-ai-bom&amp;lt;/artifactId&amp;gt;
                &amp;lt;version&amp;gt;${spring-ai.version}&amp;lt;/version&amp;gt;
                &amp;lt;type&amp;gt;pom&amp;lt;/type&amp;gt;
                &amp;lt;scope&amp;gt;import&amp;lt;/scope&amp;gt;
            &amp;lt;/dependency&amp;gt;
            ...
        &amp;lt;/dependencies&amp;gt;
    &amp;lt;/dependencyManagement&amp;gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;lt;dependencies&amp;gt;
  &amp;lt;dependency&amp;gt;
      &amp;lt;groupId&amp;gt;org.springframework.ai&amp;lt;/groupId&amp;gt;
      &amp;lt;artifactId&amp;gt;spring-ai-starter-mcp-server&amp;lt;/artifactId&amp;gt;
  &amp;lt;/dependency&amp;gt;
...
&amp;lt;/dependencies&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To access the AWS S3 service, the official Java SDK will be used. To enable this, the following dependency must be added by including the following in the pom.xml:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;lt;properties&amp;gt;
    ...
    &amp;lt;aws.sdk.version&amp;gt;2.31.39&amp;lt;/aws.sdk.version&amp;gt;
&amp;lt;/properties&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;    &amp;lt;dependencyManagement&amp;gt;

        &amp;lt;dependencies&amp;gt;
            &amp;lt;dependency&amp;gt;
                &amp;lt;groupId&amp;gt;org.springframework.ai&amp;lt;/groupId&amp;gt;
                &amp;lt;artifactId&amp;gt;spring-ai-bom&amp;lt;/artifactId&amp;gt;
                &amp;lt;version&amp;gt;${spring-ai.version}&amp;lt;/version&amp;gt;
                &amp;lt;type&amp;gt;pom&amp;lt;/type&amp;gt;
                &amp;lt;scope&amp;gt;import&amp;lt;/scope&amp;gt;
            &amp;lt;/dependency&amp;gt;
            ...
        &amp;lt;/dependencies&amp;gt;
    &amp;lt;/dependencyManagement&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;lt;dependencies&amp;gt;
  ...
  &amp;lt;dependency&amp;gt;
      &amp;lt;groupId&amp;gt;software.amazon.awssdk&amp;lt;/groupId&amp;gt;
      &amp;lt;artifactId&amp;gt;s3&amp;lt;/artifactId&amp;gt;
  &amp;lt;/dependency&amp;gt;
&amp;lt;/dependencies&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  MCP Server Configuration(STDIO)
&lt;/h3&gt;

&lt;p&gt;To configure the application as an MCP server using STDIO as the transport, the following settings should be added to the application.properties file:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;spring.application.name=s3-toolbox-mcp-server

spring.ai.mcp.server.enabled=true
spring.ai.mcp.server.stdio=true
spring.ai.mcp.server.name=s3-toolbox-mcp-server
spring.ai.mcp.server.version=1.0.0
spring.ai.mcp.server.capabilities.tool=true
spring.ai.mcp.server.capabilities.resource=false
spring.ai.mcp.server.capabilities.prompt=false

# DISABLE Spring banner and console logging to keep STDIO clean for MCP JSON-RPC only.
spring.http.client.factory=simple
spring.main.banner-mode=off
spring.main.web-application-type=NONE

logging.pattern.console=
logging.file.name=.logs/s3-toolbox-mcp-server.log
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The config under spring.ai.mcp.server is the basic setup to enable our MCP server. For more details about these params, you can refer the &lt;a href="https://docs.spring.io/spring-ai/reference/api/mcp/mcp-server-boot-starter-docs.html#_configuration_properties" rel="noopener noreferrer"&gt;official reference documentation&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Also, keep in mind there’s some extra configuration related to logging. We are basically turning off all console logging because the output should stay clean for MCP JSON-RPC interactions only. Any extra text printed to the output can break the connection between the server and the client. That’s why we’re sending all logging output to a file instead.&lt;/p&gt;

&lt;h2&gt;
  
  
  MCP Server Implementation
&lt;/h2&gt;

&lt;p&gt;The project is organized like this:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Frgnhjc69k9u80441w89x.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Frgnhjc69k9u80441w89x.png" alt="Project structure" width="316" height="135"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is a pretty common and self-explanatory way to structure Java projects with Spring. The only package that’s kind of new here is the tool package. That’s where we’re going to define our MCP primitives. In this case, the tools.&lt;/p&gt;

&lt;p&gt;Below are the main code snippets from the implementation. Some boilerplate code, such as imports and exception handling, has been omitted to keep things simple and easier to follow.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Model&lt;/strong&gt;: The following models help simplify data transport for some of the operations that will be exposed as MCP tools.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;...
import org.springframework.ai.tool.annotation.ToolParam;

public record PutS3ObjectRequest(
        @ToolParam(description = "Name of the target S3 bucket (e.g., 'my-bucket')")
        String bucketName,

        @ToolParam(description = "Optional path prefix/folder inside the bucket (e.g., 'invoices/2025/'). Leave empty or omit to upload to the bucket root.")
        String prefix,

        @ToolParam(description = "Filename to save in S3 (e.g., 'report.pdf')")
        String fileName,

        @ToolParam(description = "MIME type of the file (e.g., 'application/pdf', 'image/png')")
        String contentType,

        @ToolParam(description = "The file's binary content, provided as a Base64-encoded string and deserialized into a byte array")
        String base64Content
) {

}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It is worth noting that in this case the @ToolParam annotation is used. This annotation is part of the MCP API and is useful for providing additional detail or context about an input parameter for a given tool.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;public record PutS3ObjectResponse(String eTag) {
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;public record GetS3ObjectResponse(String fileName, byte[] content, String contentType) {
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;public record GetS3ObjectMetadataResponse(String contentType, Long contentLength, Instant lastModified, String eTag,
                                          String storageClass, Map&amp;lt;String, String&amp;gt; customMetadata) {
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Config&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;The S3Client is configured as shown below. This client is used to perform the different actions against the AWS S3 service:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;...
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.services.s3.S3Client;


@Configuration
public class S3Config {

    @Bean
    public S3Client s3Client() {
        return S3Client.builder()
                .credentialsProvider(DefaultCredentialsProvider.create())
                .build();
    }

}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Please note that for this client to work, we need to configure the following environment variables:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;AWS_ACCESS_KEY_ID&lt;/li&gt;
&lt;li&gt;AWS_SECRET_ACCESS_KEY&lt;/li&gt;
&lt;li&gt;AWS_REGION&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Service&lt;/strong&gt;:&lt;br&gt;
This layer contains the logic that interacts with the AWS S3 client to execute the defined operations.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;public interface S3ActionsService {

    List&amp;lt;String&amp;gt; listBuckets();

    List&amp;lt;String&amp;gt; listObjects(String bucketName, String prefix);

    GetS3ObjectMetadataResponse getObjectMetadata(String bucketName, String keyName);

    PutS3ObjectResponse putObject(PutS3ObjectRequest putS3ObjectRequest);

    GetS3ObjectResponse getObject(String bucketName, String key);

    void deleteObject(String bucketName, String keyName);

}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Contract implementation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;@Service
@RequiredArgsConstructor
@Slf4j
public class S3ActionsServiceImpl implements S3ActionsService {

    private final S3Client s3Client;

    @Override
    public List&amp;lt;String&amp;gt; listBuckets() {
        final var response = s3Client.listBuckets();

        return Optional.ofNullable(response.buckets())
                .orElse(Collections.emptyList())
                .stream()
                .map(Bucket::name)
                .collect(Collectors.toList());
    }

    @Override
    public List&amp;lt;String&amp;gt; listObjects(final String bucketName, final String prefix) {
        final var request = ListObjectsV2Request.builder()
                .bucket(bucketName)
                .prefix(prefix)
                .build();

        final var response = s3Client.listObjectsV2(request);

        return Optional.ofNullable(response.contents())
                .orElse(Collections.emptyList())
                .stream()
                .map(S3Object::key)
                .collect(Collectors.toList());
    }

    @Override
    public Map&amp;lt;String, Object&amp;gt; getObjectMetadata(final String bucketName, final String key) {
        final var headFuture = CompletableFuture.supplyAsync(() -&amp;gt; {
            var headRequest = HeadObjectRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .build();
            return s3Client.headObject(headRequest);
        });

        final var attrFuture = CompletableFuture.supplyAsync(() -&amp;gt; {
            var attrRequest = GetObjectAttributesRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .objectAttributes(ObjectAttributes.STORAGE_CLASS)
                    .build();
            return s3Client.getObjectAttributes(attrRequest);
        });

        return headFuture.thenCombine(attrFuture, (headResponse, attrResponse) -&amp;gt;
                    new GetS3ObjectMetadataResponse(headResponse.contentType(), headResponse.contentLength(),
                            headResponse.lastModified(), headResponse.eTag(), attrResponse.storageClassAsString(),
                            headResponse.metadata())
            ).join();
    }

    @Override
    public PutS3ObjectResponse putObject(final PutS3ObjectRequest putS3ObjectRequest) {
        final var request = PutObjectRequest.builder()
                .bucket(putS3ObjectRequest.bucketName())
                .key(putS3ObjectRequest.prefix().concat(putS3ObjectRequest.fileName()))
                .contentType(putS3ObjectRequest.contentType())
                .build();

        final var response = s3Client.putObject(request, 
                    RequestBody.fromBytes(Base64.getDecoder().decode(putS3ObjectRequest.base64Content())));

        return new PutS3ObjectResponse(response.eTag());
    }

    @Override
    public GetS3ObjectResponse getObject(final String bucketName, final String key) {
        final var request = GetObjectRequest.builder()
                .bucket(bucketName)
                .key(key)
                .build();

        final var responseBytes = s3Client.getObjectAsBytes(request);
        final var metadata = responseBytes.response();

        final var fileName = key.contains("/") ? key.substring(key.lastIndexOf("/") + 1) : key;

        return new GetS3ObjectResponse(fileName, responseBytes.asByteArray(), metadata.contentType());
    }

    @Override
    public void deleteObject(final String bucketName, final String key) {
        final var request = DeleteObjectRequest.builder()
                .bucket(bucketName)
                .key(key)
                .build();

        s3Client.deleteObject(request);
    }
}

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Tool&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;Since the goal in this case is to expose actionable operations that LLMs can use to perform the desired tasks, the @Tool primitive from the MCP API is used, as it is a better fit for this scenario.&lt;/p&gt;

&lt;p&gt;You can think of these tool definitions like how we define @Controller classes for our REST APIs. In the end, it's the way we define the tools contract for our MCP server. In other words, the actions or capabilities that our service is going to offer. Please note that @Tool comes from the MCP API.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;...
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Component;

@RequiredArgsConstructor
public class S3Tool {

    private final S3ActionsService s3ActionsService;

    @Tool(name = "list_s3_buckets", description = "Lists all the s3 buckets for the given account.")
    public List&amp;lt;String&amp;gt; listS3Buckets() {
        return s3ActionsService.listBuckets();
    }

    @Tool(name = "list_s3_files", description = "Returns a list of object keys in the specified S3 bucket, optionally filtered by a prefix.")
    public List&amp;lt;String&amp;gt; listS3Files(@ToolParam(description = "Name of the S3 bucket to search") String bucketName, @ToolParam(description = "Optional prefix to filter results (e.g. 'invoices/'). Use empty string or omit to list all objects.") String prefix) {
        return s3ActionsService.listObjects(bucketName, prefix);
    }

    @Tool(name = "get_s3_object_metadata", description = "Retrieves metadata (e.g., size, content-type, last modified) for the specified S3 object.")
    public GetS3ObjectMetadataResponse getS3ObjectMetadata(@ToolParam(description = "Name of the S3 bucket containing the object") String bucketName, @ToolParam(description = "Key (path and filename) of the S3 object to retrieve metadata for") String key) {
        return s3ActionsService.getObjectMetadata(bucketName, key);
    }

    @Tool(name = "put_s3_object", description = "Upload file bytes to S3 with optional prefix, returning ETag")
    public PutS3ObjectResponse putS3Object(PutS3ObjectRequest putS3ObjectRequest) {
        return s3ActionsService.putObject(putS3ObjectRequest);
    }

    @Tool(name = "get_s3_object", description = "Download an object from S3 by bucket name and key, returning its content and metadata such as content type, size, and ETag.")
    public GetS3ObjectResponse getS3Object(@ToolParam(description = "The name of the S3 bucket where the object is stored.") String bucketName,
                                           @ToolParam(description = "The full key (path/filename) of the object to retrieve.") String key) {
        return s3ActionsService.getObject(bucketName, key);
    }

    @Tool(name = "delete_s3_object", description = "Delete an object from S3 by bucket name and key. Returns DeleteMarker and VersionId if applicable.")
    public void deleteS3Object(@ToolParam(description = "Name of the S3 bucket that contains the object to delete.") String bucketName, @ToolParam(description = "Key (path/filename) of the object to delete from the bucket.") String key) {
        s3ActionsService.deleteObject(bucketName, key);
    }

}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here it is important to note the use of the following annotations:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;@Tool&lt;/strong&gt;: This one is used to define a specific tool we want to expose.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;@ToolParam&lt;/strong&gt;: With this annotation, we can give more details about the input parameters of a tool.&lt;/p&gt;

&lt;p&gt;It is key to write clear descriptions for both the operations and their parameters. The clearer they are, the fewer misunderstandings the models will have when trying to use them.&lt;/p&gt;

&lt;p&gt;In you want to explore this annotations in more detail please refer to: &lt;a href="https://docs.spring.io/spring-ai/reference/api/tools.html/" rel="noopener noreferrer"&gt;https://docs.spring.io/spring-ai/reference/api/tools.html/&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Finally, once the tool is created, the following configuration should be added. This ensures that Spring AI wraps the S3Tool as a ToolCallback. By defining it as a bean, the MCP server automatically registers it and makes the tool available for invocation.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;package com.github.dvindas.mcpserver.s3toolbox.config;

import com.github.dvindas.mcpserver.s3toolbox.tool.S3Tool;
import org.springframework.ai.support.ToolCallbacks;
import org.springframework.ai.tool.ToolCallback;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.List;

@Configuration
public class MCPConfiguration {

    @Bean
    public List&amp;lt;ToolCallback&amp;gt; registerTools(S3Tool S3Tool) {
        return List.of(ToolCallbacks.from(S3Tool));
    }

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Wiring Up Cursor with Our MCP Server
&lt;/h2&gt;

&lt;p&gt;In this case, given the nature of this MCP server, the connection will be made to Cursor. This setup enables seamless interaction during development, although it could also be connected to Claude Desktop or any other host that supports the standard.&lt;/p&gt;

&lt;p&gt;1- To do this, go to File &amp;gt; Preferences &amp;gt; Cursor Settings. There, we’ll see something like this:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Feb9koyf1hhdjsgb5yscj.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Feb9koyf1hhdjsgb5yscj.png" alt="Cursor Settings" width="800" height="313"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;2- Then, go to the MCP tab and click the Add new global MCP server button.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fl1cajonxfh8xjudmpfhi.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fl1cajonxfh8xjudmpfhi.png" alt="MCP tab" width="635" height="148"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;3- Once you click that button, Cursor will open the mcp.json config file, where we’re going to configure our MCP server as follows:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;{
  "mcpServers": {
    "s3-toolbox-mcp-server": {
      "command": "java",
      "args": [
      "-jar",
      "PATH_TO_MCP_SERVER_JAR\s3-toolbox-mcp-server-1.0.0.jar"
      ],
      "env": {
        "AWS_ACCESS_KEY_ID": "access_key_id",
        "AWS_SECRET_ACCESS_KEY": "secret_access_key",
        "AWS_REGION": "us-east-1"
      }
    }
  }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After adding this config, the MCP server should appear as active in Cursor:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5yayb2levpmornlmnisn.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5yayb2levpmornlmnisn.png" alt="MCP Server list" width="800" height="137"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;With that in place, the tools are ready to use directly from Cursor. No additional setup is needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  MCP Tool Discovery Flow
&lt;/h2&gt;

&lt;p&gt;Now, a high-level overview of what happens once the MCP server is configured:&lt;/p&gt;

&lt;p&gt;1- Cursor reads the config from mcp.json and starts the MCP server process locally based on the command provided. If there are environment variables defined, those will be added to the process environment when it runs.&lt;/p&gt;

&lt;p&gt;2- After that, the MCP client sends a JSON-RPC initialize request. In this case, this request is used to check the protocol version and capabilities like Resources, Prompts, and Tools. Once that's done, the session with the server is considered started.&lt;/p&gt;

&lt;p&gt;3- If the Tools capability is enabled, the client then makes a JSON-RPC 2.0 call to tools/list to discover which tools the MCP server is exposing.&lt;/p&gt;

&lt;p&gt;4- After this flow is completed, the client knows all the available tools and how to call them. The session stays open, ready to handle calls and interactions through JSON-RPC.&lt;/p&gt;

&lt;p&gt;If you want more details about this flow, refer to the official reference documentation: &lt;a href="https://modelcontextprotocol.io/docs/concepts/architecture#connection-lifecycle" rel="noopener noreferrer"&gt;https://modelcontextprotocol.io/docs/concepts/architecture#connection-lifecycle&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Showing the S3 Toolbox MCP Server in Action
&lt;/h2&gt;

&lt;p&gt;The images below show how these tools operate directly within the Cursor chat interface:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Listing S3 Bucket&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Faka0nx3i62mme3y2rnyh.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Faka0nx3i62mme3y2rnyh.png" alt="List bucket" width="553" height="1007"&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Putting an S3 Object&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fa9rufyrk6gexcj92gjba.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fa9rufyrk6gexcj92gjba.png" alt="Put object" width="550" height="1000"&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Listing S3 Objects&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh03tq3cnvmno3af2m2cu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fh03tq3cnvmno3af2m2cu.png" alt="List objects" width="544" height="1031"&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Getting object metadata&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fsrmecfx5xp0dof7zzixq.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fsrmecfx5xp0dof7zzixq.png" alt="Get metadata" width="541" height="1028"&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Getting an S3 Object&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fjxf4n8ko6sn2os0c04fl.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fjxf4n8ko6sn2os0c04fl.png" alt="Get object" width="543" height="1000"&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Deleting an S3 Object&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnu7zcvra5qfirok2ggqr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnu7zcvra5qfirok2ggqr.png" alt="Delete object" width="543" height="1000"&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;All in all, standards in technology are always a good thing. Without them, everyone starts doing things their own way. For example, when it comes to adding tools to an LLM, there are many different approaches. Some models let you register API endpoints directly and configure functionality that way. But with this open standard from Anthropic, they’re basically saying: here’s a clean and consistent way to do it, built on good practices.&lt;/p&gt;

&lt;p&gt;The adoption of this standard is growing, and several native integrations have already been announced, which is great to see. More and more services are starting to create and publish their own MCP servers as a new way to expose and interact with their features. One example of this is GitHub. Another cool thing is that there’s now an active and growing community behind this standard. There’s even a marketplace for MCP servers: &lt;a href="https://mcpmarket.com/" rel="noopener noreferrer"&gt;https://mcpmarket.com/&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Show Me the Code
&lt;/h2&gt;

&lt;p&gt;The full code for this article is available here:  &lt;a href="https://github.com/dvindas/s3-toolbox-mcp-server" rel="noopener noreferrer"&gt;https://github.com/dvindas/s3-toolbox-mcp-server&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>mcp</category>
      <category>ai</category>
      <category>java</category>
      <category>spring</category>
    </item>
    <item>
      <title>🧠 Lead Agents with Prompts</title>
      <dc:creator>Pablo Orozco</dc:creator>
      <pubDate>Mon, 28 Apr 2025 12:08:31 +0000</pubDate>
      <link>https://dev.to/imaginex/lead-agents-with-prompts-2mpc</link>
      <guid>https://dev.to/imaginex/lead-agents-with-prompts-2mpc</guid>
      <description>&lt;p&gt;Reviewing &lt;strong&gt;Andrej Karpathy&lt;/strong&gt;’s definition of "&lt;strong&gt;vibe coding&lt;/strong&gt;", I think he captured a new "&lt;strong&gt;hands-free&lt;/strong&gt;" way of building with LLMs.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4oc7rmaexdbop83t5x6g.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4oc7rmaexdbop83t5x6g.png" alt=" " width="800" height="1304"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AI Coding Tools&lt;/strong&gt; like Cursor and &lt;strong&gt;GitHub Copilot’s Agent Mode&lt;/strong&gt; are bringing that vision to life—turning the IDE into a collaborative, autonomous workflow engine powered by protocols like &lt;strong&gt;MCP&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In this post, we’ll explore how features like &lt;strong&gt;MCP&lt;/strong&gt;, &lt;strong&gt;NES&lt;/strong&gt;, and customizable agents are driving a new wave of intelligent, &lt;strong&gt;agentic development&lt;/strong&gt; across modern &lt;strong&gt;AI Coding Tools&lt;/strong&gt; such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://code.visualstudio.com/docs/copilot/chat/chat-agent-mode" rel="noopener noreferrer"&gt;Visual Studio Code&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://docs.cursor.com/chat/agent" rel="noopener noreferrer"&gt;Cursor - The AI Code Editor&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview" rel="noopener noreferrer"&gt;Claude Code&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://help.openai.com/en/articles/11096431-openai-codex-cli-getting-started" rel="noopener noreferrer"&gt;OpenAI Codex CLI&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We’ve been playing with AI completing snippets or suggesting one-liners. Agent Mode goes further—it’s closer to working with a smart developer who can write, run, and even fix things. Whether inside the IDE or through CLI tools like Claude Code or OpenAI Codex, it’s the closest we’ve gotten to truly collaborative AI in the development workflow.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F9nwardvdags5i9mr40y8.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F9nwardvdags5i9mr40y8.png" alt=" " width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  So what does Agent Mode &lt;em&gt;really&lt;/em&gt; do?
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Writes code&lt;/strong&gt; — Think full files, not just lines. You feed it intent, it returns structure.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Iterates&lt;/strong&gt; — It reviews its own output and revises.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Executes commands&lt;/strong&gt; — From installing dependencies to scaffolding projects.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Self-corrects&lt;/strong&gt; — When it fails (e.g., bad lint or broken build), it often recovers &lt;em&gt;on its own&lt;/em&gt;.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It can plan, execute, and react. That’s not a glorified copilot—that’s &lt;strong&gt;autonomous task flow&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  🧠 The Real Engine: Model Context Protocol (MCP)
&lt;/h2&gt;

&lt;p&gt;Let’s talk architecture. Agent Mode’s capabilities are powered by &lt;strong&gt;MCP&lt;/strong&gt;, and if you’re a builder, this is the rabbit hole you &lt;em&gt;want&lt;/em&gt; to go down.&lt;/p&gt;

&lt;h3&gt;
  
  
  What Makes MCP Essential for You
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/docs/copilot/chat/mcp-servers" rel="noopener noreferrer"&gt;MCP Servers&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Value&lt;/strong&gt;: You can build custom, domain-specific agents that interact with your company’s internal tools or data. The community already supports 1,000+ MCP servers: &lt;a href="https://modelcontextprotocol.io" rel="noopener noreferrer"&gt;modelcontextprotocol.io&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  🔗 What is MCP?
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://www.microsoft.com/en-us/microsoft-copilot/blog/copilot-studio/introducing-model-context-protocol-mcp-in-copilot-studio-simplified-integration-with-ai-apps-and-agents/" rel="noopener noreferrer"&gt;(MCP) in Copilot Studio&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;a href="https://code.visualstudio.com/docs/copilot/chat/mcp-servers#_add-an-mcp-server" rel="noopener noreferrer"&gt;Add and MCP Server&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;MCP&lt;/strong&gt; is an &lt;strong&gt;open protocol&lt;/strong&gt; for feeding LLMs with structured, real-world dev context.&lt;/p&gt;

&lt;p&gt;Think of it like what &lt;strong&gt;LSP (Language Server Protocol)&lt;/strong&gt; did for syntax and intellisense—MCP does for AI agents.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;It standardizes how tools like VS Code provide context to AI models.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;It connects your AI not just to code, but to:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Figma designs.&lt;/li&gt;
&lt;li&gt;Internal wikis.&lt;/li&gt;
&lt;li&gt;GitHub repos.&lt;/li&gt;
&lt;li&gt;Databases.&lt;/li&gt;
&lt;li&gt;APIs.&lt;/li&gt;
&lt;/ul&gt;


&lt;/li&gt;

&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fzwrbztvgg6kirfrhafow.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fzwrbztvgg6kirfrhafow.png" alt=" " width="800" height="960"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  🌱 Why Should You Care?
&lt;/h3&gt;

&lt;p&gt;Because with MCP, you can build custom agents that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Understand your product domain.&lt;/li&gt;
&lt;li&gt;Use internal tools.&lt;/li&gt;
&lt;li&gt;Access backend APIs or datasets.&lt;/li&gt;
&lt;li&gt;Help teammates onboard faster.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And with it being &lt;strong&gt;just HTTP or std I/O&lt;/strong&gt;, developers are shipping servers for everything: &lt;a href="https://modelcontextprotocol.io" rel="noopener noreferrer"&gt;modelcontextprotocol.io&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  🛠️ How to Plug MCP Into Your Stack
&lt;/h3&gt;

&lt;h4&gt;
  
  
  Getting Started
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Add an MCP server (e.g. GitHub) to your workspace using &lt;code&gt;.vscode/mcp.json&lt;/code&gt; or via VS Code's Command Palette.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Install methods range from Docker to Python CLI wrappers.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Security Improvements
&lt;/h3&gt;

&lt;p&gt;MCP started with JSON env var keys (yes, the usual papercut). But we’re moving toward:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Secure input variables inside VS Code.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;OAuth-enabled remote MCP servers.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Less friction, more security.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h4&gt;
  
  
  Example: GitHub MCP Server
&lt;/h4&gt;

&lt;p&gt;It allows Agent Mode to:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Create repos.&lt;/li&gt;
&lt;li&gt;Open pull requests.&lt;/li&gt;
&lt;li&gt;Search issues.&lt;/li&gt;
&lt;li&gt;Manage branches.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Everything works via &lt;strong&gt;natural language&lt;/strong&gt;. You can literally type:  &lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;"open a PR with changes from main to staging"&lt;/em&gt;&lt;br&gt;&lt;br&gt;
…and it’ll handle it.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h4&gt;
  
  
  Building Your Own Server?
&lt;/h4&gt;

&lt;p&gt;It’s easier than you think. SDKs exist for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Python&lt;/strong&gt; (my go-to)
&lt;/li&gt;
&lt;li&gt;Node
&lt;/li&gt;
&lt;li&gt;Go
&lt;/li&gt;
&lt;li&gt;C#
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Wrap your internal logic or APIs, write solid descriptions, and boom—you’ve got a tool your Agent can invoke. Test locally with tools like &lt;code&gt;mcp-inspector&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  💬 Agent Mode Features Worth Using Daily
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Unified Chat Modes
&lt;/h3&gt;

&lt;p&gt;One chat window, three gears:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Ask Mode&lt;/strong&gt;: Like Stack Overflow meets your context.  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Edit Mode&lt;/strong&gt;: Tightly scoped changes to known files.  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Agent Mode&lt;/strong&gt;: All tools, full workspace, terminal, and file edits unlocked.  &lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Switch freely mid-convo:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Plan in Ask.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Execute in Agent.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Powerful Context Tags
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;#codebase&lt;/code&gt;: Grants search across files.
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;#file&lt;/code&gt;: Targets specific files.
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;#fetch&lt;/code&gt;: Pull in web content.
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;You can also drag and drop code, images, or datasets into the chat window.&lt;/p&gt;

&lt;h3&gt;
  
  
  Prompt Files
&lt;/h3&gt;

&lt;p&gt;Save reusable &lt;code&gt;.prompt.md&lt;/code&gt; files inside &lt;code&gt;.github/prompts/&lt;/code&gt; to standardize workflows, like scaffolding components or writing tests with mocks.&lt;/p&gt;

&lt;h3&gt;
  
  
  Interact with Changes
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Keep&lt;/strong&gt; — Approve generated diffs.  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Undo/Redo&lt;/strong&gt; — Navigate iterations.  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;History&lt;/strong&gt; — All sessions saved.  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Compact Chat Window&lt;/strong&gt; — Keep the convo in your flow, not your face.  &lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Tool Permissions
&lt;/h3&gt;

&lt;p&gt;Agent Mode asks before running tools. Grant:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Per tool.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Per session.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Per workspace.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Or globally (not recommended for the careless among us).&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  📓 Jupyter Notebook Integration
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/docs/datascience/jupyter-notebooks" rel="noopener noreferrer"&gt;Jupyter Notebooks&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Agent Mode + Jupyter = 🔥&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Auto-setup environments with &lt;code&gt;uv&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Fix import errors via built-in terminal actions.&lt;/li&gt;
&lt;li&gt;Explain outputs or improve charts.&lt;/li&gt;
&lt;li&gt;Generate entire notebooks from scratch via prompt.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;Try: “Create a notebook to analyze Titanic dataset” and watch the structure roll in.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  🧠 Next Edit Suggestions (NES)
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/blogs/2025/02/12/next-edit-suggestions" rel="noopener noreferrer"&gt;Next Edit Suggestions&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is inline magic. NES predicts the next logical change &lt;em&gt;as you edit&lt;/em&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Examples:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Rename &lt;code&gt;Point&lt;/code&gt; to &lt;code&gt;Point3D&lt;/code&gt;? NES adds the z-axis logic.
&lt;/li&gt;
&lt;li&gt;Add &lt;code&gt;get_max&lt;/code&gt; after &lt;code&gt;get_min&lt;/code&gt;? NES updates your comparator.
&lt;/li&gt;
&lt;li&gt;Misspell &lt;code&gt;const&lt;/code&gt;? NES fixes it.
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Press &lt;code&gt;Tab&lt;/code&gt; to accept, &lt;code&gt;Esc&lt;/code&gt; to skip. It’s contextual, real-time, and fast.&lt;/p&gt;

&lt;h2&gt;
  
  
  🔧 Model Customization &amp;amp; BYOK
&lt;/h2&gt;

&lt;p&gt;You’re not locked into GPT-4.&lt;/p&gt;

&lt;p&gt;Agent Mode supports:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;OpenAI (03, 04 Mini, 4.1).&lt;/li&gt;
&lt;li&gt;Claude.&lt;/li&gt;
&lt;li&gt;Gemini. &lt;/li&gt;
&lt;li&gt;Ollama (local LLMs!).&lt;/li&gt;
&lt;li&gt;Anthropic, OpenRouter, Azure, and more...&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Plug in your own API keys (&lt;strong&gt;BYOK&lt;/strong&gt;) and experiment. Some models don’t support tool calling—so test before going all in.&lt;/p&gt;

&lt;h3&gt;
  
  
  Bonus: &lt;code&gt;.github/copilot-instructions.md&lt;/code&gt;
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/docs/copilot/copilot-customization" rel="noopener noreferrer"&gt;Copilot Customization&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Standardize your LLM behavior:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;"Always write unit tests"
&lt;/li&gt;
&lt;li&gt;"Use React hooks"
&lt;/li&gt;
&lt;li&gt;"Don’t use for-loops" &lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  codeGeneration.instructions.file
&lt;/h3&gt;

&lt;p&gt;Also, auto-include certain files (like schema.sql) using &lt;code&gt;codeGeneration.instructions.files&lt;/code&gt; in your workspace.&lt;/p&gt;

&lt;h2&gt;
  
  
  🔭 Looking Ahead: Project Padawan
&lt;/h2&gt;

&lt;p&gt;This is the &lt;em&gt;asynchronous agent&lt;/em&gt; future—running directly on GitHub.com.&lt;/p&gt;

&lt;h3&gt;
  
  
  How It Works:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Assign an issue to Copilot.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;It works in the background via GitHub Actions.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Opens a PR with a proposed fix.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;You review + comment → it iterates.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flwdjmksnyeskzfbzhhgg.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flwdjmksnyeskzfbzhhgg.png" alt=" " width="800" height="547"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Imagine automating your backlog, onboarding smart tasks, or killing tech debt without opening VS Code.&lt;/p&gt;

&lt;h2&gt;
  
  
  🧩 My Dev Recommendations
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Context matters&lt;/strong&gt;: Better input = better output.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Break down work&lt;/strong&gt;: Guide the agent like you would a smart dev.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Use NES&lt;/strong&gt;: Inline intelligence that complements completion.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Explore MCP&lt;/strong&gt;: Build agents tailored to &lt;em&gt;your&lt;/em&gt; stack.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Experiment with models&lt;/strong&gt;: BYOK is a playground.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Always commit&lt;/strong&gt;: Let Git be your safety net.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Stay curious&lt;/strong&gt;: This space is moving fast—ride the wave.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Agent Mode&lt;/strong&gt; is the real deal. It’s not just about AI writing lines of code—it’s about &lt;strong&gt;building with&lt;/strong&gt; you. And with &lt;strong&gt;MCP&lt;/strong&gt;, &lt;strong&gt;NES&lt;/strong&gt;, and &lt;strong&gt;customizable tooling&lt;/strong&gt;, it’s evolving into something that can actually improve how we build, collaborate, and ship.&lt;/p&gt;

&lt;p&gt;I’ve already made it part of my dev workflow—and honestly, I don’t want to go back.&lt;/p&gt;

&lt;h2&gt;
  
  
  Let’s Chat
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;What tool would &lt;em&gt;you&lt;/em&gt; wrap into an MCP server?  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;What’s your favorite Agent Mode use case?  &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Tried Padawan yet?&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>agents</category>
      <category>githubcopilot</category>
      <category>vibecoding</category>
    </item>
    <item>
      <title>Agentic AI Assistants</title>
      <dc:creator>Pablo Orozco</dc:creator>
      <pubDate>Tue, 25 Feb 2025 14:03:25 +0000</pubDate>
      <link>https://dev.to/imaginex/agentic-ai-assistants-3g31</link>
      <guid>https://dev.to/imaginex/agentic-ai-assistants-3g31</guid>
      <description>&lt;p&gt;Software development is undergoing a massive transformation with AI, yet the popular narrative of "&lt;strong&gt;AI replacing developers&lt;/strong&gt;" misses the most important point: AI's greatest value comes from &lt;strong&gt;augmenting human capabilities&lt;/strong&gt;, not replacing them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;GitHub Repository&lt;/strong&gt;: &lt;a href="https://github.com/josoroma/imaginex-pr-reviewer" rel="noopener noreferrer"&gt;https://github.com/josoroma/imaginex-pr-reviewer&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwdmlip55bp96nc43in0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwdmlip55bp96nc43in0v.png" alt=" " width="800" height="436"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Agentic AI Assistants are revolutionizing how developers work by augmenting tools and automating repetitive tasks—whether in the browser with external websites or within native system applications inside an organization.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Case of the AI Code Reviewer
&lt;/h2&gt;

&lt;p&gt;Let me show you a concrete example I've been working with. I've built an automated code review system that integrates AI to help development teams maintain code quality without slowing down velocity.&lt;/p&gt;

&lt;h2&gt;
  
  
  The system works by:
&lt;/h2&gt;

&lt;p&gt;🔹 Connecting to GitHub's API to fetch pull request data.&lt;/p&gt;

&lt;p&gt;🔹 Running code diffs through advanced LLMs (like GPT-4o or Ollama models).&lt;/p&gt;

&lt;p&gt;🔹 Generating detailed markdown reports that analyze code changes.&lt;/p&gt;

&lt;p&gt;🔹 Providing explanations, issue identification, and actionable improvements.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why This Matters for Developers
&lt;/h2&gt;

&lt;p&gt;This approach exemplifies how AI shines as a developer tool, not a replacement:&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Enhanced Capabilities, Not Substitution&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The AI reviewer doesn't replace human reviewers - it provides an initial analysis that helps developers focus their attention. Human reviewers still provide context-aware judgment and make the final decisions.&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Addressing Real Developer Pain Points&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Code reviews are often bottlenecks in development workflows. By automating the initial review, developers can:&lt;/p&gt;

&lt;p&gt;🔹 Get immediate feedback without waiting for human reviewers.&lt;/p&gt;

&lt;p&gt;🔹 Identify obvious issues before humans spend time on them.&lt;/p&gt;

&lt;p&gt;🔹 Focus human review time on more complex architectural concerns.&lt;/p&gt;

&lt;h2&gt;
  
  
  Customizability for Team Needs
&lt;/h2&gt;

&lt;p&gt;The system allows teams to use different LLMs based on their requirements:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;# Configuration supports both cloud and local models
OLLAMA_MODEL_NAME = os.getenv("OLLAMA_MODEL_NAME", "phi4")
LLM_MODEL_NAME = os.getenv("LLM_MODEL_NAME", "gpt-4o")
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Teams can choose privacy-preserving local models for sensitive code or more powerful cloud models when needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Beyond Code Review: The AI Developer Toolkit
&lt;/h2&gt;

&lt;p&gt;This pattern extends to numerous development tools that augment rather than replace:&lt;/p&gt;

&lt;p&gt;🔹 &lt;strong&gt;Pair programming assistants&lt;/strong&gt; that suggest implementations in real-time.&lt;/p&gt;

&lt;p&gt;🔹 &lt;strong&gt;Automated test generators&lt;/strong&gt; that create test cases based on implementation.&lt;/p&gt;

&lt;p&gt;🔹 &lt;strong&gt;Documentation generators&lt;/strong&gt; that produce readable explanations from code.&lt;/p&gt;

&lt;p&gt;🔹 &lt;strong&gt;Code migration tools&lt;/strong&gt; that help port between frameworks or languages.&lt;/p&gt;

&lt;p&gt;🔹 &lt;strong&gt;Performance optimization analyzers&lt;/strong&gt; that suggest improvements.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why AI Won't Replace Developers
&lt;/h2&gt;

&lt;p&gt;While these tools are powerful, they highlight precisely why AI won't replace developers:&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Context understanding&lt;/strong&gt;: Developers know the business requirements, organizational constraints, and customer needs that AI lacks.&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Tool orchestration&lt;/strong&gt;: Developers determine which AI tools to apply and when.&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Judgment calls&lt;/strong&gt;: Technical decisions often involve tradeoffs that require human judgment.&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Architectural thinking&lt;/strong&gt;: Systems design remains a deeply creative human activity.&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;Human collaboration&lt;/strong&gt;: Software development is a team sport requiring human communication.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Future: AI-Enhanced Development
&lt;/h2&gt;

&lt;p&gt;Rather than fearing replacement, developers should embrace AI as the next evolution of our toolset. Just as we moved from assembly to high-level languages, from manual memory management to garbage collection, and from physical servers to cloud infrastructure, AI represents the next abstraction layer.&lt;/p&gt;

&lt;p&gt;It is not a secret that the most successful developers won't be those who compete with AI but those who become experts at integrating AI to solve increasingly complex problems.&lt;/p&gt;

&lt;h2&gt;
  
  
  Getting Started
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Want to explore AI-augmented development?&lt;/strong&gt; The code for this review system is built with:&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;&lt;a href="https://www.crewai.com/" rel="noopener noreferrer"&gt;CrewAI&lt;/a&gt;&lt;/strong&gt; for orchestrating AI agents.&lt;/p&gt;

&lt;p&gt;🔺 &lt;a href="https://www.langchain.com/" rel="noopener noreferrer"&gt;LangChain&lt;/a&gt; for LLM integration.&lt;/p&gt;

&lt;p&gt;🔺 Support for both &lt;a href="https://platform.openai.com/docs/models" rel="noopener noreferrer"&gt;OpenAI&lt;/a&gt; and local &lt;a href="https://ollama.com/search?o=newest" rel="noopener noreferrer"&gt;Ollama&lt;/a&gt; models.&lt;/p&gt;

&lt;p&gt;🔺 &lt;strong&gt;&lt;a href="https://github.com/josoroma/imaginex-pr-reviewer" rel="noopener noreferrer"&gt;GitHub API&lt;/a&gt;&lt;/strong&gt; integration.&lt;/p&gt;

&lt;p&gt;This pattern of AI augmentation - not replacement - represents the future of software development. The goal isn't to build AI that codes, but to build AI that helps humans code better.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why did I choose CrewAI for this agentic assistant?
&lt;/h2&gt;

&lt;p&gt;When building AI-driven workflows, I wanted a framework that was both developer-friendly and efficient—and CrewAI checked all the boxes. Here’s what makes it stand out:&lt;/p&gt;

&lt;p&gt;🔹 Seamless Collaboration – CrewAI enables AI agents to work together like a well-coordinated team, making automation feel more natural.&lt;/p&gt;

&lt;p&gt;🔹 Simple &amp;amp; Intuitive – With YAML-based configuration, setting up and deploying workflows is quick and painless.&lt;/p&gt;

&lt;p&gt;🔹 Rapid Prototyping – It lets me focus on building logic instead of worrying about orchestration complexity.&lt;/p&gt;

&lt;p&gt;🔹 Highly Scalable – Whether it’s a simple automation or a complex AI-powered pipeline, CrewAI scales effortlessly.&lt;/p&gt;

&lt;p&gt;🔹 Thriving Open-Source Community – Backed by a growing community, CrewAI benefits from continuous improvements and innovation.&lt;/p&gt;

&lt;p&gt;For teams looking to seamlessly integrate AI into their workflows and boost productivity, this framework is a game-changer.&lt;/p&gt;

&lt;h2&gt;
  
  
  What’s in Your AI Toolkit?
&lt;/h2&gt;

&lt;p&gt;What AI developer tools are you using in your projects?&lt;/p&gt;

&lt;p&gt;Let’s swap ideas—drop a comment below! 👇&lt;/p&gt;

</description>
      <category>ai</category>
      <category>crewai</category>
      <category>automation</category>
      <category>agents</category>
    </item>
    <item>
      <title>DataOps Best Practices: Building Resilient Pipelines in Databricks</title>
      <dc:creator>Jean Arias</dc:creator>
      <pubDate>Thu, 23 Jan 2025 15:18:36 +0000</pubDate>
      <link>https://dev.to/imaginex/dataops-best-practices-building-resilient-pipelines-in-databricks-52g4</link>
      <guid>https://dev.to/imaginex/dataops-best-practices-building-resilient-pipelines-in-databricks-52g4</guid>
      <description>&lt;p&gt;In today's data-driven world, organizations face the challenge of managing increasingly complex data workflows. In a world where data workflows grow increasingly complex, the need for seamless operations has never been greater.&lt;br&gt;
Studies show that organizations implementing DataOps achieve up to 30% faster pipeline deployment and a 50% reduction in errors. Imagine having a seamless process to ensure your data pipelines are always reliable, scalable, and efficient. This blog explores how &lt;strong&gt;Databricks&lt;/strong&gt;, powered by Apache Spark and Delta Lake, can help you implement DataOps principles effectively. The target audience for this blog includes data engineers, data analysts, and data scientists.&lt;/p&gt;

&lt;h3&gt;
  
  
  Table of Contents
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;What is DataOps?&lt;/li&gt;
&lt;li&gt;Why Databricks?&lt;/li&gt;
&lt;li&gt;Best Practices for DataOps in Databricks&lt;/li&gt;
&lt;li&gt;Pipeline Availability&lt;/li&gt;
&lt;li&gt;Pipeline Versioning&lt;/li&gt;
&lt;li&gt;Pipeline Configurations&lt;/li&gt;
&lt;li&gt;Data Cleaning&lt;/li&gt;
&lt;li&gt;Data Aggregation&lt;/li&gt;
&lt;li&gt;Data Validation&lt;/li&gt;
&lt;li&gt;Call to Action&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  What is DataOps?
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;DataOps&lt;/strong&gt;, short for Data Operations, is a set of practices aimed at improving the quality, speed, and reliability of data analytics and machine learning pipelines. It ensures the efficient, scalable, and error-free processing of data in modern workflows, enabling organizations to extract actionable insights effectively.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why Databricks?
&lt;/h3&gt;

&lt;p&gt;Databricks, empowered by &lt;strong&gt;Apache Spark&lt;/strong&gt; and &lt;strong&gt;Delta Lake&lt;/strong&gt;, provides a unified platform for data engineering, machine learning, and analytics. This makes it an ideal foundation for implementing DataOps principles.&lt;/p&gt;




&lt;h2&gt;
  
  
  Best Practices for DataOps in Databricks
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Pipeline Availability
&lt;/h3&gt;

&lt;p&gt;Ensuring pipeline availability is essential for meeting service level agreements (SLAs) and handling disruptions without compromising performance.&lt;/p&gt;

&lt;h4&gt;
  
  
  Best Practices:
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Proactive Monitoring:&lt;/strong&gt; Utilize Databricks' Jobs UI and integrate with tools like Prometheus and Grafana for real-time pipeline monitoring and alerting. Proactive monitoring helps teams detect anomalies, measure pipeline performance, and address issues before they impact downstream processes.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fugxro3r5atxye48hvhcr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fugxro3r5atxye48hvhcr.png" alt="monitorpage" width="800" height="395"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Fault Tolerance Mechanisms:&lt;/strong&gt; Leverage Delta Lake's ACID (Atomicity, Consistency, Isolation, Durability) transactions to ensure data consistency and recoverability. For instance, atomicity guarantees that a series of data operations either complete fully or not at all, avoiding partial updates. Retry logic can handle transient errors, while error-handling mechanisms ensure pipeline stability in the event of failures.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Dynamic Resource Management:&lt;/strong&gt; Configure autoscaling in Databricks clusters to adjust compute resources dynamically based on workload demands. This reduces costs during low activity periods while ensuring peak performance during intensive tasks.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Optimized Scheduling:&lt;/strong&gt; Use Databricks Workflows for robust pipeline scheduling. Automated retries and dependency management streamline complex workflows, reducing manual intervention and ensuring timely pipeline execution.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  2. Pipeline Versioning
&lt;/h3&gt;

&lt;p&gt;Pipeline versioning enables traceability, collaboration, and reproducibility of workflows.&lt;/p&gt;

&lt;h4&gt;
  
  
  Best Practices:
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Version Control for Code:&lt;/strong&gt; Integrate Databricks Repos with Git for collaborative development and version tracking. This allows seamless collaboration, version history tracking, and the ability to roll back changes when necessary. Beginners can start by setting up a remote Git repository and connecting it through the Databricks UI.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Data Versioning:&lt;/strong&gt; Leverage Delta Lake’s time travel feature to maintain historical data versions, enabling you to access and revert to previous data states. This is particularly useful for debugging, auditing, and ensuring reproducibility in analytics workflows.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Manage different verions of your table:&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwbogwsxg5u8cznmv3o2q.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fwbogwsxg5u8cznmv3o2q.png" alt="tableversion" width="800" height="507"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;See the changelog to know what happened:&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fo3sqlvklc7qy6lq4i3i5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fo3sqlvklc7qy6lq4i3i5.png" alt="changelog" width="800" height="498"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Notebook Versioning:&lt;/strong&gt; Enable Databricks notebook version history to restore and track changes effortlessly.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Tagging and Release Management:&lt;/strong&gt; Implement a tagging strategy to mark stable versions of pipelines. Use descriptive tags for milestones such as 'v1.0-production' or 'v2.1-hotfix' to provide clarity. This ensures reliable deployments and simplifies troubleshooting or rollbacks when needed.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  3. Pipeline Configurations
&lt;/h3&gt;

&lt;p&gt;Proper configurations ensure security, scalability, and flexibility in workflows.&lt;/p&gt;

&lt;h4&gt;
  
  
  Best Practices:
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Secure Storage:&lt;/strong&gt; Use Azure Key Vault, AWS Secrets Manager, or Databricks Secrets for sensitive data.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Environment Variables:&lt;/strong&gt; Maintain environment-specific variables in centralized configuration files.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Configuration Tables:&lt;/strong&gt; Store pipeline configurations in Delta Lake for dynamic, maintainable workflows.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Version-Controlled Configurations:&lt;/strong&gt; Track configuration files in Git for audit trails and rollback capabilities.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Example of parameter table:&lt;br&gt;
&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqq8evq3236uucy79lnsx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqq8evq3236uucy79lnsx.png" alt="parameters" width="800" height="389"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  4. Data Cleaning
&lt;/h3&gt;

&lt;p&gt;Data cleaning transforms raw data into accurate, reliable datasets that drive meaningful insights.&lt;/p&gt;

&lt;h4&gt;
  
  
  Best Practices:
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Reusable Cleaning Logic:&lt;/strong&gt; Build reusable cleaning scripts using PySpark or SQL for consistent results across projects. For instance, a PySpark script can be designed to handle common tasks like removing duplicates, filling null values, and standardizing column names. This script can then be parameterized to adapt to different datasets, ensuring flexibility and consistency across workflows.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Data Profiling:&lt;/strong&gt; Leverage tools like Databricks Data Explorer or Great Expectations to identify issues such as null values, duplicates, and outliers. For example, using Great Expectations, you can define validation rules that flag missing values or inconsistent data types in real-time, ensuring your datasets meet quality standards before downstream processing.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Automated Cleaning Workflows:&lt;/strong&gt; Automate repetitive cleaning tasks using Databricks Workflows to save time and ensure consistent data quality across pipelines.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Metadata Management:&lt;/strong&gt; Utilize Unity Catalog to track and document data changes.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  5. Data Aggregation
&lt;/h3&gt;

&lt;p&gt;Data aggregation simplifies raw data into meaningful summaries, improving performance and usability.&lt;/p&gt;

&lt;h4&gt;
  
  
  Best Practices:
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Optimized Queries:&lt;/strong&gt; Write aggregation queries using Spark SQL for efficient execution. Use query hints and the Catalyst optimizer to achieve faster results.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Partitioning:&lt;/strong&gt; Use Delta Lake partitioning to optimize read and write operations, particularly for large datasets, by narrowing down the data processed.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Incremental Aggregation:&lt;/strong&gt; Apply Delta Lake’s &lt;code&gt;MERGE INTO&lt;/code&gt; for processing only new or updated records. This minimizes computational overhead and speeds up pipeline performance.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Pre-Aggregated Tables:&lt;/strong&gt; Create materialized views for frequently queried datasets to improve access speeds. For example, pre-aggregate daily sales data into weekly or monthly summaries for business dashboards.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Caching:&lt;/strong&gt; Use Spark’s caching mechanisms to store intermediate results, significantly reducing execution times for repeated queries.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  6. Data Validation
&lt;/h3&gt;

&lt;p&gt;Data validation ensures data meets quality standards and adheres to business rules before being consumed downstream.&lt;/p&gt;

&lt;h4&gt;
  
  
  Best Practices:
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Validation Frameworks:&lt;/strong&gt; Use tools like Great Expectations or custom PySpark frameworks for enforcing data quality checks.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Schema Enforcement:&lt;/strong&gt; Leverage Delta Lake’s schema enforcement to block invalid data.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Alerting Systems:&lt;/strong&gt; Set up alerts for validation failures using PagerDuty or Slack integrations.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Unit Testing for Validation:&lt;/strong&gt; Implement unit tests for transformations and validation logic using pytest or MLflow.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Call to Action
&lt;/h2&gt;

&lt;p&gt;DataOps is not just a trend; it's a necessity for organizations striving to stay competitive in today’s data-driven landscape. By applying the principles and best practices outlined in this blog, you can transform your data pipelines into resilient, high-performing systems. Databricks, with its robust tools like Delta Lake, Workflows, and Repos, provides the perfect platform to implement these changes.&lt;/p&gt;

&lt;p&gt;Don’t stop at theory—put these strategies into action. Assess your current workflows, adopt the tools discussed, and start making incremental improvements today. Your journey toward streamlined, reliable, and scalable data operations begins now. Explore Databricks Academy to empower your team and unlock the full potential of your data pipelines.&lt;/p&gt;

&lt;p&gt;Transform your approach to data and experience the difference that modern DataOps practices can bring to your organization.&lt;/p&gt;

</description>
      <category>dataengineering</category>
      <category>databricks</category>
      <category>dataops</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>Intro to Cloud: An API Gateway</title>
      <dc:creator>Joseph</dc:creator>
      <pubDate>Mon, 02 Dec 2024 14:38:16 +0000</pubDate>
      <link>https://dev.to/imaginex/intro-to-cloud-an-api-gateway-a5f</link>
      <guid>https://dev.to/imaginex/intro-to-cloud-an-api-gateway-a5f</guid>
      <description>&lt;p&gt;I tried to think of the best way to describe building an app from scratch in the cloud, but there are so many different ways you can easily get lost in all the options. So I decided I would start by showcasing what is arguably the easiest app you can make from a computing perspective. It offers the least amount of operational overhead, but at the same time, it's the most expensive option &lt;em&gt;on paper&lt;/em&gt;.&lt;/p&gt;

&lt;h1&gt;
  
  
  AWS API Gateway
&lt;/h1&gt;

&lt;p&gt;When I think of simple I think API. That might just be because I'm so used to them at this point, but there are also some direct comparisons to other compute solutions which we'll get into later in other posts as we get into more complex cloud compute ideas. Right now, though, we're going to use AWS API Gateway to deploy the first version of the API we're going to make for the rest of this series. This is a great choice if you want to spend as little time maintaining your gateway as possible. There are no load balancers to deal with. You can optionally put an edge cache (CloudFront) in front of it easily, and it connects to other serverless technology in AWS seamlessly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting up the gateway
&lt;/h2&gt;

&lt;p&gt;For the example I'm building, I'm going to actually use a terraform module that already has all the pieces we need. The first thing we need to do is add the module to our project by editing the &lt;code&gt;cdktf.json&lt;/code&gt; file to include the module.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;terraformModules&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
 &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;name&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;apigateway&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;source&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;terraform-aws-modules/apigateway-v2/aws&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;version&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;~&amp;gt; 5.2.0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;
 &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;],&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Later on, we're going to add another module but for now, that's all we need to build the gateway.&lt;/p&gt;

&lt;h2&gt;
  
  
  Defining the gateway
&lt;/h2&gt;

&lt;p&gt;I'm going to make a &lt;code&gt;serverless&lt;/code&gt; folder and put an &lt;code&gt;api.ts&lt;/code&gt; to hold all the work we'll need to get the app up and working. Inside our class, we have to import the module we made and then create the gateway.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;Apigateway&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;../.gen/modules/apigateway&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;gw&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Apigateway&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;scope&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-gw`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-gw`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;protocolType&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;HTTP&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;corsConfiguration&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;allowHeaders&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
      &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;content-type&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-amz-date&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;authorization&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-api-key&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-amz-security-token&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-amz-user-agent&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="na"&gt;allowMethods&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;*&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="na"&gt;allowOrigins&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;*&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;

  &lt;span class="na"&gt;createDomainName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;createDomainRecords&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;createCertificate&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;

  &lt;span class="na"&gt;domainName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;hostedZoneName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;hostedZone&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;

  &lt;span class="na"&gt;routes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;ANY /&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="na"&gt;integration&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="na"&gt;uri&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;lambdaArn&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="na"&gt;payload_format_version&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;2.0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="na"&gt;timeout_milliseconds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="na"&gt;$default&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="na"&gt;integration&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="na"&gt;uri&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;lambdaArn&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="na"&gt;payload_format_version&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;2.0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="na"&gt;timeout_milliseconds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In my case, I already had a hosted zone set up in my AWS account for a subdomain &lt;code&gt;aws.josephbulger.com&lt;/code&gt;. I decided to go ahead and add this gateway to that so the hosted zone in this example is &lt;code&gt;aws.josephbulger.com&lt;/code&gt; and the API's domain is &lt;a href="//apigw.examples.how2cloud.aws.josephbulger.com"&gt;apigw.examples.how2cloud.aws.josephbulger.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;You don't have to set this up with a domain. If you don't AWS will generate a random URL to host your gateway instead, and you can use that for making your API calls.&lt;/p&gt;

&lt;h3&gt;
  
  
  CORS
&lt;/h3&gt;

&lt;p&gt;I have the API's cors configuration set up to allow traffic from anywhere because, with this kind of setup, I'm assuming the front end that would potentially use this is not hosted by the same app. This is a common approach I use for my systems. I don't like to have the API backend be served by the same system that serves the front end. I like to keep them separate.&lt;/p&gt;

&lt;h3&gt;
  
  
  Routes
&lt;/h3&gt;

&lt;p&gt;Finally, you need to define the routes that the gateway will serve. In my case, I'm going to send all my traffic to a single lambda, which we'll talk about in a second. For now, we just need to know to send all the traffic over to the lambda, and let it figure out what to do with it.&lt;/p&gt;

&lt;h4&gt;
  
  
  Stages
&lt;/h4&gt;

&lt;p&gt;Additionally, we have to define the default stage. Stages in API Gateway are kind of a weird concept. It's beyond the scope of what I want to cover in this post, but for this purpose let's just say you have to define a default one, so the gateway knows which stage gets the traffic. In our case, we're just going to set it up the same way we set up the route we made.&lt;/p&gt;

&lt;h1&gt;
  
  
  Lambda
&lt;/h1&gt;

&lt;p&gt;The lambda is the part that makes this computing solution simple and easy. It does this because it's truly serverless. We don't have to worry about running a server, neither as a virtual machine an EC2 instance somewhere, nor as compute sitting on top of an orchestration plan like ECS or k8s. We just create the app, and give it to the lambda. The largest amount of operational overhead with the lambda boils down to dealing with how much CPU and memory to allocate to it, and these days you only specify the memory because the CPU is adjusted based on how much memory you choose. The price we pay for this ease of use is, well, the actual sticker price. All things considered, you can expect to pay anywhere from double to up to 6 times the price of other compute solutions in AWS for the equivalent &lt;em&gt;compute time&lt;/em&gt;, but that's on paper. In reality, there are some very compelling reasons to use Lambda, including financially.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting up your Lambda
&lt;/h2&gt;

&lt;p&gt;So to get using the lambda we first need to add the module into cdktf the same way we did the gateway.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;terraformModules&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
 &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;name&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;lambda&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;source&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;terraform-aws-modules/lambda/aws&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;version&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;~&amp;gt; 7.14.0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;
 &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;],&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Defining the Lambda
&lt;/h2&gt;

&lt;p&gt;Now we can add the lambda to our API.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;lambda&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Lambda&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;scope&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-function`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;functionName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-function`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;createPackage&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;packageType&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Image&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;architectures&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x86_64&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="na"&gt;imageUri&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ecrUri&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;:&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tag&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;publish&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;lambdaArn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;lambda&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;lambdaFunctionArnOutput&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We used the &lt;code&gt;lambdaArn&lt;/code&gt; earlier where we defined the gateway. This is actually where it came from, as an output from the lambda module. Now the last thing we need to do to connect the two is set up the lambda's triggers. The trick to them is that the lambda needs to know about the gateway, but we have to define the lambda first. What we do to get around this is just define the triggers &lt;strong&gt;after&lt;/strong&gt; we make both the lambda and the gateway and then add the triggers back into the lambda.&lt;/p&gt;

&lt;p&gt;It looks something like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;apiExecArn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;gw&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;apiExecutionArnOutput&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;triggers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;APIGatewayAny&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;service&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;apigateway&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;source_arn&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;apiExecArn&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/*/*`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="nx"&gt;lambda&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;allowedTriggers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;triggers&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now that the lambda is connected to the gateway we need to create the app that the lambda is going to run.&lt;/p&gt;

&lt;h1&gt;
  
  
  Docker
&lt;/h1&gt;

&lt;p&gt;I created a small server in &lt;code&gt;examples&lt;/code&gt; that creates a simple go web server built using docker. What's nice about Go is that you can actually deploy it from a scratch image so the overall size of the image is really small (~8 MB). Perfect for running on a lambda.&lt;/p&gt;

&lt;p&gt;The docker file looks something like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight docker"&gt;&lt;code&gt;&lt;span class="c"&gt;# syntax=docker/dockerfile:1&lt;/span&gt;

&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;golang:1.23&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;builder&lt;/span&gt;

&lt;span class="k"&gt;WORKDIR&lt;/span&gt;&lt;span class="s"&gt; /app&lt;/span&gt;

&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; go.mod go.sum ./&lt;/span&gt;
&lt;span class="k"&gt;RUN &lt;/span&gt;go mod download

&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; *.go ./&lt;/span&gt;

&lt;span class="k"&gt;ARG&lt;/span&gt;&lt;span class="s"&gt; TARGETARCH=amd64&lt;/span&gt;
&lt;span class="k"&gt;ENV&lt;/span&gt;&lt;span class="s"&gt; GOARCH=$TARGETARCH&lt;/span&gt;

&lt;span class="k"&gt;ARG&lt;/span&gt;&lt;span class="s"&gt; TARGETOS=linux&lt;/span&gt;
&lt;span class="k"&gt;ENV&lt;/span&gt;&lt;span class="s"&gt; GOOS=$TARGETOS&lt;/span&gt;

&lt;span class="k"&gt;RUN &lt;/span&gt;&lt;span class="nv"&gt;CGO_ENABLED&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0 go build &lt;span class="nt"&gt;-o&lt;/span&gt; /api

&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="s"&gt; scratch&lt;/span&gt;

&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; --from=builder /api /api&lt;/span&gt;

&lt;span class="k"&gt;EXPOSE&lt;/span&gt;&lt;span class="s"&gt; 8080&lt;/span&gt;

&lt;span class="k"&gt;ENTRYPOINT&lt;/span&gt;&lt;span class="s"&gt; ["/api"]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We'll be building it using &lt;code&gt;docker buildx bake&lt;/code&gt; so we need to set up a bake file too:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;group&lt;/span&gt; &lt;span class="s2"&gt;"default"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
 &lt;span class="nx"&gt;targets&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"api"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;variable&lt;/span&gt; &lt;span class="s2"&gt;"IMAGE_URI"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
 &lt;span class="nx"&gt;default&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"api"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="nx"&gt;variable&lt;/span&gt; &lt;span class="s2"&gt;"IMAGE_TAG"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
 &lt;span class="nx"&gt;default&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"local"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;target&lt;/span&gt; &lt;span class="s2"&gt;"api"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
 &lt;span class="nx"&gt;dockerfile&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"Dockerfile"&lt;/span&gt;
 &lt;span class="nx"&gt;context&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"."&lt;/span&gt;
 &lt;span class="nx"&gt;tags&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"${IMAGE_URI}:${IMAGE_TAG}"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
 &lt;span class="nx"&gt;platforms&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"linux/amd64"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
 &lt;span class="nx"&gt;args&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
 &lt;span class="nx"&gt;TARGETARCH&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"amd64"&lt;/span&gt;
 &lt;span class="nx"&gt;TARGETOS&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"linux"&lt;/span&gt;
 &lt;span class="p"&gt;}&lt;/span&gt;
 &lt;span class="nx"&gt;attest&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
 &lt;span class="nx"&gt;provenance&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Provenance
&lt;/h2&gt;

&lt;p&gt;One issue with deploying to lambdas, at least for now, is that they don't support multi-architecture docker images. So when we deploy our image to ECR we have to disable provenance so docker knows to only build the image in the format we specify with multi-mode turned off.&lt;/p&gt;

&lt;p&gt;That's why in our GitHub action we turn provenance off when we do the docker build and push:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Build and push&lt;/span&gt;
  &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;docker/bake-action@v5&lt;/span&gt;
  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;push&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
    &lt;span class="na"&gt;workdir&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;examples/api&lt;/span&gt;
    &lt;span class="na"&gt;provenance&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
  &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;IMAGE_URI&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;[[&lt;/span&gt;&lt;span class="nv"&gt;ECR_URL&lt;/span&gt;&lt;span class="pi"&gt;]]&lt;/span&gt;&lt;span class="s"&gt;/api&lt;/span&gt;
    &lt;span class="na"&gt;IMAGE_TAG&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.ref_name }}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h1&gt;
  
  
  That's It
&lt;/h1&gt;

&lt;p&gt;Now you have a fully functioning API. You can see the one I've deployed at &lt;a href="https://apigw.examples.how2cloud.aws.josephbulger.com/" rel="noopener noreferrer"&gt;https://apigw.examples.how2cloud.aws.josephbulger.com/&lt;/a&gt;. There's also a route you can play around with the changes in the response based on what you give it: &lt;a href="https://apigw.examples.how2cloud.aws.josephbulger.com/some/kind/of/trick" rel="noopener noreferrer"&gt;https://apigw.examples.how2cloud.aws.josephbulger.com/some/kind/of/trick&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Show me the code
&lt;/h2&gt;

&lt;p&gt;If you want to follow along with the code that I used to build this, or any other part of the "from scratch" series, check out my &lt;a href="https://github.com/josephbulger/how-to-cloud" rel="noopener noreferrer"&gt;how to cloud repo&lt;/a&gt; on github.&lt;/p&gt;

</description>
      <category>beginners</category>
      <category>aws</category>
      <category>devops</category>
      <category>serverless</category>
    </item>
  </channel>
</rss>
