DEV Community: Arctir

Artificial Intelligence with Platform Engineering

snowandcaffeine — Fri, 10 Jan 2025 14:52:12 +0000

Arguably the two biggest themes of the past few years, especially in the enterprise are Platform Engineering (PE) and Artificial Intelligence (AI). Both have become buzzwords and delivered some initial wins but haven't yet lived up to their full promise. What if the best way to meet the sky high expectations is to combine AI and PE?

As we've covered before platform engineering has emerged as the next evolutionary step to address the overload introduced by the explosion of software that has occured over the next decade, and the subsequent pressure it applied to cloud and DevOps teams. At its core, platform engineering is about creating reusable, self-service platforms and patterns that reduce developer toil and shrink the context window a developer needs to be productive. Instead of every team reinventing the wheel, platform engineers design, deploy and operate purpose built platforms that:

Standardize Workflows : Provide templates and blueprints for common tasks.
Improve Discovery: By leveraging a software catalog changes, relationships, dependencies, tooling and documentation can be rapidly understood and contextualized by developers.
Enable Self-Service : Allow developers to deploy resources, monitor applications, and troubleshoot issues without external dependencies.
Enhance Security and Compliance : Embed best practices into the platform itself, reducing risk and context switching for other teams.

For example, a platform engineering team might create an internal developer portal (IDP) powered by Flightdeck / Backstage. This portal includes at it's heart a software catalog and consolidates the view of downstream tools such as CI/CD pipelines, observability, and cloud infrastructure, offering developers a heads-up display for understanding, building and managing applications.

The Role of Artificial Intelligence in Platform Engineering

The age of artificial intelligence is transforming every aspect of technology, and platform engineering is no exception. There are two scenarios at play here; using AI within PE and building platforms for AI delivery. While there may be some overlap the two uses are different enough to warrant discussion as stand alone topics.

In the first scenario, AI is being baked into various aspects of tools and processes that the PE team uses themselves, some of which may be dog-fooding of tools they deliver to developers, others will be used primarily by the PE team.

But this raises the complex questions which led us to the second scenario. If a new agentic workflow is deployed to support developers, who is building the underlying infrastructure, specification and interfaces for that suite of capabilities? How are these new systems governed and secured?

In our view, that should be the platform engineering team. The same strategy and standards should be used here as with other platforms.

This is where we see platform engineering will be headed over the next couple of years: the platform team leverages AND provides AI capabilities to developers.

What We Call AI Platform Engineering (APE)

This presents a significant improvement over previous generations of tooling in both the developer and DevOps sphere. Instead of fighting headwinds getting developers to use new tools or adopt broken processes, developer platforms will come with intelligence baked in from day-1. Most importantly, they will provide access to the intelligence where developers and operators already work. In essence these platforms will bring the water to the horse.

The implementation and approach taken here can range from a chat-based LLM in place of search functionality, to a code generation assistant, or even a fully agentic workflow. The underlying tooling provides the same core functions, but with massively improved flexibility, performance and interfaces that leverage advances in AI.

Our opinion is that this functionality should be delivered by the platform engineering team in a workflow that is already native for developers and operations teams. Rather than yet another web interface or application, bring these capabilities directly into the IDE, terminal and collaboration tools where people already do work. This is precisely what we have already done with Flightdeck, and are actively adding even more of these types of capabilities.

Dev + AI collaboration is the future

Enhanced Context and Awareness

AI can already analyze patterns in logs, metrics, and traces to identify bottlenecks, optimize workflows, and increasingly even predict failures or redress them as they occur. This dramatically reduces the amount of toil a developer, SRE or associated team encounters.

Using a graph-backed developer platform helps provide improved context for AI and developers. This scenarios provides a few really useful implementation patterns:

Leverage Retrieval Augmented Generation (RAG), producing locally relevant results for queries.
Assist developers by summarizing status and enriching status with meaningful context from downstream systems and agents.
Suggest improvements to application code based on historical and organizational data and locally relevant patterns.
Connect disparate data sources through composable workflows.

Intelligent Context Management

As developers switch between tasks and tools they are often overwhelmed trying to find things. A given piece of software is often represented differently in each tool a developer uses. Similarly each tool often has a different workflow. All of this requires the developer to maintain a mental model of what they are working on. A well-designed and integrated AI can consolidate and present relevant context faster, reducing the burden on the developer. For example:

Summarizing downstream tools, code and discussions about a specific software entity.
Highlighting unresolved dependencies or seemingly unrelated changes in code or configuration.
Proactively surfacing documentation, code or API references related to ongoing work.

The Future of AI Platform Engineering

As AI continues to evolve, so will the demands on platform engineering. It is hard to predict the future, but we currently think this is what it looks like as these domains converge:

Hyper-Personalized Developer Tools : Platforms will leverage AI to adapt dynamically to individual workflows and preferences, offering contextual suggestions, automated task management, and real-time feedback tailored to each developer's unique needs at each moment.
Autonomous Systems : Fully self-healing and adapting platforms will become the norm, using advanced AI-driven monitoring and diagnostics to identify, resolve, and even prevent issues before they impact production. These systems will not only fix common errors but also optimize resource usage and system configurations autonomously. The governance and operations of the agents and platforms will fall to the platform engineering team.
Standardized Patterns : As AI becomes integrated into more and more developer tools and platforms, developers will likely rely on standardized patterns, frameworks and tools to ensure success.
AI-Driven Collaboration : Future platforms will act as intelligent intermediaries between teams and team members, summarizing discussions, suggesting next steps, and highlighting key dependencies across projects. By fostering seamless communication and coordination, AI will reduce friction in team-based workflows.
Continuous Learning Systems : AI-enabled platforms will incorporate learning loops that evolve based on user interactions, feedback, and emerging patterns. These systems will refine their capabilities over time, offering increasingly accurate insights, recommendations, and solutions to complex problems.

Underlying all of this is a significant question. Who runs these systems? Right now many companies are allowing the usage of AI to occur on an individual-to-individual basis with minimal controls, legal agreements or technical strategy.

In the model we propose the APE team is responsible for platform definition, operation and the standardization of communication patterns and governance.

Conclusion

Platform engineering is the natural progression of decades of innovation in systems administration and DevOps. In the age of AI, it represents an opportunity to define and govern the AI implementations leveraged by the organization. Through investment in AI Platform Engineering, organizations can empower their teams, streamline workflows, and unlock new levels of innovation – all while ensuring the security, standards and governance of AI are there from day-1.

With the right strategy and tools, platform engineering can become the foundation for success in an increasingly complex and competitive landscape.

Want to learn more about how we are extending Flightdeck for these scenarios and other roadmap items? Let's talk!

Internal Developer Platform or Internal Developer Portal? Which is Right For You?

snowandcaffeine — Wed, 18 Dec 2024 18:21:30 +0000

Key Highlights

Internal Developer Platforms (IDPlat) streamline software development across the entire Software Development Lifecycle, while Internal Developer Portals (IDPort) provide a centralized heads-up display for developers.
Both reduce cognitive load: Platform by automating tasks, Portals by consolidating access to tools and resources. Both provide golden paths to production.
The right solution depends on your specific needs and goals.
Integrating existing tools and gaining buy-in from stakeholders is crucial for successful platform and/or portal implementation.
When implemented strategically, both tools can optimize workflows and enhance the developer experience, leading to higher productivity and software quality.
You can start with one and grow into using both.

Basics of Internal Developer Platform and Internal Developer Portal

Since we launched Flightdeck we've heard a lot of confusion about which is which, where to start, and how to evolve. We believe there is no single answer. In fact, when speaking with customers we often advise reviewing the existing processes, tooling and what is working...or not. That generally highlights a few gaps that can be addressed as quick wins.

An Internal Developer Platform (IDPlat) acts as a comprehensive solution, or more often suite of solutions, that streamlines various aspects of the development workflow, offering tools for deployment, monitoring, and collaboration. Core here is automating and streamlining each step in the development lifecycle. This isn't just about deploying a continuous integration tool or orchestrator. Rather, it is reviewing the friction points in the development process from initial commit all the way to retiring software, understanding where you can standardize, what flexibility to offer, what products to buy and what to build. It is a large effort that isn't typically one shot and done. Rather when we talk about IDPlat we talk about it hand-in-glove with Platform Engineering as a wholesale way to evolve developer tooling and experience over time.

On the other hand, Internal Developer Portals serves primarily as a heads-up display where developers can access resources, documentation, and some automation to aid them in their day-to-day tasks. While there is certainly some effort in adopting a Portal it is lower effort than developing a comprehensive Platform. We view a Portal as a critical part of a platform, and typically the best starting point on that journey, as it provides quick time-to-value for most situations. If done correctly it makes the Platform journey easier.

The Functionality of an Internal Developer Platform

The platform is usually a collection of tools which comprise the whole SDLC. Because of this, we often see the Portal act as the display tier, with the platform engineering team building and operating most, if not all, of the platform components. This is not a 'all in a box' situation for most companies as the functionality spans from source control, code quality, testing, CI/CD and infrastructure all the way to running and hosting services. Heck, we've also seen companies pulling observability, AI tooling and PaaS tooling into the mix as part of this effort. Sounds like a lot? It is. That is why we generally don't advise companies to start here.

The primary goal of this effort is to reduce the cognitive overhead and the friction developers experience dealing with so many tools and processes. In turn, this allows development teams to concentrate on important tasks like value creation and innovation, instead of getting stuck in complex operations or chasing access. The net result is a modernized end-to-end system, with developers experiencing most of it from the comfort of their IDE.

The Functionality of an Internal Developer Portal

If a Platform is the umbrella then naturally the Portal fits under it. It provides developers with the display tier, a near-real-time view of their software estate, relationships, dependencies and tooling. It typically provides the entry point for golden path provisioning of new applications, via ready-made workflows using something like the scaffolder in Flightdeck.

We think of the software catalog as the most critical part of both the Platform and the Portal. It's kinda hard to manage and maintain software you don't know about or understand the relationships between software and systems. It gives developers a heads-up view to use all the different tools, services, and relevant documents in their environment. By bringing these important resources together, the portal makes the developer experience much better. Often just getting things into the software catalog is the biggest win. Chasing spreadsheets and out-of-date wiki's is a grind nobody wants to deal with.

A good portal should be easy to use and clear. It should show developers a service catalog to find and understand the services and relationships in their environment easily. The documentation should be relevant and easy to read. The whole thing should also be self-service.

This smoother developer experience reduces the time spent looking for information or waiting on help from another team. It builds a culture where developers can be self-reliant and work collaboratively.

Photo by Sigmund on Unsplash

Evaluate Current Developer Experience

Assessing your existing developer experience is crucial. Arguably this should be step one. Identify current cognitive load, workflows, tools and dependencies to gauge interaction complexity, latency and frustration.Where are developers getting stuck or facing impediments? What parts of their workflow just plain sucks? What is manual that could be automated? Map each team's workflow. Odds are that there are some dark corners here and now is likely a good time to fix that.

Admittedly, a wild card that is hard to assess is the cognitive load each team member experiences. This will vary based on their own experience level, understanding, workload and tools. No two people or teams are alike here. Based on the teams expertise and load, you may be able to bite off more or less work.

Lastly, we think ease-of-use is fundamental but ease-of-use maps to existing skills and workflows, not an obscure standard. So involve the teams in decision-making early because the current state of the teams, their learning curves, and level of cognitive load will either hinder or help streamline your journey.

This should give you a toehold to understand user needs and capabilities, which in turn informs where to start.

DevOps or Platform Engineering, Who is Running This?

We often find ourselves discussing whether it is necessary to have a platform engineering practice in place before starting this journey. Plenty of benefits if you do, but it isn't required.

Either the DevOps or Platform Engineering team plays a crucial role during the planning phase. They are probably well-versed on scalability, security, automation, and deployment requirements. Take those results and evaluate if buying or building makes the most sense for each part of the overall platform. We tend to think this ends up being a blitz and blend scenario for most companies.

The underlying infrastructure requirements will inform how to proceed with each component. For instance, some vendors can support complex tenancy and data isolation models while others anticipate you dealing with that yourself. This sort of limitation needs to be assessed against the team's proficiency in infrastructure management, automation tooling, and ability to develop complex add-on elements for the platform. You need to understand where the capabilities and gaps exist here as well.

Moreover, the DevOps or Platform Engineering team's emphasis on best-practices. such as version control, automated testing, and deployment pipelines, helps maintain code quality, reliability, and security within the Internal Developer Platform. Their ability to adapt to evolving technology trends and business requirements ensures that the platform remains agile and responsive to changing needs.

Overall, the DevOps or Platform Engineering team's dedication to driving efficiency, collaboration, and innovation makes them instrumental in shaping a dynamic and developer-friendly environment within an organization.

Selecting the Right Starting Point

Put simply, where is the largest existing pain point? For many companies this lies in the discovery of software and systems. Teams spend a surprising amount of time waiting on access and searching for up-to-date information about the dependencies, tooling, documentation and people that are working collaboratively within the software estate. It is not unusual for this discovery to be a manual processes that results in errors, omissions, or, worse, production defects.

Also evaluate the organization's capacity for change; determining which will be most or least disruptive for your team. Taking a portal-first approach is generally minimally disruptive because it doesn't initially make significant changes to the tools developers work with on a regular basis.

Integrating Existing Developer Tools

A successful platform or portal installation depends on how well it works with your organization's existing investments in tools and workflows. Platform engineers need to look closely at the current development process and plan on an integration strategy. What is the source of truth? What tools need to be integrated with what other tools? And, how are you establishing end-to-end visibility, rather than retrenching silos?

By adding current tools into the new platform, companies can continue to use what they already have. This helps avoid disruptions to existing workflows. For example, they might integrate source control systems like GitHub, CI/CD tools like Jenkins, or Infrastructure as Code solutions like Terraform.

The important part is to plan to carry out the integrations based on a clearly defined end state. Focus on automation and ease-of-use. This will help ensure a smooth change and boost overall productivity.

Getting Buy-in and Driving Adoption

As we've noted it is important to get support from everyone involved. This includes developers, platform engineers/DevOps, and business leaders. You need to clearly explain how the platform will help the business. Show how it will make work easier, enhance the developer experience, and improve overall visibility and understanding of the software estate.

Ever wonder why executives don't talk about software? It's because they can't. The visibility isn't there the same way it is for finance, HR or real estate. That needs to change.

Many companies benefit from having a product manager lead a platform initiative. Shifting to thinking about the internal elements, along the lines of a product, helps understand feedback and drive continual improvement over time. Product managers are used to consolidating feedback and providing vision. This builds trust and helps more people to adopt your collectively-chosen path. Similar to platform engineering, a product manager isn't strictly needed for a portal project but is highly advisable for a platform scaling effort.

Can an Internal Developer Portal evolve into a Platform?

Photo by EJ Yao on Unsplash

Yes, and, admittedly in our biased opinion, this is the smart approach. A portal can be integrated into a comprehensive platform later if the portal has the necessary API's and integrations to be leveraged in this manner. These capabilities help with future integration and automation and, in the end, should be a key criteria for evaluating internal developer portals.

After all, if you can't extend or automate it how long is it going to provide value? We've focused on a GitOps and API-centric approach with what we are building, and we've found most platform engineering teams prefer that to a "ClickOps" approach. This was a deliberate choice, knowing that extending into a broader platform will be cumbersome otherwise.

A portal can start by organizing documents and data and extending situational awareness. Over time, it can add functions like automated deployment pipelines via a scaffolder or integrate with a reporting tool, like Jellyfish, to provide deeper metrics about software engineering. By adding these features, the portal starts to look like a platform, and this is often how that evolution starts.

This change usually depends on what the organization needs and how it grows. As development teams grow and projects get more complicated, use cases beyond the initial portal develop. These features help improve workflows and give better control of the software development process. However, making these changes needs thoughtful planning and careful action to keep everything running smoothly.

Over the coming years we expect most of the existing development toolchain to evolve. More API's and Generative AI will play a huge part, and we've planned this out within our products so our customers will have an easier time making these adaptations. When planning out the evolution from your current starting point to the end state make sure you fit elements of what looks like the distant horizon into the mix. It's coming faster than you may think.

Conclusion & Next Steps

In conclusion, while it is important to know the difference between Internal Developer Platform and Internal Developer Portal, it is equally important to understand aspects of your current developer experience and deployment strategy for new technologies. This knowledge gives a baseline understanding to measure progress against and also helps inform the right tooling. By choosing the right starting point, using the tools you already have, and encouraging people to adopt new systems, you can make your development process more comprehensive while simultaneously reducing toil. As you plan things out, keep in mind that evolving from a Portal to a Platform is possible if you plan well and use your resources wisely. Both support your developers and increase productivity; they just take slightly different approaches.

TL; DR Summary

What is the main difference between an Internal Developer Platform and a Portal?

It is easiest to think of an internal developer platform as a collection of tools and automations that helps development teams automate the entire SDLC. An internal developer portal is primarily the display tier for the internal developer platform. It provides much of the data, via the software catalog, and integration elements needed for improved automation. But, over time, these data points and integrations help to extend into a more comprehensive internal platform offering.

How do I determine which solution best fits my team's needs?

Look at the biggest problems your team has.
Evaluate team capacity and capabilities.
If access, documentation, understanding of the existing software estate, overall developer experience, and/or cognitive load are primary pain points, make an internal developer portal a priority.
If you need to modernize large parts of the SDLC toolchain, simplify workflows and improve automation, then focus on an internal developer platform.

Kubecon Salt Lake City - Themes and Highlights

snowandcaffeine — Wed, 27 Nov 2024 04:08:05 +0000

It should surprise nobody that KubeCon North America this year (2024) was basically - Platform Engineering and AI all the things!

At the start of the week the Arctir team attended both the Backstage and Platform Engineering Co-located events followed by the big show: KubeCon itself.

BackstageCon & Platform Engineering Day (Co-located)

Backstage is moving fast and a lot of exciting changes are coming into (and recently landed in) upstream. The discussions on Tuesday fell into two camps: the contributors and user adoption. On the contributor side a key topic was the new plugin systems: the generally available backend implementation, and the still-alpha-but-getting-really-close frontend system. These are key for our platform and the project itself. Lot's of excitement about the flexibility of the new systems, but a lot of work is still needed in both the frontend system and all the downstream plugins that will consume these APIs.

We've already baked many recent changes into Flightdeck and are actively contributing to upstream. The upstream roadmap covering these topics in depth can be found here: https://backstage.io/docs/overview/roadmap/

On the adoption side we heard from a number of teams that the amount of effort and time-to-value for Backstage adoption was higher than they'd like. Two of the seemingly universal challenges here overlap with topics at Platform Engineering Day: a focus on developer enablement/education, particularly with regard to internal tool variety.

Put simply, how do you get developers bought in, and how do you build a platform around a seemingly endless variety of internal tools that developers use?

An underpinning issue here is developer overload. Folks are still being asked to learn and understand too large of a surface area. It turns out that shift-left doesn't solve everything. Our favorite talk covering some of this came from David Grizzanti at the New York Times:

Another talk pushing the edge on what is possible these days came from Hugo Smitter at FICO:

We are huge fans of both Crossplane and Dapr, so seeing this approach made us smile.

Kubecon Salt Lake City (2024)

As the larger crowd showed up, and the vendor area started getting swarmed, the hallway track built on the themes of the Co-located event with a twist.

Over the past couple of years a large number of companies have been doing more with less in light of layoffs and restructuring. As a result, many had been looking for quick wins and not necessarily large projects, but that sentiment seemed to be turning the corner with a number of large scale projects being floated by conference goers.

A number of those discussions, understandably, included AI deployments and integrations. Top of mind for everyone building an AI platform is NVIDIA, and they had a really good talk about how they maintain their GeForce NOW GPU environment (Ryan Hallisey and Piotr Prokop):

In a similar vein to building environments for AI workloads, this talk by Zoram Thanga and Peter Ableda at Cloudera gives some good considerations around how to think about requirements and approach evolution:

Both AI workloads and Platform Engineering with fairly niche even just 3-4 years ago. It was interesting and encouraging to hear how folks are thinking about how these two, previously-nascent, movements are now coming together.

A number of talks (including those above) cover what we'd define as advanced use cases, but it was clear to us, through our hallway track conversations, that the majority of teams are still very early in this journey.

And, of course, questions surrounding DevOps and Platform Engineering abounded. Is Platform Engineering an evolution of DevOps? A complementary team? Where does SRE fit in?

We have some thoughts on this, and, while there were are ton of fantastic talks on building platforms and improving specific scenarios, most of it was geared towards teams that are fairly far along in their journey. With this in mind, we are working on entry-level education content covering many of these topics, which we anticipate delivering in early 2025.

Save the best for last? We'd be remiss if we didn't mention the Backstage Maintainer talk from Ben Lambert and Patrik Oldsberg at Spotify:

We're really excited about the changes coming to Backstage, and can't wait to integrate (even more of) these changes for our Flightdeck users.

Push vs Pull - How to Succeed with an Internal Developer Platform

snowandcaffeine — Mon, 23 Oct 2023 17:16:31 +0000

Some of the most promising designs and plans software teams have come up with for managing software have struggled to gain adoption and withstand the test of time. Often this is the result of how we ask others to participate. Many times we default to “push”.

So what exactly is “push”? In this context “push” means siloed building in anticipation of future demand while relying on organizational mandates and processes to force a desired behavior. Essentially telling a developer “use this system we built without your input.”

Simple, clear purpose and principles give rise to complex and intelligent behavior. Complex rules and regulations give rise to simple and stupid behavior. - Dee Hock, Founder of Visa

One of the biggest issues we’ve all faced in trying to standardize, scale and automate how we manage systems and software is the attempt to force rules and standards onto people and teams where they don’t fit or are not well received. Mandating that software engineers enter every software component into a CMDB or to craft a Design Document that will never be read is both inefficient and demoralizing.

This frustration invariably gives rise to shadow IT: the deployment of resources outside of the approved system. It’s not going to end well: In the best case your software estate is left in organizational disarray, and in the worst case you are left to deal with attrition of your best people. Because of this, we advise that organizations adopt a “pull” strategy. Organizations should focus on creating a point of immediate value for users of internal platforms. What does your development team want? What do they need? Perhaps that is a software catalog or templates for service creation or something else entirely. Build what they need in a way that is useful for them, where the organization does not mandate usage.

We like to think of Platform Engineering as an exercise in pattern matching. The platform consists of all of the patterns that pave the golden path to production. It defines the least-friction route to successfully bring new or existing code to production. An engineering team can feel confident that their components will be fully supported by the broader engineering organization. And, as more and more patterns are identified (and eventually codified), the organization can quickly adopt more efficient ways to develop code.

This is where “pull” comes in. We’ve found that the organizations that are most successful in their adoption of Platform Engineering principles create a value chain that is continually built upon itself. For these organizations the adoption of the platform and standards outlined therein become inevitable. Not because of mandates, but because development teams are enabled by the platform’s value.

This is why we are so bullish on internal developer platforms. They are a perfect tool at the right time for helping an organization identify patterns, formalize how they may be adopted within the scope of day-to-day development needs, and even for providing insights into how these patterns may be augmented over the course of time.

Centering your internal developer platform around value delivery implies that engineering teams are granted autonomy to deviate from the golden paths defined by the broader organization. Perhaps new toolchains have been introduced, or additional commoditizable platform components have been developed. No matter the reason, a priority for an effective platform engineering team or developer platform project is to continuously codify these emerging organizational patterns.

Just as we believe organizations are best served with a “pull” model when it comes to building their platform, so too do we believe it is critical when leveraging an internal developer portal. Organizations should not be beholden to the features of proprietary solutions. They should use what works for their own needs; continuously evolving their portal to suit the current (and anticipated) needs of their development teams.

We’ve built Arctir with Backstage as the underpinning technology so that our customers can frictionlessly bring a pull model into their organizations. Consume the community components that make sense for your teams. Let us help you fill in the gaps with additional plugins.