<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Brad Micklea</title>
    <description>The latest articles on DEV Community by Brad Micklea (@bmicklea).</description>
    <link>https://dev.to/bmicklea</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F845726%2F872952ab-88e4-49c7-b552-fe131be9751d.png</url>
      <title>DEV Community: Brad Micklea</title>
      <link>https://dev.to/bmicklea</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/bmicklea"/>
    <language>en</language>
    <item>
      <title>What’s Next for the KitOps Project</title>
      <dc:creator>Brad Micklea</dc:creator>
      <pubDate>Thu, 23 Jan 2025 15:51:40 +0000</pubDate>
      <link>https://dev.to/jozu/whats-next-for-the-kitops-project-279d</link>
      <guid>https://dev.to/jozu/whats-next-for-the-kitops-project-279d</guid>
      <description>&lt;h2&gt;
  
  
  KitOps 1.0: Production Proven
&lt;/h2&gt;

&lt;p&gt;Today is an important milestone for the KitOps open source project as we release version 1.0. This isn’t just about new features (although automatically creating a ModelKit from Hugging Face or a scanned directory is exciting), it’s about the start of a journey for every KitOps user. KitOps is downloaded thousands of times a week and is used in production by global enterprises, secure government agencies, and a slew of smaller (but equally important) organizations.&lt;/p&gt;

&lt;p&gt;This 1.0 says KitOps has been proven production and by some of the most demanding organizations and it’s ready for whatever you can throw at it!&lt;/p&gt;

&lt;p&gt;Of course there are new features too thanks to our awesome community of developers, authors, and feedback providers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Generate a ModelKit by importing an existing Hugging Face repository (no need to edit the Kitfile)
&lt;/li&gt;
&lt;li&gt;Generate a Kitfile automatically by scanning a directory structure
&lt;/li&gt;
&lt;li&gt;The config object for ModelKits now stores digests with each layer, to make it easier to access a specific layer's blob
&lt;/li&gt;
&lt;li&gt;Layers are now packed to include directories relative to the context directory, making unpacking tarballs simpler
&lt;/li&gt;
&lt;li&gt;Tail logs with the &lt;code&gt;-f&lt;/code&gt; flag
&lt;/li&gt;
&lt;li&gt;The Kitfile &lt;code&gt;model&lt;/code&gt; object now has a &lt;code&gt;format&lt;/code&gt; field so you can be explicit about the model’s serialization format (e.g., GGUF, ONNX, etc…)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Despite the major version change, KitOps 1.0 is backward compatible.&lt;/p&gt;

&lt;h2&gt;
  
  
  Before What’s Next, What Started It?
&lt;/h2&gt;

&lt;p&gt;KitOps started with a simple question:&lt;/p&gt;

&lt;p&gt;“why isn’t there an open standard for packaging an AI/ML project so that it can be imported or exported into any tool or environment?”&lt;/p&gt;

&lt;p&gt;Every AI/ML tool has its own packaging format and approach, making moving projects from tool-to-tool (or worse yet, replacing one vendor’s tool with another) less fun than Carrie’s high school prom. So why wasn’t there a standard already? The answer was because tool and infrastructure vendors don’t like choice - they want customers locked into their formats and tools. Sounds great to investors, but ultimately it’s always a losing strategy because users will always need choices. That’s why PDF, GIF, Markdown, and other open standards are…well standards!&lt;/p&gt;

&lt;p&gt;Luckily, the original KitOps authors were all readers of &lt;a href="https://xkcd.com/927/" rel="noopener noreferrer"&gt;xkcd&lt;/a&gt;, so we &lt;em&gt;didn’t&lt;/em&gt; create a 15th standard, instead we built KitOps to allow teams to share and reproduce their AI/ML projects using the existing OCI standard (the container one). But…we didn’t want to just jam models, datasets, codebases, and documentation into a single container image because not everyone needs everything to do their part of the project lifecycle.&lt;/p&gt;

&lt;p&gt;Instead we packaged everything as an OCI Artifact called a ModelKit that includes models, datasets, codebases, and documentation in separate layers. We then built the Kit CLI to make it trivial for people or pipelines to pull only those layers they need to do their job.&lt;/p&gt;

&lt;p&gt;Teams using KitOps are seeing big improvements in security and reduction of risk, plus a 31.9% decrease in the time it takes to take an AI/ML project from conception to completion:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;KitOps stores, versions, and secures their AI/ML projects in the same place their containers are already kept with no additional infrastructure (everyone has a container registry already)
&lt;/li&gt;
&lt;li&gt;KitOps ModelKits are tamper-proof, secure, and can be digitally signed
&lt;/li&gt;
&lt;li&gt;KitOps ModelKits can be used with existing MLOps and DevOps tools without any changes
&lt;/li&gt;
&lt;li&gt;KitOps selective unpacking reduces time and space needed to deploy and reproduce AI/ML projects
&lt;/li&gt;
&lt;li&gt;KitOps packages are easy to turn into runnable containers or Kubernetes deployments&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What’s Next for KitOps?
&lt;/h2&gt;

&lt;p&gt;KitOps was part of a larger goal - to establish an open standard for packaging AI/ML projects that will be adopted by all vendors. We’ve made two important steps on that road:&lt;/p&gt;

&lt;p&gt;*&lt;em&gt;1/ Drafted a CNCF-sanctioned model packaging specification *&lt;/em&gt;&lt;br&gt;
The KitOps maintainers along with members of Red Hat, ANT Group, ByteDance, and Jozu have discussed and drafted a new specification that would give OCI registries and runtimes a consistent format and interfaces for handling AI/ML projects. This CNCF standard would simplify sharing and reproducing AI/ML projects in Kubernetes and its associated projects.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2/ Submitted KitOps to the CNCF as a sandbox project&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
We have submitted KitOps as a sandbox project to the CNCF, and expect to have it approved in Q1 2025. This will make KitOps community property and make it simpler for other vendors to add KitOps integrations in their products.&lt;/p&gt;

&lt;p&gt;It’s always hard letting something go out into the world that you’ve built with love and care, but we’re more excited to see how you, the AI/ML community, adopt and grow KitOps!&lt;/p&gt;

&lt;p&gt;-Brad Micklea, KitOps Project Lead &amp;amp; Jozu CEO&lt;/p&gt;

&lt;h3&gt;
  
  
  Join the KitOps community
&lt;/h3&gt;

&lt;p&gt;For support, release updates, and general KitOps discussion, please join the &lt;a href="https://discord.gg/Tapeh8agYy" rel="noopener noreferrer"&gt;KitOps Discord&lt;/a&gt;. Follow &lt;a href="https://twitter.com/Kit_Ops" rel="noopener noreferrer"&gt;KitOps on X&lt;/a&gt; for daily updates.  &lt;/p&gt;

&lt;p&gt;If you need help there are several ways to reach our community and &lt;a href="https://github.com/jozu-ai/kitops/blob/main/MAINTAINERS.md" rel="noopener noreferrer"&gt;Maintainers&lt;/a&gt; outlined in our &lt;a href="https://github.com/jozu-ai/kitops/blob/main/SUPPORT.md" rel="noopener noreferrer"&gt;support doc&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Reporting Issues, Suggesting Features, and Contributing
&lt;/h3&gt;

&lt;p&gt;Your insights help KitOps evolve as an open standard for AI/ML. We &lt;em&gt;deeply value&lt;/em&gt; the issues and feature requests we get from users in our community. To contribute your thoughts,navigate to the Issues tab and hitting the New Issue green button. Our templates guide you in providing essential details to address your request effectively.&lt;/p&gt;

&lt;p&gt;We love our KitOps community and contributors. To learn more about the many ways you can contribute (you don't need to be a coder) and how to get started see our &lt;a href="https://github.com/jozu-ai/kitops/blob/main/CONTRIBUTING.md" rel="noopener noreferrer"&gt;Contributor's Guide&lt;/a&gt;. Please read our &lt;a href="https://github.com/jozu-ai/kitops/blob/main/GOVERNANCE.md" rel="noopener noreferrer"&gt;Governance&lt;/a&gt; and our &lt;a href="https://github.com/jozu-ai/kitops/blob/main/CODE-OF-CONDUCT.md" rel="noopener noreferrer"&gt;Code of Conduct&lt;/a&gt; before contributing.&lt;/p&gt;

&lt;h3&gt;
  
  
  A Community Built on Respect
&lt;/h3&gt;

&lt;p&gt;At KitOps, inclusivity, empathy, and responsibility are at our core. Please read our &lt;a href="https://github.com/jozu-ai/kitops/blob/main/CODE-OF-CONDUCT.md" rel="noopener noreferrer"&gt;Code of Conduct&lt;/a&gt; to understand the values guiding our community.&lt;/p&gt;

&lt;h3&gt;
  
  
  Roadmap
&lt;/h3&gt;

&lt;p&gt;We &lt;a href="https://github.com/jozu-ai/kitops/blob/main/ROADMAP.md" rel="noopener noreferrer"&gt;share our roadmap openly&lt;/a&gt; so anyone in the community can provide feedback and ideas. Let us know what you'd like to see by pinging us on Discord or creating an issue. &lt;/p&gt;

</description>
      <category>opensource</category>
      <category>programming</category>
      <category>ai</category>
      <category>news</category>
    </item>
    <item>
      <title>KitOps: A Practical Approach to Accelerating AI/ML Development to Production</title>
      <dc:creator>Brad Micklea</dc:creator>
      <pubDate>Wed, 02 Oct 2024 14:12:56 +0000</pubDate>
      <link>https://dev.to/jozu/kitops-a-practical-approach-to-accelerating-aiml-development-to-production-2hoh</link>
      <guid>https://dev.to/jozu/kitops-a-practical-approach-to-accelerating-aiml-development-to-production-2hoh</guid>
      <description>&lt;p&gt;Every company and executive is talking about accelerating their success with AI/ML. But between the aspiration and reality is a huge gulf. For large tech companies they’re spending their way across: hiring data scientists, ML engineers, MLOps practitioners, and a host of other &lt;em&gt;very&lt;/em&gt; high-paid roles and putting them in new divisions or AI Centres of Excellence (AI CoEs).&lt;/p&gt;

&lt;p&gt;This new CoE looks suspiciously like the software engineering organization that it sits beside. This shouldn’t be surprising because many of the problems plaguing AI/ML development are problems that faced software development years ago: how to create effective collaboration, how to reproduce experiments across different teams and environments, how to secure projects, how to know where IP came from...&lt;/p&gt;

&lt;p&gt;The idea that companies need a whole new set of teams starting from the ground up to solve these issues for AI/ML is...odd. Why can’t we use the solutions learned from years of software development?&lt;/p&gt;

&lt;p&gt;We can...and should.&lt;/p&gt;

&lt;p&gt;The trick is identifying which parts of the existing infrastructure, tools, and processes can be adapted to work with AI/ML and which genuinely require a new approach.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;em&gt;AI/ML Project Packaging and Versioning&lt;/em&gt;
&lt;/h2&gt;

&lt;p&gt;There is no standard for packaging and versioning the various artifacts needed to reproduce an AI/ML project:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Models in Jupyter notebooks or MLOps tools
&lt;/li&gt;
&lt;li&gt;Datasets in data lakes, databases, or file systems
&lt;/li&gt;
&lt;li&gt;Code in Git repositories
&lt;/li&gt;
&lt;li&gt;Metadata (such as hyperparameters, features, and weights) scattered across different storage systems&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That’s insane - in every other area of technology there are these basic standards so teams can use different tools while collaborating: containers, PDFs, TARs, there are hundreds of other examples.&lt;/p&gt;

&lt;p&gt;Without this, the already challenging job of managing the lifecycle of AI projects is even harder, and more likely to fail.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;We created KitOps to solve this problem. It’s an open-source, standards-based packaging and versioning system designed for AI/ML projects. KitOps takes advantage of existing software standards so the tools and processes your DevOps / SRE teams use with their containerized applications, can be used with AI/ML projects.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;KitOps allows AI teams to package AI/ML models, datasets, code, and metadata into what’s called a &lt;strong&gt;ModelKit&lt;/strong&gt;. This ModelKit is a versioned bundle that can be sent to staging, user acceptance testing (UAT), or production. For DevOps and SRE teams it means they can manage AI models like any other production asset, such as containers or microservices.&lt;/p&gt;

&lt;p&gt;For companies that are using containers and enterprise registries, KitOps is a seamless fit. KitOps ModelKits use the OCI standard - because it doesn’t make sense to create a unique standard when an existing one works. With KitOps DevOps and SRE teams manage AI artifacts like models, datasets, code, and metadata from the same registry where every other artifact is.&lt;/p&gt;

&lt;p&gt;Let’s dive into how KitOps can streamline AI/ML operations and solve some of the unique challenges faced by companies adopting machine learning.&lt;/p&gt;

&lt;h3&gt;
  
  
  Level 1: Simplifying the Handoff from Development to Production
&lt;/h3&gt;

&lt;p&gt;With KitOps integrated into your CI/CD pipeline, you can automate the deployment of AI models, either manually triggered by the model development team or automatically when changes are made in the model or its associated artifacts. Here’s why this matters:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Unified Operations&lt;/strong&gt;: All assets are in one place, making it easier for operations teams to test, deploy, audit, and manage AI workloads.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Compliance and Security&lt;/strong&gt;: By keeping AI versioned packages in the same enterprise registry as other production assets, they’re easier to secure and audit, which is crucial for compliance purposes.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Vendor Independence&lt;/strong&gt;: Using the OCI standard protects your company from vendor lock-in, giving you flexibility when negotiating with vendors and adapting your MLOps or serving infrastructure as your needs evolve.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For companies dipping their toes into AI/ML, this is the first step with KitOps, but many find themselves wanting to extend its use.&lt;/p&gt;

&lt;h3&gt;
  
  
  Level 2: Adding Security to the Mix
&lt;/h3&gt;

&lt;p&gt;Security is a top concern, especially when dealing with sensitive data or compliance requirements like GDPR or HIPAA. KitOps, in combination with the open-source &lt;a href="https://github.com/protectai/modelscan" rel="noopener noreferrer"&gt;ModelScan&lt;/a&gt; tool, allows your team to scan models for vulnerabilities before they’re promoted beyond development and package them in a tamper-proof ModelKit.&lt;/p&gt;

&lt;p&gt;Security-conscious teams can create &lt;strong&gt;curated ModelKits&lt;/strong&gt; by pulling trusted models from public repositories like Hugging Face, scanning them, and storing them as ModelKits in the enterprise registry. This guarantees that only tamper-proof, verified models are used within the organization. This level of security isn’t just important for peace of mind — it’s increasingly a requirement in industries subject to strict regulatory oversight.&lt;/p&gt;

&lt;p&gt;Once a model passes scanning, it’s packaged and signed as a ModelKit, ensuring that it can’t be tampered with on its way to production.&lt;/p&gt;

&lt;h3&gt;
  
  
  Level 3: Full Lifecycle Management with KitOps
&lt;/h3&gt;

&lt;p&gt;For companies looking to mature their AI operations or meet stringent compliance standards (like in the European Union), KitOps can be used throughout the entire AI project lifecycle. Instead of waiting until production, you can start using ModelKits in development.&lt;/p&gt;

&lt;p&gt;This approach solves many common pain points:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Unified Storage&lt;/strong&gt;: All AI/ML project artifacts are stored as OCI-compliant objects, ensuring a consistent and standardized process across the organization.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Collaboration Across Teams&lt;/strong&gt;: Since data, AI/ML, and software teams work in different tools, using KitOps ensures that they can share and collaborate on artifacts securely and efficiently without compromising on their chosen toolset.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tamper-Proof Artifacts&lt;/strong&gt;: By storing development artifacts as ModelKits, you protect them from accidental or malicious tampering — an increasingly common concern in AI development.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Best of all, this process can be automated using the KitOps CLI, meaning your team won’t have to manually track every update or model change.&lt;/p&gt;

&lt;h3&gt;
  
  
  A Real-World Example: KitOps in Action
&lt;/h3&gt;

&lt;p&gt;Let’s walk through how a typical company might use KitOps from development to production.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Development&lt;/strong&gt;: Data scientists work in Jupyter notebooks, but save their work as a ModelKit at every milestone using simple KitOps commands:
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;kit pack . -t registry.gitlab.com/chatbot/legalchat:tuned  &lt;/p&gt;

&lt;p&gt;kit push registry.gitlab.com/chatbot/legalchat:tuned  &lt;/p&gt;

&lt;p&gt;This ensures that all models, datasets, and code are versioned and stored in a secure, centralized location.  &lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Integration&lt;/strong&gt;: The application team pulls the updated ModelKit:
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;kit pull registry.gitlab.com/chatbot/legalchat:tuned  &lt;/p&gt;

&lt;p&gt;They test service integration, paying attention to performance and any changes between model versions.  &lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Testing&lt;/strong&gt;: Once integration is complete, an engineer validates the model by running it with the included validation dataset:
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;kit unpack registry.gitlab.com/chatbot/legalchat:tuned --model --datasets  &lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Deployment&lt;/strong&gt;: When ready for production, the SRE team tags the ModelKit as &lt;code&gt;challenger&lt;/code&gt; and pushes it into the CI/CD pipeline for deployment:
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;kit tag registry.gitlab.com/chatbot/legalchat:tuned registry.gitlab.com/chatbot/legalchat:challenger  &lt;/p&gt;

&lt;p&gt;kit push registry.gitlab.com/chatbot/legalchat:challenger  &lt;/p&gt;

&lt;p&gt;The deployment team monitors the new &lt;code&gt;challenger&lt;/code&gt; model in production, and if it’s successful, they re-tag it as &lt;code&gt;champion&lt;/code&gt;. If there are any issues, the previous model can be rolled back with minimal friction.&lt;/p&gt;

&lt;h3&gt;
  
  
  Conclusion: Why KitOps Should Be On Your Radar
&lt;/h3&gt;

&lt;p&gt;Adopting AI/ML doesn’t have to mean adding new teams and infrastructure. KitOps offers a structured, secure, and scalable solution for managing the entire lifecycle of AI projects. It integrates seamlessly with existing container infrastructure, uses open standards, and provides critical security and auditing features that will help your team stay compliant and efficient.  &lt;/p&gt;

</description>
      <category>programming</category>
      <category>beginners</category>
      <category>ai</category>
      <category>devops</category>
    </item>
    <item>
      <title>Accelerating into AI: Lessons from AWS</title>
      <dc:creator>Brad Micklea</dc:creator>
      <pubDate>Wed, 12 Jun 2024 12:17:18 +0000</pubDate>
      <link>https://dev.to/jozu/accelerating-into-ai-lessons-from-aws-4cl2</link>
      <guid>https://dev.to/jozu/accelerating-into-ai-lessons-from-aws-4cl2</guid>
      <description>&lt;p&gt;One of the hallmarks of the best businesses is that they move fast, and consistently get the long-term strategy right. While running Amazon API Gateway, I was struck by the interplay between two of their leadership principles:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Bias For Action&lt;/li&gt;
&lt;li&gt;Right, A Lot&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To radically oversimplify: “when in doubt, start doing...but remember that huge impact come from getting the long term bets right.”&lt;/p&gt;

&lt;p&gt;AWS made the early big bet that utility computing in the cloud would change the face of software and IT. They moved fast, but never lost sight of that goal and were rewarded for it. Having spent time with the people who were there in the early days, that path was hard and uncertain because the easiest answers rarely worked for what they were building - something which is always true for big changes.&lt;/p&gt;

&lt;p&gt;I’ve been reminded of this while speaking to people about how AI is being adopted in enterprises.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Enterprise AI Divide
&lt;/h2&gt;

&lt;p&gt;Today there is a sharp divide - many organizations are taking a wait-and-see approach, but a few enterprises are building out internal AI/ML development teams that will train, tune, and manage AI models and agents tailored to their business and customers.&lt;/p&gt;

&lt;p&gt;These companies are making a long-term bet that AI will be a market changer and that those with in-house AI skills will win. Looking at previous seismic shifts like the internet and mobility, they’re likely right.&lt;br&gt;
For now, though, the road they’re walking is difficult. They’re struggling to:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Choose amongst 1,000 MLOps tools that have no standards, never work with each other, and are periodically abandoned&lt;/li&gt;
&lt;li&gt;Find and hiring strong people in not-yet-well-defined job areas like AI research, data science, ML engineering, and MLOps&lt;/li&gt;
&lt;li&gt;Establish processes to keep AI projects moving quickly and safely, without compromising enterprise data privacy and compliance regulations&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The simpler route is to use public LLMs from OpenAI, Google, Mistral, or Perplexity and simply avoiding any use cases that touch sensitive data. That’s a reasonable place to start, but it won’t be where the best companies end up, because the greatest customer impact comes from using the deepest data.&lt;/p&gt;

&lt;h2&gt;
  
  
  Too Many Tools!
&lt;/h2&gt;

&lt;p&gt;There are a host of reasons why companies are struggling to move AI projects out of the pilot phase, from a lack of strategic clarity, to worries about hallucinations, to a lack of tooling and talent. But according to a recent McKinsey study “too many tools” was the biggest reason.&lt;/p&gt;

&lt;p&gt;Focus on tools with open standards; protect yourself from vendor changes.&lt;/p&gt;

&lt;p&gt;In many cases each group, team, or division has selected a set of tools, an LLM, and started on prototypes. This is a fine way to start (after all, n solutions are better than 0 solutions), but to protect the organization there should be a focus on standards and flexibility. The hard truth is that many AI/ML tool vendors will disappear along with their tools. Focusing on standards is great protection. Unfortunately today’s crop of MLOps tools have chosen to focus on proprietary options over standards. Instead, look for solutions that let you leverage the standards in other parts of your software development chain: Terraform for infrastructure-as-code, AI project storage in your OCI registry, and open source LLMs are a good start. There are other open source projects that might help too:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.pachyderm.com/" rel="noopener noreferrer"&gt;Pachyderm&lt;/a&gt; is an open source platform for managing datasets and the workflow around their cleaning and changes.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/feast-dev/feast" rel="noopener noreferrer"&gt;Feast&lt;/a&gt; is a feature store to help teams track changes during in-house model development.&lt;/li&gt;
&lt;li&gt;
&lt;a href="http://comet.ml" rel="noopener noreferrer"&gt;CometML&lt;/a&gt; and ml&lt;a href="http://mlflow.org" rel="noopener noreferrer"&gt;MLFlow&lt;/a&gt; are popular development and experimentation tools, although some express concerns about their proprietary and weak data storage with its lack of tamper-proof guarantees.&lt;/li&gt;
&lt;li&gt;
&lt;a href="http://kitops.ml" rel="noopener noreferrer"&gt;KitOps&lt;/a&gt; lets you store all your AI project artifacts (models, datasets, parameters, code, and documentation) in a tamper-proof and versionable package you can store and share through your existing OCI / container registry. It can help protect you from costly migration work if you need to change part of your toolchain.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.kubeflow.org/" rel="noopener noreferrer"&gt;KubeFlow&lt;/a&gt; simplifies the task of deploying, running, and managing ML models on Kubernetes… which you’re probably already using.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;While it’s more “exciting” to focus on getting a chatbot deployed and playing with it and your data, companies that focus on building a repeatable, fast, and safe workflow will be able to learn faster, deploy faster, and beat their competitors. A solid and enterprise-approved AI project development lifecycle and toolset should be the first big milestone in any company’s journey with AI.&lt;/p&gt;

&lt;p&gt;What separates the best is the operational maturity around the AI project.&lt;/p&gt;

&lt;h2&gt;
  
  
  A Framework for Selecting AI Projects
&lt;/h2&gt;

&lt;p&gt;Once you have a solid foundation, then the fight about which projects to focus on begins (I know, it began long ago…). They key is prioritizing them based on customer value and risk avoidance.&lt;/p&gt;

&lt;p&gt;I’ve used the following framework to prioritize projects because it divides potential AI use cases into four quadrants based on their customer value and organizational risk.&lt;/p&gt;

&lt;h2&gt;
  
  
  A Framework for Prioritizing Enterprise AI Use Cases
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjozu.com%2Fwp-content%2Fuploads%2F2024%2F06%2FScreenshot-2024-06-11-at-8.33.09%25E2%2580%25AFAM-1024x663.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjozu.com%2Fwp-content%2Fuploads%2F2024%2F06%2FScreenshot-2024-06-11-at-8.33.09%25E2%2580%25AFAM-1024x663.png" alt="A Framework for Prioritizing Enterprise AI Use Cases" width="800" height="517"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The X-axis focuses on your customers and the amount of value they’d get from an AI-driven use case (which in a good business equates to an increase in value for your organization).&lt;/p&gt;

&lt;p&gt;The Y-axis is about your business and the amount of risk (severity and likelihood) that would result from a failure or problem with the AI-driven solution.&lt;br&gt;
The four quadrants are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Now:&lt;/strong&gt; These projects have high value but low risk so start work here. You can build MVPs using hosted LLMs’ APIs, but ultimately you want this to be handled in-house - so after the MVP is launched, use these projects as a testing ground for an in-house AI team. Can they train a model that works better, faster, and cheaper?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Next:&lt;/strong&gt; This is where your differentiation and long-term value will be unlocked. However, the risk is high so you can’t offload this to a public LLM, it will need to be built and managed in-house. This isn’t where you start, but once an in-house team has proven themselves, it’s where they need to go next.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Later:&lt;/strong&gt; This area is dangerous because it can be a distraction and resource suck. It looks appealing because it’s easy, but the customer value is low so unless you need to do something here to keep up with a competitor (and are actually losing deals because of it), then keep teams focused on the high value projects.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Stop:&lt;/strong&gt; The risk / value equation is wrong. Don’t touch these unless that materially changes.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Over time use cases will shift as their value or risk profile changes. Re-classify projects every 6-12 months.&lt;/p&gt;

&lt;p&gt;For mid-to-large enterprises, most use cases will be at the top two quadrants, because valuable data is almost always sensitive. Again, these are areas where you shouldn’t be outsourcing to a public LLM because of data privacy, but also because you will be helping your competitors by training a public model that they may be using on your unique data.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting Up an AI Product Team
&lt;/h2&gt;

&lt;p&gt;Once you decide you need an internal AI team focused on your products, how do you make it happen?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Make sure the engineering and business milestones are aligned and realistic&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Before starting, ensure you have clarity on goals and milestones (I prefer writing a document to slides, but different company cultures will dictate what is best).&lt;br&gt;
Define the Mission and Impact: It's important to establish clear goals that directly map them to the organization's business problems. Include what things the group won’t do that people may expect. Details matter here, so this should include metrics and milestones with dates for accountability. If time will be needed for experimentation (it often is with AI projects) then be clear what the milestones of learning will be along the way and treat them like any other deadline.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Secure Executive Sponsorship:&lt;/strong&gt; With a strong mission and set of objectives, gaining executive support shouldn’t be difficult. However, executives are busy so practice your “pitch” and keep it short. Executives learn by asking questions to treat your pitch as a movie trailer, not as the movie and give them time to question.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Create a Roadmap:&lt;/strong&gt; It’s tempting to do this as part of the mission and impact, but in most cases, only a high level roadmap is needed at that point. Once you have executive sponsorship your mission may shift. This roadmap should go a level or two deeper but align to the same milestones you agreed to when getting sponsorship. Classifying opportunities based on the framework above keeps things simple for stakeholders and consistent with how your team will execute.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Emphasize Agile and Iterative Approaches:&lt;/strong&gt; AI is changing at a rapid pace and no one knows the future. Don’t give an impossible sense of determinism to your future plans, instead focus on how the team will be built to adapt and iterate quickly (faster than competitors).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Creating compelling but realistic milestones can be tricky. Below is just an example, but shows how you can mix engineering milestones (we’re going to get faster, safer, and smoother over time so we can win long-term) and business milestones (we solved a customer problem faster than expected).&lt;/p&gt;

&lt;p&gt;Be up-front that your roadmap will need to balance engineering maturity and business impact milestones&lt;/p&gt;

&lt;h2&gt;
  
  
  Choosing the Organization Structure
&lt;/h2&gt;

&lt;p&gt;There are several ways to set up an AI team. The best teams, regardless of their specialization, do their best work when they are connected and focused on the customer benefit. That means that they need to be close to the “edge” where the customer is.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjozu.com%2Fwp-content%2Fuploads%2F2024%2F06%2FScreenshot-2024-06-11-at-8.32.51%25E2%2580%25AFAM-1024x312.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjozu.com%2Fwp-content%2Fuploads%2F2024%2F06%2FScreenshot-2024-06-11-at-8.32.51%25E2%2580%25AFAM-1024x312.png" alt="Be up-front that your roadmap will need to balance engineering maturity and business impact milestones" width="800" height="243"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Be up-front that your roadmap will need to balance engineering maturity and business impact milestones&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Don’t over-staff a central AI team, you need “doers” close to the customer.&lt;/p&gt;

&lt;h3&gt;
  
  
  Centralized Model
&lt;/h3&gt;

&lt;p&gt;Centralized works for small organizations with a single product, where decisions are made centrally. In this case the organization is small enough that everyone in the product organization should know the customer and their use cases, including the AI team. They should be included in key meetings not only to report on their progress, but to learn about other teams’ successes and struggles that they might learn from.&lt;/p&gt;

&lt;h3&gt;
  
  
  Hub and Spoke Model
&lt;/h3&gt;

&lt;p&gt;Hub and Spoke is best for larger organizations where there are multiple products and customers, and where decisions are made independently. In this model there’s a natural division of responsibility:&lt;/p&gt;

&lt;p&gt;The Spokes sit in each business area and should feel (and be treated) as a core part of that team. Their priorities should be set by the business, not the hub because they are closest to the product and customer. They are also closest to the ground and should be able to provide valuable data and insights back to the hub. There isn’t a lot of ambiguity in this realm, making it good for new and experienced team members.&lt;/p&gt;

&lt;p&gt;The Hub provides standards and tools that will elevate every team member in the spokes. Their primary customers are the spokes so they should listen to them and enable them. This job is hard because it’s tempting for this team to try and move more and more into standards and then push those standards down. Instead, they should listen for common problems across spokes and decide if a shared solution would benefit all. The hub is also responsible for creating a consistent career path and performance expectation. There is much more ambiguity and balance needed in the hub roles so it’s a better place for more senior and experienced team members, or for people who excel in solving ambiguous problems.&lt;/p&gt;

&lt;p&gt;One challenge with this model is that it can get "center heavy," pulling resources from the spokes which have the greatest customer impact. To prevent this, I question situations where the number of people in the hub is &amp;gt;30% of the number of people in the spokes because it can indicate an imbalance.&lt;/p&gt;

&lt;h2&gt;
  
  
  Seizing the AI Opportunity
&lt;/h2&gt;

&lt;p&gt;We are in the early stages of an AI revolution in the enterprise. Organizations that take this time to work through the challenges will be rewarded with a competitive advantage in the future. OpenAI and Google might get you started, but they won’t solve your AI problems for you - it’s time to take the reins yourself.&lt;/p&gt;

</description>
      <category>programming</category>
      <category>ai</category>
      <category>devops</category>
      <category>aws</category>
    </item>
    <item>
      <title>Why enterprise AI projects are moving too slowly</title>
      <dc:creator>Brad Micklea</dc:creator>
      <pubDate>Mon, 22 Apr 2024 14:49:06 +0000</pubDate>
      <link>https://dev.to/jozu/why-enterprise-ai-projects-are-moving-too-slowly-29ik</link>
      <guid>https://dev.to/jozu/why-enterprise-ai-projects-are-moving-too-slowly-29ik</guid>
      <description>&lt;p&gt;In AI projects the biggest (and most solvable) source of friction are the handoffs between data scientists, application developers, testers, and infrastructure engineers as the project moves from development to production. This friction exists at every company size, in every industry, and every vertical. Gartner’s research shows that AI/ML projects are rarely deployed in under 9 months despite the use of ready-to-go large language models (LLMs) like Llama, Mistral, and Falcon.&lt;/p&gt;

&lt;p&gt;Why do AI/ML projects move so much slower than other software projects? It’s not for lack of effort or lack of focus - it’s because of the huge amount of friction in the AI/ML development, deployment, and operations life cycle.&lt;/p&gt;

&lt;h2&gt;
  
  
  AI/ML isn’t just about the code
&lt;/h2&gt;

&lt;p&gt;A big part of the problem is that AI/ML projects aren’t like other software projects. They have a lot of different assets that are held in different locations. &lt;a href="https://github.com/jozu-ai/kitops" rel="noopener noreferrer"&gt;Until now&lt;/a&gt;, there hasn't been a standard mechanism to package, version, and share these assets in a way that is accessible to data science and software teams alike. Why?&lt;/p&gt;

&lt;p&gt;It’s tempting to think of an AI project as “just a model and some data” but it’s far more complex than that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Model code&lt;/li&gt;
&lt;li&gt;Adapter code&lt;/li&gt;
&lt;li&gt;Tokenizer code&lt;/li&gt;
&lt;li&gt;Training code&lt;/li&gt;
&lt;li&gt;Training data&lt;/li&gt;
&lt;li&gt;Validation data&lt;/li&gt;
&lt;li&gt;Configuration files&lt;/li&gt;
&lt;li&gt;Hyperparameters&lt;/li&gt;
&lt;li&gt;Model features&lt;/li&gt;
&lt;li&gt;Serialized models&lt;/li&gt;
&lt;li&gt;API interface&lt;/li&gt;
&lt;li&gt;Embedding code&lt;/li&gt;
&lt;li&gt;Deployment definitions&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Parts of this list are small and easily shared (like the code through git). But others can be massive (the datasets and serialized models), or difficult to capture and contextualize (the features or hyperparameters) for non-data science team members.&lt;/p&gt;

&lt;p&gt;Making it worse is the variety of storage locations and lack of cross-artifact versioning:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Code in git&lt;/li&gt;
&lt;li&gt;Datasets in DvC or cloud storage like AWS S3&lt;/li&gt;
&lt;li&gt;Features and hyperparameters in ML training and experimentation tools&lt;/li&gt;
&lt;li&gt;Serialized models in a container registry&lt;/li&gt;
&lt;li&gt;Deployment definitions in separate repos&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Keeping track of all these assets (which may be unique to a single model, or shared with many models) is tricky...&lt;/p&gt;

&lt;p&gt;Which changes should an application or SRE team be aware of? &lt;/p&gt;

&lt;p&gt;How do you track the provenance of each and ensure they weren’t accidentally or purposefully tampered with?&lt;/p&gt;

&lt;p&gt;How do you control  access and guarantee compliance?&lt;/p&gt;

&lt;p&gt;How does each team know when to get involved?&lt;/p&gt;

&lt;p&gt;It’s almost impossible to have good cross-team coordination and collaboration when people can’t find the project’s assets, don’t know which versions belong together, and aren’t notified of impactful changes.&lt;/p&gt;

&lt;p&gt;I can hear you saying... “but people have been developing models for years...there must be a solution!”&lt;/p&gt;

&lt;p&gt;Kind of. Data scientists haven't felt this issue too strongly because they all use Jupyter notebooks. But…&lt;/p&gt;

&lt;h2&gt;
  
  
  Jupyter notebooks are great...and terrible
&lt;/h2&gt;

&lt;p&gt;Data scientists work in Jupyter notebooks because they work perfectly for &lt;em&gt;experimentation&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;But you can’t easily extract the code or data from a notebook, and it’s not clear for a non-data scientist where the features, parameters, weights, and biases are in the notebook. Plus, while a data scientist can run the model in the notebook on their machine, it doesn't generate a sharable and runnable model that non-data science teams can use.&lt;/p&gt;

&lt;p&gt;Notebooks are perfect for early development by data scientists, but they are a walled garden, and one that engineers can’t use.&lt;/p&gt;

&lt;h2&gt;
  
  
  What about containers?
&lt;/h2&gt;

&lt;p&gt;Unfortunately, getting a model that works offline on a data scientist’s machine to run in production isn’t as simple as dropping it into a container.&lt;/p&gt;

&lt;p&gt;That’s because the model created by a data science team is best thought of as a prototype. It hasn’t been designed to work in production at scale.&lt;/p&gt;

&lt;p&gt;For example, the features it uses may take too long to calculate in production. Or the libraries it uses may be ideally suited to the necessary iterations of development but not for the sustained load of production. Even something as simple as matching package versions in production may take hours or days of work.&lt;/p&gt;

&lt;p&gt;We haven't even touched on the changes that are likely needed for logging and monitoring, continuous training, and deployment pipelines that include a feedback loop mechanism.&lt;/p&gt;

&lt;p&gt;Completing the model is half the job, and if you’re waiting until the model is done to start thinking about the operational needs you’ll likely lose weeks and have to redo parts of the model development cycle several times.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bridging the divide between data science and operations
&lt;/h2&gt;

&lt;p&gt;In my previous roles at Red Hat and Amazon Web Services, I faced a dilemma familiar in many tech organizations: an organizational separation between data science and operations teams.&lt;/p&gt;

&lt;p&gt;As much as the data scientists were wizards with data, their understanding of deploying and managing applications in a production environment was limited. Their AI projects lacked crucial production elements like packaging and integration, which led to frequent bottlenecks and frustrations when transitioning from development to deployment.&lt;/p&gt;

&lt;p&gt;The solution was not to silo these teams but to integrate them. By embedding data scientists directly into application teams, they attended the same meetings, shared meals, and naturally understood that they (like their colleagues) were responsible for the AI project’s success in production. This made them more proactive in preparing their models for production and gave them a sense of accomplishment each time an AI project was deployed or updated.&lt;/p&gt;

&lt;p&gt;Integrating teams not only reduces friction but enhances the effectiveness of both groups. Learning from the DevOps movement, which bridged a similar gap between software developers and IT operations, embedding data scientists within application teams eliminates the "not my problem" mindset and leads to more resilient and efficient workflows. &lt;/p&gt;

&lt;h2&gt;
  
  
  There’s more...
&lt;/h2&gt;

&lt;p&gt;Today, there are only a few organizations that have experience putting AI projects into production. However, nearly every organization I talk to is working on developing AI projects so it’s only a matter of time before those projects will need to live in production. Sadly, most organizations aren’t ready for the problems that will come that day.&lt;/p&gt;

&lt;p&gt;I started &lt;a href="https://www.jozu.com/" rel="noopener noreferrer"&gt;Jozu&lt;/a&gt; to help people avoid an unpleasant experience when their new AI project hits production. &lt;/p&gt;

&lt;p&gt;Our first contribution is a free open source tool called &lt;a href="https://www.kitops.ml/" rel="noopener noreferrer"&gt;KitOps&lt;/a&gt; that packages and versions AI projects into ModelKits. It uses existing standards - so you can store ModelKits in the enterprise registry you already use.&lt;/p&gt;

&lt;p&gt;You can read more about how we see &lt;a href="https://dev.to/kitops/streamlining-aiml-deployment-with-jozu-model-kits-innovations-and-future-directions-41ae"&gt;ModelKits bridging the Data Science / Software Engineering divide&lt;/a&gt;. Or, if you're curious to go deeper, read about how to using &lt;a href="https://dev.to/kitops/strategies-for-tagging-modelkits-208j"&gt;versioning and tagging with ModelKits&lt;/a&gt; to speed up and de-risk AI projects.&lt;/p&gt;

&lt;p&gt;If you've pushed an AI project into production I'd love to hear what your experiences were and what helped / hindered.&lt;/p&gt;

</description>
      <category>devops</category>
      <category>ai</category>
      <category>opensource</category>
      <category>aiops</category>
    </item>
  </channel>
</rss>
