DEV Community: Seal

Building Your Private ChatGPT and Knowledge Base with AnythingLLM and GPUStack

Seal — Tue, 12 Nov 2024 04:27:09 +0000

AnythingLLM [https://github.com/Mintplex-Labs/anything-llm] is an all-in-one AI application that runs on Mac, Windows, and Linux. Its goal is to enable the local creation of a personal ChatGPT using either commercial or open-source LLMs along with vector database solutions. AnythingLLM goes beyond being a simple chatbot by including Retrieval-Augmented Generation (RAG) and Agent capabilities. These features allow it to perform a variety of tasks, such as fetching website information, generating charts, summarizing documents, and more.

AnythingLLM can integrate various types of documents into different workspaces, enabling users to reference document content during chats. This provides a easy way to organize workspaces for different tasks and documents.

In this article, we will introduce how to build a personal ChatGPT with knowledge base using AnythingLLM + GPUStack.

Run models with GPUStack

GPUStack is an open-source GPU cluster manager for running large language models (LLMs). It enables you to create a unified cluster from GPUs across various platforms, including Apple MacBooks, Windows PCs, and Linux servers. Administrators can deploy LLMs from popular repositories like Hugging Face, allowing developers to access these models as easily as they would access public LLM services from providers such as OpenAI or Microsoft Azure.

Unlike Ollama, GPUStack is a cluster solution designed to aggregate GPU resources from multiple devices to run models.

To deploy the Chat Model and Embedding Model on GPUStack:

• Chat Model: llama3.1

• Embedding Model: bge-m3

And you need to create an API key. This key will be used by AnythingLLM to authenticate when accessing the models API deployed on GPUStack.

Install and configure AnythingLLM

AnythingLLM offers packages for Mac, Windows, and Linux, you can download from https://anythingllm.com/download. After installation, open AnythingLLM to begin the setup process.

Configure LLM Provider

First, configure the chat model. Search for OpenAI, select Generic OpenAI:

And fill in the details for the model deployed on GPUStack:

Save and configure embedding model.

Configure Embedding Provider

AnythingLLM includes a lightweight embedding model, all-MiniLM-L6-v2, which offers limited performance and context length. For more powerful embedding capabilities, you can either opt for public embedding services or run open-source embedding models. Here, we’ll configure the embedding model bge-m3, which is running on GPUStack. Set the embedding provider to Generic OpenAI and fill in the relevant configuration.

Then create a workspace, and we can use AnythingLLM after it's completed.

Use AnythingLLM

Chat with LLM

Select a workspace, create a new thread, and send your question to the LLM:

Fetch website content

Click the upload button next to the workspace, enter the website URL in the Fetch website box, and fetch the website content.

The fetched website content will be sent to the embedding model for vectorization and then stored in the vector database.

Check the content fetched from the website:

Documents embedding

Click the upload button next to the workspace, then click the upload box and upload a document. The document will be sent to the embedding model for vectorization and then stored in the vector database.

Check the content of embedded documents:

For more information, please read the AnythingLLM documentation: https://docs.anythingllm.com/

Conclusion

In this tutorial, we have introduced how to use AnythingLLM + GPUStack to aggregate GPUs across multiple devices and build an all-in-one AI application for RAG and AI Agents.

GPUStack provides a standard OpenAI-compatible API, which can be quickly and smoothly integrated with various LLM ecosystem components. Wanna give it a go? Try to integrate your tools/frameworks/software with GPUStack now and share with us!

If you encounter any issues while integrating GPUStack with third parties, feel free to join GPUStack Discord Community and get support from our engineers.

Building Free GitHub Copilot Alternative with Continue + GPUStack

Seal — Fri, 23 Aug 2024 17:00:00 +0000

Click here to read original post

Continue is an open-source alternative to GitHub Copilot, this is an open-source AI coding assistant that allows to connect various large language models(LLMs) within VS Code and JetBrains to build custom code autocompletion and chat capabilities. It supports:

Code parsing
Code autocompletion
Code optimization suggestions
Code refactoring
Code implementations Inquiring
Documentation online searching
Terminal errors parsing

and more. It assists developers in coding and enhancing their development efficiency.

In this tutorial, we are going to use Continue + GPUStack to build a free GitHub Copilot locally, providing developers with an AI-paired programming experience.

Running Models with GPUStack

First, we will deploy the models on GPUStack. There are three model types recommended by Continue:

Chat model: select llama3.1, this is the latest open-source model trained by Meta.
Autocompletion model: select starcoder2:3b, a highly advanced autocompletion model trained by Hugging Face.
Embedding model: select nomic-embed-text, which supports a context length of 8192 tokens, it outperforms OpenAI ada-002 and text-embedding-3-small models for both short and long context tasks.

After deploying the models, you are also required to create an API key in the API Keys section for authentication when Continue accesses the models deployed on GPUStack.

Installing and Configuring Continue

Continue provides extensions for both VS Code and JetBrains. In this article, we will use VS Code as an example. Install Continue from the VS Code extension store:

Once installed, drag the Continue extension to the right panel to avoid conflict with the file explorer:

Then, select the settings button in the bottom-right corner to edit Continue's configuration and connect to the models deployed on GPUStack. Replace the sections for "models", "tabAutocompleteModel", and "embeddingsProvider" with your own GPUStack-generated API Key:

{
  "models": [
    {
      "title": "Llama 3.1",
      "provider": "openai",
      "model": "llama3.1",
      "apiBase": "http://192.168.50.4/v1-openai",
      "apiKey": "gpustack_f58451c1c04d8f14_c7e8fb2213af93062b4e87fa3c319005"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Starcoder 2 3b",
    "provider": "openai",
    "model": "starcoder2",
    "apiBase": "http://192.168.50.4/v1-openai",
    "apiKey": "gpustack_f58451c1c04d8f14_c7e8fb2213af93062b4e87fa3c319005"
  },
  "embeddingsProvider": {
    "provider": "openai",
    "model": "nomic-embed-text",
    "apiBase": "http://192.168.50.4/v1-openai",
    "apiKey": "gpustack_f58451c1c04d8f14_c7e8fb2213af93062b4e87fa3c319005"
  }
}

Get to Use Continue

After configuring Continue to connect to the GPUStack-deployed models, go to the top-right corner of the Continue plugin interface and select Llama 3.1 model. Now you are able to use the features we mentioned at the beginning of this tutorial:

Code Parsing: Select the code, press Cmd/Ctrl + L, and enter a prompt to let the local LLM parse the code:
Code Autocompletion: While coding, press Tab to let the local LLM attempt to autocomplete the code:
Code Refactoring: Select the code, press Cmd/Ctrl + I, and enter a prompt to let the local LLM attempt to optimize the code:

The LLM will provide suggestions, and you can decide whether to accept or reject them:

Inquire About Code Implementation: You can try @Codebase to ask questions about the codebase, such as how a certain feature is implemented:
Documentation Search: Use @Docs and select the document site you wish to search for and ask your questions, enabling you to find the results you need:

For more information, please read the official Continue documentation: https://docs.continue.dev/how-to-use-continue

Conclusion

In this tutorial, we have introduced how to use Continue + GPUStack to build a free local GitHub Copilot, offering AI-paired programming capabilities at no cost to developers.

If you encounter any issues while integrating GPUStack with third parties, feel free to join GPUStack Discord Community and get support from our engineers.

Introducing GPUStack: An open-source GPU cluster manager for running LLMs

Seal — Fri, 26 Jul 2024 15:36:39 +0000

What is GPUStack?

We are thrilled to launch GPUStack, an open-source GPU cluster manager for running Large Language Models (LLMs). Even though LLMs are widely available as public cloud services, organizations cannot easily host their own LLM deployments for private use. They need to install and manage complex clustering software such as Kubernetes and then figure out how to install and manage the AI tool stack on top. Popular ways to run LLMs locally, such as LMStudio and LocalAI, works on a single machine.

GPUStack allows you to create a unified cluster from any brand of GPUs in Apple MacBooks, Windows PCs, and Linux servers. Administrators can deploy LLMs from popular repositories such as Hugging Face. Developers can then access LLMs just as easily as accessing public LLM services from vendors like OpenAI or Microsoft Azure.

For more details about GPUStack, visit:

GitHub repo: https://github.com/gpustack/gpustack

User guide: https://docs.gpustack.ai

Why GPUStack?

Today, organizations who want to host LLMs on a cluster of GPU servers have to do a lot of work to integrate a complex software stack. By using GPUStack, organizations no longer need to worry about cluster management, GPU optimization, LLM interference engines, usage and metering, user management, API access, and dashboard UI. GPUStack is a complete software platform for building your own LLM-as-a-Service (LLMaaS).

As the following figure illustrates, the admin deploys models into GPUStack from a repository like HuggingFace, and then developers can connect to GPUStack to use these models in their applications.

Key features of GPUStack

GPU cluster setup and resource aggregation

GPUStack aggregates all GPU resources within a cluster. It is designed to support all GPU vendors, including Nvidia, Apple, AMD, Intel, Qualcomm, and others. GPUStack is compatible with a laptops, desktops, workstations, and servers running MacOS, Windows, and Linux.

The initial release of GPUStack supports Windows PCs and Linux servers with Nvidia graphics cards, and Apple Macs.

Deployment and Inference for Models

GPUStack supports distributed deployment and inference of LLMs across a cluster of GPU machines.

GPUStack selects the best inference engine for running the given LLM on the given GPU. The first LLM inference engine supported by GPUStack is LLaMA.cpp, which allows GPUStack to support GGUF models from Hugging Face and all models listed in the Ollama library (ollama.com/library).

You can run any model on GPUStack by first converting it to GGUF format and uploading it to Hugging Face or Ollama library.

Support of other inference engines, such as vLLM, is on our roadmap and will be provided in the future.

Note: GPUStack will automatically schedule the model you select to run on machines with appropriate resources, relieving you of manual intervention. If you want to assess the resource consumption of your chosen model, you can use our GGUF Parser project: https://github.com/gpustack/gguf-parser-go. We intend to provide more detailed tutorials in the future.

Although GPU acceleration is recommended for inference, we also support CPU inference, though the performance isn't as good as GPU. Alternatively, using a mix of GPU and CPU for inference can maximize resource utilization, which is particularly useful in edge or resource-constrained environments.

Easy integration with your applications

GPUStack offers OpenAI-compatible APIs and provides an LLM playground along with API keys. The playground enables AI developers to experiment with and customize your LLMs, and seamlessly integrate them into AI-enabled applications.

Additionally, you can use the metrics GPUStack provides to understand how your AI applications utilize various LLMs. This helps administrators manage GPU resource consumption effectively.

Observability metrics for GPUs and LLMs

GPUStack provides comprehensive metrics performance, utilization, and status monitoring.

For GPUs, administrators can use GPUStack to monitor real-time resource utilization and system status. Based on these metrics:

Administrators perform scaling, optimization, and other maintenance operations.
GPUStack adjusts its model scheduling algorithm.

For LLMs, developers can use GPUStack to access metrics like token throughput, token usage, and API request throughput. These metrics help developers evaluate model performance and optimize their applications. GPUStack plans to support auto-scaling based on these inference performance metrics in future releases.

Authentication and access control

GPUStack also provides authentication and role-based access control (RBAC) for enterprises. Users on the platform can have either admin or regular user roles. This guarantees that only authorized administrators can deploy and manage LLMs and that only authorized developers can utilize them.

GPUStack Use Cases

GPUStack unlocks a world of possibilities for running LLMs on any GPU vendors. Here are just a few examples of what you can achieve with GPUStack:

Aggregate existing MacBooks, Windows PCs, and other GPU resources to offer a low-cost LLMaaS for a development team.
In limited resource environments, aggregate multiple edge nodes to provide LLMaaS on CPU resources.
Create your own enterprise-wide LLMaaS in your own data center for highly sensitive workloads that cannot be hosted in a cloud.

Getting Started with GPUStack

Installation

Linux or MacOS

GPUStack provides a script to install it as a service on systemd or launchd based systems. To install GPUStack using this method, execute:

curl -sfL https://get.gpustack.ai | sh -

Now you have deployed and started the GPUStack server, which serves as the first worker node. You can access the GPUStack page via http://myserver (Replace with the IP address or domain of the host you installed).

Log in to GPUStack with username admin and the default password. You can run the following command to get the password for the default setup:

cat /var/lib/gpustack/initial_admin_password

To add additional worker nodes and form a GPUStack cluster, please run the following command on each worker node:

curl -sfL https://get.gpustack.ai | sh - --server-url http://myserver --token mytoken

Replace http://myserver with your GPUStack server URL and mytoken with your secret token for adding workers. To retrieve the token in the default setup from the GPUStack server, use the following command:

cat /var/lib/gpustack/token

Or follow the instructions on GPUStack to add workers:

Windows

Run PowerShell as administrator, then run the following command to install GPUStack:

Invoke-Expression (Invoke-WebRequest -Uri "https://get.gpustack.ai" -UseBasicParsing).Content

You can access the GPUStack page via http://myserver (Replace with the IP address or domain of the host you installed).

Log in to GPUStack with username admin and the default password. You can run the following command to get the password for the default setup:

Get-Content -Path (Join-Path -Path $env:APPDATA -ChildPath "gpustack\initial_admin_password") -Raw

Optionally, you can add extra workers to form a GPUStack cluster by running the following command on other nodes:

Invoke-Expression "& { $((Invoke-WebRequest -Uri "https://get.gpustack.ai" -UseBasicParsing).Content) } -ServerURL http://myserver -Token mytoken"

In the default setup, you can run the following to get the token used for adding workers:

Get-Content -Path (Join-Path -Path $env:APPDATA -ChildPath "gpustack\token") -Raw

For other installation scenarios, please refer to our installation documentation at: https://docs.gpustack.ai/docs/quickstart

Serving LLMs

As an LLM administrator, you can log in to GPUStack as the default system admin, navigate to Resources to monitor your GPU status and capacities, and then go to Models to deploy any open-source LLM into the GPUStack cluster. This enables you to provide these LLMs to regular users for integration into their applications. This approach helps you to efficiently utilize your existing resources and deliver stable LLM services for various needs and scenarios.

Access GPUStack to deploy the LLMs you need. Choose models from Hugging Face (only GGUF format is currently supported) or Ollama Library, download them to your local environment, and run the LLMs:

GPUStack will automatically schedule the model to run on the appropriate Worker:

You can manage and maintain LLMs by checking API requests, token consumption, token throughput, resource utilization status, and more. This helps you decide whether to scale up or upgrade LLMs to ensure service stability.

Integrating with your applications

As an AI application developer, you can log in to GPUStack as a regular user and navigate to Playground from the menu. Here, you can interact with the LLM using the UI playground.

Next, visit API Keys to generate and save your API key. Return to Playground to customize your LLM by adjusting the system prompt, adding few-shot learning examples, or resizing prompt parameters. When you're done, click View Code and select your preferred code format (curl, Python, Node.js) along with the API key. Use this code in your applications to enable communication with your private LLMs.

you can access the OpenAI-compatible API now, for example, use curl as the following:

export GPUSTACK_API_KEY=myapikey
curl http://myserver/v1-openai/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $GPUSTACK_API_KEY" \
  -d '{
    "model": "llama3",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ],
    "stream": true
  }'

Join Our Community

Please find more information about GPUStack at: https://gpustack.ai.

If you encounter any issues or have suggestions for GPUStack, feel free to join our Community for support from the GPUStack team and to connect with fellow users globally.

We are actively enhancing the GPUStack project and plan to introduce new features in the near future, including support for multimodal models, additional accelerators like AMD ROCm or Intel oneAPI, and more inference engines. Before getting started, we encourage you to follow and star our project on GitHub at gpustack/gpustack to receive instant notifications about all future releases. We welcome your contributions to the project.

About Us

GPUStack is brought to you by Seal, Inc., a team dedicated to enabling AI access for all. Our mission is to enable enterprises to use AI to conduct their business, and GPUStack is a significant step towards achieving that goal.

Quickly build your own LLMaaS platform with GPUStack! Start experiencing the ease of creating GPU clusters locally, running and using LLMs, and integrating them into your applications.

How to Enhance Developer Productivity with Platform Engineering

Seal — Wed, 24 Apr 2024 15:07:00 +0000

As the cloud computing, and GenAI technologies continue to evolve, the software industry faces increasingly fierce competition. Simultaneously, software development has become more complex. Developers need to acquire more knowledge and skills while dealing with additional problems and risks.

To address these challenges, development teams must deliver valuable software products quickly, efficiently, and cost-effectively. They must also approach problem-solving with simplicity, optimization, and innovation. This is precisely why discussions around development efficiency have become a hot topic.

In this article, we will explore the definition and challenges of developer productivity, as well as how platform engineering can help organizations improve their efficiency.

What is Developer Productivity?

Developer productivity is defined as the ability of a development team to deliver higher-quality, more reliable, and sustainable business value in a more efficient manner. This is a critical focus area for both emerging technology companies and traditional software enterprises because it directly impacts competitiveness and innovation.

As the market changes rapidly, organizations that fail to adapt their development efficiency risk falling behind competitors and eventually being phased out.

Challenges in Improving Developer Productivity

However, enhancing developer productivity is a challenging endeavor. With the continuous growth in software scale and complexity, expanding development team sizes, and accelerating business requirements and market changes, the path to improving development efficiency faces several challenges:

1. Technical Complexity: As technology evolves, the technical complexity of products increases, thereby raising the technical bar for development. Modern software architectures consist of multiple layers, technologies, and services, demanding end-to-end understanding from developers. This complexity adds cognitive load and increases the risk of errors and inefficiencies. Overcoming technical complexity requires substantial resource investment to ensure both efficiency and quality.

2. Project Management Difficulty: The complexity and scale of projects inevitably lead to greater project management challenges. Enterprises require robust project management systems and tools to coordinate and manage the activities of various development teams and project timelines. Additionally, fostering efficient teamwork and communication is crucial to ensuring projects are completed on time and with high quality.

3. Technical Debt: Many organizations encounter difficulties in adopting DevOps, cloud-native technologies, and other advanced approaches due to the challenges posed by legacy systems and outdated practices. These difficulties result in technical debt and skill gaps, which in turn impede the delivery of software at a faster and more optimal pace.

4. Lack of Standardization: Enterprises often have multiple development teams using different tools and configurations for their applications and infrastructure. This lack of standardization creates silos and inconsistencies, making collaboration, sharing best practices, and ensuring quality and security more challenging.

5. Low Productivity: Developers spend significant time on non-value-added tasks such as environment setup, tool configuration, and debugging. This reduces their productivity and focus on delivering customer value.

6. Lack of Continuous Improvement and Feedback Loop:The improvement of development efficiency is a long-term project that requires continuous optimization. Without effective mechanisms and a culture of improvement and feedback within the organization, it is difficult to achieve sustained development efficiency gains.

How Platform Engineering Boosts Developer Productivity

Platform engineering is a systematic approach aimed at improving software development efficiency and quality. By building reusable, scalable software platforms, platform engineering provides development teams with standardized development frameworks and tools. It optimizes collaboration and communication, enhances software testability and maintainability, and supports rapid iteration and innovation. Let's explore these aspects in more details:

Standardized Development Frameworks and Tools:

Platform engineering offers standardized development frameworks and tools, including code libraries, components, and templates. These enable teams to develop high-quality software more quickly, reducing developers' workload and time costs.

Consistent frameworks and tools ensure everyone follows the same best practices and standards, improving efficiency, reducing errors, and minimizing technical disparities among team members. For specific industries, teams can leverage existing platforms and components without redeveloping all infrastructure, allowing developers to focus on core business logic.

Optimized Team Collaboration and Communication:

Platform engineering provides standardized development processes and specifications, unifying team development methods and approaches. This reduces communication and coordination costs, enhancing collaboration efficiency.

Centralized communication and coordination platforms (such as shared task lists, code repositories, documentation, and team discussions) allow developers to better understand each other's progress and challenges. This facilitates quick collaboration and issue resolution, ultimately improving team communication and efficiency.

Improved Software Testability and Maintainability:

Platform engineering employs a range of techniques, including automated testing, code refactoring, and performance monitoring, with the objective of enhancing the testability and maintainability of software. This approach has the potential to reduce the burden on developers and the incidence of errors, thereby improving the efficiency, quality, and reliability of software development.

Common code libraries and documentation provided by platform engineering help teams maintain and upgrade software effectively.

Support for Rapid Iteration and Innovation:

The provision of reusable templates and components by platform engineering enables development teams to implement new ideas and features in a more expeditious manner. Furthermore, it facilitates rapid iteration and updates, thereby enabling businesses to gain a deeper understanding of user needs and behavior. This, in turn, enhances the user experience and market competitiveness.

Additionally, platform engineering enhances traceability and transparency in the development process. Developers gain clearer insights into their tasks and goals, as well as the overall development status. By enabling rapid innovation and progress within teams, platform engineering facilitates enhanced development efficiency.

Conclusion

In conclusion, platform engineering offers several advantages for improving developer productivity and is a crucial approach for businesses seeking to enhance their development processes. As digital transformation continues, it can be expected that platform engineering will play an increasingly important role in enterprise development. In the future, platform engineering will continue to evolve and find applications in various areas, including multi-cloud environments, automation, and AI technology integration.

Walrus is an open-source application management platform based on IaC, that helps platform engineers build golden paths for developers and empowers developers with self-service capabilities. Its abstraction layers allow developers to leverage standardized and reusable IaC templates for self-service resource provisioning and deployments without being infrastructure expertise.

If you want to discuss more with us, welcome to join Seal Discord to share your thoughts and feedback.

Platform as a Product: Why do we need?

Seal — Thu, 18 Apr 2024 15:00:00 +0000

In today's fast-paced digital age, businesses are constantly seeking innovative ways to deliver value and drive growth. The concept of Platform as a Product (PaaP) has gained widespread attention.

With the advancement of technology, traditional product-centric approaches are being replaced by more comprehensive, platform-based strategies. This article aims to delve into the concept of Platform as a Product, exploring its meaning, characteristics, advantages, and challenges.

What is Platform as a Product?

Platform as a Product refers to a business model where a company creates and provides a platform that allows various stakeholders, including developers, third-party providers, and end-users, to build, customize, and distribute their own products or services. Unlike traditional products designed for end-users, PaaP serves as a foundation upon which others can develop and deliver their own products.

Key Characteristics of Platform as a Product

Platform as a Product represents a shift in how businesses create value and interact with developers, partners, and users. By leveraging the characteristics of Platform as a Product, businesses can gain a competitive edge. Here, we summarize the five key characteristics of Platform as a Product.

Infrastructure and Technology Stack

At the core of Platform as a Product is a robust infrastructure and technology stack. This includes the hardware, software frameworks, APIs, and developer tools that constitute the foundation for building and operating the platform.

The infrastructure must be scalable, reliable, and capable of handling the diverse and growing needs of a varied user base and ecosystem. The technology stack facilitates seamless integration, enabling developers to leverage existing platform features and providing a consistent and secure environment for application development and deployment.

Open and Collaboration

One of the key features of Platform as a Product is that it is open to external developers, partners, and users. Open fosters collaboration, knowledge sharing, and innovation within the platform ecosystem. Companies provide accessible APIs, SDKs, and developer communities and encourage participation and contribution. By embracing open, the platform nurtures a vibrant ecosystem where developers and partners can build on top of the platform, extend its functionalities, and create value-added products and services. Collaboration within the ecosystem amplifies the platform's overall value proposition and enhances its competitive advantage.

Scalability and Flexibility

Scalability is also a key characteristic of a successful PaaP model. The platform's design must be able to handle exponential growth, accommodate an increasing number of users, and support a wide range of applications and services. Scalability ensures that the platform can meet the evolving needs of its user base without impacting performance or user experience.

Flexibility is another important aspect of PaaP. The platform should offer customization options, allowing developers to tailor the platform's functionalities according to their specific requirements. Customization enhances the platform's appeal, improves user satisfaction, and supports the creation of unique applications and services that cater to different needs.

Resources and Support for Developers

To attract developers, successful PaaP models provide comprehensive developer empowerment and support. This includes documentation, tutorials, sample code, and developer communities that facilitate knowledge exchange, troubleshooting, and collaboration.

Organizations that prioritize providing resources and support for developers create an atmosphere where developers can thrive, experiment, and innovate. By providing the necessary resources and tools, the platform can attract top talent, accelerate development cycles, and drive ecosystem growth.

Advantages and Potential of Platform as a Product

The adoption of the PaaP model changes the way businesses create value, attract developers and other stakeholders, and provide innovative solutions. By adopting the PaaP model, businesses stand to gain 3 major advantages and potential.

One of the advantages that PaaP brings to businesses is tha*t it fosters accelerated innovation*. By providing platform infrastructure, tools, and APIs, developers and partners can focus on building innovative products and services. This approach allows developers to leverage existing platform features and reduces the time and effort required to develop core functionalities. As a result, PaaP enables companies to quickly bring new products to market, iterate based on user feedback, and maintain a competitive edge.

A platform as a product has the ability to attract a diverse array of developers, partners, and users, thereby creating a vibrant ecosystem. These ecosystems can foster collaboration, knowledge exchange, and the creation of value-added services. By opening their platforms to external contributors, companies can harvest and leverage a broader range of talent, ideas, and resources. The expanded ecosystem not only enhances the platform's functionalities but also opens new revenue streams, drives user engagement, and fosters community awareness.

Platform as a Product aims to deliver an excellent user experience. By integrating various services, features, and applications into a unified platform, users can access a comprehensive solution that simplifies their interactions, streamlines processes, and reduces friction. Through seamless integration, intuitive interfaces, and personalized experiences, PaaP improves user satisfaction and loyalty. Additionally, as the ecosystem expands, users benefit from the continuous innovation and enrichment brought about by the contributions of developers and partners within the platform.

Challenges Faced by Platform as a Product

While PaaP models offer many benefits and transformative opportunities for organizations, they also present unique challenges that must be strategically addressed. Implementing and managing a successful PaaP requires careful planning, continuous adjustment, and a customer-centric approach.

PaaP platforms typically involve complex technical architectures, integration challenges, and scalability requirements. Building and maintaining a robust and scalable infrastructure requires substantial resources and expertise. Companies need to invest in skilled technical teams, adopt agile development methodologies, and leverage cloud-based technologies to successfully overcome technical complexity. Collaborating with developers and partners can also help address technical challenges and ensure compatibility and interoperability within the ecosystem.

Another major challenge for PaaP is establishing effective governance and regulatory mechanisms. As platforms open to external developers, partners, and users, ensuring fair competition, content quality, data privacy, and ethical standards becomes paramount. Companies must establish policies, guidelines, and mechanisms to effectively monitor and regulate the platform ecosystem. This includes content review, dispute resolution, enforcing compliance, and maintaining user trust. Therefore, striking a balance between platform openness and governance of responsibilities is a key challenge to be addressed.

As PaaP becomes more popular, competition among platform providers will intensify. Businesses wanting to stand out in the industry need to attract and retain users and developers. Establishing a strong brand, providing an excellent user experience, and offering comprehensive developer support are basic strategies for maintaining a leading position, so businesses face the challenge of creating platform stickiness. Businesses can try to cultivate a vibrant ecosystem, foster loyalty through incentives or rewards, and continuously innovate to provide new valuable features.

Conclusion

Platform as a Product has emerged as a transformative business model. By creating a foundation on which developers and users can build their own products and services, PaaP fosters innovation, ecosystem expansion, and enhanced user experience. While challenges exist, the benefits of PaaP are undeniable, and these benefits bring advantages to businesses, making PaaP increasingly attractive to enterprises across various industries. With the continuous advancement of technology, we can expect the continued development and adoption of Platform as a Product, becoming a key driver of digital transformation.

If you are interested in Platform engineering, Welcome to our community:

Discord: https://discord.gg/fXZUKK2baF
Twitter/X: https://twitter.com/Seal_io
LinkedIn: https://www.linkedin.com/company/seal-io
Youtube: https://www.youtube.com/@Seal-io

Introducing Walrus: Streamline your Delivery with the Platform Engineering Approach

Seal — Thu, 11 Jan 2024 15:00:00 +0000

The implementation of DevOps is facing challenges due to the increasing complexity of infrastructure. It can be tough for developers to learn non-development related knowledge, which adds to their cognitive load. As a result, the Infra/DevOps teams are constantly flooded with tickets and messages.

It's time for change! In this article, we will dive into how Walrus streamlines your software delivery with the platform engineering approach that makes developers happy and DevOps happier.

Our Goal: Optimize DevOps for Developers and IT Operators

Seal hosts two fully open-source projects: Walrus, an application management platform, and Appilot, an AI agent for DevOps. Our goal with these projects is to make DevOps and developers' lives easier:

For developers:

Focus on application development and business demands
No need to navigate the complexities of Kubernetes and infrastructure
Configure once and deploy applications polymorphically on various infrastructures
Automate to create, release, start/stop environments dynamically

For IT Operators

Enable flexible orchestration of infrastructure capabilities, adhere to best practices, and ensure security compliance
Eliminate tickets and enable self-service for developers
Obtain a comprehensive view of the entire application system for efficient management and troubleshooting
Optimize resource utilization and reduce costs

How IT operators and developers collaborate with Walrus
Self-service infrastructure automation is the key to successful platform engineering. With the separation of concerns between IT operators and developers, Walrus's Resource Definition enables automated infrastructure deployment. Developers claim the resource they want in their applications system (MySQL database, Redis cache, etc.), and IT operators define how these resources should be provisioned and configured (MySQL helm charts, AWS RDS service, etc.) in different environments based on different infrastructures.

In today's tech landscape, the collaboration between developers and operators requires intricate coordination. This often leads to a series of manual processes that can hinder the pace of development. Many organizations have turned to the creation of bespoke pipelines or ticketing systems for infrastructure deployments. However, these solutions only alleviate some of the challenges without fully eliminating the need for manual processes.

This is where Resource Definition steps in, offering a unique solution: IT operators can set up Infrastructure as Code (IaC) templates (such as Terraform modules and Helm charts) that developers can leverage for self-service resource provisioning and deployments. Furthermore, Resource Definition empowers operators to establish and enforce corporate policies, dictating the usage, configuration, and deployment permissions of cloud resources.

Consequently, developers are freed from the intricacies of deploying suitable infrastructure for their applications, enabling them to concentrate on coding.

For example, an operator who sets up a Resource Definition for deploying MySQL databases that aligns with their organization’s standards: a containerized MySQL in development environment and a HA MySQL database in production environment. The operator can mandate that the IaC templates used and parameters configured based environment type, label and other attributes. Once the Resource Definition is in place, developers can deploy MySQL databases in different modes without concerning themselves with the specifics of their deployment or their configuration accuracy.

This division of responsibilities allows operator teams to extend their support to developer teams more effectively while ensuring that applications are deployed in line with organizational policies.

Here is the comparison between without and with the Walrus application model:

Conclusion

Our commitment is backed by the fact that platform engineering has been listed as one of the top 10 technology trends by Gartner for two consecutive years, 2023 and 2024. It fundamentally reimagines how developers engage with technology and how organizations shape their DevOps workflows.

Walrus dedicates itself to utilizing the platform engineering approach to simplify application delivery and declutter the cognitive load for developers. Plus, with just a one-line command, Walrus is incredibly easy to install:

sudo docker run -d --privileged --restart=always -p 80:80 -p 443:443 --name walrus sealio/walrus:v0.4.1

Give it a try and welcome to our community:

Discord: https://discord.gg/fXZUKK2baF
Twitter/X: https://twitter.com/Seal_io
Linkedin: https://www.linkedin.com/company/seal-io
Youtube: https://www.youtube.com/@Seal-io

Switching from Terraform: Integrate with OpenTofu in Walrus

Seal — Thu, 28 Dec 2023 15:21:00 +0000

What is OpenTofu?

OpenTofu is an open-source Infrastructure as Code (IaC) framework presented as an alternative to Terraform and managed by the Linux Foundation. It was developed in response to HashiCorp's decision to change Terraform's licensing from the Mozilla Public License v2.0 (MPLv2) to a Business Source License v1.1. The aim of OpenTofu is to offer a dependable and impartial option for infrastructure as code management, ensuring it remains truly open-source under a stable license.

Under the guidance of the Linux Foundation, OpenTofu seamlessly replaces Terraform v1.6.x while ensuring complete backward compatibility with Terraform v1.5.x and its predecessors.

Since its inception, it has instantly sparked considerable interest. As of the time of this writing, the OpenTofu project has over 16K GitHub stars, while the OpenTofu Manifesto project has over 36K GitHub stars.

Integrating OpenTofu in Walrus

By default, Walrus uses Terraform as its deployment engine. Since OpenTofu is a drop-in replacement for Terraform, you can seamlessly set up OpenTofu in the Walrus system without any code changes.

Start by following the Quick Start guide to deploy the Walrus Server and set up a container service in a Kubernetes cluster.

First, let's check the deployment log for the service.

It's evident that the deployment was executed using Terraform.

Now, let's switch to OpenTofu by following these steps:

Click on System Settings in the left navigation menu.
Click on the Deployment Management tab.
Press the edit button next to Basic Settings.
Change the Deployer Image to sealio/opentofu-deployer:v1.6.0-beta5-1.
Click the Save button.

That's it! Walrus will now use OpenTofu as the deployment engine. Let's create another service by following these steps:

Click on Application Management in the left navigation menu.
Enter the dev environment detail page by clicking on it.
Press the New button and select Service.
Enter the name myapp-tofu.
Choose the containerservice template.
Fill in nginx in the Image field.
Click Save and Deploy.

After the deployment finishes, let's take a look at the deployment log again.

This instance showcases OpenTofu's presence in the log. Walrus executed the deployment using OpenTofu. However, in the CLI arguments, as depicted in the image, terraform is aliased to tofu.

What's Next?

At the time of writing this article, OpenTofu is gearing up for its first stable release set for the upcoming month. The switch from Terraform to OpenTofu for end-users is not about adopting a new name that sounds more fragile :-). Rather, we're poised to witness innovative solutions from OpenTofu addressing tangible challenges in our landscape.

Welcome to Seal community!

Discord: https://discord.gg/fXZUKK2baF
Twitter: https://twitter.com/Seal_io
LinkedIn: https://www.linkedin.com/company/seal-io