DEV Community

Orquesta𝄢
Orquesta𝄢

Posted on • Originally published at orquesta.live

Orquesta CLI: Mastering Local LLM Management With Sync

#llmmanagement #orquestacli #cloudsync #localexecution

Originally published at orquesta.live/blog/orquesta-cli-local-llm-management-with-sync

Managing large language models (LLMs) locally while maintaining a reliable sync with cloud configurations can often feel like juggling two different worlds—each with its own set of complexities and constraints. With Orquesta CLI, we bridge this gap, enabling you to run and manage LLMs like Claude, OpenAI, Ollama, and vLLM locally, while ensuring seamless synchronization with a cloud dashboard.

Why Local LLM Management Matters

Running LLMs locally offers several advantages: from data privacy and controlled environment setups to reducing latency and optimizing for specific hardware configurations. However, this often comes at the cost of losing the seamless collaborative features that cloud solutions offer, such as configuration management and prompt history tracking.

Orquesta CLI changes this narrative by allowing you to harness the best of both worlds. Here’s how we do it:

  • Local Execution: Execute LLMs directly on your infrastructure, ensuring data never leaves your environment.
  • Cloud Sync: Automatically synchronize configurations and updates between your local setup and the Orquesta cloud dashboard.
  • Collaborative History: Track and audit prompt history across your organization, ensuring transparency and collaboration.

Setting Up Orquesta CLI

To get started with Orquesta CLI, follow these steps:

  1. Installation: Ensure you have the Orquesta CLI tool installed on your machine. You can do this via npm or download directly from our GitHub repository. 
npm install -g orquesta-cli
Enter fullscreen mode Exit fullscreen mode
  1. Configuration: Set up your configuration files to define which LLMs you want to run locally. For instance, you might configure Claude and OpenAI like so: 
llms:
  - name: Claude
    path: /usr/local/bin/claude-cli
  - name: OpenAI
    api_key: YOUR_OPENAI_API_KEY
Enter fullscreen mode Exit fullscreen mode
  1. Cloud Dashboard Sync: Enable sync between your local CLI and the Orquesta cloud dashboard. This involves setting up your organization’s token, ensuring all local changes are reflected in the cloud and vice versa. 
orquesta-cli sync --org-token YOUR_ORG_TOKEN
Enter fullscreen mode Exit fullscreen mode

Seamless Configuration and Prompt History Tracking

One of the standout features of Orquesta CLI is its ability to maintain an up-to-date record of configurations and prompt histories. This offers numerous benefits:

Organizational Scope

The CLI supports org-scoped tokens, which means that any configuration or prompt history is inherently tied to your organization. This facilitates a single source of truth for your LLM configurations, simplifying compliance and auditing processes.

Bidirectional Config Sync

Changes made in the cloud dashboard, such as updates to prompt configurations or API keys, can be pulled down to your local setup. Similarly, any local adjustments can be pushed to the cloud, ensuring all team members work with the latest configurations.

# Push changes to the cloud
gorquesta-cli push

# Pull latest changes from the cloud
gorquesta-cli pull
Enter fullscreen mode Exit fullscreen mode

Real-World Applications

Consider a scenario where your development team is collaborating on a project that requires the use of multiple LLM models. With Orquesta CLI:

  • Team Collaboration: Team members can submit prompts from their local environments or even from the Orquesta cloud dashboard, with all interactions logged and tracked.
  • Version Control: Every prompt and update is subject to version control, allowing team leads to review and approve changes before they go live.
  • Security: With AES-256 encryption, all data, including prompts and configuration files, is securely transmitted and stored.

Conclusion

Orquesta CLI is built with developers in mind, offering a robust solution for managing LLMs locally without sacrificing the collaborative and management features typically reserved for cloud environments. By aligning the strengths of local execution with seamless cloud sync, we empower teams to innovate faster, with more control and security.

Whether you're running sophisticated machine learning models on a single laptop or managing a fleet of LLMs across a distributed team, Orquesta CLI provides the tools you need to succeed. Embrace local autonomy without losing the benefits of cloud-based collaboration.

For more details, visit our documentation page.

Top comments (0)