Artem

Posted on Jun 19

How I built a self-hosted Algorithmic Trading Ecosystem (and why it's Open Source)

#ai #webdev #programming #python

TL;DR: Don't want to read the story? I open-sourced my self-hosted visual engine for algorithmic trading (280k LOC).
📦 View Source Code on GitHub | 🌐 Official Website

Introduction: The Problem with Centralized SaaS Bots

If you've ever dabbled in algorithmic crypto trading, you've likely encountered two fundamental problems with existing SaaS solutions.

The first pain point is security. The architectural absurdity of the modern industry is that users willingly hand over their API keys (often linked to massive balances) to third-party, closed-source servers. A database leak in such a service isn't just a leak of email addresses; it's direct access to user deposits.

The second pain point is primitive logic. Most paid solutions offer little more than basic Grid or DCA strategies. The moment you need flexibility—for example, pegging a stop-loss to an order book density, using a trailing stop that only activates after a level breakout, or dynamically filtering trades based on a higher timeframe trend—you hit the hard limits of their interface.

Ultimately, all of this pushed me to build my own project: DepthSight.

My goal was to create a self-hosted platform featuring around 40 logical blocks (order book, tape, levels, volume delta, higher timeframe checks, etc.). These blocks can reference each other's results, allowing you to build highly complex event chains. I wanted a truly flexible strategy editor where a trader could visually implement their ideas without writing a single line of code. And if assembling blocks manually feels like too much work, the built-in AI Co-pilot will do it for you.

Why Not Freqtrade, Jesse, or OctoBot?

A fair question: why write your own if there are already trading bots in the open-source space?
Indeed, there are powerful tools on the market:

Freqtrade and Jesse are magnificent frameworks. But they are built exclusively for developers and quants. You need to know how to write logic in Python, dig into config files, and manage everything via CLI. There is no visual interface for regular traders.
OctoBot has a web UI, but architecturally, it's geared towards local, personal use by a single individual. It lacks an advanced No-Code node builder and the infrastructure needed for scaling.

The advantage of DepthSight is that it takes the best of both worlds: the uncompromising power of quant frameworks and the visual comfort of top commercial SaaS solutions (like 3Commas or Bitsgap), while adding a social layer and advanced AI.

A Full-Fledged SaaS "in a Box"

DepthSight isn't just a script you run in the terminal. It's a ready-to-use infrastructure product, a SaaS "in a box" that you can deploy on your own server and immediately use to manage an investor pool or even launch your own commercial service.

Right out of the box, the project features:

Built-in Crypto Acquiring (BitCart): Accept payments in BTC, LTC, TRX, BNB without intermediaries or fees.
Billing and Quota Manager: Limit the number of bots, backtest threads, and available exchanges based on the user's subscription plan.
Full Admin Panel: User management, Role-Based Access Control (RBAC), and a support ticket system.
Affiliate (Referral) Program: A built-in hold tracker and automatic payout calculation for affiliates.
Gamification: Experience points (XP), levels, and achievements to boost trader engagement.

Architecture: How It Works Under the Hood

For the platform to handle heavy computations and run stably 24/7 in a multi-user environment, the architecture was designed from the ground up to be distributed.

Core Tech Stack:

Backend: Python 3.11+, FastAPI (REST + WebSockets).
Connectors: ccxt for unified exchange interactions.
Asynchronous Tasks: Celery (for heavy backtests and genetic optimization).
Frontend: React / Vite / TypeScript (including a PWA for mobile).
Databases: PostgreSQL (persistent storage) + Redis (state, quotas, Pub/Sub).

Architectural Magic: Market Data Fan-out

One of the main bottlenecks in multi-user trading platforms is the strict exchange rate limits on WebSocket connections. If 50 users launch bots on the BTC/USDT pair, and each opens its own socket to the exchange, you'll get banned instantly.

DepthSight implements a Market Data Fan-out pattern. A single central service (market_data_service.py) collects ticks and distributes them to all isolated worker containers via Redis Pub/Sub.

# market_data_service.py
import redis.asyncio as redis_asyncio
from bot_module.exchanges import create_exchange_executor

class MarketDataService:
    def _get_consumer(self, exchange_id: str) -> DataConsumer:
        """Dynamically creates a CCXT-based connector for the required exchange."""
        if exchange_id not in self.consumers:
            # Universal executor for any exchange
            futures_executor = create_exchange_executor(
                exchange=exchange_id,
                api_key="", api_secret="",
                session=self.session,
                market_type="futures_usdtm"
            )
            consumer = DataConsumer(
                executor=futures_executor,
                market_data_mode="direct",
                market_data_publish_callback=self._publish_market_payload
            )
            self.consumers[exchange_id] = consumer
        return self.consumers[exchange_id]

    async def _publish_market_payload(self, payload: Dict[str, Any]) -> None:
        """A single WebSocket connection receives data from CCXT 
        and broadcasts it to workers via Redis Pub/Sub."""
        stream_key = payload.get("stream_key")
        channel = f"depthsight:market_data:events:{stream_key}"

        # Publish tick to channel. Bots no longer need to ping the exchange.
        await self.redis.publish(channel, json.dumps(payload))

This solution allows the platform to scale horizontally, serving hundreds of bots with zero additional load on the exchange API.

Visual Engine and AI Co-pilot

The DepthSight interface lets you build strategies using visual blocks (nodes). But the real magic happens when a user employs the AI Co-pilot.

A trader can write a prompt in natural language perfectly matching the visual editor's capabilities:

"Create a strategy: go long on the breakout of the local high on the 15-minute timeframe, confirm with a spike in volume delta. Set the stop-loss 0.5% below the nearest large order book density, and make the take-profit floating (trailing) with a 1% activation step."

The neural network generates a ready-to-use node tree from this request.

AI Hallucination Protection (Structured Outputs)

To prevent the LLM from outputting broken JSON, the backend strictly controls the neural network's output using Pydantic:

# api/schemas.py
from pydantic import BaseModel, Field
from typing import List, Optional, Literal

class GenerateStrategyRequest(BaseModel):
    text_prompt: str = Field(..., description="Text description of the strategy from the user.")
    current_config_json: Optional[Dict[str, Any]] = Field(
        None, description="Current configuration for modification."
    )

class StrategyV2ConfigData(BaseModel):
    strategy_name: str = Field(default="VisualBuilderStrategy")
    symbol: str
    marketType: Literal["FUTURES", "SPOT"]
    filters: ConditionNode
    entryConditions: ConditionNode
    positionManagement: List[ManagementBlock]
    oracle_regime: Optional[int] = None
    use_ml_confirmation: Optional[bool] = False

# api/routes/ai.py
@ai_core_router.post(
    "/generate-strategy",
    response_model=schemas.ApiResponseData[schemas.StrategyV2ConfigData],
)
async def generate_strategy_from_text_ai_core_endpoint(
    request: schemas.GenerateStrategyRequest,
    current_user: models.User = Depends(get_current_user),
):
    # LLM (Gemini/OpenRouter) generates JSON.
    # Pydantic strictly validates the response via StrategyV2ConfigData
    generated_json = await ai_assistant.generate_strategy_json_from_prompt(
        request, current_user
    )

    # Plus user permissions check (block availability based on their plan)
    enforce_strategy_plan_restrictions(generated_json, current_user)

    return {"data": generated_json}

Federated Discovery Hub (The Social Layer)

Trading entirely alone on your private server is secure, but it often lacks a fresh perspective. That's why the Discovery Hub is built directly into the DepthSight monorepo.

Even if you are a solo trader running a personal version of the platform on an isolated VPS, you can still be part of the global community. The platform uses a federated client-server model: your independent self-hosted instance can securely connect to the central community hub.

No private keys, IP addresses, or balances ever leave your server. Users share only what they want to: logic (node trees) and beautiful PnL charts.

Essentially, it's TradingView for algo-traders:

Strategy Sharing: Like someone's result? One click on "Import", and the completed node tree is instantly copied to your local editor, ready to be launched or tweaked.
Social Mechanics: It's not just a dry exchange of backtests. You can publish trade ideas, hypotheses, leave likes, and write comments.
Hive Mind: Discuss drawdowns, share trailing stop settings, or filters for fake breakouts.

This forms a powerful network effect: the community hunts for "alpha" together and shares the best setups, yet the keys and deposits of every trader remain exclusively in their own pocket.

The Deployment Experience

I tried to make deployment as "one-click" as possible. The repository includes a deploy.sh script. It automatically sets up the firewall (ufw), spins up Docker, configures Caddy for automatic HTTPS, and safely generates all passwords:

# Snippet from deploy.sh: Generating secure environment variables
if [ ! -f .env ]; then
    echo -e "[*] Generating secure .env configuration..."
    cp .env.example .env

    # Automatically create cryptographically strong secrets
    JWT_SECRET=$(openssl rand -base64 32)
    POSTGRES_PASS=$(openssl rand -hex 16)
    FERNET_KEY=$(python3 -c "import os, base64; print(base64.urlsafe_b64encode(os.urandom(32)).decode())")

    sed -i "s|JWT_SECRET_KEY=.*|JWT_SECRET_KEY=$JWT_SECRET|g" .env
    sed -i "s|POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$POSTGRES_PASS|g" .env
    sed -i "s|API_ENCRYPTION_KEY=.*|API_ENCRYPTION_KEY=$FERNET_KEY|g" .env
fi

You don't need to manually invent database passwords or salts for your JWTs. Everything is generated locally.

The "Solo + AI" Workflow: Scaling Beyond One Person

One of the most frequent questions I get is: "How did a single developer build a 280,000-line platform in less than a year?"

The answer isn't "I let the AI do everything." The key is treating LLMs as an Architectural Force Multiplier.

I focused on the high-level system design: the Market Data Fan-out pattern, the Pydantic validation layers, and the core trading logic. The AI handled the heavy lifting—generating over 1,100 unit tests, handling boilerplate for dozens of API endpoints, and prototyping React components.

However, rapid AI-driven development has its trade-offs, primarily in the form of architectural "God Objects." While the business logic is robust and the security is tight, the boundaries between the API and the Engine are sometimes blurred. I see this as the next phase of the project: I’ve built a massive, functional core, and I’m now inviting the community to help refine, decouple, and optimize it.

OTA Updates (Over-The-Air) from the UI

Maintaining a self-hosted instance is often a chore. To solve this, I implemented an OTA update system directly in the web interface.

When a user clicks "Update" in the UI, the FastAPI backend creates a trigger file (.update_trigger) in a shared volume. A host-side cron job detects this file and executes the update.sh script, which pulls the latest code from GitHub and rebuilds the containers. This allows users to stay up-to-date without ever touching the terminal.

Conclusion and Monetization

DepthSight is released under the GNU AGPL-3.0 license. This license gives you complete freedom to use, modify, and run the product for yourself.

How do I plan to monetize this?
I believe in transparent monetization. DepthSight is an official Bybit Broker Partner. I’ve integrated my Broker ID directly into the executor code. When your bot trades on the platform, the exchange shares a small portion of its commission with me. Your trading conditions, fees, and spreads remain exactly the same—the exchange pays the fee, not you.

This creates a win-win scenario: the software remains free and open-source, while the development is sustained by the platform's actual usage.

The project is currently in Open Beta. In the future, I plan to expand our broker partnerships and refine the genetic optimization engine.

If you're interested in the architecture of complex distributed systems, Python, React, or algorithmic trading—I would love your star on the repository or your Pull Requests!

🔗 GitHub: https://github.com/DepthSight-Pro/DepthSight/

P.S. Let me know in the comments which bots you currently use and why you do (or don't) trust them with your API keys.