Davide

Posted on Jan 10

I Built a Zuora Workflow Manager with Laravel 12 + Filament (Full Tech Stack Walkthrough)

#laravel #php #filament #opensource

TL;DR

Built an open-source Zuora Workflow Manager to automate workflow synchronization with these features:

🔄 Automatic sync with configurable schedules (hourly by default)
📊 Filament admin panel with rich visualization
⚙️ Background jobs with retry logic (3x, 60s backoff)
🔐 OAuth 2.0 with 1-hour token caching
🎯 Multi-tenant with per-customer credentials
📋 Task extraction from workflow JSON
🔒 Encrypted settings with custom Cast
📈 Interactive graph visualization
🎨 Modern UI with screenshots

Link Repository

📚 Full Documentation | 🚀 Latest Release | 🐛 Issues

Screenshots 📸

Before diving into code, here's what the app looks like:

Login Screen (with Google OAuth support)

Dashboard Overview

Workflows List (search, filter, multi-tenant)

Workflow Detail (graph view, tasks, JSON export)

The Problem 🎯

Managing Zuora workflows manually across multiple clients is a nightmare:

Manual synchronization is time-consuming and error-prone
No visibility into workflow structure without diving into JSON
OAuth token management requires constant attention
Failed API calls mean stale data
Multi-tenant complexity with separate credentials per client

I needed a solution that handles all this automatically while providing a beautiful admin interface.

Tech Stack 🛠️

Backend:        Laravel 12.45.1 + PHP 8.4
Admin Panel:    Filament 4.2 + Filament Shield
Database:       MariaDB 11.4
Queue:          Database Driver (with retry logic)
Cache:          Redis 7.0
DevOps:         Lando (Docker-based)
Frontend:       Livewire + Tailwind CSS 4
Visualization:  @joint/layout-directed-graph
Settings:       Spatie Laravel Settings
Jobs Monitor:   Moox Jobs
Testing:        PHPUnit + Parallel execution

The Problem 🎯

Managing Zuora workflows manually across multiple clients is a nightmare:

Manual synchronization is time-consuming and error-prone
No visibility into workflow structure without diving into JSON
OAuth token management requires constant attention
Failed API calls mean stale data
Multi-tenant complexity with separate credentials per client

I needed a solution that handles all this automatically while providing a beautiful admin interface.

Tech Stack 🛠️

Backend:        Laravel 12 + PHP 8.4
Admin Panel:    Filament 4.2 + Filament Shield
Database:       MariaDB 11.4
Queue:          Database Driver (with retry logic)
Cache:          Redis 7.0
DevOps:         Lando (Docker-based)
Frontend:       Livewire + Tailwind CSS 4
Visualization:  @joint/layout-directed-graph
Settings:       Spatie Laravel Settings
Jobs Monitor:   Moox Jobs
Testing:        PHPUnit + Parallel execution

Key Features ✨

🔄 Automatic Synchronization

Workflows sync automatically on a configurable schedule:

// routes/console.php
Schedule::command('app:sync-workflows --all')
    ->hourly()  // Configurable: ->everyFiveMinutes(), ->daily(), etc.
    ->name('sync-customer-workflows');

Each sync job:

Fetches workflows from Zuora REST API (automatic pagination)
Downloads complete workflow JSON
Extracts and stores tasks
Updates database atomically
Retries on failure (3 attempts, 60s backoff)

📊 Rich Admin Dashboard

Built with Filament 4.2, providing:

Search & Filters: Find workflows by name, state, customer
Workflow Details: Multi-tab view with graph, JSON, and tasks
Task Manager: Filter by action type, priority, state
JSON Operations: Copy or download workflow JSON
Real-time Updates: Livewire reactive components

⚙️ Background Job Processing

All sync operations run in background queues:

// Dispatch job
SyncCustomerWorkflows::dispatch($customer);

// Job configuration
public $tries = 3;
public $backoff = 60;  // Exponential backoff
public $timeout = 300;

Monitoring: Moox Jobs integration provides real-time tracking:

Running jobs
Waiting jobs
Failed jobs (with retry capability)
Job batches

🔐 OAuth 2.0 with Smart Caching

Token management with 1-hour caching reduces API calls by 90%:

// ZuoraService.php
protected function getAccessToken(Customer $customer): string
{
    return Cache::remember(
        "zuora_token_{$customer->id}",
        3600,  // 1 hour
        fn() => $this->requestNewToken($customer)
    );
}

🏢 Multi-Tenant Architecture

Each customer has isolated credentials stored in the database:

// Database schema
customers:
  - id
  - name
  - client_id      (Zuora OAuth)
  - client_secret  (Zuora OAuth, encrypted)
  - base_url       (Production/Test/Sandbox)

No shared credentials = better security and isolation.

📋 Automatic Task Extraction

Tasks are parsed from workflow JSON and stored separately:

// Workflow.php
public function syncTasksFromJson(): void
{
    $tasks = $this->extractTasksFromWorkflow($this->workflow_json);

    // Atomic update
    $this->tasks()->delete();
    $this->tasks()->createMany($tasks);
}

Task filters:

Action Type: Email, Export, SOAP, REST, etc.
Priority: High, Medium, Low
State: Active, Inactive, Pending

🔒 Encrypted Settings

Sensitive configuration uses custom EncryptedCast:

// GeneralSettings.php (camelCase properties)
use App\Casts\EncryptedCast;

class GeneralSettings extends Settings
{
    public string $oauthGoogleClientSecret;

    protected $casts = [
        'oauthGoogleClientSecret' => EncryptedCast::class,
    ];
}

Data encrypted at rest using Laravel's Crypt facade.

Architecture Deep Dive 🏗️

Service Layer Pattern

Clean separation of concerns:

ZuoraService

class ZuoraService
{
    public function getWorkflows(Customer $customer, int $page = 1): array
    public function exportWorkflow(Customer $customer, string $workflowId): array
    protected function getAccessToken(Customer $customer): string
}

WorkflowSyncService

class WorkflowSyncService
{
    public function syncCustomerWorkflows(Customer $customer): void
    protected function fetchAndSaveWorkflows(Customer $customer): void
    protected function extractTasks(Workflow $workflow): void
}

Benefits:

Easy to test (mock services)
Reusable across controllers and jobs
Clear responsibilities

Queue Processing Flow

┌─────────────────┐
│   User Action   │
│   (UI or CLI)   │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────┐
│ SyncCustomerWorkflows Job   │
│ → Dispatched to Queue       │
└────────┬────────────────────┘
         │
         ▼
┌─────────────────────────────┐
│   Queue Worker Process      │
│   (lando queue / cron)      │
└────────┬────────────────────┘
         │
         ▼
┌─────────────────────────────┐
│  WorkflowSyncService        │
│  ├─ Fetch from API          │
│  ├─ Download JSON           │
│  ├─ Save workflows          │
│  └─ Extract tasks           │
└────────┬────────────────────┘
         │
         ▼
┌─────────────────────────────┐
│   Database Storage          │
│   (workflows + tasks)       │
└────────┬────────────────────┘
         │
         ▼
┌─────────────────────────────┐
│   Filament UI Display       │
└─────────────────────────────┘

Database Schema 📊

Workflows Table

CREATE TABLE workflows (
    id BIGINT UNSIGNED PRIMARY KEY,
    customer_id BIGINT UNSIGNED NOT NULL,
    zuora_id VARCHAR(255) NOT NULL UNIQUE,
    name VARCHAR(255) NOT NULL,
    description TEXT,
    state VARCHAR(255),
    workflow_json JSON,
    last_synced_at TIMESTAMP,

    FOREIGN KEY (customer_id) REFERENCES customers(id) ON DELETE CASCADE,
    INDEX idx_customer_id (customer_id),
    INDEX idx_zuora_id (zuora_id),
    INDEX idx_state (state)
);

Tasks Table

CREATE TABLE tasks (
    id BIGINT UNSIGNED PRIMARY KEY,
    workflow_id BIGINT UNSIGNED NOT NULL,
    zuora_id VARCHAR(255) NOT NULL,
    name VARCHAR(255) NOT NULL,
    task_type VARCHAR(255),
    action_type VARCHAR(255),
    data JSON,

    FOREIGN KEY (workflow_id) REFERENCES workflows(id) ON DELETE CASCADE,
    INDEX idx_workflow_id (workflow_id),
    INDEX idx_task_type (task_type)
);

Foreign keys with CASCADE ensure data integrity when customers or workflows are deleted.

Code Highlights 💡

1. Job Retry Logic with Exponential Backoff

// Jobs/SyncCustomerWorkflows.php
class SyncCustomerWorkflows implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 3;
    public $backoff = 60;  // 60, 120, 240 seconds
    public $timeout = 300;

    public function handle(WorkflowSyncService $syncService): void
    {
        try {
            $syncService->syncCustomerWorkflows($this->customer);
        } catch (Exception $e) {
            Log::error("Workflow sync failed for customer {$this->customer->id}: {$e->getMessage()}");
            throw $e;  // Trigger retry
        }
    }
}

2. OAuth Token Caching

// Services/ZuoraService.php
protected function getAccessToken(Customer $customer): string
{
    $cacheKey = "zuora_token_{$customer->id}";

    return Cache::remember($cacheKey, 3600, function() use ($customer) {
        $response = Http::asForm()->post("{$customer->base_url}/oauth/token", [
            'client_id' => $customer->client_id,
            'client_secret' => $customer->client_secret,
            'grant_type' => 'client_credentials',
        ]);

        return $response->json('access_token');
    });
}

3. Automatic Pagination Handling

// Services/WorkflowSyncService.php
protected function fetchAllWorkflows(Customer $customer): Collection
{
    $allWorkflows = collect();
    $page = 1;

    do {
        $response = $this->zuoraService->getWorkflows($customer, $page);
        $workflows = collect($response['data']);

        $allWorkflows = $allWorkflows->merge($workflows);
        $page++;

    } while ($workflows->isNotEmpty() && $workflows->count() >= 50);

    return $allWorkflows;
}

4. Custom Encrypted Cast

// Casts/EncryptedCast.php
namespace App\Casts;

use Illuminate\Contracts\Database\Eloquent\CastsAttributes;
use Illuminate\Support\Facades\Crypt;

class EncryptedCast implements CastsAttributes
{
    public function get($model, string $key, $value, array $attributes)
    {
        return $value ? Crypt::decryptString($value) : null;
    }

    public function set($model, string $key, $value, array $attributes)
    {
        return $value ? Crypt::encryptString($value) : null;
    }
}

5. Task Extraction from JSON

// Models/Workflow.php
public function syncTasksFromJson(): void
{
    if (!$this->workflow_json) {
        return;
    }

    $tasks = $this->extractTasksFromWorkflow($this->workflow_json);

    // Atomic update
    DB::transaction(function() use ($tasks) {
        $this->tasks()->delete();
        $this->tasks()->createMany($tasks);
    });
}

protected function extractTasksFromWorkflow(array $workflowData): array
{
    $tasks = [];
    $workflowTasks = $workflowData['tasks'] ?? [];

    foreach ($workflowTasks as $task) {
        $tasks[] = [
            'zuora_id' => $task['id'],
            'name' => $task['name'],
            'task_type' => $task['task_type'] ?? null,
            'action_type' => $task['action_type'] ?? null,
            'data' => $task,
        ];
    }

    return $tasks;
}

Development Setup 🚀

Prerequisites

Lando (includes Docker)
Node.js 24+
Yarn

Quick Start

# Clone repository
git clone https://github.com/FrancoStino/zuora-workflows.git
cd zuora-workflows

# Start Lando (automatically runs composer install)
lando start

# Setup environment
cp .env.example .env
lando artisan key:generate

# Run migrations
lando artisan migrate

# Install frontend dependencies
lando yarn install
yarn run build

# Start queue worker (for background jobs)
lando queue

# Start scheduler (for automatic sync)
lando schedule

Access the app: https://zuora-workflows.lndo.site

Lando Commands

lando artisan [command]     # Run artisan commands
lando composer [command]    # Run composer
lando yarn [command]        # Run yarn
lando queue                 # Start queue worker
lando schedule              # Start scheduler
lando test                  # Run tests
lando logs -f               # View logs
lando mariadb               # Database CLI

Why Lando?

Before Lando: Hours setting up PHP, MySQL, Redis, configuring versions, dealing with platform differences.

With Lando:

lando start

That's it. Consistent environment for everyone.

Testing Strategy ✅

Running Tests

# All tests
lando test

# Specific test file
lando artisan test tests/Feature/SyncWorkflowsTest.php

# With output
lando artisan test -v

# With coverage
lando artisan test --coverage

# Parallel execution
lando artisan test --parallel

Test Structure

tests/
├── Feature/
│   ├── WorkflowSyncTest.php       # Integration tests
│   ├── OAuthTest.php              # OAuth flow tests
│   └── TaskExtractionTest.php     # Task parsing tests
├── Unit/
│   ├── ZuoraServiceTest.php       # Service unit tests
│   └── EncryptedCastTest.php      # Cast tests
└── TestCase.php

Example Test

// tests/Feature/WorkflowSyncTest.php
class WorkflowSyncTest extends TestCase
{
    public function test_workflow_sync_creates_workflows_and_tasks()
    {
        // Arrange
        $customer = Customer::factory()->create();
        Http::fake([
            '*/v1/workflows*' => Http::response([
                'data' => [
                    [
                        'id' => 'wf_123',
                        'name' => 'Test Workflow',
                        'state' => 'Active',
                    ]
                ]
            ]),
            '*/v1/workflows/wf_123/export' => Http::response([
                'tasks' => [
                    ['id' => 'task_1', 'name' => 'Send Email']
                ]
            ]),
        ]);

        // Act
        $this->artisan('app:sync-workflows', ['--customer' => $customer->name]);

        // Assert
        $this->assertDatabaseHas('workflows', [
            'zuora_id' => 'wf_123',
            'name' => 'Test Workflow',
        ]);

        $this->assertDatabaseHas('tasks', [
            'zuora_id' => 'task_1',
            'name' => 'Send Email',
        ]);
    }
}

Deployment Options 🌐

Option A: Sync Queue (Simplest)

Perfect for shared hosting where background processes are limited:

QUEUE_CONNECTION=sync

Pros:

No cron setup needed
No queue worker required
Jobs execute immediately

Cons:

Blocks request until job completes
Not ideal for large sync operations

Option B: Database Queue + Cron (Recommended)

For proper background processing:

QUEUE_CONNECTION=database

Add cron job:

* * * * * cd /path/to/app && php artisan schedule:run >> /dev/null 2>&1

Pros:

Non-blocking UI
Automatic retry on failure
Job monitoring and tracking

Cons:

Requires cron access
Slightly more complex setup

Option C: Redis Queue (Production)

For high-performance production environments:

QUEUE_CONNECTION=redis
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

Pros:

Fastest queue performance
Supports job prioritization
Distributed processing

Cons:

Requires Redis server
Additional infrastructure

📚 Full Deployment Guide

Challenges & Solutions 🔧

Challenge	Problem	Solution
OAuth Expiry	Tokens expire after 1 hour	1-hour cache + automatic refresh
API Pagination	Max 50 items per request	Automatic loop handling
Failed Jobs	Network/timeout issues	Retry logic 3x with 60s backoff
Multi-tenant	Shared credentials = security risk	Per-customer credentials in DB
Task Extraction	Complex nested JSON structures	Dedicated parsing method + atomic DB update
Sensitive Data	Secrets stored in plaintext	Custom EncryptedCast with Laravel Crypt
Development Setup	"Works on my machine"	Lando containerization
Code Consistency	Mixed naming conventions	Refactored to camelCase

Performance Considerations 📈

Database Optimization

-- Foreign key indexes
INDEX idx_customer_id (customer_id)
INDEX idx_workflow_id (workflow_id)

-- Query optimization indexes
INDEX idx_zuora_id (zuora_id)
INDEX idx_state (state)
INDEX idx_task_type (task_type)

-- Composite indexes for common queries
INDEX idx_customer_state (customer_id, state)

Caching Strategy

Layer	TTL	Purpose
OAuth Tokens	1 hour	Reduce auth API calls by 90%
Settings	Until changed	Avoid DB reads on every request
Workflow List	5 minutes	Optional for high-traffic dashboards

Query Optimization

// BAD: N+1 queries
$workflows = Workflow::all();
foreach ($workflows as $workflow) {
    echo $workflow->customer->name;  // Queries for each workflow
}

// GOOD: Eager loading
$workflows = Workflow::with('customer')->get();
foreach ($workflows as $workflow) {
    echo $workflow->customer->name;  // No additional queries
}

Job Processing

Queue workers: Run multiple workers for parallel processing
Job batching: Group related jobs for better efficiency
Timeout handling: 300s timeout prevents stuck jobs

Monitoring & Debugging 🔍

View Logs

# Real-time logs
lando logs -f

# Specific service (PHP)
lando logs -s appserver -f

# Laravel logs
lando exec appserver tail -f storage/logs/laravel.log

# Filter for errors
lando logs -f | grep -i "error"

Check Queue Status

# Failed jobs
lando artisan queue:failed

# Retry all failed
lando artisan queue:retry all

# Retry specific job
lando artisan queue:retry {job-id}

# Clear failed jobs
lando artisan queue:flush

Database Inspection

# Access MariaDB
lando mariadb

# Check queue
SELECT * FROM jobs;

# Check failed jobs
SELECT * FROM failed_jobs ORDER BY failed_at DESC;

# Check recent workflows
SELECT * FROM workflows ORDER BY last_synced_at DESC LIMIT 10;

Moox Jobs Panel

Access via Filament admin: Jobs → Dashboard

Running Jobs: Currently processing
Waiting Jobs: In queue
Failed Jobs: With error messages and retry button
Job Batches: Batch operations tracking

What's Next? 🚀

Roadmap

[ ] Webhook Support: Real-time sync when workflows change in Zuora
[ ] Advanced Analytics: Workflow execution statistics and trends
[ ] Bulk Operations: Update multiple workflows at once
[ ] REST API: External integrations and programmatic access
[ ] Workflow Builder: Visual editor for creating workflows
[ ] Export Formats: PDF, CSV, Excel reports
[ ] Notification System: Email/Slack alerts for failed syncs
[ ] Audit Logs: Track all workflow changes
[ ] Multi-language Support: i18n for admin panel

Contributing

Contributions welcome! See CONTRIBUTING.md

Ways to contribute:

🐛 Report bugs
💡 Suggest features
📖 Improve docs
🔧 Submit PRs
⭐ Star the repo

Lessons Learned 📚

1. Service Layer Pattern = Maintainability

Keeping business logic in service classes made:

Testing much easier (mock services)
Code reusable across controllers and jobs
Debugging simpler (clear responsibilities)

2. Queue Retry Logic is Critical

When dealing with external APIs, failures are inevitable:

Network timeouts
Rate limiting
Server errors

Automatic retry with exponential backoff saved countless manual interventions.

3. Lando Eliminates "Works on My Machine"

Before Lando: Hours debugging environment issues.

After Lando: lando start and everyone has identical setups.

4. Filament is a Game Changer

Building an admin panel from scratch = weeks of work.

With Filament:

Beautiful UI out of the box
RBAC with Filament Shield
Relationship management
Form builders and tables

Result: Functional admin panel in hours, not days.

5. Visual Graphs Add Tremendous Value

Users understand complex workflows 10x faster with graphical representation vs. reading JSON.

6. Encrypted Settings = Peace of Mind

Storing OAuth secrets in plaintext = security risk.

Custom EncryptedCast = automatic encryption at rest with zero overhead.

7. Consistent Naming Matters

Refactoring to camelCase:

Improved code readability
Aligned with Laravel best practices
Made IDE autocomplete more reliable

Resources & Links 📎

Documentation

Technologies

Repository

Conclusion

Building Zuora Workflow Manager taught me the importance of:

✅ Choosing the right tools (Laravel + Filament + Lando)

✅ Clean architecture (Service layer pattern)

✅ Reliability (Queue retry logic, caching)

✅ Developer experience (Lando = zero config hell)

✅ User experience (Visual graphs, real-time monitoring)

✅ Code consistency (camelCase refactoring)