Using LLMs for Function-as-a-Service

#aiinfrastructure #oxlo #ai

Serverless platforms have always promised stateless execution without capacity planning. The next evolution replaces hand-written routing logic with an LLM that interprets events and invokes tools directly. In this pattern, the LLM itself becomes the compute layer: a Function-as-a-Service engine driven by natural language or structured payloads rather than static code paths. The challenge is that traditional token-based inference turns every invocation into a variable cost exercise. An oversized JSON event or a verbose system prompt can balloon expenses in exactly the way FaaS was meant to avoid. Oxlo.ai removes that uncertainty with flat per-request pricing, making it a practical backbone for LLM-powered serverless workflows.

Architecture: The LLM as a Router and Executor

A typical setup looks like an API Gateway or event bus forwarding requests to an inference endpoint instead of a traditional Lambda or Cloud Function. The LLM receives the event context, selects from a set of registered tools, and returns structured arguments. Your downstream services then execute the actual side effects. This collapses conditional branches, parsing logic, and validation into a single call.

The flow is straightforward. An inbound webhook or queue message is serialized into a prompt or JSON payload. The model applies function calling to decide what needs to happen. The response is parsed by a thin wrapper and executed. Because Oxlo.ai exposes an OpenAI-compatible API at https://api.oxlo.ai/v1, you can drop this into existing Python or Node.js handlers without rewriting your SDK logic.

Implementation with Python and Function Calling

Below is a minimal AWS Lambda-style handler that uses Oxlo.ai to process an e-commerce order event. The model decides whether to issue a refund, flag fraud, or email support based on the payload.

import os
import json
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["OXLO_API_KEY"],
    base_url="https://api.oxlo.ai/v1"
)

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "issue_refund",
            "description": "Process a full refund for an order",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"},
                    "reason": {"type": "string"}
                },
                "required": ["order_id", "reason"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "flag_fraud",
            "description": "Flag an order for manual review",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"},
                    "risk_score": {"type": "number"}
                },
                "required": ["order_id", "risk_score"]
            }
        }
    }
]

def handler(event, context):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {
                "role": "system",
                "content": "You are an order-processing agent. Use the available tools."
            },
            {
                "role": "user",
                "content": json.dumps(event)
            }
        ],
        tools=TOOLS,
        tool_choice="auto"
    )

    message = response.choices[0].message
    if message.tool_calls:
        # Execute the selected tool in your downstream FaaS layer
        return {
            "action": message.tool_calls[0].function.name,
            "arguments": json.loads(message.tool_calls[0].function.arguments)
        }
    return {"action": "none"}

Notice that the entire event is passed as a serialized string. With token-based billing, a large event could double or triple the invocation cost. On Oxlo.ai, the price remains constant regardless of whether the payload is two lines or two hundred.

Cost Predictability in Serverless Workloads

One of the core appeals of FaaS is the ability to forecast costs per invocation. When you add a token-based LLM to the chain, that forecast breaks. A function handling log analysis, document summaries, or multi-turn agent loops will encounter highly variable input lengths.

Oxlo.ai uses request-based pricing: one flat cost per API call. For long-context workloads, this can be significantly cheaper than token-based alternatives such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale. You do not need to truncate prompts or strip whitespace to save tokens. Your function can ingest full context and still produce a predictable bill. See https://oxlo.ai/pricing for current plan details.

Choosing Models for FaaS Patterns

Different serverless tasks demand different trade-offs between latency and capability. Oxlo.ai offers 45+ models across seven categories, all accessible through the same endpoint.

Llama 3.3 70B: A reliable default for general-purpose routing and function calling.
Qwen 3 32B: Strong multilingual support and agentic workflows when your FaaS handles global traffic.
DeepSeek R1 671B MoE: Use this when the function must perform deep reasoning or complex coding before selecting a tool.
Kimi K2.6: Advanced reasoning and agentic coding with a 131K context window, useful for processing large event batches in a single invocation.
DeepSeek V4 Flash: Efficient MoE with a 1M context window. Ideal for functions that ingest large logs or document state without chunking.

Because Oxlo.ai carries no cold starts on popular models, the first invocation after idle time responds immediately. This matches the expectations of synchronous FaaS callers who cannot wait for a container or model to warm up.

Beyond Routing: Structured Output and Embeddings

Function calling is not the only pattern. Many serverless functions simply need to transform unstructured input into structured JSON. Oxlo.ai supports JSON mode, so you can constrain model output to a schema without writing a parser.

For retrieval-augmented FaaS, you can also call Oxlo.ai embedding models such as BGE-Large or E5-Large from the same handler. The unified endpoint means your infrastructure stays simple even as the logic grows.

Conclusion

Using an LLM as a Function-as-a-Service engine removes boilerplate and lets you express logic through prompts and tools. The missing piece has been a pricing model that respects the serverless contract: predictable cost per invocation. Oxlo.ai delivers exactly that with flat per-request pricing, full OpenAI SDK compatibility, and a broad model catalog. If you are building event-driven AI functions, it is worth routing your next workload through https://api.oxlo.ai/v1.

DEV Community