Leveraging LLMs for Community Detection

#aiinfrastructure #oxlo #ai

Community detection is a foundational task in network analysis, traditionally solved with modularity-based algorithms such as Louvain or Leiden. These methods excel at finding densely connected subgraphs, but they often ignore rich textual attributes associated with nodes and edges. Large language models offer a complementary signal. By encoding semantic context, summarizing neighborhood structures, and even proposing merges or splits, LLMs can improve the coherence of communities in social, academic, and knowledge graphs. Oxlo.ai provides an inference platform that makes these workloads practical, with flat per-request pricing and long-context models that ingest entire subgraph descriptions without the cost surprises of token-based billing.

Why LLMs for Community Detection

Graph topology alone is sometimes ambiguous. Two nodes may share few edges yet belong to the same semantic community, or dense clusters may mix unrelated topics. LLMs address this through three mechanisms.

Attribute-aware clustering. When nodes carry text, such as user bios, paper abstracts, or product descriptions, an LLM can judge semantic similarity and propose groupings that pure topology misses.

Post-hoc explanation. After running a classical algorithm, an LLM can label each community, summarize its dominant themes, and flag outliers. This turns opaque cluster IDs into actionable intelligence.

Agentic refinement. In an iterative loop, an LLM reviews community boundaries, suggests merges where overlap is high, or proposes sub-communities where divergence is clear. This mirrors the manual tuning that data scientists already perform, but at machine speed.

Architecture Patterns

Integrating LLMs into a community detection pipeline does not require replacing your graph library. Most teams adopt one of the following patterns.

LLM-as-classifier. You embed each node’s neighborhood and attributes into a prompt, then ask the model to emit a community label. This works best when the number of communities is unknown and must be discovered from text.

LLM-as-descriptor. Run Louvain or Leiden to obtain hard assignments, then prompt an LLM to explain each cluster and detect semantic drift. If two topologically distinct communities share the same theme, the LLM flags a potential bridge node or data quality issue.

Agentic refinement. A stateful agent repeatedly queries the LLM with community statistics and sample node texts. The model returns operations such as MERGE, SPLIT, or RELABEL. Because this can involve dozens of turns, predictable per-request pricing becomes a significant operational advantage.

Concrete Implementation

The following example uses the OpenAI SDK to classify research papers into communities based on their abstracts. We point the client at Oxlo.ai’s API endpoint and use Llama 3.3 70B for general-purpose reasoning. Because Oxlo.ai charges a flat rate per request, we can include a large prompt with multiple examples and a partial citation graph without worrying about token length.

import openai
import json

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

def classify_papers(paper_nodes):
    """
    paper_nodes: list of dicts with 'id', 'title', 'abstract', 'neighbors'
    """
    prompt = (
        "You are a research analyst. Given the following papers and their neighbors, "
        "assign each paper to a thematic community. Return strict JSON.\n\n"
    )
    for p in paper_nodes:
        prompt += f"ID: {p['id']}\nTitle: {p['title']}\nAbstract: {p['abstract']}\n"
        prompt += f"Neighbors: {', '.join(p['neighbors'])}\n\n"

    prompt += (
        "Output a JSON object mapping each paper ID to a community label "
        "and a one-sentence rationale."
    )

    response = client.chat.completions.create(
        model="meta-llama/Llama-3.3-70B-Instruct",
        messages=[{"role": "user", "content": prompt}],
        response_format={"type": "json_object"}
    )

    return json.loads(response.choices[0].message.content)

# Example usage with a batch of 20 papers
batch = [...]  # your graph data
communities = classify_papers(batch)

For deeper reasoning over complex topology, swap the model string to deepseek-ai/DeepSeek-R1-671B-MoE or kimi/Kimi-K2.6. The response_format parameter enforces JSON mode, which lets you parse community assignments reliably into a networkx or igraph workflow.

Handling Scale and Cost

Community detection on large graphs is rarely a single prompt. It is a batch job: hundreds or thousands of nodes, iterative agent turns, or enrichment passes. Token-based providers scale costs with every word in your prompt, so including a 10,000-token neighborhood summary or a 1-million-token context window rapidly becomes prohibitive.

Oxlo.ai uses flat per-request pricing. One API call costs the same regardless of whether you send a terse node ID or an entire subgraph with long-form attributes. For long-context workloads and agentic refinement loops, this can make the pipeline significantly more predictable. You can explore the exact plan tiers on the Oxlo.ai pricing page.

Ingestion patterns that benefit from this model include:

Embedding an entire ego network into one prompt to resolve ambiguous memberships.
Passing high-resolution vision inputs, such as rendered graph layouts, to a vision-capable model like Kimi-K2.6 or Gemma-3-27B-IT.
Running multi-turn conversations where the LLM critiques its own prior community proposals.

Choosing the Right Model

Oxlo.ai hosts more than 45 models across seven categories. For community detection pipelines, we recommend matching the model to the data scale and reasoning depth.

Llama 3.3 70B: The general-purpose flagship. Use it for balanced latency and quality when classifying or explaining communities.
DeepSeek R1 671B MoE: Deploy this when the graph structure is deeply hierarchical or when you need chain-of-thought reasoning to justify a merge versus a split.
Kimi K2.

DEV Community