DEV Community

shashank ms
shashank ms

Posted on

Using LLMs for Physics Research

Introduction

We are building a lightweight physics research assistant that checks derivations, evaluates equations safely, and extracts structured metadata from paper excerpts. Because Oxlo.ai uses flat request-based pricing, feeding it multi-page LaTeX preprints does not inflate cost the way token-based billing would.

What you'll need

  • Python 3.10 or newer
  • The OpenAI SDK plus SymPy: pip install openai sympy
  • An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Bootstrap the client

First, verify that your environment can reach Oxlo.ai and that a reasoning model can solve a basic kinematics problem. I use deepseek-v3.2 here because it handles math and coding tasks reliably.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[
        {"role": "user", "content": "A 2 kg block slides down a frictionless incline at 30 degrees. Calculate its acceleration in m/s^2."},
    ],
)
print(response.choices[0].message.content)

Step 2: Define the system prompt

The system prompt constrains the model to physics reasoning and forces it to delegate calculations to our local tool rather than hallucinating arithmetic.

SYSTEM_PROMPT = """You are a physics research assistant. Your job is to help analyze problems, check derivations, and suggest experiments.

Rules:
- Always reason step by step before concluding.
- When a user provides an equation or derivation, verify it symbolically before commenting.
- If you need to perform a calculation, call the tool `evaluate_math` rather than computing in your head.
- Cite physical constants and note uncertainties where relevant.
- If a claim lacks evidence, say so explicitly."""

Step 3: Add a math tool

We will use SymPy to evaluate expressions safely. We also expose the tool schema so the LLM knows when to call it.

import json
from sympy import sympify, latex, N

def evaluate_math(expression: str, precision: int = 6):
    """Safely evaluate a mathematical expression and return LaTeX plus numeric form."""
    try:
        expr = sympify(expression)
        return {
            "latex": latex(expr),
            "numeric": str(N(expr, precision)),
            "status": "ok"
        }
    except Exception as e:
        return {"status": "error", "message": str(e)}

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "evaluate_math",
            "description": "Evaluate a mathematical expression safely using SymPy.",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": "Math expression such as 'integrate(x**2, (x, 0, 1))'"
                    },
                    "precision": {
                        "type": "integer",
                        "description": "Decimal precision for numeric output."
                    }
                },
                "required": ["expression"]
            }
        }
    }
]

Step 4: Implement the tool loop

This function sends the conversation to Oxlo.ai with the tool definitions attached. If the model requests a tool call, we execute the local handler and send the result back for a final answer.

def run_agent(user_message: str, model: str = "deepseek-v3.2"):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_message},
    ]

    response = client.chat.completions.create(
        model=model,
        messages=messages,
        tools=TOOLS,
        tool_choice="auto",
    )

    msg = response.choices[0].message

    if msg.tool_calls:
        messages.append({
            "role": "assistant",
            "content": msg.content or "",
            "tool_calls": [
                {
                    "id": tc.id,
                    "type": tc.type,
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments,
                    },
                }
                for tc in msg.tool_calls
            ],
        })

        for tc in msg.tool_calls:
            fn_name = tc.function.name
            args = json.loads(tc.function.arguments)
            if fn_name == "evaluate_math":
                result = evaluate_math(**args)
            else:
                result = {"status": "error", "message": "Unknown tool"}

            messages.append({
                "role": "tool",
                "tool_call_id": tc.id,
                "content": json.dumps(result),
            })

        response = client.chat.completions.create(
            model=model,
            messages=messages,
        )
        msg = response.choices[0].message

    return msg.content

Step 5: Structured extraction

For literature review, we need structured metadata. We use JSON mode on kimi-k2.6 to pull authors, key equations, claims, and suggested experiments out of raw LaTeX.

def extract_paper_metadata(latex_excerpt: str, model: str = "kimi-k2.6"):
    prompt = f"""Extract metadata from the following physics paper excerpt.
Return valid JSON with exactly these keys: authors, key_equations, claims, suggested_experiments.

Excerpt:
{latex_excerpt}"""

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "You are a precise metadata extractor for scientific papers."},
            {"role": "user", "content": prompt},
        ],
        response_format={"type": "json_object"},
    )
    return json.loads(response.choices[0].message.content)

Run it

The snippet below ties everything together. It runs a derivation check through the agent and then extracts metadata from a fake excerpt.

if __name__ == "__main__":
    query = """Check this derivation:
Starting from F = ma for constant force, integrate acceleration with respect to time.
Does this yield v = v0 + (F/m)*t?
Then suggest a tabletop experiment that could validate the result."""

    print("=== AGENT RESPONSE ===")
    print(run_agent(query))

    print("\n=== STRUCTURED METADATA ===")
    excerpt = r"""
\author{J. Doe, R. Smith}
We present a modified Yukawa potential $V(r) = -g^2 e^{-mr}/r$
and argue it explains anomalous scattering at low energies.
A tabletop laser interferometry setup is proposed to measure the coupling $g$.
"""
    print(extract_paper_metadata(excerpt))

Example output:

=== AGENT RESPONSE ===
The derivation is correct. Starting from F = ma, constant acceleration a = F/m. Integrating a dt from 0 to t gives v - v0 = (F/m)*t, so v = v0 + (F/m)*t.

For a tabletop experiment, use an air track with a glider of known mass. Apply a constant force via a hanging mass and pulley. Measure velocity at multiple time gates and verify linearity against the predicted slope F/m.

=== STRUCTURED METADATA ===
{'authors': ['J. Doe', 'R. Smith'], 'key_equations': ['V(r) = -g^2 e^{-mr}/r'], 'claims': ['A modified Yukawa potential explains anomalous scattering at low energies'], 'suggested_experiments': ['Tabletop laser interferometry to measure the coupling g']}

Next steps

Wire this agent to an arXiv RSS feed so it auto-processes new preprints in your sub-field. If you need heavier chain-of-thought reasoning, swap deepseek-v3.2 for llama-3.3-70b or qwen-3-32b on Oxlo.ai without changing any client code.

Top comments (0)