When Regex Isn't Enough: Building a Natural Language Parser for User Input

I was building a simple task management app. You know, the kind where you type something like "remind me to buy milk tomorrow at 3pm" and it magically creates a calendar entry. Easy, right?

I started with regex. I spent a whole afternoon crafting patterns for dates, times, and actions. It worked for my test cases. Then I gave it to a friend. She typed "pick up milk from the store" – no date, no time, just an action. My regex failed. She tried "get milk tomorrow" – that worked. Then she typed "milk tomorrow afternoon" – my regex couldn't parse "afternoon" into a time.

I was stuck. The problem wasn't just the variety of ways people express things – it was that natural language is inherently messy. I needed something that could understand intent and extract structured data without me writing a thousand rules.

What I Tried That Didn't Work

First, I doubled down on regex. I added patterns for relative dates ("next week", "in two days"), times ("morning", "evening"), and actions ("buy", "pick up", "get"). The regex grew into a monster. It was brittle and slow. One user typed "remind me at 3pm to call dentist" – my regex expected the time after the action. Fail.

Next, I tried a simple NLP library – spaCy with some custom entity extraction. I trained it on a small dataset of task phrases. It worked better, but still missed edge cases. And training it for every new pattern was tedious. I wanted something that could understand any natural language input without me writing training data.

What Eventually Worked: AI-Powered Function Calling

I realized I didn't need to parse every word – I needed to extract the intent and parameters. That's exactly what large language models (LLMs) are good at. I started using OpenAI's function calling (or tool use) to define a structured output schema. The idea is simple: you tell the model what fields you want (like task, date, time), and it returns a JSON object with those fields.

Here's the approach:

1. Define a function schema that describes the structure you want.
2. Send the user's input along with the schema to the LLM.
3. The LLM returns a JSON object (or calls the function) with the extracted data.
4. Parse that JSON and use it in your app.

Code Example (Python with OpenAI)

import openai
import json

# Define the function schema
functions = [
    {
        "name": "create_task",
        "description": "Extract task details from natural language",
        "parameters": {
            "type": "object",
            "properties": {
                "task": {
                    "type": "string",
                    "description": "The action to perform, e.g., 'buy milk'"
                },
                "date": {
                    "type": "string",
                    "description": "Date in YYYY-MM-DD format. If not specified, assume today."
                },
                "time": {
                    "type": "string",
                    "description": "Time in HH:MM format. If relative like 'afternoon', convert to 14:00."
                }
            },
            "required": ["task"]
        }
    }
]

def parse_user_input(user_input: str) -> dict:
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that extracts task information."},
            {"role": "user", "content": user_input}
        ],
        functions=functions,
        function_call="auto"  # Let the model decide when to call the function
    )

    # Extract the function call arguments
    message = response["choices"][0]["message"]
    if message.get("function_call"):
        arguments = json.loads(message["function_call"]["arguments"])
        return arguments
    else:
        # Fallback: model didn't call function – maybe it's a greeting
        return {"task": user_input, "date": None, "time": None}

# Test it
print(parse_user_input("remind me to buy milk tomorrow at 3pm"))
# Output: {'task': 'buy milk', 'date': '2025-04-02', 'time': '15:00'}

print(parse_user_input("pick up milk from store"))
# Output: {'task': 'pick up milk from store', 'date': '2025-04-01', 'time': None}

This worked surprisingly well. The model understood "tomorrow" and "3pm" without me writing any parsing logic. It even handled relative times like "afternoon" (though I had to prompt it to convert to 14:00).

Lessons Learned & Trade-offs

• Cost: Each API call costs a fraction of a cent. For a small app, it's negligible. But if you're parsing thousands of inputs per day, it adds up. You can cache common patterns or use a cheaper model like gpt-3.5-turbo.
• Latency: The API call takes 1-3 seconds. For a real-time typing experience, you might want to debounce or show a loading state. Not ideal for every use case.
• Reliability: The model sometimes returns unexpected formats or misses fields. You need to validate the output and have fallbacks. For example, if date is missing, default to today.
• Overkill: If your input is highly structured (e.g., "task: buy milk, date: tomorrow"), regex is faster and cheaper. Use AI only when you need flexibility.

What I'd Do Differently Next Time

I'd start with a hybrid approach: use regex for simple patterns, and fall back to the AI parser for complex or ambiguous inputs. That way I save cost and latency for the easy cases. Also, I'd define stricter schemas – maybe include a priority field or category – and let the model infer them.

There are also services that wrap this exact technique – like the one at https://ai.interwestinfo.com/ – that offer pre-built parsers for common domains. But the core idea is the same: use function calling to get structured data from natural language.

When NOT to Use This

• If your input is always in a fixed format (e.g., form fields).
• If you need real-time parsing with no latency tolerance.
• If you're building a high-volume system on a tight budget.
• If you can't handle occasional incorrect outputs (AI is probabilistic).

The Takeaway

Regex is great for known patterns. But user input is never fully known. By leveraging AI's ability to understand intent, you can build a parser that handles the messy, human side of input without writing a thousand rules. It's not perfect, but it's a huge step up from regex hell.

What's your approach to parsing user input? Have you tried AI-based parsing, or do you stick with rule-based systems? I'd love to hear what works for you.