Introduction
Do you need to evaluate AI models with your data or cases? Yes, absolutely. Should it be done using complex and elaborate pipelines — it depends. Every time a new version of a model comes out, it behaves differently. You have to be able to evaluate each case with the model and see the differences. Sometimes you might realize that a lightweight, maybe cheaper model can work for your case as good as the more expensive one. In Google Cloud we have Gemini Enterprise Agent Platform which provides agent evaluation tools and you might have your own workflow going through sophisticated and elaborate tests. But what if you want to quickly check the new model or compare two different models available in your Cloud SQL for PostgreSQL or AlloyDB? You can use plain SQL and create a simple, quick evaluation procedure directly in your database. Let me show you an example of a single-answer point scoring evaluation.
AI Model Evaluation Components
What do you need for this evaluation? You need sample requests, registered model endpoint(s) you want to evaluate, a scale to quantify the performance, and evaluation criteria.
Here is a set of my sample requests:
'Explain Model Context Protocol (MCP) in 50 words or less.',
'Explain write-ahead logging (WAL) in PostgreSQL in 50 words or less.',
'Explain database connection pooling in 50 words or less.',
'Explain the difference between primary keys and unique constraints in 50 words or less.',
'Explain database normalization and its 3NF form in 50 words or less.',
'Explain transaction isolation levels like Serializable in 50 words or less.',
'Explain vector embeddings and cosine similarity in 50 words or less.',
'Explain database indexes (B-Tree vs Hash) in 50 words or less.',
'Explain the purpose of Foreign Keys in relational schemas in 50 words or less.',
'Explain optimistic vs pessimistic locking in databases in 50 words or less.'
Of course, all my questions were about databases. And since we were working in database — I put all my questions into the table:
CREATE TABLE IF NOT EXISTS benchmark_prompts (
id INT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
category TEXT NOT NULL,
prompt TEXT NOT NULL,
max_words INT DEFAULT 50,
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMPTZ DEFAULT clock_timestamp()
);
-- Index for fast active prompt lookups
CREATE INDEX IF NOT EXISTS idx_benchmark_prompts_active
ON benchmark_prompts (is_active, category);
INSERT INTO benchmark_prompts (category, prompt, max_words) VALUES
('Protocols', 'Explain Model Context Protocol (MCP) in 50 words or less.', 50),
('Architecture', 'Explain write-ahead logging (WAL) in PostgreSQL in 50 words or less.', 50),
('Networking', 'Explain database connection pooling in 50 words or less.', 50),
('Data Modeling','Explain the difference between primary keys and unique constraints in 50 words or less.', 50),
('Data Modeling','Explain database normalization and its 3NF form in 50 words or less.', 50),
('Concurrency', 'Explain transaction isolation levels like Serializable in 50 words or less.', 50),
('Vector Search','Explain vector embeddings and cosine similarity in 50 words or less.', 50),
('Storage', 'Explain database indexes (B-Tree vs Hash) in 50 words or less.', 50),
('Data Modeling','Explain the purpose of Foreign Keys in relational schemas in 50 words or less.', 50),
('Concurrency', 'Explain optimistic vs pessimistic locking in databases in 50 words or less.', 50);
Next, I registered three different models in my AlloyDB database. The models were Gemini 3.5 Flash and Gemini 3.5 Flash Lite. All three models were available in AlloyDB, I just needed to register them. Here is an example of how you can register the Gemini 3.5 Flash Lite model:
CALL google_ml.create_model(
model_id => 'gemini-3.5-flash-lite',
model_request_url => 'https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-3.5-flash-lite:generateContent',
model_provider => 'google',
model_type => 'llm'
);
Don’t forget to replace the PROJECT_ID placeholder with your real project id if you use the procedure to register a model in your database.
The list of evaluation criteria depends on the nature of your data and how you use the AI models. In my case, I wanted to put some basic quality criteria for the model’s response:
1. Accuracy: Is the response technically accurate regarding the subject described in the prompt?
2. Word compliance: Is the response 50 words or less?
3. Quality: How well does it answer the question?
I wanted all my evaluations to be scaled from 1 to 10, where 10 would be the best response.
How did I evaluate the responses? I registered one more model in my database just for the evaluation. I used the Gemini 3.1 Pro Preview as the Pro model should provide higher-quality evaluations. But nothing stops you from using any other models. For example, you can test it with our partner models from Anthropic or Meta which are available in our Model Garden.
To save some tokens and time I put a limit of 50 words for each response to a question.
And last but not least, I count the total tokens for each request and average them at the end because cost is important and can help to choose the right model for each case.
SQL Function for Evaluation
Here is an example of a SQL function with all the evaluation conditions directly in the SQL code. The function takes three parameters: model to evaluate, model which will be doing evaluation and number of executions:
CREATE OR REPLACE FUNCTION benchmark_model_quality(
p_target_model TEXT,
p_evaluator_model TEXT,
p_num_runs INT DEFAULT 10
)
RETURNS TABLE(
total_runs INT,
avg_quality_score NUMERIC,
median_quality_score NUMERIC,
min_quality_score NUMERIC,
max_quality_score NUMERIC,
avg_tokens_per_prompt NUMERIC
) AS $$
DECLARE
v_all_prompts TEXT[];
v_selected_prompts TEXT[] := '{}';
v_current_prompt TEXT;
v_predict_result JSON;
v_target_response TEXT;
v_tokens INT;
v_total_tokens INT := 0;
v_eval_prompt TEXT;
v_eval_raw TEXT;
v_score INT;
v_scores INT[] := '{}';
v_pool_size INT;
BEGIN
-- Input validation
IF p_num_runs <= 0 THEN
RAISE EXCEPTION 'p_num_runs must be greater than 0';
END IF;
-- Load prompts directly from the table
SELECT array_agg(prompt)
INTO v_all_prompts
FROM benchmark_prompts;
v_pool_size := cardinality(v_all_prompts);
IF v_pool_size IS NULL OR v_pool_size = 0 THEN
RAISE EXCEPTION 'No prompts found in benchmark_prompts table';
END IF;
-- 1. Select exactly p_num_runs prompts WITH replacement from table pool
SELECT ARRAY(
SELECT v_all_prompts[floor(random() * v_pool_size + 1)::INT]
FROM generate_series(1, p_num_runs)
) INTO v_selected_prompts;
-- 2. Loop through each of the selected subjects and execute exactly ONCE
FOREACH v_current_prompt IN ARRAY v_selected_prompts LOOP
-- A. Query target model
SELECT google_ml.predict_row(
model_id => p_target_model,
request_body => json_build_object(
'contents', json_build_array(
json_build_object(
'role', 'user',
'parts', json_build_array(
json_build_object('text', v_current_prompt)
)
)
)
)
) INTO v_predict_result;
-- B. Extract response text and tokens
v_target_response := v_predict_result -> 'candidates' -> 0 -> 'content' -> 'parts' -> 0 ->> 'text';
v_tokens := COALESCE((v_predict_result -> 'usageMetadata' ->> 'totalTokenCount')::INT, 0);
v_total_tokens := v_total_tokens + v_tokens;
-- C. Build adaptive judge prompt
v_eval_prompt := format(
'You are an expert technical evaluator. Evaluate the following generated response to the prompt: "%s"
Generated Response to Evaluate:
"%s"
Instructions:
Evaluate the response based on:
1. Accuracy: Is the response technically accurate regarding the subject described in the prompt?
2. Word compliance: Is the response 50 words or less?
3. Quality: How well does it answer the question?
You must respond ONLY with a raw JSON object conforming to the following schema, and no other text or Markdown formatting:
{
"score": <integer from 1 to 10>,
"reasoning": "<short description of why the score was given under 50 words>"
}',
v_current_prompt,
v_target_response
);
-- D. Query evaluator model
SELECT google_ml.predict_row(
model_id => p_evaluator_model,
request_body => json_build_object(
'contents', json_build_array(
json_build_object(
'role', 'user',
'parts', json_build_array(
json_build_object('text', v_eval_prompt)
)
)
)
)
) -> 'candidates' -> 0 -> 'content' -> 'parts' -> 0 ->> 'text' INTO v_eval_raw;
-- E. Clean up markdown code blocks
v_eval_raw := regexp_replace(v_eval_raw, '^```
json\s*', '', 'i');
v_eval_raw := regexp_replace(v_eval_raw, '\s*
```$', '', 'i');
v_eval_raw := trim(v_eval_raw);
-- F. Parse and store score
BEGIN
v_score := (v_eval_raw::json ->> 'score')::INT;
v_scores := array_append(v_scores, v_score);
EXCEPTION WHEN OTHERS THEN
RAISE WARNING 'Failed to parse JSON score from evaluator output: %', v_eval_raw;
END;
END LOOP;
-- 3. Compute aggregate stats over successfully parsed scores
RETURN QUERY
WITH sorted_scores AS (
SELECT s,
row_number() OVER (ORDER BY s) as rn,
count(*) OVER () as total_count
FROM unnest(v_scores) AS s
),
stats AS (
SELECT
avg(s)::NUMERIC(10,2) as v_avg,
min(s) as v_min,
max(s) as v_max
FROM unnest(v_scores) AS s
),
median_stat AS (
SELECT avg(s)::NUMERIC(10,2) as v_median
FROM sorted_scores
WHERE rn IN (floor((total_count + 1)/2.0), ceil((total_count + 1)/2.0))
)
SELECT
cardinality(v_scores),
stats.v_avg,
median_stat.v_median,
stats.v_min::NUMERIC,
stats.v_max::NUMERIC,
(v_total_tokens::NUMERIC / NULLIF(cardinality(v_scores), 0))::NUMERIC(10,2)
FROM stats, median_stat;
END;
$$ LANGUAGE plpgsql;
There are some potential improvements to the procedure and the method depending on use case. For example you can expand it with “Golden” responses for each question for evaluation. It is useful when you are working with your own domain knowledge and know exactly how the best answer should sound like.
Testing
I tried it on two different models and compared the results.
The first test was for the Gemini 3.5 Flash model. I set the test to run 45 times:
SELECT * FROM benchmark_model_quality('gemini-3.5-flash', 'gemini-3.1-pro-preview', 45);
The Gemini 3.5 Flash showed really good results:
total_runs | avg_quality_score | median_quality_score | min_quality_score | max_quality_score | avg_tokens_per_prompt
------------+-------------------+----------------------+-------------------+-------------------+-----------------------
45 | 9.96 | 10.00 | 8 | 10 | 916.27
Then I ran the same test for Gemini 3.5 Flash Lite:
total_runs | avg_quality_score | median_quality_score | min_quality_score | max_quality_score | avg_tokens_per_prompt
------------+-------------------+----------------------+-------------------+-------------------+-----------------------
45 | 9.82 | 10.00 | 7 | 10 | 77.11
For that sample set, Gemini 3.5 Flash Lite showed almost the same quality and cost much less.
If you compare the tokens per request, you can see an 11x reduction in token count. And those tokens are 3.5 times cheaper than Gemini 3.5 Flash. So, you can effectively pay 38 times less for the same quality response.
Conclusion
There are two things I want to say in this article. If you are working in AlloyDB or Cloud SQL and need to quickly evaluate some AI models you can try it using SQL function thanks to the direct integration with AI in Google Cloud databases.
And maybe you can use a less expensive model for your workload and get acceptable results saving some money.
Happy testing and let me know if you think such kind of simple recipes are useful for you.
Top comments (0)