DEV Community: Kesavaram Rathnasingam

Can AI Be Trusted to Code Financial Software? I Built a Benchmark to Find Out

Kesavaram Rathnasingam — Thu, 08 Oct 2026 04:10:23 +0000

This is a submission for the Kaggle Benchmarking Challenge

What I Benchmarked
BankSafe-Bench: Can AI Safely Code Financial Software?

I built BankSafe-Bench, a benchmark designed to evaluate how well AI coding models handle realistic financial software engineering tasks.

Instead of testing models with generic programming questions, the benchmark focuses on situations where “code that looks correct” is not necessarily correct.

The benchmark covers tasks across:

C# / ASP.NET Core backend development
SQL Server queries and data operations
Financial calculations and business rules
Edge-case handling and validation
Database transactions and consistency
Concurrency and failure scenarios
Secure coding practices
Legacy-code modification
Debugging and code review

For example, a task may provide a financial calculation with several business rules and edge cases and ask the model to implement it correctly. Another task may present a multi-step database operation and ask the model to identify what could happen if one operation fails.

The goal is to measure something more useful than whether an AI can generate code:

Can an AI coding model produce software that is actually safe and reliable when financial correctness matters?

This interests me because modern AI coding assistants are increasingly used for production software development, while financial applications have requirements where small mistakes in calculations, authorization, transactions, or data handling can have significant consequences.

Models Tested

I selected models representing different approaches to AI-assisted software engineering, including:

A leading general-purpose model
A reasoning-focused model
A coding-focused model
A strong open-weight model
A smaller/efficiency-focused model

The models were evaluated using the same benchmark tasks and scoring criteria so that their performance could be compared on identical problems.

The benchmark does not assume that the largest or most popular model will perform best. The goal is to discover which capabilities actually matter for financial software engineering.

Findings

The benchmark is designed to evaluate models across several dimensions rather than using a single “correct/incorrect” score.

Each task is scored across:

Category Weight
Functional correctness 30%
Financial/business-rule correctness 25%
Edge-case handling 15%
Security 15%
Code quality 10%
Explanation 5%

The most interesting part of this benchmark is the difference between code generation ability and engineering reliability.

I will report:

Overall model scores
Performance by task category
Financial-rule accuracy
Security failures
Edge cases missed
Transaction/concurrency failures
Examples of seemingly correct solutions that fail under realistic conditions
What surprised me

Rather than focusing only on which model achieves the highest overall score, I want to investigate where models fail.

For example:

Can a model produce code that compiles but violates a financial rule?

Can it recognize a transaction-consistency problem that isn't obvious from the happy path?

Can it distinguish a technically valid SQL query from one that produces incorrect financial results?

These failures are potentially more important than simple syntax or compilation errors.

What I would measure next

Future versions could expand the benchmark to include:

Multi-turn debugging
Realistic API integration
Codebase-level changes
Automated test generation
Performance optimization
Database deadlock diagnosis
Authentication and authorization
AI-generated code review
Agentic coding workflows

The long-term goal is to develop a benchmark that measures engineering reliability, not just code-generation capability.

My Benchmark

Kaggle Benchmark:
BankSafe-Bench — Can AI Safely Code Financial Software?

The benchmark is designed to be reproducible so that additional models and new versions of existing models can be evaluated as AI coding capabilities evolve.

LankaLens — AI Nature Explorer

Kesavaram Rathnasingam — Thu, 08 Oct 2026 04:06:30 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built
🌿 LankaLens — AI Nature Explorer

I built LankaLens, a mobile application that encourages people to put their phones down and explore the nature around them.

The idea is simple: when you're walking, hiking, gardening, or exploring outdoors, you can take a photo of a plant, tree, flower, insect, or other part of nature. LankaLens uses open-source AI to help identify and explain what you found.

But the goal isn't to keep people looking at the app.

The app gives you a short explanation and an observation challenge, such as:

"Look around you and see if you can find another tree with similar leaves."

Then the phone goes back into your pocket.

I designed it for people who are curious about nature but don't necessarily know the names of the things they see around them.

The longer-term goal is to make it particularly useful for exploring Sri Lankan plants, trees, birds, insects, and local biodiversity, with support for English, Sinhala, and Tamil.

Demo

🎥 Demo video: [Coming soon]

The demo will show the application being used outdoors with the device disconnected from the internet, demonstrating the local-first AI workflow.

Code

💻 GitHub: [Coming soon]

The project will be open source so others can experiment with the models, improve the identification pipeline, add local species knowledge, and adapt it to their own regions.

How I Built It

The application is built around the idea that AI should work for the outdoor experience rather than become the experience.

The mobile application handles:

📷 Camera-based nature observations
📍 Location-aware exploration
💾 Offline storage
🌿 Nature observations and history
🧭 Small outdoor exploration challenges
🌐 English, Sinhala, and Tamil support

At the core is an open-weight AI model running locally rather than sending every photograph to a proprietary cloud AI API.

The architecture is designed so that the AI model can be replaced or upgraded without rebuilding the entire application.

         Mobile App
             │
    ┌────────┼────────┐
    │        │        │
 Camera     GPS    Offline DB
    │
    ▼

Local AI Inference
│
▼
Open-Weight Model
│
▼
Nature Identification
│
▼
Short Explanation
│
▼
Exploration Challenge
│
▼
📱 → Pocket
🌳 → Explore
Why Does Open Innovation Matter?

For this project, open AI isn't just a technology choice. It changes what the application can do.

Nature exploration often happens in places where mobile connectivity is unreliable or unavailable. A cloud-only AI application would make the experience dependent on an internet connection and an external service.

With an open-weight model and local inference, the goal is to make the core experience possible without an internet connection.

It also means users don't have to upload every photograph they take to a third-party AI service.

More importantly, an open approach gives the project room to evolve.

I can experiment with different models, improve the prompts and inference pipeline, add local knowledge about Sri Lankan species, and eventually fine-tune or replace components without being locked into a single proprietary API.

For this project, open innovation makes AI more accessible, private, customizable, and useful in the places where people actually go outside.

My Agent Session

Coming soon.

I will include the DevRelay session showing how the project was designed and built.

Prize Categories
Open-Source AI
Touch Grass