Great point. I built something similar for my own app a while back. I usually rely on Cloudflare AI Gateway for this because it offers features like request caching for significant savings, per-request costs, rate limiting, request retries, model fallbacks, and detailed logs. However, having a custom dashboard definitely provides more flexibility for specific needs.
Building AI products from Gaza. Cybersec student turned AI builder. Creating Provia — an AI sales platform. I document every bug, fix, and lesson. The journey is the content. Follow along.
Gary — Cloudflare AI Gateway is exactly the kind of comparison I should have addressed in the article. The tradeoff I see: gateway-level tools are great for infra concerns (caching, retries, fallbacks) but they treat all calls as equal. They can't tell you "this 100-token call costs $0.0001 and this 1,820-token call costs $0.02" for the same user-facing feature. That tenant/feature/conversation breakdown has to live closer to your application code, where you have the labels.
So I think the right architecture is probably both: gateway for infra-level wins, custom dashboard for product-level cost attribution. Did you find yourself running both in parallel, or did the Gateway end up being enough for your use case?
Thanks for the detailed response, Ali! You've raised an interesting point about cost attribution.
From my experience with Cloudflare AI Gateway, it actually handles per-request logging quite well. It shows the exact token count and estimated cost for every single call in the logs. Regarding product-level attribution, I found that using the custom metadata headers (cf-aig-metadata) allows you to tag requests with IDs from your app, which bridges the gap between infra-level logs and product-level analytics.
To answer your question, I actually built my own calculator mainly out of curiosity to compare my internal logs with the Cloudflare AI Gateway data. I ran them both in parallel for a while and, to my satisfaction, they matched up perfectly. While the Gateway ended up being sufficient for my production needs, building a custom tool was a great way to verify the accuracy of the data.
Building AI products from Gaza. Cybersec student turned AI builder. Creating Provia — an AI sales platform. I document every bug, fix, and lesson. The journey is the content. Follow along.
Gary — that's genuinely useful, and I have to update my mental model. The cf-aig-metadata header is exactly the missing piece I assumed didn't exist; if it lets you propagate tenant/feature/conversation IDs from your app into the gateway logs, then Cloudflare does solve the attribution problem cleanly.
The honest revised take: my dashboard isn't an alternative to AI Gateway, it's what you build when you don't know AI Gateway has metadata headers. The "I built it because I had to" framing in the article only holds if you're not on Cloudflare's stack. For anyone already on it, your approach (Gateway as source of truth, custom tool for verification) is the better starting point.
Adding a follow-up note to the article. Appreciate the correction.
Building AI products from Gaza. Cybersec student turned AI builder. Creating Provia — an AI sales platform. I document every bug, fix, and lesson. The journey is the content. Follow along.
Great point. I built something similar for my own app a while back. I usually rely on Cloudflare AI Gateway for this because it offers features like request caching for significant savings, per-request costs, rate limiting, request retries, model fallbacks, and detailed logs. However, having a custom dashboard definitely provides more flexibility for specific needs.
Gary — Cloudflare AI Gateway is exactly the kind of comparison I should have addressed in the article. The tradeoff I see: gateway-level tools are great for infra concerns (caching, retries, fallbacks) but they treat all calls as equal. They can't tell you "this 100-token call costs $0.0001 and this 1,820-token call costs $0.02" for the same user-facing feature. That tenant/feature/conversation breakdown has to live closer to your application code, where you have the labels.
So I think the right architecture is probably both: gateway for infra-level wins, custom dashboard for product-level cost attribution. Did you find yourself running both in parallel, or did the Gateway end up being enough for your use case?
Thanks for the detailed response, Ali! You've raised an interesting point about cost attribution.
From my experience with Cloudflare AI Gateway, it actually handles per-request logging quite well. It shows the exact token count and estimated cost for every single call in the logs. Regarding product-level attribution, I found that using the custom metadata headers (cf-aig-metadata) allows you to tag requests with IDs from your app, which bridges the gap between infra-level logs and product-level analytics.
To answer your question, I actually built my own calculator mainly out of curiosity to compare my internal logs with the Cloudflare AI Gateway data. I ran them both in parallel for a while and, to my satisfaction, they matched up perfectly. While the Gateway ended up being sufficient for my production needs, building a custom tool was a great way to verify the accuracy of the data.
Gary — that's genuinely useful, and I have to update my mental model. The cf-aig-metadata header is exactly the missing piece I assumed didn't exist; if it lets you propagate tenant/feature/conversation IDs from your app into the gateway logs, then Cloudflare does solve the attribution problem cleanly.
The honest revised take: my dashboard isn't an alternative to AI Gateway, it's what you build when you don't know AI Gateway has metadata headers. The "I built it because I had to" framing in the article only holds if you're not on Cloudflare's stack. For anyone already on it, your approach (Gateway as source of truth, custom tool for verification) is the better starting point.
Adding a follow-up note to the article. Appreciate the correction.
I appreciate the follow-up, Ali. Just wanted to share some insights from my workflow. Glad you found it helpful!
Genuinely. Threads like this are why I keep writing here.