The latest articles on DEV Community by Sarkar-AGI (@sarkaragi). https://dev.to/sarkaragi https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3946047%2F2a7f29fe-1483-4f31-b59c-ad0699b65871.png https://dev.to/sarkaragi en Sarkar-AGI Fri, 22 May 2026 12:07:24 +0000 https://dev.to/sarkaragi/titancore-core-1-trillion-parameter-llm-training-infra-in-ccuda-with-zero-3-5lc https://dev.to/sarkaragi/titancore-core-1-trillion-parameter-llm-training-infra-in-ccuda-with-zero-3-5lc <p>Hi</p> <p>I built TitanCore Core-1, a lightweight core infrastructure (around 75+ files) written in C++ and custom CUDA kernels to address the VRAM bottleneck in trillion-parameter LLM training.</p> <p>By implementing Fully Sharded Data Parallelism (FSDP) via ZeRO-3 and bypassing standard framework overhead with fused kernels, I managed to hit 890 GB/s memory bandwidth utilization ($2.6\times$ speedup compared to traditional pipelines).</p> <p>The code is fully open-source. I would love to get your feedback on the custom memory handling and activation checkpointing logic!<br> GitHub link <a href="https://github.com/Sarkar-AGI/Core-1" rel="noopener noreferrer">https://github.com/Sarkar-AGI/Core-1</a></p> cpp llm performance showdev