The latest articles on DEV Community by Emre Can Kartal (@emreckartal). https://dev.to/emreckartal https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F2812485%2Fdd9484ae-2d2d-49b5-bca9-ad00cece5abe.jpg https://dev.to/emreckartal en Emre Can Kartal Tue, 18 Feb 2025 11:38:17 +0000 https://dev.to/emreckartal/jan-v0515-more-control-over-llamacpp-settings-advanced-hardware-control-and-more-5he8 https://dev.to/emreckartal/jan-v0515-more-control-over-llamacpp-settings-advanced-hardware-control-and-more-5he8 <p>If you're hearing about Jan for the first time: <a href="https://jan.ai/" rel="noopener noreferrer">Jan</a> is a desktop app that runs models locally. It's fully free, open-source, and as simple as ChatGPT in UI.</p> <p>You can now tweak llama.cpp settings, control hardware usage and add any cloud model in Jan. We just released a major update, adding some of the most requested features from local AI communities. Thanks for all the feedback!</p> <h2> New llama.ccp settings </h2> <p>You can now tweak llama.cpp settings directly in Jan's UI.<br> Also, no more waiting for us to update the Jan to bump the engine - you can now update the engine version yourself.</p> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fldfxv5ek59rcpw9d7b04.gif" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fldfxv5ek59rcpw9d7b04.gif" alt="Image description" width="800" height="580"></a></p> <p>Settings you can control over:</p> <ul> <li>llama.cpp backends</li> <li>Continuous Batching</li> <li>Parallel Operations</li> <li>CPU threads</li> <li>Flash Attention</li> <li>Caching</li> <li>KV Cache Type</li> <li>mmap</li> </ul> <h2> Advanced hardware controls </h2> <p>Hardware control got a big upgrade. You can now activate/deactivate GPUs and see all hardware details in Settings → Hardware.</p> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ffgd0sdao4387f9meny3c.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ffgd0sdao4387f9meny3c.png" alt="Image description" width="800" height="594"></a></p> <h2> Remote models update </h2> <p>Managing cloud models is now easier. Instead of manually adding them, you can install custom remote engines via Settings → Engines. </p> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fg38f2d142afbi5lqodx0.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fg38f2d142afbi5lqodx0.png" alt="Image description" width="800" height="619"></a></p> <p>API support for Gemini and DeepSeek is also available.</p> <p>These updates (and more) are now live in v0.5.15. Update your Jan or grab the latest version here:</p> <ul> <li>Web: <a href="https://jan.ai/" rel="noopener noreferrer">https://jan.ai/</a> </li> <li>GitHub: <a href="https://github.com/janhq/jan" rel="noopener noreferrer">https://github.com/janhq/jan</a> </li> </ul> <p>We'd appreciate all feedback and are happy to hear what you'd like to see next!</p> ai llm opensource llamacpp Emre Can Kartal Tue, 04 Feb 2025 03:20:38 +0000 https://dev.to/emreckartal/a-tip-from-deepseek-on-running-r1-locally-7n9 https://dev.to/emreckartal/a-tip-from-deepseek-on-running-r1-locally-7n9 <p>DeepSeek team recommends adding <strong>\n</strong> to the prompt so deepseek-r1 always starts with thorough reasoning.</p> <blockquote> <p>Additionally, we have observed that the DeepSeek-R1 series models tend to bypass thinking pattern (i.e., outputting "\n\n") when responding to certain queries, which can adversely affect the model's performance. To ensure that the model engages in thorough reasoning, we recommend enforcing the model to initiate its response with "\n" at the beginning of every output.</p> </blockquote> <p><a href="https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-file#usage-recommendations" rel="noopener noreferrer">Source<br> </a></p> ai machinelearning