The latest articles on DEV Community by Dmitrii Shakhov (@dmitrii_s). https://dev.to/dmitrii_s https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1504995%2F27053ca4-fa00-4b79-9c13-ad5d3fa8f26f.jpeg https://dev.to/dmitrii_s en Dmitrii Shakhov Wed, 27 May 2026 09:20:56 +0000 https://dev.to/dmitrii_s/i-built-a-simple-pytest-plugin-for-test-observability-need-your-help--2kgp https://dev.to/dmitrii_s/i-built-a-simple-pytest-plugin-for-test-observability-need-your-help--2kgp <p>Guys, I need your help 😅</p> <p>As I’ve noticed, many QA Engineers (also devs) do not measure how stable or fast their tests are. Even teams that do care about test observability often build their own internal tooling to analyze test statistics.</p> <p>People sometimes ask me how they can organize their own analytical system for test observability. But I never had a clear answer. Just "develop your own system". This approach might work for large teams, but it can be expensive for small and medium teams. Googling didn't help me. I could not find any tools that I would confidently recommend to others. And...</p> <p>I decided to create a simple solution for people who use Python and pytest. It is a pytest-plugin designed to improve test observability without requiring a TMS or other paid solutions. It exports test results into ClickHouse.</p> <p>Key features:</p> <ul> <li>Automatic CI detection</li> <li>Allure report support</li> <li>Basic Grafana configuration provided out of the box</li> <li>No impact on pytest execution</li> <li>No additional overhead if export to ClickHouse is disabled</li> <li>Integrates with pytest</li> </ul> <p>I’m looking for people who can test it, provide feedback, and suggest feature requests. Let’s build a useful tool for the QA community together 🙃</p> <h2> How to test it locally </h2> <p>If you want to see the dashboards, I added a docker-compose file so you don't need to configure a DB from scratch.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>git clone https://github.com/shakhov-dmitrii/pytest-test-observer.git <span class="nb">cd </span>pytest-test-observer docker compose up <span class="nt">-d</span> </code></pre> </div> <p>Then just install the plugin and run your tests:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>pip <span class="nb">install</span> <span class="s2">"pytest-test-observer[allure]"</span> pytest <span class="nt">--ch-url</span><span class="o">=</span>localhost:8123 <span class="nt">--ch-table</span><span class="o">=</span>pytest_results </code></pre> </div> <p>You can check the results in Grafana at <a href="http://localhost:3000/d/pytest-test-observer-overview" rel="noopener noreferrer">http://localhost:3000/d/pytest-test-observer-overview</a> (admin / admin).</p> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fsb1vivgibisyr31nu2ql.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fsb1vivgibisyr31nu2ql.png" alt=" " width="800" height="620"></a></p> <p>Here is the repository: <a href="https://github.com/shakhov-dmitrii/pytest-test-observer" rel="noopener noreferrer">https://github.com/shakhov-dmitrii/pytest-test-observer</a></p> <p>Having historical metrics for tests has been a massive quality of life upgrade, but I want to make it better.</p> qa python pytest ci