The latest articles on DEV Community by Andrew Graham-Yooll (@andrewgy8). https://dev.to/andrewgy8 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F119618%2F22555f81-b409-49c1-afbc-59948f42f0a0.jpg https://dev.to/andrewgy8 en Andrew Graham-Yooll Tue, 29 Sep 2020 09:31:47 +0000 https://dev.to/andrewgy8/rele-1-0-0-1d7i https://dev.to/andrewgy8/rele-1-0-0-1d7i <p>We are pleased to announce <a href="https://github.com/mercadona/rele">Relé</a>, a comprehensive Python solution to get you up and running with Google Cloud’s Pub/Sub as quickly and easily as possible. Meant to ease development workflow, Relé comes built-in with key components for managing a project with Pub/Sub enabled services.</p> <p>Out-of-the-box, Relé provides:</p> <ul> <li>Powerful Publishing API</li> <li>Highly Scalable Worker</li> <li>Intuitive Subscription Management</li> <li>Easily Extensible Middleware</li> <li>Optional Django and Flask Integration</li> <li>And much more!</li> </ul> <p>Allowing our products to scale with ease, Pub/Sub is a vital piece of infrastructure at <a href="http://mercadonatech.es/en/">Mercadona Tech</a> and we lean on Relé to manage publishing and subscribing to millions of messages.</p> <h2> Show me the data! </h2> <p>In the last 30 days we have:</p> <div class="highlight"><pre class="highlight plaintext"><code>Published 20 million+ messages Consumed 125 million+ messages managed 120+ topics and 300+ subscribers </code></pre></div> <p>And it’s growing every day!</p> <h2> Use Case </h2> <p>For many years, Python’s go-to task queuing library has either been Celery or RQ. Although on the face of it, Pub/Sub’s design may look similar, it is solving a different problem.</p> <p>While Celery and RQ solve task queues within an application, Pub/Sub shines when it comes to managing task queues between many services. Leveraging Relé with Pub/Sub maintains an elegant API much like Celery and RQ’s, but instead allows for scalable messaging within a service oriented architecture.</p> <h2> Reasoning </h2> <p>Since the Google Pub/Sub client does not come built-in with solutions to database connection management, publisher management, declaring subscribers, and running workers, the Mercadona Tech team decided to coalesce around an open source solution.</p> <p>To reduce complexity and risk when adopting Pub/Sub, we organized our experiences from the Python client into one comprehensive library to solve common use cases and patterns suitable for a highly-available production environment.</p> <p>If any of this sounds interesting, checkout our <a href="https://mercadonarele.readthedocs.io/en/latest/">docs</a> and <a href="https://github.com/mercadona/rele">repo</a>.</p> <p>Cheers!</p>