DEV Community: Andrew MacKenzie

Introducing the new PubSub+ Messaging API for Python

Andrew MacKenzie — Thu, 18 Feb 2021 18:05:22 +0000

Solace is pleased to announce the release of version 1.0.0 of a new Python client API for PubSub+ messaging applications. The API is available on our downloads page or through PyPi. To install via pip, simply type: pip install solace-pubsubplus

Depending on whose list you look at, Python is anywhere from 1^st to 3^rd in popularity, demand for developers, and use in popular projects among programming languages. This new API fits in nicely with the existing Solace native APIs in C, Java, .Net/C#, and JavaScript/NodeJS.

While the popularity of Python was certainly a factor in creating this new API, as well as customer demand, it’s also important as a way of opening up support for a number of use cases popular in the Python community such as event streams to analytics packages for real-time machine learning, AI, and data analysis.

An event mesh using Solace event brokers enables the distribution and federation of streaming events across your organization, regardless of where the applications live or the language you’re using. With the addition of PubSub+ Messaging API for Python, the event mesh can deliver events to your Python applications in Pandas, NumPy, TensorFlow, Keras, and more. Data is the lifeblood of data science, and event streaming data that is distributed across your event mesh makes that data real-time, up-to-date, and fresh.

If you are looking for a tutorial to learn how to use the Solace PubSub+ Messaging API for Python, you can get started by:

Read our official documentation on our docs site
Access our samples, developer resources and API reference docs
Get help from your peers and participate in the discussions on the Solace Community

You can also subscribe to our channel on YouTube. There you will get a mix of topics such as using the PubSub+ Platform, event-driven architecture, customer testimonials, and, of course, technical content related to writing event-driven applications and the features of PubSub+ Event Broker.

Recently Tamimi, one of our terrific developer advocates, did a live coding session featuring an early access version of the Python API. It is well worth a view.

In addition to the resources above, it is interesting to look at ways that developers are using Python in event-driven applications in general and how they could be written using the Solace PubSub+ Messaging API for Python instead.

Just recently, Solace Developer Community member, Daniel Curtis wrote a blog post “Visualizing Live Streamed Data with Python and Solace PubSub+”. In it, Daniel articulates a use case of tracking NYC taxis using Pandas DataFrames, fed by a streaming subscription to a PubSub+ Event Broker topic, and visualized using Bokeh on a dashboard featuring a map. Daniel used the Paho MQTT Python API for this as the Solace PubSub+ Messaging API for Python was not yet available. He could just as easily have used the new API. The advantages of using Solace PubSub+ Messaging API for Python over an open standard protocol library like Paho MQTT is the ability to use advanced features of the broker that can’t be easily expressed using another open standard library.

Do you have an event-driven use case that would be best implemented in Python and fully utilize the power and breadth of the PubSub+ Platform? It is time to use our new PubSub+ Messaging API for Python!

And this 1.0.0 version of Solace PubSub+ Messaging API for Python is just the beginning. Future releases will add even more features such as request/reply helpers, Kerberos client authentication, session transactions and more. Solace developers are working hard on the Python API to enhance the feature set and cover more use cases. Stay tuned for future releases of this brand new API.

The post Introducing the new PubSub+ Messaging API for Python appeared first on Solace.

Introducing the Solace PubSub+ for PCF Tile Version 2.2.0

Andrew MacKenzie — Mon, 07 Jan 2019 17:35:37 +0000

Note: In September 2019, Pivotal changed the name of Pivotal Cloud Foundry (PCF) to Pivotal Platform.

A growing number of customers are choosing to bind their Pivotal Cloud Foundry (PCF) applications to Solace PubSub+ event brokers. PubSub+ is a great match for PCF application developers, many of whom are developing event-driven microservices with Spring. The Solace PubSub+ for PCF tile makes deploying PubSub+ in PCF simple, manageable and repeatable.

The tile has been available for over a year and has matured over various releases. In September, with version 2.1 we added support for Solace PubSub+ Standard Edition, a forever-free enterprise-grade message broker that supports publish/subscribe, queueing, request/reply, and streaming, along with integrated high availability (HA) and disaster recovery (DR).

Now, just in time for the holiday season, we are releasing version 2.2.0 of the PubSub+ for PCF tile, which is chock-full of useful, customer-requested features.

Summary of new features:

On-demand service broker
Support for application security groups (ASGs)
Service instances are ready for dynamic message routing
Configurable webhook triggered by service lifecycle changes (create, update, delete)
Export/import service instance configuration settings for service backup and restore for migration
Operator-configurable service plans

The release includes other features and fixes you can learn about in the release notes, but I’d like to explain the most noteworthy additions and enhancements.

On-Demand Service Broker

The on-demand service broker (ODB) allows the service instances to be created dynamically. It provisions resources for the service “on-demand” rather than allocating services from a pool of pre-provisioned PubSub+ event brokers on virtual machines (VMs).

ODB provides more elastic and instance-specific resources to the services created. This is made much easier by having the underlying VM resources allocated on demand.

The tile operator can have a combination of services that use both the pre-allocated broker/VMs and the new on-demand service instances.

Application Security Groups (ASGs)

Support for ASGs allows the PubSub+ tile to automatically manage the required ASGs for the operator.

Pivotal defines ASGs as “a collection of egress rules that specify the protocols, ports, and IP address ranges where app or task instances send traffic.”

The Solace tile support for ASGs means that when an app is bound to a PubSub+ service instance, the necessary rules are automatically created to allow that app to properly communicate with and use the service, and when an app is unbound or deleted, so is the ASG. This eliminates any guesswork or trial and error in configuring an app to communicate with PubSub+ services.

Dynamic Message Routing (DMR)

Dynamic Message Routing is a recently introduced feature that enables PubSub+ event brokers to be easily configured into an intelligent, self-routing mesh that automatically distributes events between any number of globally-distributed applications and devices as if they were connected to a single broker.

The PubSub+ for PCF tile 2.2.0 supports DMR by creating service instances that are ready to be configured by the PubSub+ user to participate in this event mesh. The event mesh can be amongst other PubSub+ for PCF service instances or with external, non-PCF PubSub+ event brokers running in other clouds, on-premises, or wherever… or some combination of all of these!

Configurable webhook triggered by service lifecycle changes

This release has a configurable webhook that will make an outbound REST call to your chosen endpoint after certain lifecycle events for your service such as service creation, edit or deletion.

The webhook enables automation and orchestration of custom tasks related to the lifecycle of your PubSub+ PCF service, and will find a natural home within DevOps CI/CD pipelines.

Export/import service instance settings

Sometimes you want to be able to be able to backup (and possibly restore) the configuration of your PubSub+ for PCF services.

You may also want to be able to template the configuration of a service so that it’s repeatable across service instances. For example, you want your development, test, and production services to have the same queues and topics.

The export/import feature allows you to export a service configuration to an external file. You can also choose to import it into another new service to set the configuration to what is in the exported configuration.

Note that this feature is released as beta as there is some functionality still under development and some known limitations. The feature is usable and Solace would like your feedback on it while it is in beta. The beta tag will be removed in a future version of the PubSub+ for PCF tile.

Operator-configurable service plans

The PubSub+ for PCF tile comes with various pre-configured service plans that can be deployed as-is. In some cases, the tile operator might want to customize these plans to offer different service options to their developer applications. They might also want to create new custom plans for their users to complement (or replace) the Solace-supplied service plans.

The 2.2.0 release allows you to create new service plans, customize the VM types, number of message VPNs, and more. The PubSub+ tile will automatically apply scaling tiers to match the resource allocations in service plans you create.

Summary

We’ve developed this new version of the Solace PubSub+ for PCF tile in close consultation with many Solace/Pivotal customers. We think it includes some terrific new features and some operational improvements that make it a must-have for those looking to use PubSub+ on the Pivotal stack.

We hope you agree, so please let us know how it goes after you install or upgrade!

The post Introducing the Solace PubSub+ for PCF Tile Version 2.2.0 appeared first on Solace.

How to Unlock the Potential of Spring Cloud Stream Apps with an Advanced Event Broker

Andrew MacKenzie — Fri, 21 Sep 2018 05:00:00 +0000

Developers building apps with the Spring Cloud Stream framework can now connect their microservices to the Solace PubSub+ event broker and take advantage of easy, full-featured messaging-based integration with other applications, cloud services and connected devices.

Events are becoming the backbone of today’s modern microservices interactions. Message exchange patterns such as publish/subscribe and event streaming offer many advantages over the API-centric request/response model, such as the ability to decouple services and independently scale services as usage increases and decreases. Each of these advantages provide tangible business value in the form of greater agility, faster time to market, and a more rich and responsive user experience.

Pivotal recently introduced Spring Cloud Stream, a new event-driven framework designed to make it easier to write message-based microservices.

In their own words:

“Spring Cloud Stream is a framework for building highly scalable event-driven microservices connected with shared messaging systems. The framework provides a flexible programming model built on already established and familiar Spring idioms and best practices, including support for persistent pub/sub semantics, consumer groups, and stateful partitions.”

Spring Cloud Stream builds on the “opinionated configuration” approach of other Spring frameworks (like Spring Boot) to abstract the nuances of messaging middleware away from the application developer and allow them to focus on the business logic of their microservice. Developers no longer need to learn event broker client APIs, protocols or the complex semantics of asynchronous communication.

Spring Cloud Stream presents a declarative, annotation-based interface that is standardized across vendor broker implementations.

Increasingly, the challenge of having complex event/data integration is reducing developer productivity. These developers are using modern frameworks such as Spring Cloud Stream to accelerate the development of event-driven microservices, but that efficiency is hindered by the inability to access events flowing out of legacy systems, systems of record or streaming from mobile/IoT devices.

Developers frequently face challenges even when simply trying to exchange events across organizational boundaries. All these new, greenfield microservices are producing events and insights that in many cases must be consumed by non-Spring Cloud Stream applications in a variety of ways (such as streaming, queueing and replay).

What these organizations really need is an architectural layer known as an event mesh.

An event mesh gives enterprises the ability to support event-driven architectures, from the smallest of microservices deployments to extending applications to hybrid cloud in a governed, performant, robust, secure, well-architected manner. It provides the ability to integrate legacy applications, modern microservices, devices, data stores, SaaS – dynamically and in real-time.

An event mesh gives application developers and architects a foundation on which to build and deploy distributed, event-driven applications wherever they need to be built and deployed.

Solace PubSub+ event brokers are uniquely capable of helping organizations create an event mesh and, when combined with Pivotal’s Spring framework, are a great way for developers to create production-grade, reactive microservices that can consume any event from anywhere, even across long-distance WAN links and organizational boundaries.

Spring Cloud Stream allows developers to define the specifics of the event broker in configuration (such as broker host/port and credentials/certificates) and simply “bind” to the middleware using Spring annotations.

You can find examples of using the Solace PubSub+ Spring Cloud Stream binder in our samples repository. These samples are a subset of the Spring Cloud Steam samples and focus on the Solace PubSub+ configuration and basic functionality.

Microservices are either Sources, Sinks or Processor and specify the destination of the events in which they intend to consume from and/or produce to. Uniquely to Solace, these destinations and their corresponding events can be produced to one part of the mesh (i.e., onto one event broker or Pivotal Platform service instance) and can be consumed by applications/microservices connected to some other part of the mesh (i.e., a different broker).

Modern applications are extremely dynamic, as microservices are increasingly transient. Businesses may initially be running an application on-premises, but due to changes in utilization may need to deploy it into the cloud (e.g., cloud bursting). The Solace-provided capability of dynamic, intelligent routing is critical to enable both business agility and overall time to market.

Let’s look at an example of a Spring Cloud Stream app:


@SpringBootApplication
@EnableBinding(Sink.class)
public class LoggingConsumerApplication {

    public static void main(String[] args) {
        SpringApplication.run(LoggingConsumerApplication.class, args);
    }

    @StreamListener(Sink.INPUT)
    public void handle(Person person) {
        System.out.println("Received: " + person);
    }

    public static class Person {
        private String name;
        public String getName() {
            return name;
        }
        public void setName(String name) {
            this.name = name;
        }
        public String toString() {
            return this.name;
        }
    }
}

From an event broker point of view, Spring has defined a “binder” interface with standard service provider interfaces (SPIs) across all the binder implementations that do the work of connecting to brokers “under the covers.” This ensures that Spring Cloud Stream applications are event broker agnostic: they behave the same across broker vendors and versions, and don’t lock you in to any one broker.

The highlighted code above shows how messaging-specific annotations can be used to bind an application to a binder (in the @EnableBinding annotation) and receive messages from a channel (in the @StreamListener annotation). Behind the scenes, Spring Cloud Stream will invoke the correct binder classes to execute the requested actions. Developers can easily invoke a different binder or use a different input/output channel simply by changing the binder configuration.

Solace has fully embraced the Spring Cloud Stream approach and released a Spring Cloud Stream binder that lets applications leverage the high performance and flexibility of topics, queues and a wealth of other event routing capabilities on Solace PubSub+ software and hardware brokers. The binder is open source and is available on GitHub as well as Maven Central for adding as a dependency to Spring Boot applications.

We have also developed some Spring Cloud sample applications which you can download and experiment with.

Thanks to our strong partnership with Pivotal (we recently won their 2018 “Do The Right Thing” Award), we’re one of the first vendors to release a Spring Cloud Stream binder. We’re excited to see what applications our customers can build using Spring Cloud Stream, as well as Spring Cloud Data Flow, a powerful toolkit for creating rich data integration and real-time data processing pipelines.

Streams can be easily constructed using either a GUI designer or a DSL which is based on a UNIX-like, pipeline syntax.

Get started with the Solace Spring Cloud Stream Binder and PubSub+ Event Broker to unleash the power of your reactive streams and microservices!

The post How to Unlock the Potential of Spring Cloud Stream Apps with an Advanced Event Broker appeared first on Solace.