DEV Community: Marc DiPasquale

Publishing Github Java Packages to Maven Central for a New Domain

Marc DiPasquale — Fri, 19 Mar 2021 18:41:05 +0000

Overview

🎉 The Solace Developer Relations team recently launched our SolaceCommunity github organization as a home for open source projects that are authored, maintained and supported by the amazing Solace Community. If you're interested in joining our community you can read more about it here, but that won't be the focus of the rest of this blog. The focus of this blog is how I enabled publishing of Java packages to Maven Central for this new github community and how you can do so for your own domain. Since Java is one of our most used languages, and most Java projects now leverage Maven or Gradle, I wanted to enable developers contributing their Java projects to be able to easily publish to Maven Central to make it easier for everyone in the community to easily use them.

Over the past few weeks I went through the steps necessary to setup publishing of Java packages to Maven Central for the https://solace.community domain. This was trickier than I thought so I figured I'd share what I did to hopefully help others :)

I'm going to try to keep this short and to the point but feel free to let me know if you have any questions.
I used the following resources when figuring out these steps so props to their creators!

Setting up OSSRH (The Repository)

There are a handful of approved repository hosting options specified by Maven Central. You can find the entire list here, but the easiest approach seems to leverage Open Source Software Repository Hosting (OSSRH) so that's the approach I decided to take.

Note that if you're just trying to publish a few personal projects on github and are fine publishing under io.github then you don't need to follow all of these steps so I'd suggest heading over to google and doing a few more searches :)

Get An Account

The first step down this road is to register a OSSRH account. This account will be used to prove ownership of your domain (for publishing packages), but also to manage your repositories in the future. Sign up for an account here

✅ Account Created

Prove Domain Ownership

Now that you have an account the next step in the process is to prove ownership of the domain that matches the group that you'd like to publish to. Usually this is your domain name in reverse, so something like com.company if your domain is company.com. Since our developer community is at solace.community this meant we would publish to the community.solace group.

To prove that we own this domain I had to execute a few simple steps:

Open a New Project Ticket with OSSRH.
Follow the instructions request in the ticket to addd a DNS TXT record to our domain.
Wait a few hours (it says it could take up to 2 business days) for the DNS TXT record to be verified.
Check the ticket for confirmation that domain ownership has been confirmed.
Make a note to comment on this ticket after your first release to enable syncing to maven central!

✅ Domain ownership proven

Create Your User Token

Now that we have permission to publish to our domain we need to create a user token for publishing. This token will be used as part of the publishing process.

To get this token do the following:

Login to the OSSRH Nexus Repository Manager w/ your OSSRH account
Go to your profile using the menu under your username at the top right.
You should see a list menu that is on the Summary page; change it to User Token. You can create your token on this page.
Copy & Paste this token info so you can use it later! (Keep it private!)

✅ OSSRH is now ready to go and we have the info we need

Configuring the Maven pom

The next step is to setup our maven pom for publishing. Note that I used copy and paste programming to figure this out so I'm by no means an expert here :)
If you are reading this and think I should include more detail here or my explanations are confusing please drop a comment and let me know.

An entire example pom can be found here

Here is what I did:

Ensured that my groupId starts with the reverse domain that we have approval to publish to! For example this is what we used, note that the groupId starts with community.solace.
```
<groupId>community.solace.spring.integration</groupId>
<artifactId>solace-spring-integration-leader</artifactId>
<version>1.1.0-SNAPSHOT</version>
```
What to know about the version! Okay this is important. When publishing maven projects you have releases and you have snapshots. A "release" is the final build for a version which does not change whereas a "snapshot" is a temporary build which can be replaced by another build with the same name. Go ahead and google this if you want to learn more :)

Once you know the difference you're ready to set your version. Use a version ending in a number, e.g: 1.1.0, for a "release" and end it in -SNAPSHOT, e.g: 1.1.0-SNAPSHOT for a snapshot.

Include a description name, description and url pointing to your repository. For example,

<name>Solace Spring Integration Leader</name>
<description>This project allows for Spring Integration Leader Election using Solace Exclusive Queues</description>
<url>https://github.com/solacecommunity/spring-solace-leader-election</url>

Include a license, source control info scm, developers and organization(I believe this is optional) information.

<licenses>
    <license>
        <name>MIT License</name>
        <url>https://github.com/solacecommunity/spring-solace-leader-election/blob/master/LICENSE</url>
        <distribution>repo</distribution>
    </license>
</licenses>

<developers>
    <developer>
        <name>Solace Community</name>
        <email>community-github@solace.com</email>
        <organization>Solace Community</organization>
        <organizationUrl>https://solace.community</organizationUrl>
    </developer>
</developers>

<organization>
    <name>Solace Community</name>
    <url>https://solace.community</url>
</organization>

<scm>
    <url>https://github.com/solacecommunity/solace-spring-integration-leader.git</url>
    <connection>scm:git:git://github.com/solacecommunity/solace-spring-integration-leader.git</connection>
    <developerConnection>scm:git:git@github.com:solacecommunity/solace-spring-integration-leader.git</developerConnection>
    <tag>HEAD</tag>
</scm>

Add a profile for OSSRH which includes the snapshotRepository info, the nexus-staging-maven-plugin, and the maven-gpg-plugin. Note in the example below I have this profile activeByDefault so you don't have to specify it when running maven commands, however you may not want to do this. Depends on your use case :)

<profile>
        <id>ossrh</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <properties>
            <gpg.executable>gpg</gpg.executable>
        </properties>
        <distributionManagement>
            <snapshotRepository>
                <id>ossrh</id>
                <name>Central Repository OSSRH</name>
                <url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
            </snapshotRepository>
        </distributionManagement>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.sonatype.plugins</groupId>
                    <artifactId>nexus-staging-maven-plugin</artifactId>
                    <version>1.6.7</version>
                    <extensions>true</extensions>
                    <configuration>
                        <serverId>ossrh</serverId>
                        <nexusUrl>https://s01.oss.sonatype.org/</nexusUrl>
                        <!--  Change to true once we're good! -->
                        <autoReleaseAfterClose>false</autoReleaseAfterClose>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-gpg-plugin</artifactId>
                    <version>1.5</version>
                    <executions>
                        <execution>
                            <id>sign-artifacts</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>sign</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                      <gpgArguments>
                        <arg>--pinentry-mode</arg>
                        <arg>loopback</arg>
                      </gpgArguments>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>

Include the maven-release-plugin, the maven-javadoc-plugin, the maven-source-plugin and the flatten-maven-plugin plugin.

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-release-plugin</artifactId>
            <version>2.5.1</version>
            <configuration>
                <tagNameFormat>@{project.version}</tagNameFormat>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.apache.maven.shared</groupId>
                    <artifactId>maven-invoker</artifactId>
                    <version>2.2</version>
                </dependency>
            </dependencies>
        </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-javadoc-plugin</artifactId>
            <version>2.10.4</version>
            <executions>
                <execution>
                    <id>attach-javadocs</id>
                    <goals>
                        <goal>jar</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <source>8</source>
                <detectJavaApiLink>false</detectJavaApiLink>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-source-plugin</artifactId>
            <version>3.0.1</version>
            <executions>
                <execution>
                    <id>attach-sources</id>
                    <goals>
                        <goal>jar</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>flatten-maven-plugin</artifactId>
            <version>1.1.0</version>
            <configuration>
                <updatePomFile>true</updatePomFile>
                <flattenMode>oss</flattenMode>
                <pomElements>
                    <distributionManagement>remove</distributionManagement>
                    <repositories>remove</repositories>
                </pomElements>
            </configuration>
            <executions>
                <!-- enable flattening -->
                <execution>
                    <id>flatten</id>
                    <phase>process-resources</phase>
                    <goals>
                        <goal>flatten</goal>
                    </goals>
                </execution>
                <!-- ensure proper cleanup -->
                <execution>
                    <id>flatten.clean</id>
                    <phase>clean</phase>
                    <goals>
                        <goal>clean</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

✅ The maven pom is now ready to go!

Create GPG Keys

The next step is to create a GPG key for signing the published packages. This allows for users of your project to verify the published package's authenticity and trust using it in their own projects. In order to create a GPG key you will need to execute a few steps.

Create Your Key

The first step is to Create Your Key. You can do this locally using a tool such as gpg. Be sure to keep your private key private. If others get ahold of this they can become a trusted publisher of your packages.

Install the gpg tool; on mac you can do this by executing the command below. If you aren't using a mac check out the instructions here
```
brew install gpg
```
Generate your key pair. You will be prompted for a "Real Name" and "Eamil Address" that you want to use with the key
```
gpg --gen-key
```

✅ GPG key created!

Share Your Public Key

Now that you've generated your key pair, which consists of a private and a public key, you need to share the public piece. The public key will be used by developers to verify the package's authenticity. You can also do this with the gpg command. It will share your key to a keyserver which tools such as maven know how to query to retrieve the keys for automated verification.

Get your keypair identifier. To do this you need to list your keys. The key will have an identifier that looks like a random string of characters, something like C48B6G0D63B854H7943892DF0C753FEC18D3F855. In the command below I've replaced it with MYIDENTIFIER to show it's location.

MJD-MacBook-Pro.local:~$ gpg --list-keys
/path/to/keyring/pubring.kbx
----------------------------------------
pub   rsa3072 2021-03-11 [SC] [expires: 2023-03-11]
      MYIDENTIFIER
uid           [ultimate] solacecommunity <community-github@solace.com>
sub   rsa3072 2021-03-11 [E] [expires: 2023-03-11]

Distribute to a key server using the identifier found in the previous step. Note that you may want to publish to a different keyserver. The one that worked for me was hkp://keyserver.ubuntu.com:11371

gpg --keyserver hkp://pool.sks-keyservers.net --send-keys MYIDENTIFIER

✅ We've now shared our public key!

Configure the Github Secrets

Okay at this point our project is looking pretty good and we could run a deployment locally using the mvn --batch-mode clean deploy command, however we actually want to perform our releases via a Github action. Shoutout to sualeh for creating this gist which helps me navigate the next few steps! In order to make the release from a Github Action we need to make our GPG private key and OSSRH user information available to the Github actions while also keeping it private. We can do this using Github Action Secrets.

I created secrets at the Github Organization level so I followed the steps below to keep my OSSRH_GPG_SECRET_KEY,OSSRH_GPG_SECRET_KEY_PASSWORD,OSSRH_USERNAME and OSSRH_PASSWORD secret. If not clear by the name these are my GPG Secret Key, my GPG Secret Key password, my OSSRH Username (from the token we generated earlier) and the OSSRH password (from the token we generated earlier).If that screenshot feels out of date you can find the docs here

✅ Secrets configured!

Setting up the Github Action

Now that the Github Action Secrets are available let's go ahead and configure the Github Action itself. To give credit where credit is due, I created this github action using this gist as a starting point. You can find the Github Action workflow file here.

You'll see below that this Github Action will run on the latest ubuntu and execute the publish job which has several steps. The steps will setup the maven central repository info, install the secret key and then run the command to publish to OSSRH.

name: Publish package to the Maven Central Repository
on:
  release:
    types: [created]
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Maven Central Repository
        uses: actions/setup-java@v1
        with:
          java-version: 11
          server-id: ossrh
          server-username: MAVEN_USERNAME
          server-password: MAVEN_PASSWORD
      - id: install-secret-key
        name: Install gpg secret key
        run: |
         cat <(echo -e "${{ secrets.OSSRH_GPG_SECRET_KEY }}") | gpg --batch --import
         gpg --list-secret-keys --keyid-format LONG
      - id: publish-to-central
        name: Publish to Central Repository
        env:
          MAVEN_USERNAME: ${{ secrets.OSSRH_USERNAME }}
          MAVEN_PASSWORD: ${{ secrets.OSSRH_PASSWORD }}
        run: |
          mvn \
            --no-transfer-progress \
            --batch-mode \
            -Dgpg.passphrase=${{ secrets.OSSRH_GPG_SECRET_KEY_PASSWORD }} \
            clean deploy

✅ Github Action ready to go

Running the Github Action - let's test it with a Snapshot release

Now you're ready to run the Github Action. I'd recommend testing out the publishing of a snapshot release first. To ensure you do this make sure your version in the pom ends in -SNAPSHOT as discused earlier.

Once the version on your main branch includes -SNAPSHOT go ahead and run the github action workflow. You should see the publish job successed in a few minutes depending on how long your build takes. Ours took 1m 20s the first time.
After the deploy job successfully runs you can head over to https://s01.oss.sonatype.org/content/repositories/snapshots/ and navigate to your project to verify that the snapshot has successfully deployed. It should look something like this:

✅ Snapshot published!

Running the Github Action - now let's make an actual release

Now that the snapshot has successfully deployed let's go ahead and make a real release. Make sure your code is ready before doing this part of course :)

Change your version in the pom to remove the -SNAPSHOT, so it should end in a number. Something like 1.1.0.
Re-Run the Github Action workflow. Wait for the publish job to succeed :)

✅ At this point our project will be staged for release. There is just one more step!

Approve the Deployment in Nexus

Now that our project is staged for release we need to login to the OSSRH Nexus Repository Manager to promote the release.

Login to the OSSRH Sonatype Nexus Repository Manager
Navigate to the Staging Repositories in the menu on the left hand side.
Examine the contents of the release and if everything looks good close the release. More information is available here
The OSSRH deployment is now complete, but if this is your first release remember to go back and comment on your "New Project Ticket" we created earlier so your project will sync to Maven Central.

✅ Deployment Complete!

Check out the deployed project!

After ~24 hours head over to Maven Central's search and you should be able to find our project by typing in your groupId or artifactId.

Conclusion

Hope this was useful! Feel free to leave a comment if you have any questions :)

Getting Started with Spring Cloud Stream using Spring Initializr

Marc DiPasquale — Fri, 11 Sep 2020 15:02:04 +0000

Thanks to Spring Cloud Stream and Spring Initializr it is easier than ever to develop event-driven microservices. Even with partner-maintained binders like Solace PubSub+! If you’re not familiar with Spring Cloud Stream or Spring Initializr, this post will provide some quick background info as well as a tutorial so you can try it yourself.

The Spring Cloud Stream Framework

There are a number of Spring projects for building applications of all types. When developing scalable event-driven microservices, Spring Cloud Stream is often the framework of choice as it provides a flexible programming model and allows developers to create event-driven microservices without having to learn messaging APIs. Instead of learning messaging APIs, Spring Cloud Stream just requires developers to learn basic message-driven concepts such as publish-subscribe and consumer groups. The framework leverages cloud stream binders, which exist for several message brokers including Solace PubSub+, Apache Kafka, and RabbitMQ.

PubSub+ and Spring Initializr

The Solace PubSub+ Spring Cloud Stream binder is automatically added to Spring Initializr projects when you choose both the Solace PubSub+ and Cloud Stream dependencies. For those who are not familiar with start.spring.io, it allows developers to generate Spring Boot projects pre-populated with the dependencies they need to get started quickly.

There are a couple scenarios that could play out when generating a project with Solace PubSub+ as a dependency using Spring Initializr:

Your dependencies include Solace PubSub+, but do not require a binder.
Your dependencies include Solace PubSub+ AND Cloud Stream, which states that it requires a binder.

In scenario 1, Spring Initializr adds the solace-spring-boot-starter to your project. This starter allows you to use Spring Boot with either the Solace JMS API or the Solace Java API (JCSMP).

In scenario 2, Spring Initializr realizes you’re trying to create a Spring Cloud Stream microservice and automatically includes the Solace Spring Cloud Stream Binder Starter in your project.

Using Solace with Spring enables microservices created using the Spring ecosystem to communicate with both Spring and non-Spring applications across multi-cloud, hybrid-cloud, and no-cloud environments.

5-Minute Tutorial (seriously, only 5 minutes)

Do you already have a JDK, Maven, and Docker installed? If so, it will only take you 5 minutes. Video evidence in this youtube video! So, why not give it a try for yourself!

If for some reason you can’t use Docker, check out this codelab for how to do it using PubSub+ Cloud.

Let’s Begin…

Deploy a Solace PubSub+ Event Broker via Docker

docker run -d -p 8080:8080 -p 55555:55555 -p:8008:8008 -p:1883:1883 
-p:8000:8000 -p:5672:5672 -p:9000:9000 -p:2222:2222 --shm-size=2g 
--env username\_admin\_globalaccesslevel=admin 
--env username\_admin\_password=admin 
--name=solace solace/solace-pubsub-standard

Go to start.spring.io and add the Solace PubSub+ and Cloud Stream dependencies. Yes, that link added them for you 😜.
Click Generate
Unzip the downloaded zip file and import into your IDE as a Maven project

Open DemoApplication.java and add the following Spring bean

@Bean
public Function<String, String> uppercase()
{
  return v -> {
      System.out.println("Uppercasing: " + v);
      return v.toUpperCase();
  };
}

*Save and Run
Navigate to the Try-Me! menu item in your local PubSub+ Manager which deployed with the Docker container in Step 1. Your username/password are admin/admin by default.
On the Subscriber side, click Connect, type uppercase-out-0 in the Subscribe to a topic to receive direct messages box and click Subscribe.
On the Publisher side click Connect, type uppercase-in-0 as the topic to publish to and click the Publish button.
Boom! You’re done! Was that simple enough?

*You may be thinking, “What! No connection info needed?!” That is correct. By default, the Solace Binder knows how to automatically connect to the local docker container!

Behind the Scenes

Okay, so you followed the steps above, but what exactly happened? Let me explain:

You deployed a Spring Boot microservice which contained a java.util.function.Function bean in your code and both the cloud stream and a cloud stream binder as dependencies. This told Spring Boot that you wanted to create an event-driven microservice using Spring Cloud Stream.
After Spring Boot realized you wanted a Cloud Stream microservice it knew it needed to create cloud stream bindings to exchange events with the available broker. So, based on your function name of uppercase, it created the default input binding of uppercase-in-0. The Solace Cloud Stream binder then subscribed to the uppercase-in-0 topic on the Solace PubSub+ Event Broker.
When you published an event in the Try-Me! tool, you published an event to the uppercase-in-0 topic on the Solace Broker that contained a payload of “Hello world!”
Your microservice received this event via the binding previously set-up, identified the payload as a String type, and called your uppercase bean for processing which converted the payload of “Hello world!” to “HELLO WORLD!” and returned a String.
Spring Cloud Stream included that returned String as the payload in an outbound message to the Solace Binder which in turn published it to the uppercase-out-0 topic on the Solace Broker.
The Try-Me! tool was then able to receive the processed message and display it to the screen.

What should you try next?

Read about how to use AsyncAPI to generate a Cloud Stream app
Learn how Solace works with Spring Initializr at start.spring.io
Check out this codelab for some more Cloud Stream Fundamentals and to extend what you did here with Function Composition:
Code for the open source Solace PubSub+ Spring Cloud Stream Binder can be found in the github repository.

SolaceProducts / solace-spring-cloud

Supports all solace spring cloud products
Watch our recent Live Coding session featuring Oleg, a lead developer on the Spring Cloud Stream project.

The post Getting Started with Spring Cloud Stream using Spring Initializr appeared first on the Solace Blog.

AsyncAPI Code Generation: Microservices Using Spring Cloud Stream

Marc DiPasquale — Tue, 28 Apr 2020 17:39:34 +0000

Code generation is no simple feat. There are a lot of complexities when it comes to generating useful application code. In this post, I am going to walk you through generating your own microservices using Spring Cloud Stream and the AsyncAPI Code Generator. These tools should help to simplify things when defining and implementing your asynchronous applications. I explained the same idea in a video you can watch here, and all of the artifacts are available in GitHub.

AsyncAPI: What Is It?

Before we dive into code generation let’s start with the basics – what is AsyncAPI? Over the past few years, AsyncAPI has emerged as the industry standard for defining asynchronous, event-driven APIs; you can think of it as OpenAPI for the asynchronous world. It is an open source initiative that provides both a specification to describe and document your asynchronous applications in a machine-readable format, and tooling (such as code generators) to make life easier for developers tasked with implementing them.

I’m not going to go into great detail about the specification, but for context you should know that it defines metadata about your asynchronous API, the channels available for sending/receiving messages, and components – such as schemas – that define the messages that are being exchanged. For more information about the specification you can read about it here.

Defining the Application That You Want to Develop: The AsyncAPI Document

The first step in doing code generation with AsyncAPI is obtaining an AsyncAPI document that defines the application that you want to develop. Per the specification, this document is represented as JSON objects and must conform to the JSON standards. YAML, being a superset of JSON, can also be used. There are two main ways of going about obtaining this document: manually create the document or use an event portal.

If you decide to manually create the document after familiarizing yourself with the specification, don’t worry – you won’t be starting with a blank slate. The AsyncAPI initiative has provided a handy, interactive tool called the AsyncAPI playground to make this easier. On the left side of the playground you can familiarize yourself with the specification and make changes to a real AsyncAPI document, and as you do so the right side of the screen updates to show how the document is parsed into a more human-readable format.

The second way is to use an event portal. Solace PubSub+ Event Portal, for example, allows for architects and developers to collaborate using a GUI to design your event-driven architecture. The team would define the applications that exist in the system, as well as the events that are exchanged and the schemas which define them. Having a catalog of well-organized channels and events for reuse will also save you both time and headaches while collaborating, instead of having to comb through a bunch of files in various locations.

Once the design is in place, PubSub+ Event Portal allows the developer to choose the application they are responsible for developing and download the AsyncAPI document in JSON or YAML.

* *

Create Event-Driven Microservices Using Spring Cloud Stream Without Learning Messaging APIs

Now that we have our AsyncAPI document that describes our application it’s time to develop the application. The AsyncAPI Code Generator supports templates to generate code for a variety of different languages and protocols, but for this example we’re going to use the Spring Cloud Stream template. One should note that the template generates a Maven project.

The Spring Cloud Stream framework provides an easy way to get started with event-driven microservices by providing binders that allow the developer to create their microservices without having to learn messaging APIs.

Download and Run the AsyncAPI Generator

The first step is of course to install the AsyncAPI generator itself. If you have NodeJS installed this takes just one easy npm command as seen below. You can find the required versions in the Code Generator on github.

npm install -g @asyncapi/generator

Once you have the generator installed you can run it using the ag command. At a minimum you must specify the AsyncAPI document to run it against and the template to use as shown below.

ag ~/AsyncApiDocument.yaml https://github.com/asyncapi/java-spring-cloud-stream-template.git

In most cases you’ll want to take advantage of the parameters and specification extensions that are specified by the template being used. For example, the Spring Cloud Stream template that I’m using in this example allows me to configure many options, including the Spring Cloud Stream binder I want to use – for example, the Solace binder.

Other parameters include:

Maven information: artifactId and groupId
Java package: javaPackage
Broker connection Info: host, username, password and msgVpn

Using these options, my ag command might look something like this, where -o specifies the output directory:

ag -o ExpenseIntegration -p binder=solace -p view=provider -p actuator=true -p artifactId=ExpenseIntegration -p groupId=acme.rideshare -p javaPackage=acme.rideshare.expense -p host=localhost:55555 -p username=default -p password=default -p msgVpn=default ~/Downloads/ExpenseIntegration.yaml @asyncapi/java-spring-cloud-stream-template

After running, the output will look something like this:

Add Your Business Logic

At this point the generator has created an ExpenseIntegration directory that contains the Maven project. We can use the IDE of choice and import the Maven project to add business logic.

As seen in the image below, once imported, the project looks like a regular Spring Boot Java project with generated classes under the javaPackage that was defined earlier and an application.yml file for configuration. Generated classes under javaPackage include Plain Old Java Objects (POJOs) defined from the schemas in the AsyncAPI document and Application.java which contains the actual Spring Cloud Functions where we’ll add our business logic.

The generated POJOs, like RideReceipt in the image above, define your data model per the schemas included in the AsyncAPI document. These POJOs contains variables with getters and setters for each attribute defined to allow both for developers to get coding quickly without having to manually create the objects themselves, but also for Spring Cloud Stream to automatically convert messages directly to POJOs.

Then we have the Application.java class, which can be renamed using the javaClass parameter. The generator will add functions to handle messages delivered on the channels defined in the AsyncAPI document as described in the template.

In the example below we can see a single java.util.function.Consumer bean since our AsyncAPI document describes our application as a subscriber to messages whose payload is defined by the RideReceipt schema. Note the comment that states // Add business logic here; this is where the developer can add their business logic.

@SpringBootApplicationpublic class Application {private static final Logger logger = LoggerFactory.getLogger(Application.class);public static void main(String[] args) {SpringApplication.run(Application.class);}@Beanpublic Consumer<RideReceipt> acmeRideshareBillingReceiptCreated001Consumer() {// Add business logic here.return null;}}

You might say: “Marc, that’s great, but how the heck is that function actually binding to the messaging channels!?” This is where the application.yml file comes into play.

The generated application.yml file defines several things as specified in the AsyncAPI document or from the parameters passed into the generator. First, it defines the list of functions it wants Spring Cloud Stream aware of under spring.cloud.stream.function.definition. Second, it tells Spring Cloud Stream which channels to bind those functions to under spring.cloud.streams.bindings. Lastly, it contains connection information to the messaging system. The connection info is specified by different parameters depending on the binder you choose but, in this case, it’s defined under solace.java.

spring: cloud: stream: function: definition: acmeRideshareBillingReceiptCreated001Consumer bindings: acmeRideshareBillingReceiptCreated001Consumer-in-0: destination: acme/rideshare/billing/receipt/created/0.0.1solace: java: host: 'localhost:55555' msgVpn: default clientUsername: default clientPassword: defaultlogging: level: root: info org: springframework: info

Note that all of this was done for the developer so they didn’t have to track down which SCSt parameters needed to be set, map the functions to the bindings, etc. They just have to add their business logic in place of the project and hit run! In this case since it’s a Spring Boot project you can “run as a Spring Boot app” in your IDE or even run from the command line using mvn spring-boot:run.

Helpful Parameters and Specification Extensions for Creating Microservices Using the AsyncAPI Spring Cloud Stream Template

As I mentioned, there are a lot of complexities when it comes to generating useful application code from a microservice. Because of these complexities, I thought I’d call out some tips, tricks, and painpoints of using the AsyncAPI Spring Cloud Stream template.

There are a bunch of different parameters and specification extensions that you should consider when generating your code. All of them can be found here, but I’ll go over a few of the parameters that I use quite often:

The binder parameter allows you to specify the Spring Cloud Stream binder that you’d like to use. Currently the generator supports kafka, rabbit and solace.
The info.x-view specification extension can be set at the info level in your AsyncAPI document. This extension allows for you to define how the document should be viewed from an application perspective. By default an AsyncAPI specification takes a client view where operations (publish/subscribe) defined in a document represent what an application accepts (or how you would communicate with that application). However, for code generation you may want to generate what an application actually does. This is where setting the view parameter comes in. If you set view to a value of provider the operations defined in the document will be treated as what an application actually does. Note that this extension can also be set using the view parameter on some generator templates, such as the Java Spring Cloud Stream one.
The operation.x-scs-function-name specification extension can be set on your publish or subscribe operations in the AsyncAPI document, allowing you not only to name the generated function, but also tie two operations together to form a function that subscribes to one channel and publishes to another when the same name is used. For example, if your AsyncAPI document looked like the image below a java.util.function.Function bean called “calculatePercentage” would be generated which subscribes to the input channel and publishes to the output channel.

channels: 'input': subscribe: x-scs-function-name: calculatePercentage message: $ref: '#/components/messages/CovidTracking\_SingleStateCurrentDataUpdate' 'output': publish: x-scs-function-name: calculatePercentage message: $ref: '#/components/messages/CovidTracking\_SingleStateTestPercentagesUpdate'

The x-scs-destination specification extension can be specified on a subscribe operation, allowing you to override the default destination value which usually matches the channel. This is useful when you are using the Solace binder and you are following the Solace pattern of publishing to topics and consuming from queues. In this case the x-scs-destination value would be treated as the name of the queue which your microservice will consume from and the channel name in the AsyncAPI document will be added as a topic subscription to that queue.
The x-scs-group specification extension can also be specified on a subscribe operation, allowing for the addition of a group to the generated Spring Cloud Stream binding. This allows for the use of consumer groups and will end up in a durable queue being created when using the Solace binder.

Tips For Using The Code Generator To Create Event-Driven Microservices Using Spring Cloud Stream

Besides configuration options there are a few more things to keep in mind when using the generator to create event-driven microservices using Spring Cloud Stream.

Make sure generated POJOs have the Java types you would expect for generated variables! For example, if your JSON schema defines an attribute type as a number or integer those are being mapped to a Double or Integer in Java respectively. If you would like another type, such as a float or long, you’ll want to make that change. It is also important to make sure you pay close attention to data that represents dates and/or times as those will likely end up just being represented by a String by default.
Dynamic topics are not yet supported by the AsyncAPI SCSt Code Generator. We’ll be looking to enhance them both to support dynamic topics in the future but for now you’ll want to remove dynamic pieces of the topic from your channels in the AsyncAPI document and add them into the code afterwards.
When creating a Spring Cloud Stream microservice that does not contain a java.util.function.Supplier include a web server so the microservice continues running and listening for messages to process. This can be done by including the -p actuator=true parameter to include Spring Actuator functionality which itself requires a web server, and also provides some cool management and monitoring capabilities. Alternatively, you can just add the spring-boot-starter-web starter to your pom after it’s been generated. Note this is not an issue with the AsyncAPI generator template, but just a bug with the Solace Spring Cloud Stream binder which will be relevant to people using the generator.

I hope those tips are helpful and save you some troubleshooting time!

Conclusion

I hope this post was useful and you’re able to quickly dive in to generating your own event-driven microservices using Spring Cloud Stream and the AsyncAPI Code Generator after exploring the example described above.

You can get started right away and use the Solace PubSub+ Event Portal to generate your AsyncAPI document for FREE by signing up for a new cloud account!

If you have more questions or want to share your experience with the tools, you can let us know in the Solace Community Forum or consider joining us in contributing directly to the AsyncAPI initiative.

The post AsyncAPI Code Generation: Microservices Using Spring Cloud Stream appeared first on Solace.

Why Developers Need an Event Portal; Creating Applications that Disseminate Real-Time COVID-19 Data

Marc DiPasquale — Fri, 03 Apr 2020 13:24:27 +0000

As developers, we are asked to create software that solves hard problems in a way that’s scalable, secure, and can be modified as needs evolve – all with deadlines expressed in terms of how long ago they wish they’d had this. Does “We need this yesterday!” ring a bell? Things change so quickly these days, in business and the world at large, that the applications we deliver need to help people achieve positive outcomes very quickly, based on the latest information. That’s where event-driven architecture comes in.

Letting applications produce, consume, and act on information that’s sent to and fro based on user actions and changes in condition (i.e. “events”) instead of periodic request/reply polling, makes it possible to identify and act on opportunities as they emerge, in real-time, instead of looking at them in the rear-view mirror.

Event driven architecture presents developers with some interesting challenges throughout the development process. These challenges include:

How to capture the design and collaborate with the architects
How to implement the design in a simple and consistent way
How to understand the flow of information from an operational perspective in order to alert and solve problems

I’d like to explain these challenges in terms of how they’re addressed today, and how they can be dealt with now, thanks to our new product PubSub+ Event Portal.

I find that using an example is the best way to expose challenges and solutions, so I’ll frame my explanation in terms of a project some colleagues and I took on last week. Simply put, we used PubSub+ Event Portal to create event streams related to the COVID-19 pandemic so those looking to report on, track, or help with the pandemic could do so with access to real-time information.

Creating an Application to Provide Real-Time COVID-19 Data

Lots of developers out there are looking to do their part by helping government agencies, journalists, and – most importantly – scientists and public health authorities process and analyze data related to the COVID-19 pandemic. We decided to help them do so by making it easier to overcome the first challenge each of them faces – how to access the data and understand what it looks like.

Historically, architects have used tools like Confluence and PowerPoint, etc. to describe and document the design of events, applications, and data flow we’re being asked to implement. This can be challenging for a variety of reasons, leaving us asking questions like:

Where do I connect?
What are the payload schemas so I can serialize/deserialize the data?
Where do I send the output of my newly created application/microservice?
If I’m using an event generated outside of my team, who do I contact for more info?
What do I do if the documentation gets stale?

API Management solutions like Mulesoft provide a great experience for Synchronous RESTful interactions, but there isn’t an equivalent for event-driven or streaming interactions. Well, there wasn’t, anyway. We at Solace felt your pain and have come up with a solution that overcomes some of the challenges described above so developers like us can get the answers we need. Our product PubSub+ Event Portal is the first solution that lets you collaboratively create and manage events and event-driven applications.

Collaborative Design and Documentation

Getting back to the COVID-19 project, our first step was to document the design. Our Architect colleague (Jonathan Schabowsky, who wrote about his experiences here), kicked off the project by identifying the data source we would access (Johns Hopkins CSSE) and the applications we would need to acquire the raw data, look for changes, and publish smaller, more fine-grained events that represent changes to regions or cases so the community can consume just the state changes they‘re interested in.

They design the system they envisioned with Solace PubSub+ Event Portal so we, as developers, could understand and make comments. For example, we had some interesting discussions around how the topic hierarchy would work and how to apply best practices. In addition, we collectively decided to split up the logic into different Spring Boot microservices so they were more purpose-specific and manageable. All these changes were decided upon in Slack meetings, and updates were made in the event portal. It was awesome to be able to collaborate, make updates, and be in sync before we even started to write code.

The architecture was kept up to date in the event portal as we implemented it so we could easily provide the COVID-19 data feeds to the technical community. Collaboration and design documentation: Check!

Simple, Consistent Development

Moving beyond collaboration and documentation, one of the biggest pains I experience as a developer advocate at Solace is taking the design, topic hierarchy, asynchronous API (JMS, Solace, Spring Cloud Stream, Paho, etc.) and consistently developing applications.

These problems don’t come up with REST because OpenAPI provides a machine-readable API specification that lets code generators create a client (API consumer) or server (API provider) that adheres to the spec. This ensures consistent code, conformance to the design, and a simple development experience. Once again, the event-driven world hasn’t had anything like this… until now!

AsyncAPI is a machine-readable specification for event-driven applications that also provides code generators. We at Solace recently contributed a Spring Cloud Stream code generator for AsyncAPI and PubSub+ Event Portal supports the exportation of AsyncAPI files that represent the asynchronous application interfaces.

Going back to the usage, our architect designed the event-driven interactions within our event portal and assigned us different microservices to implement. Once we exported the AsyncAPI file for each application, we ran the code generator, reviewed the data models, made small tweaks, and added the business logic (like splitting the data by cases or regions) we had running.

Consistent and scalable microservices within minutes –a huge step forward for developers!

What’s going on with my application?

It’s not all sunshine and roses though, and as the philosophy of DevOps has become the norm we as developers increasingly own the applications throughout their lifecycle. Not only do we design and implement applications, but we also have a hand in deploying, monitoring, and managing them. This includes troubleshooting and resolving problems when the inevitable rainy day occurs. Although we try to avoid bugs, they always seem to show up eventually. The challenge is that when problems pop up we need to be able to efficiently identify the cause and push out a fix.

This has been a problem with event driven architecture for ages. The decoupling of applications, while it provides many advantages, makes it hard to trace the flow of events through the system. All we hear is, “Where did my message go?” and we have to start tracking and tracing to figure out where the “hose” got kinked – application failed, data parsing issue, what have you.

So, it’s after hours and you’re hunting down that pesky design document you last reviewed 6 months ago… #badmoment. You find it, realize it has not been updated, and wonder if it’s even up to date: #worsemoment! The point is, we can now use PubSub+ Event Portal to understand the implementation, know where to look for problems, and diagnose them without the stress and anxiety of having to “reverse engineer” the implementation.

Conclusion

I get that sometimes it feels like you could add another tool to your development environment every day, but if you’re developing event-driven applications development, Solace PubSub+ Event Portal is a no brainer. Seeing as how it helps with documentation (which isn’t personally my favorite part of being a developer), automates code generation, and lets you see what’s going on in the production environment, the benefits for developers like you and me are enormous. Take it for a spin on your next event-driven application project and see for yourself! I’m confident you’ll be glad you did.

From everyone at Solace, we would like to thank everyone out there working to alert and inform the public during this pandemic. You are truly our heroes!

If there is a project or event-driven application you have in mind, we encourage you to try PubSub+ Event Portal. We’d love to see what you can do with this toolset and are eager to receive any feedback you may have on our product.

The post Why Developers Need an Event Portal; Creating Applications that Disseminate Real-Time COVID-19 Data appeared first on Solace.

Reflecting on my First Hackathon

Marc DiPasquale — Thu, 20 Feb 2020 20:16:59 +0000

The Hackathon: MakeUofT

This past weekend I attended my first hackathon (well a make-a-thon in this case) as a sponsor and mentor since starting my role as a Developer Advocate. I have to admit that going in I had pretty low expectations. I was traveling from Orlando, FL to Toronto, Ontario on a holiday weekend to what I thought would be a pretty boring booth duty gig, and then to judge some projects that after only 24 hours of work would be half thought out and maybe partially working, if at all.

Well, the weekend has come and gone and wow was I wrong. MakeUofT not only surprised me, but it absolutely blew my mind and reminded me how far technology has come in the past decade. All while being a blast to participate in. According to their website, MakeUofT, at the University of Toronto, is Canada's Largest Makeathon and is a Major League Hacking member event. It brought together teams of students from across North America for a weekend of hacking and creativity. And not just software hacking, the event had a wide variety of hardware available for participants to use which ranged from Raspberry PIs to Qualcomm Dragonboards, sensors, cameras, 3D printers and more.

My Reflection

So why was I so surprised? As a software engineer for the past decade I knew in my mind that we have had many technological advances, but I think it's natural to see and assess them one by one as they happen. Raspberry PI became a thing, the cloud became the way to go, software as a service became the new norm, etc. And I started to research and use subsets of each of the advances one at a time as my projects called for it, e.g: at my previous company my team replaced physical racks of servers with AWS, used the Atlassian SaaS ecosystem to manage and track the software development process and went from installing & configuring proven software, such as the Elastic Stack and MongoDB to using their SaaS offerings. However, seeing how quickly the hackathon teams were able to go from idea to architecture to mostly working proof of concepts made me step back and really see all the technological advances as a whole. What many teams accomplished would have been nearly impossible just 10 short years ago.

Of course it helps that the students were super tech savvy. With most of them probably being born after the dot-com bubble in the late 1990s they likely never knew a time without computers and it showed. All of the teams that I helped mentor were really sharp: they knew their way around their IDEs and code in general, were able to quickly pick up on new technical concepts, slap together some code and see it run. On top of the extra tech savviness, the advancements that I mentioned above are the baseline for them. They expect to be able to just sign up for a service like Solace PubSub+, have it "just running", and use it to stream events immediately; whereas my first experience with that same Solace technology about ~9 years ago was to navigate the website, contact the sales team, have some meetings to determine feasibility, have hardware shipped, rack and stack the hardware, cable it up and get it added to the network, configure security & access control, and probably some other steps I'm forgetting about before I could even get around to using it. What took me probably a few weeks at an absolute minimum now takes them 5 minutes!

And why was my mind blown? The projects were amazing. Way better than I expected. And in many cases they actually worked 🤯!? Not just the software, but the hardware as well. The students really took advantage of the speed of technology in 2020. They knew that they could develop their proof of concepts quickly so they took their time at the beginning of the event to create well thought out ideas and a feasible architecture instead of just diving into the implementation. A large percentage of the teams even started to think about a potential business plan for where they could take their ideas beyond the event itself! At the end of the 24 straight hours of hacking (yes, very few of the students took time out to sleep) teams set up their projects in the main auditorium where they hosted science fair style demonstrations and judges, mentors and participants got to go around and see what the teams came up with. Seeing the demos was an absolute blast and there was definitely excitement in the air as you looked around and saw everything from drones and rovers to robotic arms and intelligent trash receptacles. If you want to check out some of the projects you can find them on DevPost here which I've also shown an image of below.

The Business Value of a Hackathon

At the end of the weekend it's great that I had a good time this weekend, but let's face it, our companies don't sponsor events for us to have a good time. So what do companies get out of sponsoring hackathons?

First and foremost we get to support the up and coming developer community while raising awareness at the same time. It was a pleasure to work with these students and see them begin to bring their ideas to life while also helping them learn about event-driven development and using our companies' product to do so.

This awareness has multiple benefits: It shows students that your company not only exists, but that it cares about the developer community. These types of events attract the best and brightest up and coming students and this is an excellent away to help identify them and hopefully add them to your team. The awareness also gets students using your technology, and hopefully, pending a good experience, they will continue to use your companies' technology for future projects as well.

Another great reason for companies to sponsor hackathons is to get candid developer feedback. The students participating in these hackathons are there to have fun and learn, but most of them also want to win. This means they have a short period of time to create something really awesome that will help set their project apart. As sponsors they are going to use your companies' technology and they expect it to work. They expect to get started fast and be able to use it to accomplish what you told them it is meant to achieve. They aren't just going to take a look at some sample code, walk through a tutorial and nod their head. They're going to really dig into the code, pull pieces out and use it to create their project. If there is confusion or speed bumps in your getting started experience they'll find it!

For this reason alone I would highly recommend any Developer Advocate or for that matter anyone who "owns" an API get your company to sponsor a hackathon. On this note, don't be afraid to ask for feedback. When going around to judge the "Best use of Solace PubSub+" prize I asked each team that used our technology about their experience. Was it enjoyable? Would they use it again? Did they have any pain points? It's not everyday that you get a chance to collect this much feedback in person so don't pass it up!

Thank You!

After such a great weekend I'm definitely looking forward to the next time I'll get to participate in a hackathon! Thank you to the University of Toronto IEEE and Major League Hacking for putting on MakeUofT, as well as to the sponsors that made it possible: Solace, Telus, Qualcomm and Huawei, among others. And last, but definitely not least, thank you to the students for coming out with such great energy and dedication.

Spring Initializr, Meet Your Perfect Match for Event-Driven Microservices: PubSub+

Marc DiPasquale — Wed, 06 Nov 2019 22:27:35 +0000

If you’re not familiar with start.spring.io, it hosts the Spring Engineering team’s opinionated version of Spring Initializr, which allows developers to generate Spring Boot projects pre-populated with the dependencies they need to get started quickly.

Spring Developer Advocate Josh Long calls start.spring.io the “second-best place on the internet,” and it just got better! Why do I say that? Because Solace PubSub+ is now available on start.spring.io, enabling developers to create event-driven microservices.

As a developer who has been creating event-driven applications with a combination of the Spring Framework and Solace PubSub+ for about eight years, I love how the developer experience keeps improving. This is the next step.

When selected as a dependency on Initializr at start.spring.io, the `Solace PubSub+’ option informs Initializr to add the Solace JMS Spring Boot Starter to the maven pom or build.gradle file as a dependency. This enables autoconfiguration of a ConnectionFactory or JndiTemplate, allowing developers to use Spring Boot to easily create event-driven microservices with PubSub+ and the JMS API. Using PubSub+ to create event-driven microservices empowers developers to use the best message exchange patterns for their use case, such as publish/subscribe, queuing, request/reply and it even provides the ability to store & replay. The use of PubSub+ also allows developers to take advantage of all the benefits of having events flow over the Event Mesh, such as being able to communicate with applications using other APIs and protocols and built-in high availability and disaster recovery.

Maven Example:

<dependency> <groupId>com.solace.spring.boot</groupId> <artifactId>solace-jms-spring-boot-starter</artifactId> <version>3.2.0</version> </dependency>

Want to give it a shot? Spring Initializr offers several ways to generate your Spring Boot project. You can do it using the Web UI, from your favorite IDE or even from the command-line.

Web: Use this link to spring.io to use the Initializr Web UI, and yes it even offers a dark mode! You can choose your project type (Maven or Gradle), programming language, project dependencies and more. You’ll find the Solace PubSub+ dependency by looking under the “Messaging” section, searching for “solace” or if you use the link above, we’ve pre-populated the Solace PubSub+ dependency for you ;). Once you’ve made your choices, click the Generate button to download a zip file of your Spring Boot project.
IDE: You can also create Spring Boot projects from your favorite IDE. These IDEs currently support Initializr: Spring Tool Suite, IntelliJ IDEA Ultimate, NetBeans (with this plugin) or VSCode (with the vscode-spring-initializr plugin). Which Initializr do you think they use by default? If you guessed the one at start.spring.io, you’re correct. For example, when using the Spring Tool Suite IDE for development, you can create a new Spring Starter Project and under the covers Initializr will generate your project. Check out the options available in the image below. After choosing those initial options and selecting Next, you will be asked to choose dependencies. You’ll find Solace PubSub+ under the Messaging dropdown. Choose it, click Finish, and a Spring Boot project is generated right in your IDE.
CLI: Not a fan of a GUI? No worries, you can generate Spring Boot projects from the command line as well. For example, if you want to generate a zip file containing a project with Java 11 and Solace PubSub+ as a dependency, you could execute: curl https://start.spring.io/starter.zip -d dependencies=solace -d javaVersion=11 -o demo.zip Or to see all available options, execute curl -XGET https://start.spring.io as seen in the image below.

Now that you’ve used Spring Initializr to generate your Spring Boot project with Solace’s Spring Boot JMS Starter as a dependency, it’s time to code your event-driven microservice. You can learn more about using our Spring Boot JMS Starter using the following resources:

If you have any questions, feel free to ask them in the Solace Developer Community with the Spring tag.

And if you love start.spring.io so much that you want to learn more about how it works, watch Stéphane Nicoll’s talk from SpringOne Platform 2019: Creating “The Second Best Place on the Internet” with Spring Initializr

The Spring + Solace developer experience has come a long way in the last eight years and I’m excited to see it continuously improve. Check out solace.com/spring to see all the ways you can use Spring with Solace and stay tuned for more upcoming Spring + Solace news.

Hint: SolaceYSpring & SolaceYAsyncAPI. Until next time, Happy Eventing!

The post Spring Initializr, Meet Your Perfect Match for Event-Driven Microservices: PubSub+ appeared first on Solace.

Considerations for Developers Building Event-Driven Microservices

Marc DiPasquale — Mon, 19 Aug 2019 12:00:31 +0000

A while back my colleague Jonathan Schabowsky recently wrote a paper called The Architect’s Guide to Event-Driven Microservices that’s turned out to be very popular, and for good reason. In it he explains the benefits of combining event-driven architecture and microservices, and candidly describes how decomposing applications admittedly makes life a little more…as he put it…”interesting.” With this post I’ll summarize the elements of the architectural view that are most important for developers to know as they build event-driven application and microservices.

Are You Ready to Develop for EDA?

Many organizations migrate to microservices for one reason: Agility. The ability to very quickly create and modify components in a way that offers bottom line business value is mission critical in a world where your competitors are a click away and time to market is everything. So, you’re a developer and you have been tasked with delivering on this migration…what do you need to know to be successful?

The first thing to understand is the fallacies of distributed computing:

The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
Topology doesn’t change
There is one administrator
Transport cost is zero
The network is homogeneous

They’re all relevant, but I’ve bolded the ones that are especially important in the world of microservices, because the smaller you make each microservice, the larger your service count, the more the fallacies of distributed computing impact stability and user experience/system performance.

The challenge is that microservices require connectivity/data in order to perform their roles and provide business value, and data acquisition/communication has been largely ignored, so much so that the tooling has severely lagged behind. For example, API management/gateway products only support synchronous, request/reply exchange patterns, which when coupled with the fallacies noted above exacerbates the challenges of distributed computing. Many of these challenges can be minimized by properly implementing an Event-Driven Architecture.

Think Event-Driven

Unlike service-oriented design, which are synchronous and blocking, event-driven design is non-blocking and uses lots of callbacks. Event-driven design is not a new concept, and you may even be familiar with it from using programming patterns such as the “Reactor Pattern” or the “Observer Pattern” which is one of the “Gang of Four” design patterns. The concepts used by these programming patterns can be applied to event-driven microservice architectures using architectural patterns, such as Publish-Subscribe, to achieve service decoupling, independent scalability, and one-to-many bi-directional communications.

The first step towards adopting the event-driven mindset is to change the way you think about designing and architecting solutions. Initially the tendency is to think about all interactions between services as a series in a sequence of request/ reply service calls. In fact, if you or your team uses the terminology of “invoking,” “requesting,” or “calling” then it is a sure sign you are still thinking in the request/reply paradigm.

Instead, try these: “What events should my service process?” and “What events will my service emit?”

Stop Orchestrating, Start Choreographing

Once you adopt event-driven thinking, you need to make the shift from orchestration to choreography. It is common for developers to think in terms of “service A will call service B which will call service C” and then implement that model through a chain of invocations (a->b> c) or by creating an orchestrator service such that x->a, then x->b, then x->c. Both approaches will cause chaos when the realities of distributed computing kick in, especially when you start to scale.

Orchestration vs. Choreography, Source: Thoughtworks

The alternative is to follow the philosophy of choreography. Services should react to changes in their environment and the benefits are immense:

Better agility: Agile development teams are more independent and are significantly less impacted by changes to other services.
Services are smaller/simpler: Each service is not required to have complex error handling for downstream service or network failures. Who likes writing error handling??? Not me!
Less service coupling: Services have no knowledge about the existence of other services.
Enables fine-grained scaling: Each service can be independently scaled up or down based on demand. This ensures a good user experience and is less wasteful of compute resources.
Easy to add new services: Due to less coupling, a new service can come online, consume events and implement new functionality without changes to any other service.

This list of benefits doesn’t come for free; there is no silver bullet…

Consistency of state then becomes an area of focus, because a service, being temporarily down, means that the event state changes may not be processed immediately. Fundamentally, how do we deal with this negative side effect?

Embrace Eventual Consistency

Eventual consistency is the idea that consistency will occur in the future, and it means accepting that things may be out of sync for some time. It’s a pattern and concept that lets developers avoid using nasty XA transactions. It’s the job of the eventing/messaging platform to ensure that these domain change events are never lost before being appropriately handled by a service and acknowledged.

Some think the only benefit of eventual consistency is performance, but the real advantage is the decoupling of the microservices since individual services are merely acting upon events that they are interested in.

Pick an Event-Driven Microservices Development Framework

As developers, we want the ability to easily write clean code which enables us to deliver functionality quickly. Thus, the development language and associated tooling becomes important. While event-driven architectures allow us to decouple our microservices and choose different technologies to implement each one it is recommended to choose a standard language/framework that best fits your team’s skill sets and deviate only when necessary.

For many organizations this skill set includes experience using the Spring Framework with Java (and other Spring supported, JVM-based languages). This experience has led to Spring Boot becoming the most popular framework for building microservices. Its adoption continues to grow and best of all, it has built in support for event-driven microservices through Spring Cloud Streams. Spring Cloud Streams is a framework for building highly scalable, event-driven microservices connected using shared event brokers. On top of providing an implementation framework, Spring Cloud Streams also defines vendor agnostic terminology for the event-driven community, including defining source _and _sink _applications that send or receive events over _channels.

Conclusion

While event-driven microservices may seem difficult initially, they are the future of most microservices and IT strategies. Become a ninja at their development by following these steps and make your life easier and simpler!

The post Considerations for Developers Building Event-Driven Microservices appeared first on Solace.

How to Develop Microservices: 7 Event-Driven Considerations

Marc DiPasquale — Mon, 19 Aug 2019 12:00:31 +0000

What are the are most important things for developers to know when it comes to microservices application development? Many organizations migrate to microservices for one reason: Agility. The ability to quickly create and modify components in a way that offers bottom-line business value is mission critical in a world where your competitors are a click away and time to market is everything. However, you, a developer in your organization, also know that performance of your operational systems matters and no matter how agile microservices allow your team to be you are tasked with both delivering quickly AND ensuring the operational system can perform under peak load. To achieve both agility and performance many development teams are embracing the use of event-driven microservices. So, what do you need to know to be successful?

In a nutshell, you need to know:

The basics of messaging – its benefits and the various communication patterns
The common false assumptions of distributed computing that developers make
How event-driven architecture can help you
Why to choose choreography over orchestration for your microservices
Why you should embrace eventual consistency
Frameworks that support event-driven microservices development
Tools to visualize and choreograph your microservices

Note: This blog post was recently updated on July 7, 2020

1. The Basics of Messaging

Capitalizing on the benefits of microservices application development within your enterprise requires a broader understanding of messaging patterns for event-driven microservices and when to use them.

Using messaging to communicate between microservices allows you to realize these benefits:

Simple, scalable connectivity
Simple, high availability
Simple producer/consumer scalability
The enablement of publish/subscribe, message filtering, routing and fanout
Decoupling of your producers and consumers

To learn more details about these benefits, I suggest you read this blog post.

2. The False Assumptions of Distributed Computing

It has been more than 20 years since Peter Deutsch and James Gosling developed the 8 fallacies of distributed computing, but these eight false assumptions are still made today by developers new to distributed applications.

All eight fallacies are relevant, but I’ve bolded the ones that are especially important in the world of microservices application development, because the smaller you make each microservice, the larger your service count, and the larger the service count, the more these fallacies will impact your stability and user experience/system performance.

The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
Topology doesn’t change
There is one administrator
Transport cost is zero
The network is homogeneous

The challenge is that microservices require connectivity/data in order to perform their roles and provide business value, and data acquisition/communication has been largely ignored, so much so that the tooling has severely lagged behind. For example, API management/gateway products only support synchronous, request/reply exchange patterns, which when coupled with the fallacies noted above, exacerbates the challenges of distributed computing. Many of these challenges can be minimized by properly implementing event-driven architecture.

3. The Advantage of Event-Driven Application Architecture

Unlike service-oriented design, which are synchronous and blocking, event-driven design is non-blocking and uses lots of callbacks. Event-driven design is not a new concept, and you may even be familiar with it from using programming patterns such as the reactor pattern or the observer pattern. The concepts used by these programming patterns can be applied to event-driven microservice architectures using architectural patterns, such as publish-subscribe, to achieve service decoupling, independent scalability, and one-to-many bi-directional communications.

The first step towards adopting the event-driven mindset is to change the way you think about designing and architecting solutions. Initially the tendency is to think about all interactions between services as a series in a sequence of request/reply service calls. In fact, if you or your team uses the terminology of “invoking,” “requesting,” or “calling” then it is a sure sign you are still thinking in the request/reply paradigm.

Instead, try these: “What events should my service process?” and “What events will my service emit?”

4. The Benefits of Choreography in Microservices

Orchestration vs. Choreography, Source: Thoughtworks

The alternative is to follow the philosophy of choreography. Services should react to changes in their environment and the benefits are immense:

Better agility : Agile development teams are more independent and are significantly less impacted by changes to other services.
Services are smaller/simpler : Each service is not required to have complex error handling for downstream service or network failures. Who likes writing error handling??? Not me!
Less service coupling : Services have no knowledge about the existence of other services.
Enables fine-grained scaling : Each service can be independently scaled up or down based on demand. This ensures a good user experience and is less wasteful of compute resources.
Easy to add new services : Due to less coupling, a new service can come online, consume events and implement new functionality without changes to any other service.

This list of benefits doesn’t come for free; there is no silver bullet…

5. Eventual Consistency in Microservices

6. Event-Driven Microservices Development Frameworks

As a developer, you want the ability to easily write clean code which enables you to deliver functionality quicker. Thus, the development language and associated tooling becomes important. While event-driven architectures allow us to decouple our microservices and choose different technologies to implement each one it is recommended to choose a standard language/framework that best fits your team’s skill sets and deviate only when necessary.

For many development team, their skill set usually includes experience using the Spring framework with Java (and other Spring-supported, JVM-based languages). This experience has led to Spring Boot becoming the most popular framework for microservices application development in organizations. Its adoption continues to grow and best of all, it has built in support for event-driven microservices through Spring Cloud Stream.

Spring Cloud Stream is a framework for building highly scalable, event-driven microservices connected using shared event brokers (a Solace binder is available here. On top of providing an implementation framework, Spring Cloud Stream also defines vendor agnostic terminology for the event-driven community, including defining source and sink applications that send or receive events over channels.

If you’re interested, I dive deeper into creating microservices using Spring Cloud Stream and AsyncAPI Code Generator in this post and this video.

7. Model and Visualize Microservices in Your Event-Driven Process

As discussed earlier, one of the benefits of choreographing your microservices using an event-driven architecture is to achieve less service coupling. But this benefit introduces some challenges of its own. Specifically, when your producers and consumers don’t care about the existence of each other it can make it challenging to model your architecture at design time and even harder to understand what events are flowing where at run-time.

Attempts to solve this challenge have been made in many different ways by many different organizations, but until recently there hasn’t been an all-encompassing solution available to the masses. The introduction of the Solace PubSub+ Event Portal solves these challenges by allowing you to visualize, manage, and choreograph your microservices at design time and providing features that allow you to discover, audit and secure your run-time environments.

Read more about Why Developers Need an Event Portal, where I walk through the steps I took to create consistent and scalable microservices within minutes.

Conclusion

The post How to Develop Microservices: 7 Event-Driven Considerations appeared first on Solace.

DEV Community: Marc DiPasquale

Publishing Github Java Packages to Maven Central for a New Domain

Overview

Setting up OSSRH (The Repository)

Get An Account

Prove Domain Ownership

Create Your User Token

Configuring the Maven pom

Create GPG Keys

Create Your Key

Share Your Public Key

Configure the Github Secrets

Setting up the Github Action

Running the Github Action - let's test it with a Snapshot release

Running the Github Action - now let's make an actual release

Approve the Deployment in Nexus

Check out the deployed project!

Conclusion

Getting Started with Spring Cloud Stream using Spring Initializr

The Spring Cloud Stream Framework

PubSub+ and Spring Initializr

5-Minute Tutorial (seriously, only 5 minutes)

Let’s Begin…

Behind the Scenes

What should you try next?

SolaceProducts / solace-spring-cloud

Supports all solace spring cloud products

AsyncAPI Code Generation: Microservices Using Spring Cloud Stream

AsyncAPI: What Is It?

Defining the Application That You Want to Develop: The AsyncAPI Document

Create Event-Driven Microservices Using Spring Cloud Stream Without Learning Messaging APIs

Download and Run the AsyncAPI Generator

Add Your Business Logic

Helpful Parameters and Specification Extensions for Creating Microservices Using the AsyncAPI Spring Cloud Stream Template

Tips For Using The Code Generator To Create Event-Driven Microservices Using Spring Cloud Stream

Conclusion

Why Developers Need an Event Portal; Creating Applications that Disseminate Real-Time COVID-19 Data

Creating an Application to Provide Real-Time COVID-19 Data

Collaborative Design and Documentation

Simple, Consistent Development

What’s going on with my application?

Conclusion

Reflecting on my First Hackathon

The Hackathon: MakeUofT

My Reflection

The Business Value of a Hackathon

Thank You!

Spring Initializr, Meet Your Perfect Match for Event-Driven Microservices: PubSub+

Considerations for Developers Building Event-Driven Microservices

Are You Ready to Develop for EDA?

Think Event-Driven

Stop Orchestrating, Start Choreographing

Embrace Eventual Consistency

Pick an Event-Driven Microservices Development Framework

Conclusion

How to Develop Microservices: 7 Event-Driven Considerations

1. The Basics of Messaging

2. The False Assumptions of Distributed Computing

3. The Advantage of Event-Driven Application Architecture

4. The Benefits of Choreography in Microservices

5. Eventual Consistency in Microservices

6. Event-Driven Microservices Development Frameworks

7. Model and Visualize Microservices in Your Event-Driven Process

Conclusion