DEV Community: Dublin Anondson

February 15th: Week in Review

Dublin Anondson — Sun, 16 Feb 2020 01:04:40 +0000

Hello again! Finally got around to posting one of these on time finally.

Things I learned/did this week:

I joined a new team 🎉🎉
I started learning Kotlin and implemented the Builder Pattern for the project I'm working on ✋, it's a great step away from the Java development I'm used to
Started converting my You Don't Know JS notes into posts that I'll be sharing soon 🤓📖
Continuing to wrap my head around react hooks

Some articles I read:

Saw this great video of a guy who created a nerf dart face tracking turret using Python
Found out dicts in Python are now ordered as of 3.7
Some good ol' Kotlin docs

Gotta say this week has been pretty solid for me, I hope everyone else's was as well. If you did anything cool you want to share tell me about it in the comments. Be seein' ya 🤠🐮

February 8th: Week in Review

Dublin Anondson — Mon, 10 Feb 2020 01:28:35 +0000

2nd week of February is done, hope everyone had a good time.

Things I learned/did this week:

I got promoted to Senior Software Engineer 🎊🎊
Started the movie-search react project based on Simon Holdorf's post: 9 Projects you can do to become a Frontend Master in 2020
Built out the artist profile for basic-music-player, I'm super stoked on how it turned out.

Articles I've read:

Learned about functional react components at programmingwithmosh react is blowin' my mind so far and I'm excited to delve into it more.
I wanted to get a clearer idea of JAMStack so I read about it at jamstack.wtf
evernoise is a neat sight that maps music genres, from main stream to ultra niche. Highly recommend checking it out if you need some new music.

I'm working on a series based off my notes from reading You Don't Know JS, so be on the lookout for them.

If you did anything cool you want to share let me know in the comments. Be seein' ya 🤠 🐮

February 1st: Week in Review

Dublin Anondson — Sun, 09 Feb 2020 02:37:51 +0000

*blows dust off of blog

Not like I missed a post or anything. Hope everyone has been doing well this past week.

Things I learned/did last week:

I learned how to leverage retries in rxJava for web service calls. There's a handy guide on rxJava error handling operators here if you need it. But what I used boiled down to:


Callable.fromAction(() -> webserviceClientOrWhatever.makeHttpCall())
    .retry(3)
    .subscribe(() -> 
               log.info("Look mom I'm on Dev!"));

I'm looking forward to leveraging it more in future projects now. It's some powerful stuff.

I learned the differences between Subject types in rxJS.
- Subject: doesn't keep a history of events.
- BehaviorSubject: when subscribed to, gives you the last emitted value.
- ReplaySubject: keeps a history of values so a new subscriber can replay play those.
I learned to how to setup TravisCI for basic-music-player. Excited to build out the CI pipeline.

Some articles I read:

Once a month I read a post about emotional intelligence. I think it's a good reminder to be more mindful of yourself.
Found out about Pop OS, a new Linux distro built by System76 from this dev article by Jeremy Morgan.

Hope everyone had a good week, if you did anything cool you want to share let me know in the comments. Be seein' ya 🤠 🐮

January 25th: Week in Review

Dublin Anondson — Sun, 26 Jan 2020 00:19:57 +0000

I would have posted a review for last week but I was out camping 🏕️ 🔥, and camping is cool.

It's the last weekend of January 2020, we made it!

Some things I learned/did this week:

Glide.js is a thing (made a note of it for future projects for sure).
Being totally isolated from my profession and the rest of the world is a great mental reset for the beginning of the year, highly recommend winter camping if you can manage. ☃️
In Angular, you can't import the same component into two sibling components, move that import into the parent one of course!
A talk submission of mine got accepted for an internal conference my job is putting on next month. 🎉🎉

Some interesting articles I read:

A Personal Website Journey from Adrian Bece. It's cool to see someone's style evolution over time, not to mention the neat tech behind each of them.
Some good advice on how to NOT read tech books by Miranda. It was a good reminder to not make reading tech books a slog (I like to finish books I start but tbh most tech ones are really dry).
I've seen this one going around the past week but finally got around to reading it. Goodbye, Clean Code, a short anecdote on why clean code isn't always the best code.

Hope everyone has a great next week 🌵🤠

January 11th: Week in Review 🎉🎉

Dublin Anondson — Sun, 12 Jan 2020 02:15:37 +0000

Been a minute since I published my last week in review. Time got away from me (read: I'm winding down a Red Dead Redemption 2 binge that went on for a little too long).

The year ended and the new one began, we did it! 👏 👏

Things I learned/did this week:

Deleting an entry out of an array in JS doesn't cause the length of the array to get updated, thanks You Don't Know JS
I picked up coding basic-music-player again, nice to get back to personal project coding
Java has a var keyword 🤷

I read one whole article this week but I enjoyed it, Adrian's post on the Revealing Module Pattern in JS.

Hope your year is starting off well. I'm hoping to write some tech articles this year on Kubernetes and JS. See ya soon! 👋

Rubber Ducks: Do they even Work?

Dublin Anondson — Tue, 07 Jan 2020 13:06:43 +0000

I replied to a comment on Ben Halpern's post on getting out of debugging ruts and it got me thinking about how I've always had more success talking to a real person over an inanimate object.

I know talking to rubber ducks is a often recommended way to talk through problems by yourself. But I feel like it lacks some pizazz that makes it useful for me.

To all you dev folks out there, has a rubber ducking worked for you, or was talking to a person significantly better?

December 7th: Week in Review 🎉🎉

Dublin Anondson — Sun, 08 Dec 2019 23:15:21 +0000

The first week of December is over. We did it!! 👏👏

Some things that happened:

I got my Spotify end of year wrap up and found out I listened to 37,495 minutes of music and 20 full hours of that was BROCKHAMPTON. Here's my 2019 Top Tracks Playlist if you want to judge me.
I turned 27 🎂
I refactored a huge chunk of my basic-music-player app to use handlebars templates rendered from the backend and got the routing setup. I can more easily build out new features now 💻 💯

Some good written word:

I read a thought provoking Agile article by Charles Lamdin, Dear Agile, I’m Tired of Pretending
Not an article, but I got an audible subscription and started listening to Extreme Ownership by Jocko Willink and Leif Babin. It's been real helpful in getting me thinking about how I can improve and develop as a leader.

If you had a sweet top tracks playlist or an audiobook recommendation, I'd love to hear it. Recommendations are always better with human backing 😄

Hope everyone is having an awesome last month of the year. You got this!

November 30th: Week in Review 🎉🎉

Dublin Anondson — Sat, 30 Nov 2019 19:30:58 +0000

Hope everyone out there is having a good weekend. I wanted to start a week in review series where I pass on posts I've read and talk about anything interesting I learned over the past week.

Some of the things I learned:

While working through the Ruby exercism.io track, I learned how to make immutable constants using the freeze keyword.

I finished the Object and Classes book in the You Don't Know JS series and gained a much better idea of how prototypes and objects work. Behavior delegation kinda weirds me out but it seems way less gross than trying to jam inheritance into JS.
I started learning how to use Adobe XD for creating wireframes for websites.

Some articles I read:

Martin Splitt's overview of CORS
Java with a Clojure mindset by Dan Lebrero
Benjamin Liu's article on Twelve Factor Apps

Is there anything cool you learned last week? Let me know down below 👇

Changing Primary Language/Stack Experiences

Dublin Anondson — Sun, 10 Nov 2019 15:48:19 +0000

I've been a Java/Spring Boot developer for the majority of my professional career and I'm working to broaden my skill-set beyond that so I've been learning JS and web development in my spare time.

The web development ecosystem seems large to say the least, and I feel like it's a tough move out of the back-end space given the problem sets and things to learn.

What's been the dev communities experiences changing primary language/tech stack, has it been easy for you to switch it up?

Getting Eventful With Azure Event Hubs: Part Two

Dublin Anondson — Sun, 06 Oct 2019 15:22:11 +0000

In part one of this series, I went over creating an event hub in azure, connecting it up to my spring boot web service, then send messages to it. Now I'm going to go over reading those messages from event hub and processing them.

Like in part one, the completed example can be found here.

Creating a Storage Account in Azure

Why do we need a storage account?

We need a storage account setup so we can store partition leases and checkpoints.

Head back to the azure portal and click on storage accounts.

Then click the Add(+) button and you'll be taken to the storage account creation screen.

Fill that out and click 'Review + Create' at the bottom. We should get the OK from Microsoft, if something did go wrong, make the needed corrections and try it again.

Once the OK is gotten, we can click 'Create' and azure will spin up our storage account.

Getting Access Key Info

Once the storage account is created, navigate to it and select the 'Access Key' option in the side menu and copy over either of the connection strings.

Creating an Event Processor Host

In our EventHub config class, I've created an EventProcessorHost bean.


import com.microsoft.azure.eventhubs.EventHubClient;
import com.microsoft.azure.eventhubs.EventHubException;
import com.microsoft.azure.eventprocessorhost.EventProcessorHost;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.io.IOException;
import java.util.concurrent.Executors;

@Configuration
public class EventHubConfig {

    @Value("${eventHub.connectionString}")
    private String connectionString;

    @Value("${eventHub.name}")
    private String eventHubName;

    @Value("${eventHub.storage.consumerGroupName}")
    private String consumerGroupName;

    @Value("${eventHub.storage.hostNamePrefix}")
    private String hostNamePrefix;

    @Value("${eventHub.storage.storageConnectionString}")
    private String storageConnectionString;

    @Value("${eventHub.storage.storageContainerName}")
    private String storageContainerName;

    @Bean
    public EventHubClient setupEventHubConnection() throws IOException, EventHubException {
        return EventHubClient.createFromConnectionStringSync(connectionString, Executors.newScheduledThreadPool(1));
    }

    @Bean
    public EventProcessorHost createEventHubProcessorHost() {
        return EventProcessorHost.EventProcessorHostBuilder
                .newBuilder(EventProcessorHost.createHostName(hostNamePrefix), consumerGroupName)
                .useAzureStorageCheckpointLeaseManager(storageConnectionString, storageContainerName, null)
                .useEventHubConnectionString(connectionString, eventHubName)
                .build();
    }
}

For this example I'm using the EventProcessorHost builder and passing in the needed config for it. I've defined the config in my application.yml below.


eventHub:
  connectionString: 'connections string here' #Event Hub connection string goes here
  name: 'event hub name here' # The name of the event hub
  storage:
    consumerGroupName: $Default #This is the default consumer group for event hub, but it can be customized
    storageConnectionString: 'storage connection string here'
    storageContainerName: 'storage container name here'
    hostNamePrefix: 'some unique prefix' # Identifies the instance of the EventProcessorHost

Now that we have an EventProcessorHost bean, we can inject it into our EventProcessorHostService.

import com.dublin.eventhub.demo.exception.ErrorNotificationHandler;
import com.dublin.eventhub.demo.processor.EventProcessor;
import com.microsoft.azure.eventprocessorhost.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.util.concurrent.ExecutionException;


@Component
public class EventProcessorHostService {

    private final EventProcessorHost eventProcessorHost;
    private Logger log = LoggerFactory.getLogger(EventProcessorHostService.class);

    @Autowired
    public EventProcessorHostService(EventProcessorHost eventProcessorHost) {
        this.eventProcessorHost = eventProcessorHost;
    }

    @PostConstruct
    public void run() throws ExecutionException, InterruptedException {
        log.info("Setting up event hub {}", eventProcessorHost.getHostName());
        EventProcessorOptions options = new EventProcessorOptions();
        options.setExceptionNotification(new ErrorNotificationHandler());
        eventProcessorHost.registerEventProcessor(EventProcessor.class, options).get();
    }
}

There's a little to unpack here, so let's go over what's going on.

I'm using constructor injection to pass in the EventProcessorHost bean I defined earlier.
I've defined a basic EventNotificationHandler class that handles errors not related to specific partitions (such as initialization failures). I pass that to the EventProcessorHost by wrapping it in an EventProcessorOptions object.
I'm registering the EventProcessor, this registers the host with the event hub and obtains leases on some of the partitions so we can start processing messages. For each partition lease, an instance of EventProcessor is created for that partition.
Finally, I use @PostConstruct to initialize the connection to event hub.

Below is the ErrorNotificationHandler.

import java.util.function.Consumer;

import com.microsoft.azure.eventprocessorhost.ExceptionReceivedEventArgs;
import lombok.extern.slf4j.Slf4j;

@Slf4j
public class ErrorNotificationHandler implements Consumer<ExceptionReceivedEventArgs> {

    @Override
    public void accept(ExceptionReceivedEventArgs t) {
        log.error("SAMPLE: Host " + t.getHostname() + " received general error notification during " + t.getAction() + ": " + t.getException().toString());
    }
}

We now have our EventProcessorHost configured and ready to run, but in order for us to process messages, we also have to define an event processor class that reads off the event hub and handles the checkpointing.

We need to extend the IEventProcessor interface, which gives us access to the PartitionContext for checkpointing and an Iterable of the EventData. There's a couple things going on here.

I initialize the EventPayloadProcessor class in the onOpen method.
In the onEvents method, I've created a for loop to iterate through the events, and a try catch in the loop body, so if an event errors out while processing, I'm able to process the remaining events.
If each event processes successfully, I checkpoint right after processing.

NOTE: Since checkpointing is asynchronous and not the fastest, I recommend checkpointing after some number of events processed. Since this example is small I'm not bothering to.

import com.dublin.eventhub.demo.model.EventPayload;
import com.microsoft.azure.eventhubs.EventData;
import com.microsoft.azure.eventprocessorhost.CloseReason;
import com.microsoft.azure.eventprocessorhost.IEventProcessor;
import com.microsoft.azure.eventprocessorhost.PartitionContext;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import org.springframework.util.SerializationUtils;

@Slf4j
@NoArgsConstructor
@Service
public class EventProcessor implements IEventProcessor {

    private EventPayloadProcessor eventPayloadProcessor;

    @Override
    public void onOpen(PartitionContext partitionContext) {
        eventPayloadProcessor = new EventPayloadProcessor();
    }

    @Override
    public void onClose(PartitionContext partitionContext, CloseReason closeReason) {

    }

    @Override
    public void onEvents(PartitionContext partitionContext, Iterable<EventData> iterable) {
        for(EventData event: iterable) {
            try{
                EventPayload eventPayload = (EventPayload) SerializationUtils.deserialize(event.getBytes());
                eventPayloadProcessor.process(eventPayload);
                partitionContext.checkpoint(event);
            } catch (Exception e) {
                log.error("An Error occured when processing event data, exception: ", e);
            }

        }
    }

    @Override
    public void onError(PartitionContext partitionContext, Throwable throwable) {
    }
}

And finally, I've defined the EventPayloadProcessor below.

import com.dublin.eventhub.demo.model.EventPayload;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

@Slf4j
@NoArgsConstructor
@Service
class EventPayloadProcessor {

    void process(EventPayload eventPayload) {
        log.info("Hello! My name is {} and my favorite food is {}", eventPayload.getFirstName(), eventPayload.getFavoriteFood());
    }
}

And that's it. Everything should be ready to roll. 🤠

Running the Application

If you've been following along since part one. There should be some messages out in event hub for us to process. So we should be able to start up the app and have the payloads get processed.

2019-10-06 08:12:18.242  INFO 42015 --- [nio-8080-exec-1] c.d.eventhub.demo.controller.Controller  : Eventhub send endpoint called, sending EventPayload(firstName=Dublin, lastName=Anondson, email=null, favoriteFood=Nanas) to event hub..
2019-10-06 08:12:18.247  INFO 42015 --- [nio-8080-exec-1] c.d.e.demo.service.EventHubService       : Sending message to the event hub event-hub-test
2019-10-06 08:12:18.253  INFO 42015 --- [pool-1-thread-4] c.m.azure.eventhubs.impl.MessageSender   : clientId[EC_493aee_1570367538249_MF_41070c_1570367523209-InternalSender], path[event-hub-test], operationTimeout[PT1M], creating a send link
2019-10-06 08:12:18.258  INFO 42015 --- [pool-1-thread-4] c.m.a.eventhubs.impl.MessagingFactory    : messagingFactory[MF_41070c_1570367523209], hostName[dublin-rest-demo.servicebus.windows.net], getting a session.
2019-10-06 08:12:18.258  INFO 42015 --- [pool-1-thread-4] c.m.azure.eventhubs.impl.SessionHandler  : onSessionLocalOpen connectionId[MF_41070c_1570367523209], entityName[cbs-session], condition[Error{condition=null, description='null', info=null}]
2019-10-06 08:12:18.259  INFO 42015 --- [pool-1-thread-4] c.m.a.eventhubs.impl.SendLinkHandler     : onLinkLocalOpen senderName[cbs], linkName[cbs:sender], localTarget[Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
2019-10-06 08:12:18.259  INFO 42015 --- [pool-1-thread-4] c.m.a.eventhubs.impl.ReceiveLinkHandler  : onLinkLocalOpen receiverName[cbs], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
2019-10-06 08:12:18.327  INFO 42015 --- [pool-1-thread-2] c.m.azure.eventhubs.impl.SessionHandler  : onSessionRemoteOpen connectionId[MF_41070c_1570367523209], entityName[cbs-session], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
2019-10-06 08:12:18.328  INFO 42015 --- [pool-1-thread-2] c.m.a.eventhubs.impl.SendLinkHandler     : onLinkRemoteOpen senderName[cbs], linkName[cbs:sender], remoteTarget[Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
2019-10-06 08:12:18.328  INFO 42015 --- [pool-1-thread-2] c.m.a.eventhubs.impl.ReceiveLinkHandler  : onLinkRemoteOpen receiverName[cbs], linkName[cbs:receiver], remoteSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
2019-10-06 08:12:18.329  INFO 42015 --- [pool-1-thread-2] c.m.a.e.impl.RequestResponseOpener       : requestResponseChannel.onOpen complete clientId[MF_41070c_1570367523209], session[cbs-session], link[cbs], endpoint[$cbs]
2019-10-06 08:12:18.398  INFO 42015 --- [pool-1-thread-4] c.m.a.eventhubs.impl.MessagingFactory    : messagingFactory[MF_41070c_1570367523209], hostName[dublin-rest-demo.servicebus.windows.net], getting a session.
2019-10-06 08:12:18.398  INFO 42015 --- [pool-1-thread-4] c.m.azure.eventhubs.impl.SessionHandler  : onSessionLocalOpen connectionId[MF_41070c_1570367523209], entityName[event-hub-test], condition[Error{condition=null, description='null', info=null}]
2019-10-06 08:12:18.464  INFO 42015 --- [pool-1-thread-4] c.m.azure.eventhubs.impl.SessionHandler  : onSessionRemoteOpen connectionId[MF_41070c_1570367523209], entityName[event-hub-test], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
2019-10-06 08:12:18.464  INFO 42015 --- [pool-1-thread-4] c.m.a.eventhubs.impl.SendLinkHandler     : onLinkLocalOpen senderName[EC_493aee_1570367538249_MF_41070c_1570367523209-InternalSender], linkName[LN_ddb0ee_1570367538464_5324_G9], localTarget[Target{address='event-hub-test', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
2019-10-06 08:12:18.531  INFO 42015 --- [pool-1-thread-4] c.m.a.eventhubs.impl.SendLinkHandler     : onLinkRemoteOpen senderName[EC_493aee_1570367538249_MF_41070c_1570367523209-InternalSender], linkName[LN_ddb0ee_1570367538464_5324_G9], remoteTarget[Target{address='event-hub-test', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
2019-10-06 08:12:18.531  INFO 42015 --- [pool-1-thread-4] c.m.azure.eventhubs.impl.MessageSender   : onOpenComplete - clientId[EC_493aee_1570367538249_MF_41070c_1570367523209-InternalSender], sendPath[event-hub-test], linkName[LN_ddb0ee_1570367538464_5324_G9]
2019-10-06 08:12:18.678  INFO 42015 --- [b4f183ddd]-1-14] c.d.e.d.processor.EventPayloadProcessor  : Hello! My name is Dublin and my favorite food is Nanas

But gosh, that's a lot of noise in the logs, don't you think? Let's fix that by setting the log level of the event hub packages to ERROR.

logging:
  level:
    com.microsoft.azure.*: ERROR

And try it again, much cleaner don't you think?

2019-10-06 08:55:48.241  INFO 47531 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2019-10-06 08:55:48.246  INFO 47531 --- [           main] com.dublin.eventhub.demo.Application     : Started Application in 5.37 seconds (JVM running for 5.792)
2019-10-06 08:56:01.700  INFO 47531 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2019-10-06 08:56:01.700  INFO 47531 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2019-10-06 08:56:01.709  INFO 47531 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 9 ms
2019-10-06 08:56:01.817  INFO 47531 --- [nio-8080-exec-1] c.d.eventhub.demo.controller.Controller  : Eventhub send endpoint called, sending EventPayload(firstName=Dublin, lastName=Anondson, email=null, favoriteFood=Nanas) to event hub..
2019-10-06 08:56:01.822  INFO 47531 --- [nio-8080-exec-1] c.d.e.demo.service.EventHubService       : Sending message to the event hub event-hub-test
2019-10-06 08:56:19.581  INFO 47531 --- [352fbae26]-1-14] c.d.e.d.processor.EventPayloadProcessor  : Hello! My name is Dublin and my favorite food is Nanas

And that's it, we've connected to event hub and read off our message!

Final Thoughts

In this tutorial we've gone through creating an azure storage account, setting up a connection to event hub in our spring boot service, and reading off the messages we've previously sent to event hub. I hope this guide was helpful, let me know what you think in the comments.

Happy coding!

Getting Eventful With Azure Event Hubs: Part One

Dublin Anondson — Wed, 25 Sep 2019 02:03:38 +0000

Event hub is a big data ingestion offering from Microsoft, it leverages AMQP, HTTPS, and Apache Kafka under the hood. Event hub offers benefits like partitioning and check-pointing on the data stream, plus all the scalability your wallet can handle.

In this article I'll be going over how to setup a connection and send data to event hub using a spring boot web service. You can view the completed example here.

Setting up an Azure Account

If you haven't yet, you'll need to signup for an azure dev account. They hook you up with enough credit for what we need to do in this guide.

Creating the Event Hub Namespace

Log in to the azure portal and search for 'event hub' in the search box at the top. You'll want to select the 'Event Hubs' option.

From there click on 'Event Hubs', and click on the Add (+) icon.

Fill out the create form, for this example I'm using the barest possible settings. You'll probably need to create a resource group for this event hub as well.

Boom! We've got an event hub namespace setup. The namespace acts as an organizational directory for your created event hubs.

Creating the Event Hub

Navigate to your event hub namespace and click on the Event Hub Add (+) icon.

Give your event hub a snazzy name, scale the partition count as you need but for this guide I'm keeping mine at 2. Click create at the bottom of the form and we're good to go.

You should see a message saying that your event hub is getting created, once it's done, navigate to it from your namespace menu. You'll come upon a dashboard that shows all kind of sweet metrics, like throughput and message counts.

In the side menu, navigate over to 'Shared access policies' and click Add (+). Give it a name and give it manage access (which gives us read and write).

Creating a Shared Access Policy gives us the keys to the event hub castle. Click on the policy you just created and make a note of the connection string.

Sending Data to Event Hub

Adding in the event hub dependencies.

First things first, get those dependencies added to your pom file. As of this writing the latest versions of the dependencies are 3.0.0.

      <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-eventhubs</artifactId>
            <version>3.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-eventhubs-eph</artifactId>
            <version>3.0.0</version>
        </dependency>

Next, let's create a config class and create an EventHubClient.

Creating a client and hooking it up.

import com.microsoft.azure.eventhubs.EventHubClient;
import com.microsoft.azure.eventhubs.EventHubException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.io.IOException;
import java.util.concurrent.Executors;

@Configuration
public class EventHubConfig {

    @Value("${eventHub.connectionString}")
    private String connectionString;

    @Bean
    public EventHubClient setupEventHubConnection() throws IOException, EventHubException {
        return EventHubClient.createFromConnectionStringSync(connectionString,
                              Executors.newSingleThreadScheduledExecutor());
    }
}

In this class I'm pulling in the connectionString from a configuration file (in my case the application's yml file), injecting into the config class, then passing it as-is to the createFromConnectionStringSync method, which takes in a connection string and a ScheduledExecutorService object. Since we're not doing anything fancy here I'm using a single threaded executor.

Here's how I've defined my yml file.

eventHub:
  connectionString: 'connections string here'

Now that we have an Event Hub Client bean, let's go ahead and create a service component that uses it.

import com.dublin.eventhub.demo.controller.Controller;
import com.dublin.eventhub.demo.model.EventPayload;
import com.microsoft.azure.eventhubs.EventData;
import com.microsoft.azure.eventhubs.EventHubClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.SerializationUtils;

import java.util.Objects;

@Service
public class EventHubService {
    private final EventHubClient eventHubClient;
    private Logger log = LoggerFactory.getLogger(Controller.class);

    @Autowired
    public EventHubService(EventHubClient eventHubClient) {
        this.eventHubClient = eventHubClient;
    }

    public void sendEvent(EventPayload test) {

        byte[] bytes = SerializationUtils.serialize(test);

        log.info("Sending message to the event hub {}", eventHubClient.getEventHubName());
        eventHubClient.send(EventData.create(Objects.requireNonNull(bytes)), test.toString());
    }
}

We'll use constructor injection to inject the client into our service, from there I define the sendEvent method which will use the client to send the data.

To send data to event hub, we need to serialize the message into a byte array, wrap it in an EventData object, then pass it to the client's send method.

Creating a data class

For this example I've defined a simple data class.

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.io.Serializable;

@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class EventPayload implements Serializable {
    private String firstName;
    private String lastName;
    private String favoriteFood;
}

It's important to note that the data you're working with for event hub needs to be Serializable, so make sure to implement Serializable on the data class you're working with, otherwise you'll get a java.lang.IllegalArgumentException: Failed to deserialize object message.

Next, we'll define an endpoint to post our data to.

Building an Endpoint

import com.dublin.eventhub.demo.model.EventPayload;
import com.dublin.eventhub.demo.service.EventHubService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

@RestController
public class Controller {

    private final EventHubService eventHubService;
    private Logger log = LoggerFactory.getLogger(Controller.class);

    @Autowired
    public Controller(EventHubService eventHubService) {
        this.eventHubService = eventHubService;
    }

    @PostMapping(path = "/eventhub/send")
    public ResponseEntity sendEvent(@RequestBody EventPayload payload) {
        try {
            log.info("Eventhub send endpoint called, sending {} to event hub..", payload.toString());
            eventHubService.sendEvent(payload);
        } catch (Exception e) {
            log.error("An error arose sending a message to event hub: " + e);
            return new ResponseEntity<Exception>(HttpStatus.INTERNAL_SERVER_ERROR);
        }
        return new ResponseEntity(HttpStatus.OK);
    }
}

I've created the POST method "/eventhub/send" and defined the request body to be our data class. From there it just sends to the service.

Once we have all of that hooked up we should be able to run our application and see it successfully connect to event hub.

2019-09-24 13:42:21.670  INFO 16113 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.24]
2019-09-24 13:42:21.734  INFO 16113 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2019-09-24 13:42:21.734  INFO 16113 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1063 ms
2019-09-24 13:42:21.983  INFO 16113 --- [pool-1-thread-1] c.m.a.eventhubs.impl.MessagingFactory    : messagingFactory[MF_41d4fd_1569350541964], hostName[dublin-rest-demo.servicebus.windows.net], info[starting reactor instance.]
2019-09-24 13:42:21.999  INFO 16113 --- [pool-1-thread-1] c.m.azure.eventhubs.impl.ReactorHandler  : name[MF_41d4fd_1569350541964] reactor.onReactorInit
2019-09-24 13:42:22.002  INFO 16113 --- [pool-1-thread-1] c.m.a.eventhubs.impl.ConnectionHandler   : onConnectionInit hostname[dublin-rest-demo.servicebus.windows.net], connectionId[MF_41d4fd_1569350541964]
2019-09-24 13:42:22.003  INFO 16113 --- [pool-1-thread-1] c.m.a.eventhubs.impl.ConnectionHandler   : onConnectionLocalOpen hostname[dublin-rest-demo.servicebus.windows.net:5671], connectionId[MF_41d4fd_1569350541964], errorCondition[null], errorDescription[null]
2019-09-24 13:42:22.101  INFO 16113 --- [pool-1-thread-1] c.m.a.eventhubs.impl.ConnectionHandler   : onConnectionBound hostname[dublin-rest-demo.servicebus.windows.net], connectionId[MF_41d4fd_1569350541964]
2019-09-24 13:42:23.157  INFO 16113 --- [pool-1-thread-4] c.m.a.eventhubs.impl.ConnectionHandler   : onConnectionRemoteOpen hostname[dublin-rest-demo.servicebus.windows.net:5671], connectionId[MF_41d4fd_1569350541964], remoteContainer[100db877ccad41b1a689c5a458bf1fbc_G6]
2019-09-24 13:42:23.268  INFO 16113 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2019-09-24 13:42:23.449  INFO 16113 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 2 endpoint(s) beneath base path '/actuator'
2019-09-24 13:42:23.491  INFO 16113 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2019-09-24 13:42:23.495  INFO 16113 --- [           main] com.dublin.eventhub.demo.Application     : Started Application in 3.126 seconds (JVM running for 3.493)

Now that we're all setup, let's send some data to it. Pull up your favorite HTTP client and post some data to it. For this example I'm using Insomnia.

And in the logs I should see that the message got sent to event hub successfully!

2019-09-24 13:43:25.806  INFO 16113 --- [nio-8080-exec-1] c.d.eventhub.demo.controller.Controller  : Eventhub send endpoint called, sending EventPayload(firstName=Johnny, lastName=Carson, email=null, favoriteFood=Potatoes and Molasses) to event hub..
2019-09-24 13:43:25.808  INFO 16113 --- [nio-8080-exec-1] c.d.eventhub.demo.controller.Controller  : Sending message to the event hub event-hub-test

We did it, we're serializing and sending data to event hub. 😁

Recap

In this guide I walked through creating an event hub in Azure, setting up an event hub client and service, along with an endpoint to post data to.
In part 2 I'll walk through consuming events.

I hope this guide helps, let me know what you think in the comments.

Three Ideas for Scoping Work Safely

Dublin Anondson — Sun, 25 Aug 2019 21:06:59 +0000

We've all been asked at some point by a team member or our manager, "Hey how long is this work going to take, can you give an estimate on it?".

Scoping is hard and not always as intuitive as one might think. Below are some ideas for scoping in a safe and practical way.

Fully understand the work AND the impact it could have.

When scoping, it's important to consider applications outside of the ones being modified. If you're updating the contract of a return payload for a service, does that break other services using it? Is there a data store needing to be modified to support this new value as well? Adding something to the contract might not seem so easy now.

Ask as many questions as you can up front to get a clear picture. Dig into the code, does it seem clean enough to get in there and make the change easily? Make sure you understand the ins and outs of the system being modified before putting a number on it.

Don't be Optimistic

I think Fred Brooks summed it up best in his book, The Mythical Man Month, All programmers are optimists. Perhaps this modern sorcery especially attracts those who believe in happy endings and fairy god-mothers.

Optimism can be dangerous, we might start out thinking that the work in front of us is a piece of cake, only to be humbled later by unforeseen consequences.

It's important to understand how to strike a balance between optimism and realism. As a rule, some amount of padding should be built into the scope. It's difficult to pin down how much extra will be needed needed, but depending on the complexity of the change I like to double whatever I initially came up with and then shave off some amount of time appropriate to the change.

It's better to over scope and delivery early than to under scope and deliver late.

Consider Process

Take for example, enterprise development. Things can be siloed and segmented to a point where even talking to someone on another team might require a meeting (I sincerely hope it's not like that for you, dear reader).

If the change is reliant on a modification to a data store, do you have to have someone make that change on your behalf? What does their schedule look like for being able to get that work done?

Time can get chipped away in places you might not expect. So it's worth considering if you know there's going to be some process to follow ahead of you.

In the end, scoping, no matter how you slice it, can be tricky. There's almost never a magic number and it's probably never going to be pinpoint accurate. But we can scope safely and save ourselves some headache by thinking around the problem and being pragmatic in our estimate.