DEV Community: Jamil Ahmed

Unlocking the Power of Alibaba’s Machine Translation Service with JMS, HTTP and MQTT

Jamil Ahmed — Fri, 20 Sep 2019 20:29:31 +0000

Alibaba Cloud offers a language translation service powered by machine learning. You can use this to easily present your e-commerce site in multiple languages, enable multi-lingual search and even translate support chats in real-time.

The service is currently made available as an SDK called “Machine Translation,” also known as Ali MT. Users are required to integrate the SDK into their application components that need to perform language translation. One approach would be implementing a client-server model with the SDK implemented in just the designated ‘server’ process(es) with the clients making translation requests to it. This offers some abstraction in your architecture by avoiding the need to have every process that wants to use the translation service be familiar with the full Machine Translation SDK.

For example, in a modern event-driven microservices architecture you can envision a set of elastically scaled microservices that offer the translation capability to all other microservices.

In many enterprises, JMS brokers are a very common way of connecting applications and services. As an easy to use messaging API, JMS is embedded in many integration products. If you have an enterprise service bus (ESB) in use today, a JMS provider is probably part of the stack.

Looking across your enterprise IT infrastructure, you may need to expose new cloud-based services like this to existing applications that are connected to an ESB, in addition to connecting them to new applications in a microservices architecture or functions-as-a-service (FaaS) deployment. You may also have some Internet of Things (IoT) devices that need to access them.

Wouldn’t it be great if you could deploy just one set of services and handle all these different client and protocol needs? With the help of an event broker like PubSub+, you can.

Starting with JMS for existing legacy applications

With JMS being an API specification only, and not tied to any particular product, it is possible to code an application against the API so that it works with any JMS-compliant JMS broker. The client libraries that contain vendor-specific detail can be resolved at runtime from the classpath. To eliminate the need for even this vendor specific library dependency in your application, the Advanced Message Queuing Protocol (AMQP) defines a standards-based connectivity protocol so an open source library can connect to brokers from multiple vendors without so much as a library change.

To support JMS connectivity to Alibaba MT, I have created a wrapper around the SDK and made it available here on Github. There are two demo JMS applications (as runnable JARs) in the repository that are implementing the request-reply pattern:

A “Requester” program that makes requests to translate English text to Chinese, sent over a JMS destination.
A “Replier” program that listens to requests on a JMS destination, translates the text to Chinese and responds to the original requestor over another JMS destination.

The Replier program is in the server role in the client-server pattern and is the only program needing to implement the Machine Translation SDK; the Requestor program is completely unaware of the SDK.

The repository by default builds with support for the AMQP JMS libraries from Apache Qpid, so it’s immediately ready to connect to a JMS broker such as Solace PubSub+ that supports AMQP.

To run the programs you need to update the contents of two files that are also in the checked out repository:

jndi.properties – This tells the programs how to connect to the JMS broker and what destination to use
alibaba-mt.properties – This tells the Replier program how to connect to the Alibaba Machine Translation service.

Get your Solace PubSub+ Connection Details

The easiest way to get started is to sign up for a free developer plan of PubSub+ Cloud. Once you have your service, jump down into the AMQP connectivity details from the “Connect” tab like so:

You can then modify the jndi.properties file to add the connection URI, username and password into the jndi.properties file like so in the “connectionfactory.ConnectionFactory” line:

connectionfactory.ConnectionFactory = amqp://your-service-here.messaging.solace.cloud:5672?jms.username=solace-cloud-client&jms.password=YourPasswordHere&jms.clientIDPrefix=NLPTranslationSample-&

For the purposes of this demo, also update the JMS destinations names like so:

topic.nlp-translation-requests-send = jms/nlp/translation/requests topic.nlp-translation-requests-receive = \*/nlp/translation/requests

Get your Alibaba Machine Translation Service Details

Next, we need to provide the details of your Machine Translation service to the programs. You can get these details by following the instructions here.

In alibaba-mt.properties enter your service details like so:

service-region=cn-hangzhou access-key-id=IDGoesHere access-key-secret=SecretGoesHere

If you don’t have access to the service yet, you can also run the programs in offline/simulation mode. Just add the following line to the file instead:

simulation-mode=true

Start Translating

Now you are ready to run the programs and access Machine Translation over JMS. Start the replier program first:

java -jar AlibabaNLPReplier.jar -j ./jndi.properties -a ./alibaba-mt.properties

Then start the requesting program:

java -jar AlibabaNLPRequestor.jar -j ./jndi.properties

The requesting program will collect keyboard input as you type and submit each new line as a request for translation.

Adding HTTP support for new microservices

For lightweight microservices running in a Functions-as-a-Service manner without a lot of library dependencies, communicating over HTTP is a popular mechanism. So let’s add HTTP support for the translation service next.

With the Java replier program running, we can actually have the same process respond to requests delivered over HTTP without deploying any further services or components. By selecting Solace PubSub+ as the JMS provider, its REST Microgateway provides access to the same JMS destinations over HTTP.

The Solace PubSub+ REST connectivity feature can operate in two modes: messaging and gateway. The default setting for a newly created event broker or cloud service is messaging but we need it operating in ‘gateway’ mode for this exercise.

Follow the instructions here to change the mode for your broker.

Get your REST connectivity details

Going back to the Connection tab of your PubSub+ Cloud instance, now collect the details on how to connect the same instance via HTTP REST:

Set up cURL command example

Assuming a Linux shell environment, we can use the curl client to make a translation request over the HTTP POST method. To facilitate this, we’ll put the relevant connectivity details into shell variables for convenience like so:

$ USER=solace-cloud-client $ PASS=your-password-here $ REST\_ENDPOINT="http://your-instance-here.messaging.solace.cloud:9000/nlp/translation/requests"

You’ll note the REST endpoint adds the nlp/translation/requests path as a way of matching up to the JMS program and the destination it is already listening to.

When the JMS programs receive requests and respond to them, they expect a CorrelationID to be present to properly respond to the requests. For the HTTP POST we can pass this in the header, so let’s setup a variable for this too:

$ CORRELATION="Solace-Correlation-ID: aRandomString"

Lastly, another header is required to specify that this is plain text being sent and not, say, binary. This final shell variable can set this up too:

$ TYPE="Content-Type: text/plain"

That’s all that is required. We are ready to run the curl command with the relevant arguments:

$ curl -u $USER:$PASS -d "This request is via HTTP POST" -H "$TYPE" -H "$CORRELATION" -X POST $REST\_ENDPOINT

Here is an example response with it all put together:

On the Replier logs you can see messages being handled from both the JMS and HTTP sources:

Adding MQTT support for IoT devices

MQTT is a low power and low bandwidth protocol which makes it a very popular choice for IoT systems. With these devices typically being supported by applications and services running elsewhere (such as on-premises or in a cloud) there is a need to get bidirectional data flows from these devices using MQTT to applications and services using different protocols. Fortunately, PubSub+ event brokers will easily do this protocol translation for you.

Keeping the same Java program running, let’s get a request for translation sent using the MQTT protocol so that the Java program can consume it over JMS.

Get your MQTT connectivity details

Just as before, go back to the Connection tab of your PubSub+ Cloud instance and this time collect the details on how to connect to the instance via MQTT:

Connect your MQTT Client

As MQTT is a standard connectivity protocol, any MQTT client API can publish to the PubSub+ event broker (i.e. There is no API required to be fetched from Solace.) Therefore, we can use the online hosted JavaScript-over-WebSocket MQTT utility by Eclipse Paho for this demonstration.

Copy in the connectivity details into the online client:

Publish a message from the web client:

Note: This request can be sent to the MQTT specific topic of mqtt/nlp/translation/requests

Review the request arrival in the JMS Replier logs:

You will recall that our JMS programs are utilizing a ‘request-reply’ pattern of message exchange, with a ‘reply-to’ topic being essential to see the translation response. As this hosted MQTT client is not carrying out all these necessary steps, the request will not be properly dealt with. However, evidence of the message arrival and processing of the content can be seen in the logs.

While the lightweight MQTT protocol does not explicitly define a request-reply mechanism, it can very easily be simulated using custom defined topics and headers. For further information on how you may build your own MQTT sender to properly implement the request-reply pattern to successfully get a translation response back to the MQTT sender, read the Request/Reply guide.

That’s all, folks!

That’s the end of the tutorial. If you have any questions or comments, you can find me at solace.community.

The post Unlocking the Power of Alibaba’s Machine Translation Service with JMS, HTTP and MQTT appeared first on Solace.

Multi-Cloud for the Digital Silk Road?

Jamil Ahmed — Sun, 05 May 2019 17:48:42 +0000

Map of the Digital Silk Road: London to Beijing Cloud Connectivity with Alibaba, Tencent and Huawei

For Western companies doing business in mainland China, digital connectivity can be a huge challenge when you’re having to contend with the “Great Firewall of China” restricting or slowing down cross-border Internet connectivity.

Could a multi-cloud solution help? I decided to investigate…

> What is the Digital Silk Road?

The “Silk Road” was an ancient network of trade routes that connected East and West, dating from around the year 114 BC. More recently however the Silk Road term has been rebooted to represent the massive $800 billion foreign investment programme by the Chinese government to strengthen international trade and connectivity.

Within that overall initiative, the Digital Silk Road is a $200 billion wide-ranging project focusing on the digital aspects such as new undersea Internet cables, advancements in Artificial Intelligence, quantum computing, and even alternative satellite navigation systems.

> The connectivity challenge

In spite of all this investment to foster and improve international trade, the ‘ Great Firewall of China ’ that affects all cross-border Internet connectivity remains firmly in place. This firewall enables the censorship and blocking of certain traffic or content, which has the net effect of severely slowing down all data flows in and out of mainland China.

(As a tourist to Beijing in 2017, I had to bear with my wife’s constant frustrations with her social media access being slow!)

A photo at one of the watch towers of the Great Wall of China (Mutianyu)

Now picture a hypothetical Western company wishing to do business in China. Say a European manufacturer of smart, Internet-connected cars that are now driving on the roads of busy Beijing. To support the digital services and applications in that vehicle, it needs healthy bidirectional connectivity to applications and services deployed in existing on-prem data centres. One hypothetical service is your vehicle being able to re-route your journey to avoid a known accident further ahead. Information such as the location of the accident and the real-time location of your vehicle will need to be processed without undue delay for this to be effective.

Vehicles with smart, internet connected services

Some of those digital services may be delivered from a regional ‘hub’ data centre close by in Asia, and others delivered from a data centre in Europe closer to company headquarters.

Having a digital presence within China — as a first stop before reaching the on-prem data centres — would be required to effectively deliver responsive services to those connected cars.

> Enter Alibaba, Tencent and Huawei Clouds

Just as Amazon AWS, Microsoft Azure and Google are the dominant players in the West for public cloud services, for China you can consider Alibaba, Tencent and Huawei for the same.

With each of them maintaining data centres (or “cloud regions”) in many international cities, the business of facilitating digital access for foreign companies into mainland China is a ‘low hanging fruit’ use-case.

Alibaba and Tencent are the largest players and make up a duopoly locked in a constant battle for overall dominance:

“Forget Google versus Facebook. Forget Uber versus Lyft. Forget Amazon versus … well, everybody. The technology world’s most bruising battle for supremacy is taking place in China.” — NY Times

With my focus here being connectivity from London to Beijing, Alibaba Cloud actually inches ahead a little over Tencent. Alibaba Cloud has 11 regions outside of China versus 10 from Tencent. More importantly, the nearest cloud region in Europe is also in London for Alibaba, versus Frankfurt for Tencent and Paris for Huawei.

Summary of international ‘cloud regions’ availability for Alibaba, Tencent and Huawei

(When Alibaba launched their London region in late 2018, I performed an analysis on how they compared with AWS, Azure and Google. You can find that here: Alibaba Cloud — A Data-Driven Analysis.)

> The Investigation Parameters

As part of looking at this issue I was interested in the following items:

1. What exactly does “slow” connectivity mean?

This will be quantified by mimicking a simple application interaction using a popular communication pattern (request-response), that is stretched between a pair of services deployed in our London ‘on-prem’ data centre and the Beijing ‘cloud’ data centre.

This will take place at a regular 5 second interval to build up a long term picture of how long the full ‘round-trip’ of those interactions take to complete. i.e. The latency of the connection at an application level.

Request-Response interaction facilitated by Solace PubSub+ Event Brokers deployed in London and Beijing

Furthermore, with the help of a multi-cloud “event mesh” architecture for this connectivity, the ‘responding’ service in Beijing can be deployed in triplicate across the three cloud providers — in each of their Beijing cloud ‘region’. This will have the effect of ‘racing’ the interaction across the three in parallel so we can also assess which offered the fastest connectivity at that time and determine the arrival ranking in that race.

Applications being deployed in triplicate across Alibaba, Tencent and Huawei clouds in a multi-cloud event-mesh.

2. Does an indirect connectivity via the cloud provider add value?

There are two different ways we can connect up our London application with the applications in Beijing:

The first is a direct connection from our London on-prem network to the Beijing cloud provider’s network.
The second is an indirect path by adding an extra interim hop to the cloud provider’s “Europe” region first, to then connect onwards to their Beijing region.

Direct vs Indirect Event Mesh connectivity architectures for London to Beijing

To use an airplane analogy, the options here are either a direct flight to Beijing or one with a change in Europe first.

Counter-intuitively, the extra hop may actually help to reduce the latency of the overall communication and provide a more predictable service. Instead of being treated as cross-border Internet traffic straight away, the hop may direct the traffic via a different, more optimised path across the border.

The architecture of the event mesh has this new dimension now. As well as providing competing service deployment across three cloud providers, we can also compete across the two connectivity paths for each provider.

3. Do increases in latency affect all 6 traffic paths to Beijing at the same time?

This item is the crux of the overall investigation. If dealing with the ‘Great Firewall’ is an unavoidable aspect of doing digital business with China, can a multi-cloud approach help to protect (or “hedge”) against it?

To put it another way, if one connectivity path and deployed service is suffering from a sudden slow down, would an alternate path and service be performing better at that same moment? If there is no correlation across the paths during those moments of slow down, enterprises that are particularly sensitive can choose to operate a “multi-path” approach to their connectivity to mainland China.

> The Live Results

The results of the continuous application ‘request-response’ interaction described above, along with the event-mesh architecture with multiple clouds and multiple paths, can be viewed in a live demo here:

http://london.solace.com/multi-cloud/digital-silk-road.html

Example results with the actual round-trip latency in milliseconds are available below:

An example result showing Tencent Cloud via Frankfurt as the lowest latency path to Beijing.

Another example result showing Alibaba Cloud via their London region as the lowest latency path to Beijing.

> The Findings

1. Even snails move faster than this!

This investigation and live demo builds on earlier work measuring AWS, Azure and Google with global communications spanning UK, US and Singapore. After deploying it against China here, the latency was at times so high it broke the charting logic of my demo! Embarrassingly, the high latency exposed a logic bug that was easily fixed, but a great example of how previously working applications can fail in strange ways when faced with such challenging connectivity circumstances.

The logic of the application was as follows:

Send a single request message that the PubSub+ Event Brokers replicate across the multiple paths and cloud providers
Wait up to 5 seconds for the multiple responses to arrive
Calculate the latency of each response and determine an overall ranking of the order in which they arrived
Repeat with a new request message

What I failed to anticipate is the response messages arriving more than 5 seconds late. From all the experience so far with the other global regions, latency rarely breaks the 1 second mark, let alone 5.

The fastest responses seen from China were around the 160 millisecond mark. (Less than a fifth of a second.) At the upper end however responses can be 100x higher in the 10’s of seconds like the 14 seconds example below:

Example latency result in the 10’s of seconds range.

We just had the London Marathon last weekend so will use that to help explain the logic bug. With the latency being so late past the 5 second measurement window, the effect was like a marathon runner starting at the weekend but arriving very late the next month to finish the race. At that point all the other participants have gone so when he arrives at the finish line he is the only one present. Then when put into an ordered rank of arrival, he appears to be “first” by virtue of no one else being there!

The simple fix can also borrow from the marathon analogy. Each race has a “pacer” response that will always be present. When determining the final ranking, you only get one assigned if the pacer is present at the time of your arrival.

2. Don’t take the direct flight

One result that looks unanimous is that the direct connection to Beijing from London ‘on-prem’ is by far the slowest.

In the latency chart below, the broken dashed line represents the indirect connectivity and the solid line being direct.

For both Alibaba and Tencent the best connectivity path is to add the hop via the nearest cloud region in Europe. Latency can be up to 36% lower with this approach.

Example result of the stark differences in latency for the direct vs indirect modes.

Note that for Huawei, this comparison of direct vs indirect could not be made. While the public website lists Paris as an available region for their “Elastic Cloud Server”, upon inquiry I was told this region is not fully available at the moment. So the demo retains the placeholder for Huawei connectivity via Paris should it become available.

Huawei Cloud’s missing “EU-Paris” region even though listed on their website.

3. It’s Yes to Multi-Cloud

The main finding here is that while there are some cases of the connection slowdown (latency spikes) aligning perfectly across multiple providers, for the most part they do not correlate. i.e. If one cloud provider experiences a spike, in that moment the others often appear to be fine.

An example when the high latency spike was seen across Huawei and Tencent at the same time:

Example result where two different providers (Alibaba and Huawei) experience a large latency spike at the same time.

You can review this recording of a 3 hour replay of latency results to decide for yourself:

Once again, you can view the live results and trigger your own replay window here: http://london.solace.com/multi-cloud/digital-silk-road.html

> Next Steps

These are the preliminary findings being shared and the investigation can continue further. Adding Huawei Cloud’s Paris region when it becomes available is one pending step.

Once that is in place, the three will be on a fairer footing to try additional services available from the cloud providers. Mainly, the ability to purchase dedicated and low latency bandwidth between the cloud regions to further improve performance and connectivity.

In the meantime, I hope you found this an interesting read. Please do share your thoughts in the comments section below.

Do follow me for more content like this.

If you liked this article, you might like this one too: Comparing Machine Learning through BBC News Analysis

Article image for ‘Comparing Machine Learning through BBC News Analysis’

Originally published at https://www.linkedin.com.

Stop taking standards for granted

Jamil Ahmed — Wed, 09 Jan 2019 00:57:33 +0000

What has an angry Michelin starred chef got to do with standards and multi-cloud? Let me explain…

Picture the scene: It’s that hazy period between Christmas and New Year’s where you know there are days in the calendar but you are not sure what is happening where or what day it currently is.

Do check out @instachaaz on Instagram

There’s loads of family over and we’re all at various stages of a food coma.

My brother-in-law suddenly surprises me with:

“Oh I read your last blog [on Alibaba Cloud], very interesting.”

Now this is a surprise because he is a Pharmacist by profession and does not really consider himself technically inclined. i.e. Not exactly my target audience for that particular article in the first place.

Turns out he has been suffering from insomnia of late and so reading articles on random topics is on his quest to find the sleep that eludes him. Now I could take offence here. I mean I don’t write to help put insomniacs to sleep! Alas, I am a glass half-full guy so the fact it engaged him enough to still remember the blog next morning, I am taking this as a win.

“This multi-cloud idea, isn’t that just standard?”

This was the question that was posed to me from this non-technical reader. I guess everyone can appreciate why being multi-cloud has advantages, whether you are in the field or not. I was just stumped at explaining why it is sadly not a given that everyone does this.

It is the word standard that I decided to focus on when trying to explain the situation. Not the adjective meaning of the word as posed in the question, but the noun:

standard[stan-derd]

noun:

“something considered by an authority or by general consent as a basis of comparison; an approved model.”

adjective:

“usual, common, or customary”

Enter the Angry Chef…

“Imagine a chef in a very busy Michelin starred restaurant”, I began…

Diners are plenty, waiters are taking orders and the kitchen is abuzz. There is suddenly something wrong though. The supply of gas to the premises has failed. The whole kitchen is stuck on pause. Our imagined Chef is now very angry at not being able to deliver for the hungry diners.

No matter how skilled the chef or the business are at what they do, a fundamentally crucial element to their function is missing and there is no way around it. The gas service is a utility that is taken for granted when there, but absolute catastrophe strikes when gone.

Fortunately the restaurant switches to another more reliable supplier and service is restored. They were able to do this so easily due to standards. Standards mean that whichever the supplier, the gas is the gas. It will operate and behave the same, regardless of who is supplying it. The industry as a whole can independently agree on what mixture of hydrocarbon gasses is acceptable to be coming through that pipe and kitchen appliances can be designed to operate with those expectations too. (Insomniacs, click here for more reading on gas standards!)

What if there were no standards?

Our angry chef and restaurant decide to ditch their current supplier and select another. Another company happily arrives to accept their business.

“Oh but… Our gas is different…”

It is of a different quality. So you’ll need to change your kitchen equipment.
It also arrives at a different pressure. So you’ll need to put new pipes in.
It also burns at a higher temperature. So you’ll need to adjust your recipes.

How angry is our hypothetical chef now?

Where are the standards in cloud computing?

I explained how the reason why multi-cloud is not a standard approach is because there are no widely accepted standards to help users.

There is currently a need for standards at many levels. From how resources in the cloud can be discovered and used, to how applications can be deployed and moved around across providers.

Sadly the space is not mature enough to have these be in place for the greater good of the users. Or perhaps not enough of us are demanding them.

At least I could offer up one consolation: All applications need to communicate with each other. The unrestricted movement of data is one of those fundamental utilities that is taken for granted when working but it’s catastrophe when it breaks.

An Event-Mesh Architecture powered by Solace PubSub+ allows applications to be built against an Open Standard that provides the protection to easily switch if needed.

It is of course just one small piece of a much bigger puzzle, but it’s a start.

With that, I hope I gave my brother-in-law enough peace of mind to finally get a good night’s sleep!

Alibaba Cloud — A Data-Driven Analysis

Jamil Ahmed — Wed, 05 Dec 2018 12:01:00 +0000

Alibaba Cloud — A Data-Driven Analysis

Multi-Cloud Arbitrage: Sample Response Tracker

First up, this post touches on Brexit politics, peculiarities of Internet traffic crossing the Atlantic, Mo Farrah running the 5000m race, and of course cloud computing.

So if your reading interest falls in the intersection of these sets, do carry on reading and make yourself known with a clap or comment.

> What’s all the hype with Alibaba?

I’m sure you’ve had Alibaba frequently popping up in your feed. It’s a hot topic, covered often in both business and technology news.

Alibaba is China’s online shopping giant that is often compared to Amazon. Just as Amazon has a cloud computing business with AWS, so does Alibaba.

While AWS is the more established player with the largest market share and estimated revenues of $17bn last year, it is the rapid growth rate of Alibaba that is actually making people take notice:

“In 2016 the cloud-computing business of the Chinese e-commerce behemoth grew by 126%, to $675m. Growth is unlikely to slow soon.” — The Economist

“Alibaba Cloud growing like gangbusters, but still far behind AWS and other market leaders.” — Techcrunch

So it’s all about where Alibaba is projected to be in a few years, not necessarily where they are today.

> Alibaba Cloud Launches London, UK Region

Last month Alibaba added to its global network of data centres with a UK “Region” to some fanfare:

Alibaba UK Region Launch

A London location launch means they are courting UK businesses in sectors such as public and financial services, where “data localisation” matters. It is also the kind of event that in the current political climate gets inevitably co-opted to mean something more:

“It will be seen as a pre-Brexit vote of confidence in the strength of the UK tech industry which has worried about levels of investment in the run up to leaving the European Union.” — ZDNet

Yay or Nay for Brexit?

I found it interesting that while the new region is outwardly called “UK (London)” like so:

Alibaba EU and UK Regions

The reality is that under the covers, you can tell that Alibaba saw the UK as an extension of their presence in the EU, with the “ID” of this new UK region being exposed in various places as EU-West-1 :

“UK” or is it “EU-West-1”?

With the only other region in the EU being Frankfurt, added 2 years ago, it is obvious that this new UK region is more likely the continuation of long laid plans and not a new investment decision taken following the referendum.

Now that’s enough of Brexit talk… Moving on…

> How can we fairly compare Alibaba Cloud with the big three? (AWS, Azure and Google)

Earlier this year I built a demo to describe a way of enabling Multi-Cloud Arbitrage for applications. The idea is to build and deploy applications on an event-mesh architecture layer that allows them to move easily between different cloud providers. Then based on various ‘arbitrage’ factors such as the cost of compute, actual running performance of your application, or even idle CPU capacity on existing provisioned machines, you dynamically move your application to take advantage of those differences across the providers in real-time.

Avoiding “cloud lock-in” and being multi-cloud ready is key to achieving this kind of arbitrage potential and maintaining freedom of choice. This growing desire by enterprises to achieve multi-cloud readiness can also explain IBM’s $34 Billion acquisition of Red Hat:

“The thing about IBM is, we’ve been around long enough to know this is a multi-cloud world.” (Ginni Rometty, IBM CEO) — Reuters

The live demo itself works by deploying:

Solace PubSub+ Brokers across three regions within AWS, Azure and Google, taking care of inter-application communication in an ‘any-cloud, any-region’ event-mesh architecture.
An identical Java application on a “burstable” virtual machine type in the same regions of each cloud.
Finally, a single Java application running “on prem” in a London data centre.

The on-prem application sends request messages that are delivered identically to all the applications running within the three clouds, for which they all process the request and respond back individually.

This setup allows the on-prem application to calculate how long each responding application took (i.e. the ‘latency’ in milliseconds) to process the request and send the response. In effect, determining a “holistic performance” of the responding application that can be compared with its peer in the same region, but running in a different cloud provider. A ranking can then be determined on which application (running in either AWS, Azure or Google) came back first.

Show me the money.

An example result with this multi-cloud and multi-region setup visually depicted can be found below:

Example Result of Multi-Cloud Arbitrage Demo — 13th Nov 2018 04:49:45 GMT

The ability to deploy the above applications to identical machine sizes, operating systems, and geographical locations in Alibaba Cloud to match what I have already in AWS, Azure and Google is the first test of maturity and ease of use in this comparison.

Secondly, I can see how the actual application results compare across the 4 cloud providers and whether there are any interesting findings there.

Just deploy it already!

Turns out that deploying the Java application and Solace PubSub+ Broker to Alibaba Cloud was fairly simple. The equivalent virtual machine types and regions selected across the four clouds are available to review below:

Equivalent Virtual Machine Size and Region Selections (AWS, Azure, Google and Alibaba)

> The Internet has a personality too…

The result for Alibaba in the UK was showing approximately 73 milliseconds for the response to arrive back to the Solace on-prem data centre. This latency number is very odd as it’s comparable to the latency of responses received from the US region in the other three clouds — so my initial thinking was that I must have deployed the “UK” virtual machine for Alibaba in the wrong place!

Unusually high latency result for the newly deployed Alibaba UK Region

To understand this result, it was time to break out the go-to tool for any network troubleshooting: traceroute.

(For those who are unfamiliar with it, traceroute allows you to see all the hops that make up the total route of your network traffic between your own IP address and the target IP address. )

London to London, the scenic route.

Results from the traceroute troubleshooting tool.

The traceroute results showed that our Internet Service Provider (ISP) was indeed sending the data to the United States first, for it to use an Internet Peering Exchange located in New York, to eventually get to the Alibaba network via the ISP: “China Mobile International Limited”.

What was even more puzzling is that a similar traceroute test to the same Alibaba London server using a different Internet connection from London did not show the same high latency.

All this shone a bright light into the interesting world of Internet traffic routing across different ISPs:

Explanation of result from UK ISP used by Solace

Put simply, as the Internet is really a network of networks, ‘peering agreements’ between these different networks controls how data may pass from one to another. Individual ISPs will have numerous agreements with other ISPs and large organisations with the aim of eventually reaching all parts of the Internet in an optimal manner.

Show me the way to Ali…baba.

When the ISP used by Solace in London is trying to figure out “How do I send data to this Alibaba IP address?”, the result is “China Mobile knows how to get there, and I am peered with China Mobile in New York, so will hand it over to them there.”

If you want to understand this space in more detail, a good write-up is here. If you want to learn more about just how peculiar Internet routing can be, read about the time a single ISP in Pakistan took out YouTube for most of the world here.

Keeping an eye on the mileage.

This experience does raise an important point though: If data locality matters to you, just selecting to use a cloud service in a given country is not the full picture. Unless you are an enterprise that is purchasing private network circuits from your own data centre to the cloud provider, double-check just how the traffic over the Internet is travelling for you and your end-users to those services.

A service you think is hosted in the UK may as well be in a different country as far as network connectivity and user experience is concerned…

> So how does Alibaba compare?

After working with Alibaba Cloud Support and our own ISP to resolve this issue of peering in London, we can finally look at results on a fair footing.

The best way to do this is through the three historical graphs (on the demo page) that are a rolling window on a per-region basis on what order the application responses arrived.

If you imagine a closely-run long-distance race, like the Men’s 5000 metres from the London 2012 Olympics, you’ll notice that even though Mo Farrah won Gold, his position throughout the race varied greatly:

Tracking Gold Medal Winner Mo Farrah’s position through the 5000m race in the 2012 Olympics

Those graphs can be thought of as the continuous position tracker of four runners, in a race that never ends.

Ladies and Gentlemen, place your bets.

In this contrived demonstration of applications being deployed across on-prem and multiple clouds, and only the fastest response being of interest, you can visually see that at present there is a lot of competition between the four players for responses from the UK and US.

In the UK, responses from AWS and Google are closely vying for the top position. Alibaba and Azure are then in close competition to determine the third and fourth place.

Application response arrival tracker — 4th December 2018, 16:35 GMT.

For responses from the applications in the US region, Google responses are predominantly in first place at this point in time. Then there is a lot of competition between AWS, Azure and Alibaba for the remaining positions.

In contrast the results for the Singapore region look very calm and serene when reviewed graphically like this. The positions in the race are very set and not showing a lot of movement. While Google Cloud was often the front runner for the earlier two regions, it is in firm fourth place here.

Alibaba Cloud responses are in third place, trailing behind AWS and Azure.

In summary, no single cloud provider is a clear winner when seen holistically across the three regions. It is also not a surprise that while the newcomer Alibaba Cloud is not taking any top positions yet in these particular results, the competition is indeed fierce for the second or third arrival position. — An apt metaphor of Alibaba’s real-world position perhaps.

> To conclude…

With cloud computing, this is a race with the players expected to change positions throughout as circumstances and capabilities change.

The only real course of action is to pick your favorite runner with the data you have available up to that point in time. Then as things inevitably change, just be ready to quickly pick your new favorite!

That is the essence of multi-cloud arbitrage.

Happy evaluating!

Comparing Machine Learning through BBC News Analysis

Jamil Ahmed — Wed, 21 Nov 2018 07:01:02 +0000

Comparing two Cloud-based Machine Learning capabilities: Image Analysis and Sentiment Analysis

Leveraging Machine Learning (or Artificial Intelligence) capabilities for your application is easier than it ever has been. The major cloud computing providers offer these capabilities in an easy to consume manner, so you can start building applications quickly.

However, did you know the actual results across the providers can vary greatly?

I decided to build a demo application that uses two Machine Learning capabilities to analyse BBC News articles and compare results across the big three cloud providers: Amazon’s AWS, Microsoft’s Azure and Google’s Cloud Platform.

You can try the application yourself here.

What is Machine Learning?

Machine Learning (ML) is a branch of Artificial Intelligence (AI), although often the terms are used interchangeably. It describes the ability of computers to “ learn” how to perform an action, without being explicitly told how to do it, as would have been the case for computers in the past. It is unlocking a whole range of new possibilities. I came across this description recently that summed up very well this step-change in computing:

“Computers have been able to read text and numbers for decades, but have only recently learned to see, hear and speak.”

For my demo application I focused on two ML capabilities: Image Analysis (a.k.a Computer Vision) and Sentiment Analysis (a.k.a Natural Language Processing).

What is Image Analysis?

Going back to the earlier quote, put simply, Image Analysis is the capability for computers to now “see”. More concretely, this ‘Computer Vision’ is how faces can be detected within photos. Or it is how a Self-Driving Car may continually process it’s surroundings to detect the pedestrian crossing the road ahead.

In my application I use this capability to produce ‘labels’ to annotate the central photo that is accompanying the BBC News article. In effect, describe the contents of the photo if you could not see it.

What is Sentiment Analysis?

Sentiment Analysis is simply categorising a piece of writing on whether it is positive, neutral or negative in its content. An example use would be an application automatically processing product feedback left by a user and flagging it for follow-up if it seems negative. A marketing department may also wish to monitor live tweets and have it’s team intervene if there are negative tweets affecting their brand.

In financial services, political and business news can move markets, so sentiment analysis of live news may be an input into automated trading engines that respond to world events in real-time.

In my application I use this capability to ‘read’ the headline and description of the BBC News article and provide it’s sentiment. In other words, is the news about a positive event, neutral or negative?

Observations so far…

Not All Machines Are Created Equal

As mentioned in the beginning, I was surprised to see the differences in results across the three providers. Below is an example result where it was not unanimous between them on what the article sentiment is:

An example of an inconsistent ML result across the three providers.

As humans, it is quite clear to us that this is a sad and distressing piece of news, yet Google’s Natural Language service deemed it positive in sentiment. In fact the score is 0.9 where 1.0 would represent the maximum for a positive sentiment. It is surprising why words such as “fresh fighting”, “growing violence” and “conflict” have not led it firmly into the negative sentiment territory. Azure’s Text Analytics did the best in this example.

The neutral sentiment determined by AWS’s Comprehend is also worth highlighting here. That service seems to often play it safe with a ‘on-the-fence’ assessment of ‘neutral’ being its dominant stance:

An example of AWS alone providing an outlier result.

On a more happier note, it does seem quite universal though that everyone loves the news of a new Royal Baby:

An example of a unanimously agreed result.

A Picture Paints How Many Words? Well I’ll Tell You…

As you’ve probably noticed, the previous three examples also show quite a difference in the Image Analysis capability too.

The trend appears to be AWS’s Rekognition yielding many more labels per image compared to Azure’s Computer Vision and Google’s Vision services. A quick scan also shows most of the labels to be relevant to the image too. In fact compared to its ‘on-the-fence’ approach to Sentiment Analysis, the Image Analysis capability is a lot more aggressive and offers up more ‘inferred’ labels that are likely to be associated to the image.

For example, the Royal Baby article has an image of the Royal Horse Artillery firing a 41-gun salute. While just the fired gun, smoke plume, and 3 service personnel are seen, it produced a whopping 27 labels for the image:

“Human, People, Person, Cannon, Weaponry, Smoke, Airport, Weapon,Airfield, Engine, Locomotive, Machine, Motor, Steam Engine, Train, Transportation, Vehicle, Aircraft, Airplane, Jet, Military, Army, Military Uniform, Soldier, Team, Troop, Warplane”.

Azure and Google seem to be more comparable with each other in that a lesser number of (more focused?) labels are produced. That is, if the service is not completely stumped and produces nothing, as Azure did here:

An example result of Azure being an outlier with zero labels produced.

To Conclude…

This was a very high-level summary of my findings so far. It is in no way a comprehensive review. The demo application is available for you to submit your article choices for analysis and better form your conclusions:

http://london.solace.com/cloud-analytics/machine-learning.html

As always, the correct selection of Machine Learning provider will come down to your own usage requirements. For instance, those requirements will determine if a verbose set of labels produced for an image is useful, or whether less is actually more.

One thing we can perhaps take from this is that Machine Learning has made great leaps forward, but we are a long way from intelligent robots taking all our jobs! The systems still need a little more stirring:

xkcd.com on Machine Learning

Please do let me know what you thought. I have a program on standby to perform sentiment analysis of the comments on this post and automatically reply with the relevant emoji.

Originally published at https://www.linkedin.com.