DEV Community: Luca Bianchi

AII4DEVS #10: Diverse knowledge is the key to grow the next generation of ML practitioners into AI engineers.

Luca Bianchi — Sun, 04 Jul 2021 12:38:46 +0000

Diverse knowledge is the key to grow the next generation of ML practitioners into AI engineers. Our weekly newsletter becomes bi-weekly, with more content, deep dives, and lands on Medium's most relevant publication.

The road so far…

When the AI4DEVS newsletter started, it was just an experiment following a gentle nudge from people asking to have a reference about what’s new on AI and machine learning. The fun part of the story is that all these requests came mainly from peers with no prior expertise in data science willing to get started somewhere.

The last few years saw an unprecedented evolution in the technical role of machine learning experts, growing to a mere wizard pulling complex spells out from their arcane books into an engineering profession, with workflows and best practices to bring any machine learning project to a successful production. The well-known Facebook anecdote of “less than 10% of our AI projects makes it to production” is not acceptable anymore if these competencies are becoming mainstream if we’re approaching the Fourth Industrial Revolution.

AI4DEVS was born out of an attempt to empower people with the knowledge they needed to start their journey. Week after week, a lot of feedback coming to the newsletter better shaped this feeling about what people need: a newsletter that is a mere collection of URLs is good enough if you have time, but it is not great enough to make people deep dive into the content. People, especially technical folks with no time to experiment on their own, need someone to shed light on these results, in the form of curated comments and hints of discussion, to build a strong opinion about what could and what couldn’t work them. This is why AI4DEVS took a few weeks off and comes back with a new format that will hopefully support people to leap AI.

A new beginning

The AI4DEVS newsletter is evolving with its tenth issue, becoming longer and shifting to a bi-weekly publication. Content will be curated by a team of data scientists, developers, and guest experts willing to support this initiative. Moreover, this initiative is thought from the community, for the community so that the obvious choice would be to Our final goal remains the same: to empower people adopting machine learning and AI in what they love, thus building the deep transformation we believe these technologies can bring to our lives.

Real-life use cases

One of my favorite approaches to machine learning is the practical “learn by doing.” Hence AWS offers a great use case with this post about **vessel time arrival prediction*. Maybe ship tracking is not your job, but this concrete use case offers a great example of handling tabular structured data. Moreover, the post showcases how to address a machine learning project, framing the problem, understanding the data, and engineering features. In this scenario, vessel position is computed as well as vessel efficiency. In the end, this post is a nice and comprehensive use case for *scikit-learn.

One of the most underrated capabilities of Amazon Rekognition is Custom Labels, which enable image classification customization on specific datasets. A great use case for this managed service is Ostervall which uses this amazing technology to classify claims to insurance companies from driver images. Amazon Rekognition is used to classify the type of damage and identify where the damage is within an image.

On the Natural Language Processing side, a great application comes from Daniel Wellington’s service department implementation of Amazon Translate to reduce translation costs for their customer care services, which led to providing answers to customer issues in their home language. An application developed in less than two weeks reduced operational costs by 90% and improved team response time.

Frameworks and libraries

To all folks in love with Rust programming language, **linfa** is a promising library to check out: a complete porting of the well known scikit-learn library, which enables common preprocessing tasks and classical ML algorithms such as clustering, linear learners, logistic regression, and decision trees as well as support vector machines and Bayesian algorithms such as Naive Bayes. We all know that Python has the 98% of the machine learning languages market share, but if I looked to something else, a super-fast Rust implementation would be my first stop.

Engineering Machine Learning

In a recent panel, Ryan Keenan joined together Andrew Ng, Robert Crowe, Laurence Moroney, Chip Huyen, and Rajat Monga to discuss how MLOps shapes the future of machine learning production engineering.

Speaking of MLOps, Julien Simon, technical evangelist at AWS, presents Amazon SageMaker comprehensive set of applications within their platform to enable project management. It’s pretty scary to understand how many tools can be used to address every aspect, from data cleaning to feature engineering, model training, and bias evaluation.

Where to go from here?

AI4DEVS is coming back in two weeks with a new issue full of machine learning use cases, MLOps stuff, and outlines from the world of AI.

If you are interested in joining the team, send me a DM, and we’ll start a discussion.

AI4DEVS: Issue #7

Luca Bianchi — Thu, 29 Apr 2021 21:56:30 +0000

It's SageMaker month!

Last issue presented AI in medicine, NLP, and serverless adoption AI. This week we start unwrapping what NVidia GTC, space, and food have in common, while we enter the SageMaker month!

Here are our top picks of the week!

Just at the beginning of its annual conference, NVidia CEO Jensen Huang, announced the availability of its massive parameters NLP model. Since the cool thing in language processing nowadays are transformer models, this trillion parameters model could have no other name than Megatron.

This pandemic situation made clear we need more empathy and emotions to be shared. The Washington Post tries to fill this gap joining poetry and generated Art using GANs

Cooking can be art on its own, but mixing different flavors is not something for every taste. Now FlavorGraph uses AI to predict how two ingredients will pair togheter

This application of computer vision classification, to catalog Distant Galaxies from Penn University, is a cool massive leap into a better understanding of our not so close neighbors.

Upcoming events and announcements

It's the SageMaker month

Starting from April 20th for one month, AWS is going to publish content, announcements, and use cases about Amazon SageMaker. Be sure to check out the updated recap every day or come back here for selected top picks!

AWS Machine Learning Summit Registration now open

This long-awaited event is back in a fully virtual fashion, on June 2nd in the Americas and June 3rd in Europe and Asia with a day full of top-level speakers. Be sure to reserve your spot!

And this is all for this week. Stay tuned and subscribe to be always up to date with the latest news in AI and machine learning applications.

Thank you for reading, see you next week!

About the AI4DEVS

AI4DEVS is a weekly selection of curated AI news selected from the internet by a community of professionals to help you add a bit of AI in your next project or better understand its applications.

This newsletter would haven’t been possible without ML specialists’ joint efforts and AWS Heroes Gillian Armstrong, Pavlos Mitsoulis Ntompos, and Alex Schultz.

A special mention to Cameron Peron and Alex Casalboni for their steady support and incentive to raise the bar for the community, pushing more people to embrace Artificial Intelligence.

Thank you for reading, see you next week!

Cheers,

Luca

%%[ga]

AI4DEVS: Issue #6

Luca Bianchi — Thu, 15 Apr 2021 21:05:05 +0000

Serverless and AI, a match made in Heaven

Last week talked about winning your customer base, detecting objects and diseases using AI. This week we explore how to improve medical diagnosis management, introduce some fundamentals of NLP, and what does means serverless to AI.

Here are our top picks of the week!

Health and Medical AI applications are becoming the new normal in diagnosis, but this is just half of the story. A compelling AI system should be able to infer from previous reports, thus detecting the usage of medical terms. Ram Vegiraju explains how AWS Comprehend Medical can be used for Medical Entity Detection using

Comprehend is one of the powerful tools we can leverage to implement Natural Language Processing or NLP for short. to deep dive into these practices, Ketan Doshi does a gentle introduction to NLP

I am a huge fan of serverless, and Jeremy Daly does a great job interviewing Alexandra Abbas in this new Serverless Chats episode exploring how AI can match it to raise the bar of your next project

Finally, an advanced example of machine learning model handling, through Sagemaker Autopilot. Model hyperparameters fine-tuning is not an easy task. Still, this use case is explanatory of the potential and wide range of applications.

Upcoming events

We're just wrapping up a week full of AI with nVidia GTC conference. We'll talk about all of them in a future episode, but there is a lot of juice coming from this conference.
If you wish to approach Machine Learning on AWS, I could suggest attending this workshop on getting started with Amazon Sagemaker on April 23rd.

And this is all for this week. Stay tuned and subscribe to be always up to date with the latest news in AI and machine learning applications.

Thank you for reading, see you next week!

About the AI4DEVS

AI4DEVS is a weekly selection of curated AI news selected from the internet by a community of professionals to help you add a bit of AI in your next project or better understand its applications.

This newsletter would haven’t been possible without ML specialists’ joint efforts and AWS Heroes Gillian Armstrong, Pavlos Mitsoulis Ntompos, and Alex Schultz.

A special mention to Cameron Peron and Alex Casalboni for their steady support and incentive to raise the bar for the community, pushing more people to embrace Artificial Intelligence.

Thank you for reading, see you next week!

Cheers,

Luca

AI4DEVS: Issue #5

Luca Bianchi — Thu, 08 Apr 2021 23:15:01 +0000

Harnessing the power of customers with AI

Last week presented some interesting advancements in the NLP domain, which is living a renaissance. This week we talk about winning your customer base, detecting objects and diseases using AI.

Here are our top picks of the week!

Building a strong customer base means predicting and preventing customer churn, then maximizing your customer lifetime value. Recent applications in the retail space are shifting from basic product recommendation systems based on previously bought items. Amazon Personalize addresses a lot of use cases with its existing managed service. Still, this is a great article to start building your customized recommender system.

Looking for a detailed description about what is Customer Lifetime Value and how you could detect this score for your customer base, this is a great place to land.

One year ago, Facebook Research released a top-level, state-of-the-art machine learning model for object detection, named Detectron2. Today this model is available as a pre-built notebook on Amazon SageMaker. Just run the notebook, and you can use it within minutes.

Computer vision is empowering every market, and even agriculture can deploy a crop disease detection model with Amazon Rekognition Custom Labels.

Your next doctor will probably be a machine learning model on the research landscape that now can read your laboratory data and detect your disease better than an average medical practitioner. Healthcare is an amazing field for data science, and we're just scratching the surface of what will be possible in the next few years.

Upcoming events

Many of you asked for upcoming events and speaking opportunities in the machine learning world. This week two of the worldwide top events have been announced!
nVidia GTC conference is coming online next week (April 12-15th), and it is definitely the place to be whether you want to deep dive into deep learning stuff and have a glimpse of the near future. Never mind, you are not a skilled data scientist, be sure to attend at least the main keynotes: it's a great value for your time.
A bit ahead of time, AWS announced the upcoming AWS Machine Learning Summit which is happening on June 2nd. This is a huge one-day event joining together researchers, visionaries, and state-of-the-art applications. Stay tuned for more info when registration opens.

And this is all for this week. Stay tuned and subscribe to be always up to date with the latest news in AI and machine learning applications.

Thank you for reading, see you next week!

About the AI4DEVS

AI4DEVS is a weekly selection of curated AI news selected from the internet by a community of professionals to help you add a bit of AI in your next project or better understand its applications.

This newsletter would haven’t been possible without ML specialists’ joint efforts and AWS Heroes Gillian Armstrong, Pavlos Mitsoulis Ntompos, and Alex Schultz.

A special mention to Cameron Peron and Alex Casalboni for their steady support and incentive to raise the bar for the community, pushing more people to embrace Artificial Intelligence.

Thank you for reading, see you next week!

Cheers,

Luca

%%[ga]

AI4DEVS: Issue #1

Luca Bianchi — Fri, 12 Mar 2021 18:42:51 +0000

The evolution of artificial intelligence from magic art to engineering practice

In the last few years, machine learning has become mainstream and has been widely adopted by data scientists and a growing number of developers, architects, and decision-makers. This new age of AI makes tools and models more accessible to everyone without data science skills, but understanding these techniques will soon be the core foundation of any software project.
When practitioners shift from a wizard pulling magic spells from an arcane book to an engineering professional, a change of mind is required to discover a new world of opportunities and many use cases.

AI4DEVS is a weekly selection of curated AI news selected from the internet by a community of professionals to help you add a bit of AI in your next project or better understand its applications.

Here are our top picks of the week!

Natural Language Processing (NLP) and Natural Language Understanding (NLU) have always been some of the most promising fields of AI, with many applications, even making a contact center able to diagnose COVID-19 using managed services.

AI is becoming a tool to be used in everyday applications. Nothing more than a showcase about adding text analytics capabilities into an existing workflow makes the point.

Finally, if you want to be amazed by what AI can do today, check out this video about OpenAI playing hide and seek in a simulated environment. Impressive!

About the Authors

This newsletter would haven’t been possible without ML specialists’ joint efforts and AWS Heroes Gillian Armstrong, Pavlos Mitsoulis Ntompos, and Alex Schultz.

Thank you for reading, see you next week!

Cheers,

Luca

AWS Heroes reactions to Swami Sivasubramanian keynote on Machine Learning

Luca Bianchi — Wed, 09 Dec 2020 06:11:13 +0000

This year re:Invent has been quite far from usual: firstly, the conference is entirely online. Secondly, it is lasting three weeks in a row, with a bonus update scheduled for mid-January. Bonus point, we just had a dedicated Keynote entirely focused on machine learning delivered by Swami Sivasubramanian, VP Amazon AI.

Since talking about machine learning is super cool, and AWS is announcing a lot of new features and services, Swami is not the only keynote with AI stuff in it: Andy Jassy delivered a couple of mind-blowing news last Tuesday, and Dr. Matt Wood joined Swami to further deep dive into presented topics.

Being an AWS Hero, with interests in machine learning and serverless, makes my job wonderful these days.
Here a few reactions of fellow heroes:

We had wonderful recaps from Gillian Armstrong, Zamira Jaupaj and Or Hiltch about all the announcements.

SageMaker announcements

A lot of features have been announced related to SageMaker. As Mike Chambers pointed out, it appears clear that AWS is trying to onboard data scientists with no prior cloud expertise and provide machine learning capabilities to developers without any data science knowledge.

One step towards this ambitious goal has been the announcement last week of SageMaker Data Wrangler, SageMaker Feature Store, and **SageMaker Pipelines.

This week, Swami presented another interesting feature, focused on model bias detection and explainability: Amazon SageMaker Clarify, which has been received with enthusiasm:

"It helps you to detect biases in your dataset. I think it's fantastic; it's just surfacing that this problem exists. A lot of people don't even realize this a problem at all. It's pretty amazing!"
Jeremy Edberg

This feature could improve a lot the application of machine learning to many fields:

"I think it's a big step forward getting people thinking about ethics and AI and responsible AI"
Gillian Armstrong

Another common issue when dealing with machine learning models is related to profiling and debugging. Here comes Amazon Deep Profiling for SageMaker Debugger, which offers the capability to drill down into how a model performs with his focus centered on visibility.

" Visibility is key. Machine learning, especially when you are starting, is a kind of magic black box. I'd like things providing this sense of how things work"
Mike Chambers

Another enabling technology is easing things out for data scientists when dealing with deployments (or the so-called MLOps). Sagemaker Pipelines

_"AWS is surfacing services that are elsewhere and make them available within SageMaker."
Mike Chambers

In this direction moves also Amazon SageMaker Edge Manager that aims to simplify deployments to edge devices from the cloud.

"Hopefully now you can actually manage huge fleet of edge (deep learning) devices"
Guy Ernest

Bringing ML to databases

In an effort to remove friction to developers using ML where data is, today AWS announced the support of its machine learning bridge for two additional databases: Amazon Redshift ML and Amazon Neptune ML.

New services

A re:Invent keynote couldn't be over without the announcement of completely new services, and there they came: Amazon Lookout for Metrics complements AWS solution stack in the anomaly detection domain, adding a flexible service for time series analysis which is probably at the core of Amazon Lookout for Equipment, Amazon Lookout for Vision, and Amazon Monitron. Lookout for metrics exposes the capability to track multi-dimensional metrics and how they change through time.

"We've building anomaly detection systems for decades, but this makes things easier because you essentially need just to have an anomaly and figure out to detect it, but it was a lot of back and forth. But now it's just magic."
Jeremy Edberg

" Applying ML to real-world use cases such as gluing a Lookout sensor to a manufacturing machine or connecting a Panorama appliance to a network of existing cameras is the right way to get ML to be useful for more people and companies"
Guy Ernest

Final thoughts

AWS Heroes are amazed by the direction AWS is leading, pushing data scientists to the cloud and developers without prior expertise to the complex domain of artificial intelligence.

"What we have seen so far in reInvent 2020, and what we see again in Swami's keynote, is AWS focusing on getting real business and social value out of ML and AI, not just jumping on the latest shiny thing. We see this with practical end-to-end hardware powered solutions like Monitron and Panorama, through to high level enabling announcements like Amazon HealthLake."
Mike Chambers

A complete round-up of first impressions is also available in this panel held by Mike Chambers right after keynote end. Definitely to check out!

This year machine learning keynote saw the rise of Amazon SageMaker Studio as a complete IDE for data scientists and the deployment of vertical solutions addressing industrial domains such as healthcare and industrial anomaly detection. It means AI and machine learning are becoming mainstream components in any real-world application, not just a promising research field. In this scenario, serverless or managed services offer a straightforward and clear advantage compared to on-premise deployment, either for model training, operation, and no model at all, which represent the real competitor to players such as AWS.

Cover photo by Gabriel Bassino on Unsplash

Clean or dirty HVAC? Using Amazon SageMaker and Amazon Rekognition Custom Labels to automate detection

Luca Bianchi — Tue, 17 Nov 2020 16:36:28 +0000

ML classification: Amazon Rekognition or Amazon SageMaker?

Computer vision problems have been tackled using neural networks in recent years, obtaining unprecedented results and continuously raising the bar of accuracy to near-human performances.
In this article, the focus is set on image classification in an uncommon context related to a Neosperience customer that provided the opportunity to compare two different approaches: Amazon Rekognition Custom Labels and Amazon SageMaker custom model.
Both approaches have advantages and could find their spot in a given context, but Amazon Rekognition Custom Labels offer an interesting tradeoff between time-to-market and cost.

Alisea

Machine Learning applications are steadily shifting from research domains to industry, opening a wide range of applications from simple object detection to people tracking in dangerous environments.
In this scenario, brands decide to innovate their target market, introducing smart products with features made possible by modern machine learning applications.
Alisea has led the Heating, Ventilation, and Air Conditioning (HVAC) systems sanitization market for almost two decades with over 3000 customers in Italy and abroad. Back in 2005, Alisea was born with a single mission: to offer the market the best HVAC hygienic management service with state-of-the-art innovations, without compromise.

Alisea customers range from corporate offices to malls, supermarkets, and hotels. In recent years, in the US, private houses also adopted HVAC technologies to control air temperature due to better efficiency than standard air conditioning systems.
Every HVAC plant requires periodic inspection, usually once a year, to provide a minimum hygiene level and reduce risks people.
Unfortunately, in crowded environments, this cannot provide good enough guarantees about risk biological management: hazardous agents such as bacterias, dust, and viruses tend to reform faster than expected.
To ensure the safest environment for people, air quality and plant status need to be checked frequently to avoid the spread of respiratory diseases, but it is an expensive procedure.

Introducing Remotair

A few years ago, Alisea decided to innovate its market, start developing a new smart product, detect in near real-time duct status, and trigger alarms when air quality and cleanliness fall below a risk threshold.
The system comprises a hardware board with pressure, humidity, CO2 sensors, and two cameras. Snapshots of duct status are taken periodically and should be processed by a machine learning model to understand whether or not the plant is dangerous to people. This feature has been called Visual Clean

Implementation with a ResNet on Amazon SageMaker

The first implementation of the Visual Clean machine learning model consisted of a ResNet50 neural network, trained using PyTorch and FastAI library with transfer learning and loss function adjustments. Also, a considerable effort was required through the phases of data preparation.
The training phase required a couple of hours and a hundred epochs to reach a final accuracy score of 0.90 ~ 0.92.

The result has been a deep learning model, served through Amazon SageMaker, and invoked as an HTTP endpoint.
It required having data scientists available to handle the task, prepare the dataset, build the network, and fine-tune the training phase to reach that score.

Amazon Rekognition Custom Labels

As soon as AWS released Rekognition Custom Labels, we decided to compare the results to our Visual Clean implementation to the one produced by Rekognition.

Upload images

The first step to create a dataset is to upload the images to S3 or directly to Amazon Rekognition.

Recently, the capability to upload images into the console has been added. However, reproducibility is fundamental in an industrial project, so we decided to upload data into an S3 bucket, divided into a folder for each label to predict, in our case, dirty and clean. Massive data uploads can be achieved through the command line with

aws s3 sync <source_path> s3://<destination_bucket>

Once images have been uploaded, access policy must be set on the bucket to ensure Amazon Rekognition will access data. This can be done by adding to the bucket policies (in bucket properties within S3):

Create a dataset

Once images have been uploaded, a dataset can be created, providing a manifest file describing the available type of data.
The manifest consists of a set of JSON lines added to the same file (note: it is not a JSON file itself, but a plain text file containing JSON fragments). Each line has the structure:

{
    "source-ref": "s3://dataset/images/Slot123_clean.png",
    "visual-clean-dataset-cleanliness": 1,
    "visual-clean-dataset-cleanliness-metadata": {
        "confidence": 1,
        "job-name": "labeling-job/visual-clean-dataset",
        "class-name": "clean",
        "human-annotated": "yes",
        "creation-date": "2020-06-26T17:46:39.176",
        "type": "groundtruth/image-classification"
    }
}

To make things easy, images can be labeled through Amazon SageMaker Ground Truth or with a pre-defined folder structure (one for each label).

The overall result is shown as follows:

Model training and evaluation

Once a dataset has been built, just hitting the "Train Model" button triggers a training job, asking how to obtain a testing set. The simplest way is to let Rekognition split the existing dataset in 80% training and 20% testing.

After that, the training job starts and required a couple of hours to complete

Once the model has been trained, it is reported into the "Projects" section of Amazon Rekognition Custom Labels with its evaluation.

Discovering that the Rekognition Custom Label model reached the same accuracy score of 0.92 was mind-blowing. The overall training time stating that 1.066 hrs were used a clear insight that Rekognition tries many different models and parameters simultaneously to find the best one.

Once the model has been trained, it can be invoked directly through API, just passing an image to its endpoint:

aws rekognition detect-custom-labels \
  --project-version-arn "arn:aws:rekognition:eu-west-1:XXXXXXXXXX:project/alisea-visual-clean/version/alisea-visual-clean.2020-10-26T23.00.38/1603749XXXXX0" \
  --image '{"S3Object": {"Bucket": "dataset.alisea","Name": "clean/Slot_620.JPG"}}' \
  --region eu-west-1

Since Amazon Rekognition Custom Label has an hourly price for the model, it can be stopped and started whenever required to reduce costs when no inference is required or to pack data processing efficiently.

Conclusions

Amazon Rekognition offers a viable solution to machine learning model development every time a custom classification model (either binary and multi-class) is required. It doesn't require a dedicated data scientist and can provide the same outcome. The pricing model is hourly based on a fixed cost of $1 for each training hour and $4 for each inference hour. Better profile cost can be achieved by optimizing the model active hours, thus reducing the working window.

Building a Neural Network on Amazon SageMaker with PyTorch Lightning

Luca Bianchi — Mon, 13 Jul 2020 23:17:40 +0000

Leverage the power of Amazon SageMaker and PyTorch Lightning to build ML models avoiding to manage boilerplate code and infrastructure.

Continue reading on Towards Data Science »

Neosperience Cloud empowering product development on AWS

Luca Bianchi — Mon, 06 Jul 2020 15:16:00 +0000

Neosperience aims to become the foundation of partners' product strategy and the backbone of our services. Photo by Mirko Blicke from Unsplash

Some of the companies joining us in the last few months brought non only strong domain expertise and technical competencies in their field (i.e., MIkamai, LinkMe) but also widely adopted products, like Workup RubinRed Digital Commerce platform. Many to come in the next months as soon as we find top performers in their fields, willing to scale up and join our family. On a product strategy basis, it has not been easy to think which evolution model was the best to provide the best value for our customers.

On one side, we could adopt a centralized model with firm product feature decisions coming from a steering committee, including domain experts from our subsidiaries, at the risk of losing some particular point of view that gets flooded into an enormous backlog. On the other side, a decentralized model could empower teams taking ownership, but integration issues could become an issue.

The ruling or serving dilemma

Company growth often means acquiring new teams that join your dream. The most challenging part of this process is to set up the relation between the leading company and its subsidiaries. Here at Neosperience, we developed through the years a sound technological structure that empowers several innovative solutions and their evolution through years while remaining with a modern approach and a strong knowledge of our customer experience domain. Every acquisition we made has been on an industrial basis, forecasting which companies we would like to join us bringing their domain knowledge to our family. Domain knowledge is an incredible value we would like to build into our products. So it comes to how we could make these reality express their full potential into Neosperience products.

We decided to make Neosperience Cloud the backbone of our products while letting them be great at what they already are with no external constraints. Our microservices architecture and wide adoption of technological standards make us able to provide powerful tools to our teammates. They could make a product more robust and interoperable.

Neosperience as the backbone of our products

Neosperience Cloud is the foundation of solutions such as Unbreakable Community, Image Memorability and Nudging, and Gamification. We plan to make this happen for every product within our family. In the next months, we are going to expose Neosperience Cloud architectural services and design principles to let other products integrate with us, obtaining a set of ready-to-use AI services while getting seamless integration between every product.

Neosperience Cloud becomes the foundation of Neosperience products, standardizing principles, libraries, and tools to ensure interoperability between our solutions.

In 2020, we plan to let Neosperience become the backbone of our products, developing support among four distinct paths: guidelines, events, architecture, and tools. An ambitious outcome achieved through four pillars: guidelines, events, architecture, and tools.

1. Guidelines

Neosperience Cloud is the expression of many best practices in software engineering carved through the last decade of product development. Such expertise is a value we have to share with our peers. Sometimes these guidelines should be suggested even vigorously because they seem time-consuming at first sight but become valuable in the long run. We propose the adoption of the following modern software development principles:

a shared unified Design Library , to provide our customers with a consistent experience through our solutions. Interaction principles adopt easy to use metaphors, and a consolidated design of UI components makes our user feel comfortable with any of our product flattening the learning curve.
An architecture made of microservices , to encourage integrations either between Neosperience solutions and with 3rd party ecosystem. Microservices allow adopting service composition into new products and projects. Breaking monolithic services into smaller components means increased flexibility and interoperation.
The use as much as possible of managed computing resources as containers or, even better, serverless is suggested . It avoids the need to manage infrastructure scalability and focus on providing the most value to customers instead of handling servers. This requirement matches a strong focus on cloud providers, especially Amazon Web Services (AWS), which offers a set of quick out-of-the-box services to build modern apps.
A shared layer for authentication is required , leveraging OAuth2 to authenticate users and a JWT token with a standard structure for authorization. The usage of Amazon Cognito is encouraged since it eases the need to write a lot of boilerplate code.
The support of events emitted and consumed by every entity within a product architecture to allow other solutions (either Neosperience or 3rd party) is encouraged. Services can react to them, getting notified of data changes without the need for direct communication. Events represent the preferred way of communication and provide many advantages such as standardization, validation, and fast webhooks development. AWS EventBridge is the reference technology for event handling within Neosperience. Still, in some specific contexts, Apache Kafka, Amazon SNS, or MQTT could be used to support event transport when AWS is not a viable choice.
An API First approach that makes every product expose a set of APIs. We ask our teams to start designing a product or a feature from the business and move downward to its interfaces. If the domain is unclear to every stakeholder, they could use mapping techniques such as event storming. Defining APIs means stating what happens between our product and the external world. It also has the benefit of easing a lot of work with integrations because no one has to build them on top, specifically for clients. A robust API strategy also empowers the separation of concerns and pushes companies to avoid monolithic architectures.

A summary of the six pillars within the guidelines domain

These guidelines can be adopted incrementally, having a minimum impact on product roadmap but enable a lot of the benefits we would like to leverage within Neosperience products and 3rd party partners.

2. Interfaces

Integration between different software products means building a mechanism to exchange data between them. Interfaces play a fundamental role because they guarantee a standard communication pattern to transfer data. In Neosperience Cloud, we plan to have two kinds of interfaces: REST APIs and Events.

Application Programming Interfaces (API)

Web APIs allow clients to invoke HTTP URLs and obtain data in a pull fashion. REST emerged in the last few years as a standard for domain entities representation using simple URL paths and HTTP methods (GET, POST, DELETE, PUT). Upon this standard, software developers defined some protocols to overload simple web links into full-fledged APIs. One of the most adopted protocol is OAuth2 for data access control. Neosperience Cloud exposes its API with a well-documented structure, utilizing a standard fashion to describe API. Every API within Neosperience Core and Business Services should expose a regularly maintained Swagger file, describing API structure and how to invoke any of them: params, expected data in and out as well as authorization support and error codes expected when performing operations.

An example of a Swagger definition of an API

Beside APIs, there is the need to document and also standardize the structure handled by services. Since the de-facto syntax of data follows JSON as a representation protocol, we adopted JSON-schema as its validation language. JSON-like syntax and a defined set of properties describe semantically the structure of a payload of data expected or returned by a service within Neosperience Cloud.

An example of JSON-schema to enforce payload structure validation.

Events

Web and mobile applications usually interact with a backend service, often via an API. Many frontend applications pass requests for processing, wait for a result, and then display this to the user. A synchronous architecture is only one way to handle messages, but modern applications have alternatives to provide a better user experience.

In recent years, reactive software architectures obtained a lot of hype due to their strong reliability and the efficient usage of server resources. One of the main cornerstones of reactive designs is to pull API invocations, in favor of message-passing solutions,

Neosperience Cloud has an event-based architecture implementing communication between different services. Each domain entity publishes events in response to state changes. Events are small JSON payloads that resemble a flexible structure.

Example for “product added to cart” event

Neosperience Cloud exposes events for several actions on domain objects: e-commerce, chatbot, the voice of the customer, notification, user-insight, customer, content, store, profile.

Publishing and subscribing to events allows communication between different services and data synchronization with an eventual consistency pattern. Event-based architectures are preferable to direct remote calls through RPC because they offer many advantages in scalability and fault tolerance as well as allow for interesting patterns such as event sourcing and fan-in / fan-out.

APIs and events represent the interfaces between Neosperience Cloud and Solutions

3. Services

Neosperience Cloud becomes the foundation of any SaaS strategy, offering several ready available infrastructure services that enable cross-cutting behavior between different products. Thanks to the adoption of the Infrastructure-as-Code (IaC) pattern, we can leverage AWS capabilities in cloud-native application deployments. The ubiquitous adoption of serverless technologies makes these services readily available, scalable, and cost-effective independently from their usage. We were able to build some infrastructure services that empowered Neosperience Solutions in the last decade and evolved throughout all the technological advancements of cloud computing. Today we’re opening up these services to our partners and companies, allowing a fast time-to-market, interoperability with Neosperience existing business services such as Image Memorability, Nudging/Gamification, and Unbreakable Community.

Specifically, we are going to make our core services available such as:

nsp-identity : a service developed specifically to handle service authorization independently from the entity being authorized. You could be a human, a backend service, or a toaster and can call every service with identity, roles, and permissions. Identities implement all OAuth2 fashion to offer access control to any REST service through HTTP Authorization header tokens. nsp-identity also provides service helpers to get tokens from social accounts, AWS credentials, username and passwords, third party providers, and machine-to-machine. Tokens can also be exchanged between all of them. In 2018, we added support for Auth0 credential providers, thus enabling 40+ additional providers. It relies on Amazon Cognito Identity management to allow data segregation but adds a standard layer that supports OAuth2 header management, roles, and token standardization through JSON Web Token (JWT).
nsp-user: user credentials management is the foundation of many Neosperience services, which is provided as a standard layer on Amazon Cognito User Pools, thus making its internal workflow entirely transparent to services, even when they do not rely on AWS. Moreover, login/registration flow is implemented as well with frontend components wrapping all the required service calls.
nsp-organization: Neosperience built a B2B2C platform, which means data should be segregated within different instances of any service, and a given identity must belong to only one of them. At the same time, a user could join multiple instances. This result is achieved through the development of the Neosperience Organization, which supports multi-tenant data handling and offers the capability to create a new organization, subscribe users, add licenses to them, and org-specific metadata (such as company-wide preferences and configurations) which can be retrieved by specific services. Neosperience Organization makes SaaS scalability independent for its underlying AWS account, thus allowing for multi orgs within the same account which relies on better management of cloud resources.
Customer Data Platform (CDP): is a data platform capable of storing and querying user data even in unstructured form. It supports an event-based collector and management, and data can be stored raw or aggregated in events within a user profile. Users can be queried by their attributes or timelines, thus offering and aggregated time-based evolution of a customer base. CDP support sources such as Salesforce, Shopify, SAP, and many more, as well as Neosperience Solutions, through a number of connectors. Integrations can be leveraged through service APIs, custom events (either published or subscribed by CDP), and Zapier.
Neosperience Cloud offers a service registry to provide relevant information about deployed microservices. It stores service health and status information about Neosperience as well as third-party services, through a health update and health check event bus and Amazon CloudWatch log analyzer which extracts metrics and usage insights. A Service Health Dashboard is offered to publish a summarized overview about service quality for a given account or organization.
An integrated Developer Portal offers third party teams the capability to obtain their Neosperience Cloud AccessKeyId and SecretAccessKey as well as to access service documentation.

On the deployment aspect, we offer to every team support through Neosperience CI/CD architecture, built on Gitlab and AWS CloudFormation with AWS Cloud Development Kit and Serverless Framework.

4. Tools

Neosperience aims to become the starting point for product development leveraging serverless technologies and forecast this important duty through an offering than includes not only ready to use services, but also tools to speed up teams building their next product.

Neosperience provides its partners support in adopting the aforementioned guidelines, with a set of libraries , offering services middleware, and project bootstrap code to ease starting building stuff. On the frontend, our complete design library offers support to UX/UI development with a consistent look and feels as well as interaction principles. Language-specific SDK s are the starting point for teams to integrate our services without the hassle of handling APIs and authentication. We plan to support React, NodeJS, iOS, and Android. Finally, we envision the release of a Command-Line Input (CLI) tool to be used by teams to scaffold a new project (either client or service) with middleware, libraries, and our design library with no need to dive into dependency management or library download and built-in support for service deployment within a Neosperience Cloud dedicated instance.

Where to go from here?

In 2020 Neosperience plans to take a huge leap towards partner ecosystem support while maintaining our strong focus on our solutions making our technology more accessible and inclusive to diverse teams. We value the personal point of view everyone could provide to the ambitious goal of bringing empathy in technology and we can’t wait to see what amazing things our partners are going to build together.

My name is Luca Bianchi. I am Chief Technology Officer at Neosperience and, the author of Serverless Design Patterns and Best Practices. I have built software architectures for production workload at scale on AWS for nearly a decade.

Neosperience Cloud is the one-stop SaaS solution for brands aiming to bring Empathy in Technology, leveraging innovation in machine learning to provide support for 1:1 customer experiences.

You can contact me via Twitter and LinkedIn.

A practical application of Amazon Leadership Principles

Luca Bianchi — Tue, 30 Jun 2020 07:06:01 +0000

Photo by Ian Schneider from Unsplash.

Since we started scaling up Neosperience from a single company to a group, while onboarding people with different stories and experiences, making everyone feel comfortable has been one of our top priorities.

We know this posed several challenges we had to consider to reach the ambitious goal of having everyone “feel like being at home” within Neosperience.

We know that the three keys driving people’s motivation are autonomy, mastery, and purpose. Such principles are good at keeping a single person engaged with a company or a team. Unfortunately, they do not provide sufficient enough guidelines about the strategies to help employees participate in a shared vision. Moreover, they do not explain an inclusive and supportive behavior between different teams. Furthermore, onboarding teams with different cultures and history but excellent at providing value to customers means being able to match strategies that effectively works and the principles behind them.

Be inspired by great stories, make them real.

Stories make people live exciting lives. We have been fortunate enough to work in unprecedented times when companies in just a few years grew up from a small startup to international enterprise companies. So we looked to them and tried to understand what principles they follow daily and the outcome of such tenets in company growth, team satisfaction, and customer value.

We decided to deep dive into Amazon Leadership Principles to understand how to apply them in practical situations and how far this takes us. At first naive reading, they could sound abstract and not concrete, but such distilled knowledge needs to be meditated and elaborated before becoming actionable. Leadership Principles are not prescriptions nor legal articles, and instead, they can be intended as guidelines to strive for to inspire and be inspired in everything every employee does at Amazon. Customer Obsession means, for example, to avoid developing features just for technical pleasure or “why we can” principles, but validate early your ideas with customers. At the same time, Learn and Be Curious is an input to invest in R&D and explore new paths, avoiding to fall on the status quo.

One of our primary goals is to develop products that raise the bar of performance of our customers. We applied these principles through our product strategy, contextualizing them to product development to shed light on what is in our future and how every employee at Neosperience can help to make our dream of a more empathic world become a reality:

Customer Obsession : our solutions have and must have a direct connection to customers’ needs to provide them the best value. We should engineer our product’s features from the feedback provided by customers. One great idea is to start from the end of the process, with an internal press release about new features or products, and understand how it works moving backward. It is a powerful tool in product development, called “working backwards.” Every team within Neosperience has to understand what is happening in a continuously evolving market. One customer request does not make a feature, but it’s an idea that deserves to be explored to understand the impact it can produce on a similar business. Monitoring competition could be useful, but it is not our prime focus because we’re here to fill a need and serve customers, not do better than others.
Ownership : we’ve been lucky enough to onboard teams with strong domain expertise. They trusted our vision of empathic technology. Now it’s our turn to support them in acquiring ownership in their application domain within the group and make the difference. We prefer technologies such as Infrastructure as Code that put in the hands of builders all the tools they need to own a solution. Ownership is balanced by the availability of shared tools that ease repetitive tasks and ensure interoperability.
Invent and Simplify : remove the complexity. As Calvino explains much in his Six Memos for the Next Millennium book, “lightness” is what remains when we remove everything not necessary. It means the effectiveness, exceptional value we want to deliver. We have to engineer our releases to make things easier for our customers. It starts with the integration of our products to provide a seamless experience to customers within our products portfolio.
Are Right, A Lot : we value instinct and ideas, especially when they come from a different history or culture. They enrich us all, forcing us to change our point of view. We look forward to onboarding people and companies with keen judgment and knowledge.
Learn and Be Curious : an excellent complement to avoid doing things in the same inefficient way is being curious about new ideas, markets, and technologies. Our growth opens up the road to sharing ideas and having the possibility to learn new skills both within and outside Neosperience. This passionate work has a direct impact on the quality and innovation level of our products. Passion and curiosity drive the best of us into exploring new paths, technologies, and ideas, then share them with the team. We have launched the Neosperience Academy webinar series (open to everyone) to encourage this sharing within our organization and the community.
Hire and Develop the Best : every company and team who join our forces makes us a step closer to our goal of making the difference. Every release of our products must be better than the previous one. A new version of our products could improve stability or reliability, but it must also raise the bar of performance for our customers.
Insist on the Highest Standards : every software has bugs and can be improved, but our products must consistently progress the quality curve. We do not accept poor quality releases as well as temporary solutions or “workarounds.” We ask everyone to focus on what makes the product excellent. Teams should continuously ask themselves, “what is best for the customer?”
Think Big : our products must solve problems for good. Customers expect us not to be the cheapest alternative to market leaders but to work with and integrate their products to provide a differentiating value. Our products are not an all-or-nothing solution, but we design them to work at their best when integrated into an ecosystem. It means providing connectors, APIs, and interoperability by design.
Bias for Action : we prefer writing code in favor of complex analysis because it is only building things that you can figure out the strengths and weaknesses of an idea. Moreover, requirements tend to be vague, especially for product features, when you work on cutting edge technologies in an uncertain market. People are free to make mistakes, but we, as managers and architects, have to reduce the impact of failure when things go wrong.
Frugality : there are no extra points for wasting people’s time in endless meetings. Time is a constrained resource for everyone and must be dedicated as much as possible to our customers instead of discussing resource allocation or administrative tasks. Efficiency is always a bonus point every time it does not compromise quality. Frugality does not mean to be cheap or stingy, but to make the best use of the resource you have. They could be a team of 2, 10, or a hundred people and a multi-millionaire budget or just a few thousand dollars; it does not matter regarding frugality. The outcome will be different, but efficiency still applies to the bill. Time is the most precious resource because it’s scarce.
Earn Trust : prefer listening rather than speaking. Respect every point of view. Integrate into our products the best ideas even if they are not your own. Confront and discuss with teammates and colleagues. Be supportive and, if not possible, explain the big picture to earn people’s Trust.
Dive Deep : products win and fail through data. Measure as much as possible a product strategy or implementation. Provide data and discuss interpretation. Have a deep understanding of the technology you’re using, explore and anticipate your customers’ questions, to have answers, and frankly admit when you don’t.
Have Backbone; Disagree and Commit : one of the best aspects of being part of a growing group of excellent professionals is you have plenty of confrontation opportunities. Every domain owner must listen to challenges to their ideas because it is only discussing the status quo we can improve. When an owner makes a decision, however, everyone must commit to that, stop debating, regretting, and work hard to realize it. Owners should have checkpoints, named retrospective in agile, to consider whether a decision was right or not, sharing the knowledge with the team but carrying the responsibility of everyone’s commitment to that choice. Everyone accountable for a decision must also be in charge of making it.
Deliver Results : bad things happen, especially when building compelling and innovative products that strive to challenge the status quo and evolve customers providing them unprecedented value. Setbacks occur in the form of requirement pitfalls or immature technology adoption in a production release or wrong choices when storing a massive amount of data on purpose-built storage or database, then having to migrate it. Failure means knowledge, and this is a value we have to share with teammates and the community. We are not upset when a product or a solution misses its goal, nor we do not persist in a wrong path. Teams know and understand very well many factors can impact the success of a product, and that some of them are outside of their control, and accept it. Nonetheless, they are always engaged in finding the way through obstacles to deliver outstanding solutions at market speed.

The road ahead

These principles are a strong foundation of our management policies and should be the reason behind every choice we make. Companies are not the perfect expression of the theoretical tenets, and Neosperience makes no exception: our teams are encouraged to strive for them. At the same time, we support and coach them when this does not happen. Leadership principles should be our main guideline in Neosperience for product development and be enforced through our decisions. We’ll share more details about how we architected our products following these principles in the next months.

Stay tuned and build great stuff!

Neosperience Cloud is the one-stop SaaS solution for brands aiming to bring Empathy in Technology, leveraging innovation in machine learning to provide support for 1:1 customer experiences.

You can contact me via Twitter and LinkedIn.

Migrating Image Memorability service to AWS Lambda using a shared Elastic File System volume

Luca Bianchi — Wed, 17 Jun 2020 14:31:47 +0000

Image analysis ML models can be run smoothly from a serverless function, democratizing access to AI. Photo by Alisa Anton on Unsplash

On June, 16th Amazon Web Services released the long-awaited feature of AWS Elastic File System (EFS) support for their AWS Lambda service. It is a huge leap forward in serverless computing, enabling a whole set of new use cases, and could have a massive impact on our AI infrastructure, thus reducing machine learning inference costs down to pennies for a wide number of applications.

The most exciting feature of Elastic File System is its capability to be mounted both on EC2 virtual machines, Fargate containers, and AWS Lambda. This feature is not anything new on the EFS domain. It has been primarily used by many application sharing stored data, helping customers evolve their applications towards stateless services: a couple of EC2 or containers could save vast amounts of data on an EFS volume and share them with producers and consumers. It avoids the complexity (and latencies) of storing objects on S3, then downloading them every time they are needed. Moreover, EFS throughput can be configured up to 1024MB/s speed, thus making it work seamlessly as a local file system.

Since AWS SageMaker involves EC2 to run Jupyter Notebooks, mounting shared storage between different instances is quite easy to set up and can avoid a lot of pain when managing huge deep learning models with millions of parameters and size of hundreds of megabytes.

The case for Image Memorability

Image analysis, classification, object detection, and picture rating through machine learning were applications unimaginable only a few years ago. Now they are commoditized services offered by any cloud provider, such as Amazon Rekognition, through an API interface that can easily be invoked with no need to understand the complexities of how a neural network works

Nevertheless, some specific image analysis tasks are far from being solved by general standardized services and require a context-specific approach, which means building a customized neural network to process images and extract meaningful insights. One of the most exciting services within Neosperience Cloud is related to image scoring on metrics that can be useful to marketers to decide whether or not it could be used in a campaign. The wrong image, or merely a picture not making the viewer focus on the message, can make your company waste a lot of money, lowering the return on investment to a poor outcome. Focus groups have been used since forever with the constraint of testing a campaign on a tiny number of potential users. Standing on the shoulders of giants such as Netflix, techniques such as A/B testing tried to do their best to find the most suitable image for the vast majority of users.

Following our experience in the last decade of digital marketing, we understood that all these methods are flawed because they do not consider the emotional impact of an image, which is deeply connected to its success. To fill this gap for our customers and provide a set of tools that support empathic analysis of content, we started building our Neosperience Image Services leveraging serverless technologies offered by AWS. One of the most interesting is our Image Memorability scoring system. Starting from a preliminary work published by MIT in 2015, we moved forward building a memorability dataset, but with a focus on images usually involved in marketing campaigns and product showcases to increase the model accuracy. After that, we used Amazon SageMaker to train an ensemble of neural networks in detecting a memorability score for a given image and extract the features of the picture contributing to that score.

Service architecture before EFS support

The overall service architecture is outlined in the image below. A simplified version of this architecture is discussed in an episode of This is My Architecture (in Italian)

Image Memorability services architecture

In this article, we focus on the part of the architecture related to when a trained model is uploaded from SageMaker to Amazon S3, then served either to on-premise customer servers or SageMaker endpoints, to support a variety of use cases:

Introducing AWS EFS allow reducing the part of an architecture related to model storing a management

Upgrading inference to serverless

Thanks to EFS support for AWS Lambda, it is now possible to remove the need for Amazon Sagemaker endpoints and shift the load to a Lambda function, triggered by an image saved to Amazon S3. It is a huge leap forward for many reasons:

removing the Amazon EC2 instances from the architecture resulted in lower costs by one order of magnitude
AWS Lambda scalability is more granular than Amazon EC2 Autoscaling groups and requires less time to scale up/down, with an impact on service response time
using S3 triggers to Lambda function, we can remove the need to poll an SQS queue containing processing jobs
since there are no more EC2 instances to manage in cloud, there is no need for an instance pool DynamoDB Table anymore (used to coordinate different workers on the same data)

The final architecture is much more simplified, using fewer AWS resources to accomplish the same task

The final architecture after adopting EFS as a shared volume between on-premise servers, Amazon SageMaker training instances and Lambda processing images for inference

About performances and costs

The usage pattern for Image Memorability often occurs when a marketer uploads a bunch of pictures from her laptop, preparing the next campaign. With an average of 15–20 images for each upload, our service incurs in peaks related to different users uploading sets of different photos during the same timeframe (i.e., working hours). For each batch, a marketer identifies some of the most impactful images, then computes memorability heatmaps upon them to ensure the remembered details are what she wants.

To serve our customers with a responsive platform, we dedicated an inference worker to everyone. Such choice is necessary to avoid spinning up an EC2 instance every time a batch is loaded, resulting in poor user experience. The drawback is in an EC2 autoscaling group dedicated to a given customer, with a minimum number of one p2.xlarge instance per customer within their cluster.

Adopting AWS Lambda with EFS support, we were able to shift all the load related to image scoring (which is the most common operation done by customers) to serverless while maintaining a shared cluster of p2.xlarge workers to compute heatmaps when required.

Such improvement made our score computing time grow from 5 seconds to 10/12 seconds per image. Still, it allowed us to lower the overall batch execution time to less than 20 seconds, thanks to lambda parallelization: a noticeable improvement compared to a batch processing time of more than 90 seconds achieved by the previous service version.

On the cost side, we were able to remove the dedicated p2.xlarge autoscaling group, which is used only for heatmaps, in favor of a shared group: this results in a considerable saving on the overall customer base and better resource optimization.

Where to go from here?

AWS Lambda support for Elastic File System is a game-changer improvement in serverless computing, enabling to overcome Lambda storage size limit of 512MB and S3 transfers bottlenecks to the function runtime. ML models are often quite heavy (especially if they work on images), good advice is always to load the model into VM memory. It can be done initializing the Neural Network object in your deep learning framework, and make this alive within Lambda cache for the life of the container. Such a simple trick could speed up further inferences, even more, avoiding the wait for 100+MB transfer from the file system to the memory.

At this time, we migrated only Image Memorability services related to image scoring. However, Memorability Heatmaps still need GPUs to backpropagate score through the network and extract features, which unfortunately is not yet available on AWS Lambda instances.

Image Memorability was migrated to this architecture in just a few days because configuring a new mount point for EFS is an easy task if your functions are already within a Virtual Private Cloud (VPC). Computing cost reduction by one order of magnitude means we’re able now to consider some improvements to our service. Different prediction models could be used for the same image, sending back to users different scores at the same time together with their performances to let them choose which suits best to their use case.

Neosperience Image Memorability service is free to try and more information is available on our product page to help marketing teams maximize their campaign ROI.

To get started with AWS Lambda and EFS support please refer to this great blog post by Danilo Poccia, while details can be found in the documentation. Starting with AWS EFS and Lambda is quite easy while shifting to modern architectures to leverage serverless and microservices requires support by an experienced team. Feel free to reach out to our consultancy team at Mikamai to get help building your next cloud-native solution on AWS.

Neosperience Cloud is the one-stop SaaS solution for brands aiming to bring Empathy in Technology, leveraging innovation in machine learning to provide support for 1:1 customer experiences.

You can contact me via Twitter and LinkedIn.

Architetture a microservizi, tra hype e casi d’uso — parte 1

Luca Bianchi — Tue, 16 Jun 2020 07:01:01 +0000

Architetture a microservizi, tra hype e casi d’uso — parte 1

Majestic Reflections of Shaik Zayed Grand Mosque. One of the world's most astonishing example of Islamic architecture. Photo by Photo by Farhan Khan.

Il mondo IT non è esente dal fascino delle “mode del momento”: soluzioni che magari sono state adottate da aziende considerate punti di riferimento e che assumono nell’immaginario collettivo il ruolo di “no brainer” diventando il nuovo status quo, almeno per alcuni mesi, in attesa che la nuova “moda” prenda il posto della precedente. Il più delle volte si tratta di soluzioni che hanno più di un razionale forte nella loro adozione da parte delle aziende che le hanno proposte per prime. Si tratta, più in generale, di metodologie, architetture o tecnologie che vale la pena prendere in considerazione, ma è necessario avere ben chiari i problemi che si desidera risolvere ed il rapporto costi/benefici offerto da queste tecnologie.

Nessuna scelta tecnologica è esente da “costi”: compito di un buon architect è valutarne l’impatto in relazione ai benefici che si desidera ottenere.

Perché sono interessanti le architetture microservizi?

In questo scenario è opportuno valutare adeguatamente l’impatto delle cosiddette architetture a microservizi. Vediamo cosa caratterizza le scelte tecnologiche alla base di questa metodologia per la definizione di architetture. In particolare un fattore fondamentale di queste soluzioni è legato alla scalabilità intrinseca a queste architetture, soprattutto in presenza di pattern di scalabilità differenti per le diverse componenti del nostro sistema. Consideriamo, ad esempio, un portale e-commerce. Sappiamo ormai da quasi un decennio che il primo passo per garantire una buona scalabilità è la separazione quantomeno della componente client dalla parte backend dell’architettura. Questa scelta che oggi appare intuitiva, non lo era affatto solo pochi anni fa: numerose tecnologie affermatesi nel tempo, incentrate sulla semplicità ed il rapido time-to-market hanno invece proposto un approccio diametralmente opposto. Si tratta di framework come Django, Rails e soluzioni analoghe che solo recentemente hanno abbracciato quantomeno un’architettura multi-tenant, separando il client (ormai sempre più una Single Page App o “SPA”) dalle componenti relative al backend, richiedendo lo sviluppo di endpoint. In questo scenario si è affermato sempre più lo standard di sviluppo di backend che espongano verso i client, endpoint REST. Non si tratta ancora di architetture a microservizi, ma di una prima suddivisione di componenti con requisiti e molteplicità profondamente diversi: grazie all’avvento del mobile, oggi si considera sempre più spesso lo sviluppo di differenti client, magari costruiti con la medesima tecnologia, ma rivolti ad utenze diverse. Il sistema backend, in questo contesto diventa un oggetto software che ha il compito di esporre degli endpoint verso i client attraverso cui inviare o ricevere informazioni utili al soddisfacimento dei nostri requisiti. Tornando all’esempio del sto e-commerce, il backend offrirà ai client quantomeno endpoint relativi al catalogo, al carrello ed alla procedura di pagamento dei prodotti acquistati.

Progettare la scalabilità di un sistema richiede un’attenta analisi della variazione nel tempo delle differenti componenti dell’architettura

Ovviamente dal punto di vista della scalabilità questi insiemi di endpoint avranno requisiti molto diversi tra loro: il catalogo subirà un traffico legato ad eventuali campagne promozionali, mentre il carrello (oppure un’eventuale wish list) avranno traffico necessariamente minore. In molti casi si sperimenta non solo un differente volume di chiamate, ma anche pattern tra loro alquanto diversi, soggetti alle iniziative del business che hanno lo scopo di servire: ad esempio offerte “flash” da Black Friday generano traffico molto diverso da normali periodi di esercizio. Supportare differenti pattern di accesso con una medesima architettura impone di trovare un minimum comune denominatore tra le varie esigenze. Le architetture a microservizi consentono di separare i vari endpoint, in base alle necessità imposte dai requisiti business per meglio soddisfarle. Si accetta quindi un maggiore costo “gestionale” giungendo a differenti configurazioni di deploy, così da consentire una gestione più efficiente del sistema con un migliore profilo di costi. L’impatto di queste scelte risulta ancora più evidente quando si considera la persistenza dei dati relative alla soluzione realizzata.

A ciascun servizio la sua persistenza

La separazione delle varie componenti di un’architettura in microservizi offre la possibilità di valutare per ciascuna di esse il miglior database disponibile in base alla struttura dati. Si tratta di un cambiamento quasi epocale nella gestione dei dati, poiché per quasi tre decadi l’unico approccio disponibile, considerato da molti un vero e proprio “silver bullet”, è stato l’utilizzo di modelli di dati strutturati, che venivano poi calati su database relazionali (spesso impropriamente detti SQL per via del linguaggio di query utilizzato da questi DBMS). Prodotti di mercato del calibro di Oracle, IBM DB2 e Microsoft SQL Server, affiancati negli anni da soluzioni open source come MySQL e PostgreSQL, hanno suggerito come unica soluzione alla gestione della persistenza dei dati la riduzione del modello nella cosiddetta “forma normale” così da minimizzarne la ridondanza informativa. Questi database provengono da un’epoca in cui lo storage aveva costi decisamente maggiori della potenza computazionale che, nella maggior parte dei casi applicativi, si trovava ad essere largamente sotto-utilizzata. L’avvento del cloud e la comparsa dei primi servizi di computazione on-demand, unitamente alla disponibilità di memorie a basso costo, ha ribaltato questa prospettiva.

Non si tratta però di un solo fattore economico: negli ultimi vent’anni l’adozione in modo pervasivo di tecnologie mobile e la digitalizzazione di interi comparti industriali hanno portato alla nascita di casi d’uso che mettevano in crisi il modello unico relazionale su temi di scalabilità e consistenza di dati spesso acceduti e modificati da varie parti del globo. In particolare la pubblicazione nel 2000 del teorema CAP ha dimostrato come un sistema software sia sempre il risultato di una serie di compromessi che ne determinano l’architettura e l’impatto che tali scelte hanno sulla persistenza dei dati rischia di essere non trascurabile. In particolare negli ultimi anni, si è affermato il modello dei purpose-built database, contrapposti alla scelta di un solo DBMS per tutto il nostro applicativo. In questo scenario diventa puerile la diatriba SQL vs NoSQL che ha appassionato architect e developer per quasi un decennio: il database migliore dipende dai requisiti di un sistema che deve soddisfare un determinato caso d’uso e, nel caso di architetture a microservizi, non esiste una risposta univoca. In particolare queste architetture offrono la possibilità di scegliere il miglior database per un determinato insieme di endpoint, incoraggiando quindi una pluralità di modelli dati all’interno della stessa soluzione. All’interno del perimetro di ciascun servizio, è possibile valutare le esigenze degli specifici schemi di accesso ai dati ed in base alle esigenze della logica di business, scegliere la persistenza più adatta.

Cicli di vita e rilasci incrementali

Un beneficio interessante spesso sottovalutato delle architetture a microservizi è la capacità di parallelismo del ciclo di vita delle varie componenti: ciascun team è in buona parte indipendente dagli altri e quindi può procedere con iterazioni di sviluppo più brevi e minori rischi di regression poichè la base di codice da manutenere risulta più piccola. Questo significa che i componenti del team non devono necessariamente conoscere il dettaglio di tutti gli aspetti del sistema, mentre un maggiore valore è dato al ruolo dell’architect che svolge una funzione importante della condivisione del know-how necessario alla realizzazione dei macro-requisiti di business.

Questo aspetto evidenzia anche alcuni punti di attenzione che le architetture a microservizi presentano: la separation of concern è garantita by design da questa metodologia richiede ovviamente un’attenzione particolare alla gestione della comunicazione tra i vari microservizi.

Nella seconda parte..

Nella seconda parte di questo articolo vedremo quali criticità pongono le architetture a microservizi e come possono essere superate mediante l’adozione di tecniche quali API Façade e comunicazione event-based.

Informazioni sull’autore

Sviluppatore e Cloud Architect da oltre un decennio, Luca Bianchi segue in qualità di CTO, lo sviluppo della tecnologia e dei prodotti Neosperience, Technology partner AWS, con particolare focus su serverless e machine learning.

Dal 2015 è co-organizer del Serverless Italy Meetup e dal 2016 collabora all’organizzazione del capitolo italiano di ServerlessDays. Autore di numerose pubblicazioni, talk e video è da sempre appassionato di tecnologia, con particolare attenzione allo sviluppo di applicativi serverless, ai modelli di machine learning e al mondo IoT.

Potete contattare Luca su Twitter, LinkedIn e Medium

DEV Community: Luca Bianchi

AII4DEVS #10: Diverse knowledge is the key to grow the next generation of ML practitioners into AI engineers.

The road so far…

A new beginning

Trending topics

Real-life use cases

Frameworks and libraries

Engineering Machine Learning

Where to go from here?

AI4DEVS: Issue #7

It's SageMaker month!

Here are our top picks of the week!

Upcoming events and announcements

It's the SageMaker month

AWS Machine Learning Summit Registration now open

About the AI4DEVS

AI4DEVS: Issue #6

Serverless and AI, a match made in Heaven

Here are our top picks of the week!

Upcoming events

About the AI4DEVS

AI4DEVS: Issue #5

Harnessing the power of customers with AI

Here are our top picks of the week!

Upcoming events

About the AI4DEVS

AI4DEVS: Issue #1

The evolution of artificial intelligence from magic art to engineering practice

Here are our top picks of the week!

About the Authors

AWS Heroes reactions to Swami Sivasubramanian keynote on Machine Learning

SageMaker announcements

Bringing ML to databases

New services

Final thoughts

Clean or dirty HVAC? Using Amazon SageMaker and Amazon Rekognition Custom Labels to automate detection

ML classification: Amazon Rekognition or Amazon SageMaker?

Alisea

Introducing Remotair

Implementation with a ResNet on Amazon SageMaker

Amazon Rekognition Custom Labels

Upload images

Create a dataset

Model training and evaluation

Conclusions

Building a Neural Network on Amazon SageMaker with PyTorch Lightning

Neosperience Cloud empowering product development on AWS

The ruling or serving dilemma

Neosperience as the backbone of our products

1. Guidelines

2. Interfaces

3. Services

4. Tools

Where to go from here?

A practical application of Amazon Leadership Principles

Be inspired by great stories, make them real.

The road ahead

Migrating Image Memorability service to AWS Lambda using a shared Elastic File System volume

The case for Image Memorability

Service architecture before EFS support

Upgrading inference to serverless

About performances and costs

Where to go from here?

Architetture a microservizi, tra hype e casi d’uso — parte 1

Architetture a microservizi, tra hype e casi d’uso — parte 1

Perché sono interessanti le architetture microservizi?

A ciascun servizio la sua persistenza

Cicli di vita e rilasci incrementali

Nella seconda parte..

Informazioni sull’autore