DEV Community: jaeyow

Going to Production with Github Actions, Metaflow and AWS SageMaker

jaeyow — Sun, 14 Aug 2022 04:38:00 +0000

A scalable (and cost-effective) strategy to transition your Machine Learning project from prototype to production

Fully working source code is freely available here.

Three is a crowd

Github Actions, Metaflow and AWS SageMaker are awesome technologies by themselves however they are seldom used together in the same sentence, even less so in the same Machine Learning project.

Github Actions is every software developer's favorite CI/CD tool, from the makers of Github. It is used by millions of developers around the globe, and is very popular among the Open Source community and is mostly free.

Metaflow, the Data Science framework developed in Netflix, is a very attractive proposition as it allows one to easily move from prototype stage to production, without the need for a separate specialist team of platform engineers, enabling your data science team to be more productive.

And lastly, AWS SageMaker has been in the ML game for a while, and AWS is still the leading Cloud provider (for Cloud services and ML workloads) in the world.

But why use the three together?

It all boils down to the use of Metaflow and its inherent scalability. You can start using Metaflow even when you are still running your experiments on your laptop.

But when you want to move to the cloud, you will need a basic (and free/cheap) scheduler/orchestrator to kick-off your workflows - enter Github Actions. There are some limitations with using Github Actions as your workflow scheduler/orchestrator, but for simple projects, or in the case when you're leaning on SageMaker for your training and deployment workloads, it is more than sufficient.

And as to AWS SageMaker, teams that are already on the AWS ecosystem will appreciate the almost unlimited deployment possibilities in their fingertips. Typically SageMaker workflows are created and controlled through the AWS console, but to enable us to go to Production in this article, we will be using the AWS Python SDK, through Metaflow.

What does Going to Production mean?

A typical Data Science project starts with the Prototyping stage, and the tool which is the perennial favorite among data professionals is Jupyter Notebooks. And this is also my personal playground when I start looking at DS and ML problems, and it's awesome for quick iterations.

It allows me to quickly play with the data, create visualizations, transform data, train and deploy them to an extent. It is awesome, and it has become my favorite too. However, when and if the project needs to progress to production, we need a better tool and work practice. Many teams still deploy to production with their laptop and Jupyter Notebook setup, and though you can, you really shouldn't.

The main characteristic of production workflows is that they should run without a human operator: they should start, execute, and output results automatically. Note that automation doesn’t imply that they work in isolation. They can start as a result of some external event, such as new data becoming available. --Ville Tuulos, Effective Data Science Infrastructure

Because my background is in Software Engineering, I've been honed to the process of continuous integration and deployment for virtually all my software projects. That all projects should have the ability to be built and deployed at a click of a button, or as a result of a schedule or event, even multiple times a day if required.

Everything automated, everything typically stress-free unremarkable events.

This is exactly what we want to happen when we want to push ML projects to Production. We want it to be automated, and we want it to be a stress-free unremarkable process.

The Sample Project

As I'm currently studying towards the AWS ML Specialty certification, I wanted to play a bit more with the built-in algorithms that SageMaker has in its portfolio. One of the first algorithms I checked out was Linear Learner. It's a type of supervised learning algorithm for solving classification and regression problems.

I did not create the notebook from scratch by myself, however I based it from the examples in AWS SageMaker - An Introduction to Linear Learner with MNIST.

I basically ported it into Metaflow DAGs as seen in the flow image below:

I've leaned on Python for all my ML work, and Metaflow is based on Python (you can use R too), and AWS SageMaker has an extensive SageMaker Python SDK at your disposal.

Looking at the above flow, we are able to do an end to end ML workflow including model training on AWS SageMaker, leveraging on AWS ML instances for ML training workloads, to endpoint creation with your required AWS instance type.

For brevity, I have skipped the part of serving that endpoint to the public, but this can easily be done by serving the endpoint behind an API Gateway and a Lambda. Perhaps we can do this operation in a separate article. Also take note that we have deleted the endpoint, before we end the flow to avoid the dreaded AWS bill specially now as we are just checking out the service.

In this sample project, I have also leveraged on Github Actions as our workflow scheduler/orchestrator, this is really to prove that Github Actions, even in the free tier can really be used in 'production', albeit in a very simple flow.

Although it's worth mentioning that AWS Step Functions is being pushed by Metaflow as the workflow orchestrator of choice if we really want one that that is robust and easy to use. Metaflow has an official Step Functions integration available. Yes this is also a topic that I'd like to cover in a future post.

Metaflow performs the SageMaker operations - from model training, to model creation, to creating and testing the endpoint, all through the Python SDK, where we really have the power of SageMaker in our fingertips.

Summary

In this article we've ported the Python code from the MNIST AWS example, trained our ML model in AWS SageMaker, built, and deployed the model in a SageMaker endpoint in production. This is all automated by Github Actions (in this example the workflow gets triggered by a repository push), and brought all together and made possible by the awesome Metaflow.

Thanks, I've made the source code freely available here.

Let me know if you have any concerns, or any questions, or if you want to collaborate!

Resources

Small to Reasonable Scale MLOps - An Approach to Effective and Scalable MLOps when you're not a Giant like Google

jaeyow — Thu, 28 Jul 2022 09:56:00 +0000

Quick recap of the AWS Summit ANZ

For me, one of the more memorable presentations in the recently concluded AWS Summit 2022 ANZ was the one where Carsales described their strategy in scaling their AI/ML Operations. They did not have a large Data Science team, but as they had more and more Data Science projects, they needed an effective strategy for scaling their AI operations.

It's undeniable that leadership is instrumental in any company and project success, however I was intrigued with one of their ML tool choices that helped them reach their goal. I was so curious about this choice that I just had to learn more about it, so in this article will be talking about a sound strategy of effectively scaling your AI/ML undertaking and a tool that makes this possible - Metaflow.

Why Metaflow

Metaflow was created in Netflix where they used it internally in demanding real-life data science projects and was open-sourced in 2019. And because of its tight experience with AWS, it plays really well with many AWS services, in fact these are all described in detail here.

As I researched about it and used it in a project, I came to the realization that its secret I think is in its simplicity. But don't let this simplicity fool you.

In a nutshell, Metaflow allows you to create DAGs (Directed Acyclic Graph), so we are now touching graph theory here, but in the end it is really just a fancy term for a workflow, but one that doesn't form a closed loop.

These DAGs, combined with Python, Serverless and the Open stack, is a very powerful combination. This results in the democratization of the Machine Learning function, making it easier than ever to kick off that personal ML project, or scale your company's capability in the ML Space.

Scalability

The typical machine learning process starts with simple experiments, mostly done on laptops or PCs, and this can easily be done with Metaflow. The realization of being resource constrained will come pretty quickly as soon as more complex algorithms or gigantic volumes of data come into the picture.

However, using the same Metaflow Python scripts (oh yeah you can use R too), plus a sprinkling of decorators, you will have the ability to leverage almost infinite cloud compute (both GPU and CPU), gigabytes of memory, and a long list of mostly open source SAAS/PAAS tools.

Analysis Paralysis

Remember the feeling of trying to select a tool, but feel paralyzed of picking one for fear of being stuck with it forever? Metaflow will not only enable you to easily pick and integrate an ML tool to your project, it will also allow you to abandon that choice relatively easily once a better one comes along, and be rest assured that you are using a tool that was battle tested in Netflix.

This article comes with a simple example project and although the algorithms it needs don't require the resources that a more complicated model requires, it represents real world data, and was originally created when I started following Formula 1 more regularly and was looking for something that I can learn Data Science and ML with, and wouldn't mind spending countless of hours with.

However, if you want a more realistic problem, more worked-out open source examples are found here, and here, and hopefully you will believe me that you don't need to be the size of Google to be able to tackle these types of Data Science problems.

Example ML project with Metaflow

The example project I have here is a very simple workflow that although consumes real-world dataset, was really only created for learning purposes. Following image shows some of the technologies I used to get it working.

Most of the code is from my General Assembly capstone project - where I go through the process of consuming data I have pulled from a public API, do a bit of feature engineering, integration with another popular Machine Learning tool called Comet ML and Github Actions, then train multiple algorithms in parallel, all repeatable since Metaflow keeps track of all experiment metadata.

The intention is to show how easy it is to leverage Metaflow not just for orchestrating the parallel workflow, but also for enabling the repeatability of your experiments. It finishes after the parallel training and testing of models, however in reality you could do a myriad of tasks after this, such as model selection, model deployment or even scheduling for retraining.

Code is freely available here.

Summary

The main disconnect with many new to Data Science and Machine Learning is the difficulty of shipping models to production. Many Data Science courses may teach you the basics of the whys and hows of using algorithms and model building, however, throw you to the wolves with deployment and scalability. With Metaflow, it is easy to create ML pipelines for development when you're working on your laptop, and when it is time to push it to production, there will be minimal work involved in moving that workload on the AWS cloud.

The example project shows that it was easy to create a workflow that performs feature engineering, model training and testing in parallel, and easily integrate 3rd party tools. This is only the tip of the iceberg, it really does enable one to have the ability to do ML using the open stack, 3rd party SAAS offerings, many free tools, and do ML at par with the big boys.

Resources

AWS Summit 2022 Australia and New Zealand - Day 2, AI/ML Edition

jaeyow — Fri, 20 May 2022 08:57:55 +0000

AWS Summit 2022 - AI/ML Edition

This is my Day 2 report for the AWS Summit 2022 AU and NZ.

I make no secret that my goal in the next 12 months is to build up Machine Learning skills and find projects that will exploit these newfound skills. So many of my learning activities this past year have been about this field, and the effort is still continuing. Imagine my surprise when I found out that the AWS Summit has a good chunk of sessions focusing solely on AI/ML!

Day 2 AI/ML Sessions

Day 2 Keynote

In the Day 2 Keynote, Simon Elisha - ANZ Chief Technologist, was talking excitedly about a few cloud advancements but it was these AWS Partner's presentations that attracted my attention:

Franky - the stroke rehabilitation robot

Franky is an at-home rehabilitation robot for stroke patients that uses AWS IOT and Machine Learning among other technologies.

In early trials, Franky is showing the benefits of using AWS IoT by saving patients travelling time, improving data collection, and avoiding physical contact with patients during the pandemic.

Swoop Aero - making access to the skies seamless

Swoop Aero is one cool company, using cool technology. It has been used all around the world to as a world-leading drone logistics service, delivering anywhere on the planet, all from there operation HQ in Melbourne!

All made possible by proprietary drone aircraft - Kite, AI technology and digital twin based on AWS.

Reduce bushfire risk with machine learning at scale

The damage and loss caused by bushfires every year is in the excess of tens of millions of hectares and would have cost billions of dollars to the economy. It has been identified that one of leading cause of bushfires originate from the energy supply network, ie. the millions of kilometers of power lines all around the country.

To improve the process of asset inspection of these power poles nationwide, Machine Learning was used to replace the current method to ensure that the operation gets completed quicker and cheaper. The current method was very labor intensive and can't possibly scale easily nationwide.

Sepsis detection using model ensembles on Amazon SageMaker

Through a project collaboration between the NSW Government and University of Sydney a project was undertaken to tackle the problem of diagnosing Sepsis in the emergency room. Sepsis is a leading cause of death in Australia, greater than the annual national road toll, breast cancer, prostate cancer, or colon cancer.

However if it can be diagnosed early enough, then most of these deaths will have been avoided. This session demonstrated the use of model ensembles (in this case 3 models - XGBoost, Linear Learner and Logistic Regression), and AWS SageMaker, Amazon's machine learning platform.

Traffic signal optimization with Machine Learning on AWS

This next session was quite interesting as it combines IOT, Edge, reinforcement learning and computer vision all in the same project, aiming to replace the current decades old technology being used in our roads.

There is a need to optimize the current state of things because our city's livability has now been affected. Pollution, Noise, accidents, and congestion, there has to be something that can be done with these.

AWS IoT Greengrass allows one to manage their IOT Edge devices, download ML models locally, so that inference can then be also be done locally.

MLOps with Amazon SageMaker: Carsales journey to scale AI

The last (and definitely not the least), is the talk from Carsales on how they scaled their Data Science/ML capability. As they don't have a large DS team but their projects that required Data Science were on the rise, they have demonstrated how they were able to:

democratize the Data Science skill
democratize the ML Engineer skill
allow them to experiment at scale
allow them to do flexible model deployment
allow the reusability of models across teams

As a result of their new DS framework (based on a Metaflow - a DS framework built at Netflix and AWS SageMaker Pipelines), they were able to free up their DS resources so that Software Developers were now trained and equipped to tackle their normal DS projects, at a ratio of 70% DS/ML work was now completed by developers. This leaves the 30% meatier and more difficult problems for the Data Scientists to tackle.

Summary

A couple of highlights for today, first was the demo from the team at Swoop Aero, and to think that they do all their world operations from their Melbourne HQ is just mind boggling.

And next is Carsales as they were able to democratize the Data Science and Machine Learning, they magically enabled their developer team to tackle Data Science and Machine Learning projects. They developed a Data Science framework that enabled normal developers to work on DS/ML projects with little input from the DS team, which frees up the Data Scientists to work on the more difficult problems facing the business.

Resources

AWS Summit 2022 - Australia and New Zealand

AWS Summit 2022 Australia and New Zealand - Day 1, AI/ML Edition

jaeyow — Thu, 19 May 2022 06:59:00 +0000

AWS Summit 2022 - AI/ML Edition

This is my Day 1 report for the AWS Summit which is an all Online event this year.

Even if the event was a real face-to-face conference, I would still have been attending the sessions online, since coincidentally, I've very recently contracted Covid, and I am still recovering as I'm typing this report.

Out of about 104 sessions, 28 are about AI/ML, I thought that was great, so this for year's AWS Summit, that's where my focus will be. Obviously I could not have attended all 28 sessions, just the ones that I fancied and handpicked from the agenda.

Day 1 AI/ML Sessions

Jumpstart the machine learning journey with Canva

The first session is a peek into how Canva approached scaling ML in their company.

This can be summarized into 5 main points, please see image below. When the AWS Summit videos are made available, I will post their links here.

Team Enablement - involves defining culture, team training (and hackathon), and tools
Data Strategy - involves training, best (data) practice
Proof of Concept - involves training, business alignment, cross functional teams
Repeatability - involves training, tools, cloud enablement, MLOps
Scale - involves training, Center of Excellence, optimization

Solving business challenges with AI - powered by Intel on AWS

AWS and Intel have a deep partnership and together provide solutions to real world customer problems. Many of the these are related to AI and ML, such as in the development of hardware chips that provide faster, more powerful performance that is cheaper and more sustainable to run.

End-to-end MLOps for architects

This session introduces the concept of Machine Learning Operations (MLOps) in Data Science/Machine Learning teams. As the team's AI maturity improves, it becomes more and more important to integrate MLOps practices into your workflow to avoid the problems of POCs not making it into production.

Issues like manual intervention for changes, work stuck in Jupyter notebooks, not enabling autoscaling, or having no feedback on model performance are only some of the issues that contribute to the failure of your ML system.

We were then presented with 3 MLOps architectures, from Small, Medium and Large, which represents architectures that one would adopt, depending on your team's AI maturity, and budget.

Sustainability powered by AI and machine learning

In re:Invent 2021, AWS announced that Sustainability as the latest pillar added to the 5 existing: Operational Excellence, Security, Reliability, Performance Efficiency and Cost Optimization. It was nice to see sustainability being more and more involved in presentations, so it was one that I was looking forward to hear.

Leadership is crucial if sustainability initiatives are to succeed. To adopt a sustainability mindset, direction from leaders is a requirement. From there the company can enable teams, adopt a powerful data platform such as AWS, and finally choosing the right first project.

The session also showed an example in an industrial setting of one Amazon.com's buildings. The problem was to improve the current system's power consumption by 8% and achieved that by creating a model based on Reinforcement learning and Amazon SageMaker suite of tools.

Kick-start your ML journey: Get hands-on with AWS DeepRacer

Not everyday does one have the chance to develop systems used in autonomous vehicles and AWS have just given every developer the ability to do so in DeepRacer! Using Amazon SageMaker and Intel's OpenVINO toolkit, Reinforcement learning can be used to train your models, after which it can be run in the DeepRacer 3D racing simulator.

Also, AWS has purpose-built a real 1/18th scale race car, so that you can use it to download your models to, and race against the autonomous racing league.

Summary

That's a wrap for Day 1 of the AWS Summit 2022 Australia and New Zealand. I've got two highlights for today.

First is seeing how AWS SageMaker is being positioned by Amazon at the center of their AI/Machine Learning story - as an end to end MLOps framework. When I learned about Data Science in a course I completed towards the end of 2021, I was not made aware of such tool available, and rightly so. It would have made the course more difficult than it was.

And next was seeing DeepRacer - truly making Machine Learning (Reinforcement Learning) accessible to developers the world over.

Keep an eye out for my Day 2 wrap later!

Resources

AWS Summit 2022 - Australia and New Zealand

Micro-frontends building blocks: Webpack Module Federation

jaeyow — Sat, 23 Apr 2022 06:41:17 +0000

Meet Module Federation

The next micro-frontends building block we will cover is Module Federation.

It was initially created to enable asynchronous loading of Javascript bundles, so one can easily share and consume code in the Javascript ecosystem, ie. any Javascript project, think browser, Node.js or Electron.

Later on, it was extended to support server-side rendering scenarios. Since the method I'm going to explore in building my micro-frontends is client-side composition using React.js, I've identified Module Federation as the perfect technology to use.

Module Federation was created by Zack Jackson, a Webpack core maintainer himself, and was integrated as a flagship feature of Webpack 5 in October 2020.

But, what seems to be the problem?

Scalability - its the problem that Module Federation tries to solve.

So that large applications can be split easily where each part can then be shared among other parts and may be developed by separate teams.

Module Federation is a game-changer in Javascript Architecture because before this, sharing code was clunky and just didn't feel smooth enough. The usual module sharing method that we have been using for years is using NPM packages, enabling developers to create build-time dependencies. Although NPM works, it cannot help us when we need to load dependencies at run-time.

As micro-frontends evolved, we now have a need to resolve Javascript modules at run-time. This is where Module Federation shines.

Module Federation super powers

Good build performance

Because Module Federation encourages you to arrange your application into separate projects so that you can build and deploy them independently (therefore in parallel), each project can be built and deployed in isolation and may be done by different teams.

Good web performance

The problem with the usual NPM module composition is that it typically increases the application size as the number of dependencies increase. Module Federation provides you an option to lazy load bundles, to avoid loading them when your application loads, but only loading them on demand. This results in a better web performance as it avoids having to download modules before they are actually needed.

Good management for shared dependencies

And because Module Federation provides excellent dependency management, it efficiently resolves vendor and third-party dependencies so that only one version of a library is ever loaded by your application. How good is that?

Import code from other builds, at runtime

Instead of sharing code and thinking of "libraries" when using NPM package approach, we can think of applications that use Module Federation not dissimilar to APIs. Now web applications can expose functionality to other applications, the same way that it can also consume from other applications.

Deploy independent code, without needing to re-deploy consumers

The ability to have evergreen functionality is very appealing to the developer. There will be no need to redeploy the consumers anymore when exposed dependent functionality will have changed. I have to say that this in itself is a very powerful feature, which will need very careful consideration to avoid unintended results.

Redundancy and self-healing capabilities

With shared dependencies, Module Federation maintains a dependency graph for your whole application, so that even though applications fail to declare a dependency or when there are network issues, it knows the needed dependencies so that it takes care of downloading it as required.

Micro-frontends will work like a monolith

Bringing in shared functionality to your application is quite simple - either synchronous loading by importing the bundle as usual. Or asynchronous loading by using lazy loading, to only load the dependencies when required.

Developer experience improved, maintaining customer experience

Using Module Federation will feel very familiar to any Javascript developer, since it is available as a Webpack plugin starting with Webpack 5. If we think about that for a bit, this is actually quite powerful and exciting.

Think about all the things that Webpack can bundle - scripts, assets, styles, images, markdowns, and more by using third party Webpack loaders. All these can be federated and shared though Module Federation, great, right?

Caveat

However, as great as Module Federation is, it is important to remember that it is not a framework, and as such it does not handle any of the implementation details for you. For example, you could use awesome client-side composition libraries such as Single SPA, or Next.js and just leverage Module Federation to do the module loading over the wire for you.

Conclusion

In this article we have introduced Module Federation as an excellent option for building your micro-frontend application.

It promotes scalability by enabling multiple teams to work on a separate applications by allowing applications to share and consume functionality at runtime.
It makes applications more compact through the use of shareable dependencies.
It enables evergreen functionality so that you won't need to build and deploy your consumers when the shared functionality changes.
Developer experience is great since many developers are already familiar with the Webpack ecosystem
Once configured, your application will work like a monolith which is awesome.

Resources

Originally published at Fullstack Developer Tips

Micro-frontends building blocks: Monorepos

jaeyow — Tue, 05 Apr 2022 00:51:49 +0000

A monorepo, what's that?

A while back I was finishing work with an employer and was due to join my new team. I remember feeling excited when I heard that the new project will be using monorepos. I did not have a clue what monorepos were, only I've heard that that's what the big boys were using.

What is a Monorepo, image from Voiceflow.com

Definitely not mainstream (still not these days) they were supposed to promote better collaboration, and give the teams using it full autonomy and are perfect for when you're anticipating the project to scale.

Yeah, OK that sounds good, however, I just didn't understand how just by co-locating all your sub-projects into a single repository will magically give you all the advantages mentioned.

Monorepo is code co-location

Yes, Monorepo is code co-location, but there is more to it than that.

Just putting several projects in one repo does not make it a monorepo. In fact, if that's the only thing that we do, we are creating a monolith. A good monorepo enables you to create distinct and separate projects, that have well defined relationships and dependencies.

This is a very common misconception in software engineering, where as far as I can remember, we have been conditioned to put everything in its own repository to encourage team autonomy. But it also promotes isolation that creates silos within the team and that essentially kills collaboration.

Monorepo promotes dependencies and relationships

Monorepos excel with dependency management now that all the code is co-located in one spot. Because a monorepo can contain many disparate projects that may have deep and non-trivial relationships, monorepo tooling are optimized for build speed.

It can cache previously performed operations, so that running them again will skip the work that has already been done to produce substantial time savings.

The node_modules folder needs no introduction, its that deep dark place where all project dependencies end up in. Current monorepo tooling can save a substantial amount of disk space by hoisting common dependencies to the top level, so that all lower level dependencies are satisfied from there instead.

Monorepo is one version for all

With monorepos, version configuration is simple, there is only one version, and this essentially is a snapshot of the system removing the need to manage the multiple different versions of your application and its dependencies.

Monorepo is atomic

Imagine all your system in one repo. Web frontend, IOS and Android mobile app, shared libraries, APIs, Lambdas, IaC, scripts, etc. The ability to be able to work on a task and do everything in the same commit, that is possible with monorepos.

Monorepo is intelligent tooling

Many of the monorepo benefits are not possible without the advancements in current monorepo tools.

Tools like Lerna, Bazel, Nx, Rush, Turborepo, to name a few. Lerna is probably the grand daddy of all monorepo tools. CRA, Babel, Jest are a few projects that use it. Bazel has been refined and tested for years at Google to build heavy-duty, mission-critical infrastructure, services, and applications. Turborepo is the monorepo for Vercel, the leading platform for frontend frameworks. These tools can help keep your monorepo workspaces fast, understandable and manageable.

Monorepo.tools is an excellent resource detailing the many intelligent features of these tools that help manage your monorepo projects.

Monorepo is collaboration

Because all the team's code (maybe even the whole organization's) exist in the same repository, monorepos encourage code sharing, transparency and cross team collaboration. This does not come for free, though. There will be more noise, but with good management and the help of efficient tooling, this is all possible.

Monorepo is people

Software engineering is as much technical as it is people. Yes monorepo is a technical strategy to structure your project and workspace.

However, it is actually much more than that. Because it encourages people to collaborate effectively and work efficiently together, it represents a change that probably many software teams need today.

Conclusion

Moving to monorepos represents a paradigm shift in software engineering. There are reasons why many organizations have made the shift. There are obviously many gains with using monorepos that made these organizations move. But because it represents a fundamental change and shift in thinking, there are also many detractors.

Because on the surface what may look like a simple change in project and workspace structure, is actually an organizational change, and that can be hard in any industry.

Resources

Originally published at Fullstack Developer Tips