DEV Community: Marzee

Connecting Stripe with AWS Serverless holy grail: Cognito, Lambda and DynamoDB

Marzee — Tue, 11 Jul 2023 15:16:59 +0000

Payment gateways are the lifeline of any business, and it doesn't take much to understand why, healthy businesses generate revenue by providing products and/or services to customers, these revenue streams cannot exist without a reliable and secure gateway that abstracts away all the pains of processing payment transactions.

When it comes to payments a couple services spring to mind: Paypal, Stripe, Amazon Pay and Square (honorable mentions to Adyen and Braintree). We chose Stripe because it has a wonderful developer experience, the documentation is very clear and their libraries are easy to use and integrate.

If you've read some of our blogposts in the past you'll know that we make great use of the Serverless model, the "Impulso" project like many others is powered by the AWS Serverless holy grail: Cognito, Lambda and DynamoDB, which map to Authentication, Compute and Storage, respectively, having this model in mind, we needed a payment gateway that could easily integrate with it, Stripe proved to fulfill that role very well.

There are three parts to this problem:

Integrating Stripe into our authentication/authorization system
Syncing latest payment/billing information from Stripe
Providing billing management to users
In Stripe's API the most important resource is the "Customer", we had to fit this into AWS Cognito's lifecycle to create a 1:1 mapping, luckily Cognito has this thing called "Triggers" that allow you to hook custom code (Lambda functions) to different events, this was a lifesaver because it meant that we could create and associate a Stripe "Customer" as soon as a user registered, thus solving part one of the problem which is identity, from now on whenever a user is logged in we can be certain that there is an attribute of that user we can use to identify him within Stripe's system.

Next, we move on to authorization, the second part of the problem, this is one of the most crucial aspects for this platform in specific, as this is a subscription based system, so we need to make sure users don't get access to paid features if their subscriptions expire, and of course, guarantee that they do get access for as long as their subscription is active, I think we can all agree that there's nothing worse in terms of user experience as a buggy system that works well enough to take your money... but not to provide you with the services you paid for, the horror!

Luckily, Stripe has got our back, all we need to do is create a special endpoint on our side, then we let Stripe know of its existence, this is commonly known as a "webhook" (or "web callback"), from that point forward every time some important event happens within Stripe's system we get notified, isn't that great? The event could be a new subscription, a failed subscription renewal, a chargeback, and many more types depending on what you're trying to achieve, the possibilities are endless.

Last but not least, management, after a user registers they most certainly need a portal to manage their payment details and iniciate the billing cycle if they desire to, and let me tell you, we were delighted to find out that not only does Stripe provide a ready to use and customizable hosted UI but also makes it a breeze to setup, and that's how we saved ourselves multiple hours designing and implementing a billing UI, which ultimately made the project cheaper for our client, textbook example of a win-win situation.

In the end we were very happy with the technical solution, it's amazing how much value we added to the "Impulso" project by leveraging Stripe, especially taking into consideration that this integration wasn't part of the original design when we started the project, it was genuinely surprising how straightforward it was to integrate with the existing system.

Blogpost originally posted in Marzee's website: https://marzee.co/blog/connecting-stripe-with-aws

How an open mindset can boost the evolution of a continuous development platform

Marzee — Thu, 06 Aug 2020 10:01:04 +0000

This blogpost was originally posted on Marzee Labs blog.

A fruitful partnership can be decisive for the success of a project. Finding the right balance between both sides is extremely important, but more than that, promoting an environment where all actors involved strive for the same goals, can really make a difference.

We've already talked about how it can be difficult for a business to find the right tech partner to develop/improve their web platform and drive the business to success. The same is true for companies providing web development services, like us. Not all partners see web development companies as part of the team, but rather as merely executors and service providers.

When we do find that dreamy partner, one that really makes us feel like part of the team, it feels like we found a soulmate. This is happening with us and we are very happy to have established a partnership with Parlia.com.

We've started to work with them last year, being responsible for evolving their web platform, back then still in a Beta phase. Since then, we have strengthened our relationship and we have made meaningful progress in the platform.

In this post, we aim to share a successful example of how an open mindset can empower the team involved and boost the evolution of a continuous development platform.

What is Parlia?

Parlia is an encyclopedia of opinion. They believe there is a finite number of arguments about everything and, with the help of a global community, they want to map them all. More than that, Parlia wants to help people to understand their own beliefs, and the beliefs around them.

They want to make this a community effort and most importantly to have it available to everyone. What better place to do this than the internet, right?

At Parlia's website, you can find a list of questions about relevant global topics. All the opinions are well structured on a map. Each question has a set of different opinions, organized in what they call positions. To support each position, there is a list of detailed arguments.

From the beginning, it was very important to make the web platform as intuitive as possible, so people can relate to it. User Experience and performance are both significant aspects of this web platform.

The technical requirements of the Parlia web platform

When we started working at the Parlia website, it was in a beta phase. From a technical point of view, we had many challenges to tackle and opportunities for improvement. At this moment, it was key to have direct communication with the people behind Parlia to understand their priorities, their short term and long term goals and thus draw a plan of action for the following months in close collaboration with the Parlia team.

As you can imagine, the requirements and challenges of such a web platform are diverse. It’s not enough to have an optimized structure and smooth experience for people interacting with the platform, it's important to assure a strong technological ground as well.

It was clear that we had a lot of work to enhance consistency throughout the platform. Both from a User Interface and User Experience point of view and from technical architecture in order to prepare the platform for fast scalability.

Main improvements

The first step was to set up a cloud application platform as a server and to create a staging environment. The staging environment enables developers, designers and QA teams to perform tests and, most importantly, to gather feedback from all actors involved before pushing it to production. It's key for any type of project, but it becomes crucial for continuous development projects.

In terms of priorities, having a visual appealing web platform was very important, but it was even more urgent to improve the overall experience of the end-user when interacting with questions, positions and arguments.

As an example, we aimed for a more intuitive user interface for the maps of opinions, making sure that the end-user can easily navigate through positions and arguments and still be aware of where he is in relation to the question/opinion.

In the overall, it was key to follow with User Interface/User Experience (UI/X) improvements both to editors and end-users in order to facilitate the use of the website and make it easy to understand the information shared.

Promoting consistency between the frontpage and admin sections was a big upgrade for the platform and very important to provide a better experience for the people working behind the curtains. More than that, it was key to improve the administrative functions, aiming for a better management experience for admin accounts and enabling them to keep track of new content and new users.

All of these improvements were made in close collaboration between the Parlia and Marzee Labs teams. Parlia's openness to receiving and following our suggestions made the whole process smooth and grateful.

"Far from purely delivering the work we asked for, they (Marzee Labs) kept coming back with new ideas and suggestions for optimising our design and approach, almost all of which we took on board and implemented."

Source: Parlia, Founder.

This open environment and partnership mindset also enabled a revamp of the platform technical structure in order to improve the developer experience and prepare the platform for a new phase.

As we moved forward with the priorities set by the Product Owner, our team pressed for a significant code revamp. We were very clear in stating that functional code is fundamental to achieving better performance, speed and greater results on search engines.

The transparent relationship we had made the difference here and we were able to follow with:

modernization of JS/CSS infrastructure;
markup and styles cleanup which improved consistency for the end-user and maintainability;
website optimization for a reliable and quick loading experience;
consistency of image uploads system;
implement an automatic build process to optimize deployments and releases, having the process less subjected to human error.

These steps didn't just make the developer experience much better, it had a direct impact on the website performance and on Search Engine Optimization (SEO).

Why a tech partner?

During this process, the key ingredient for the smooth progress and high-quality results we achieved is the environment that was created from the beginning.

The openness from the Parlia team to receiving suggestions and changing priorities to achieve bigger goals made a difference in making us feel like part of their team.

Parlia and Marzee Labs are working as part of the same team towards the same final goal. This is what makes this collaboration so successful.

Are you ready for a tech partner?

Why you should advocate for automated tests

Marzee — Thu, 02 Jul 2020 14:14:19 +0000

How many times have you deployed a new release in a project, and only after merging it to production an urgent bug is detected? This usually happens for lack of testing, for following an incomplete test guide, or for human error.
For continuous delivery projects, where you are delivering new code constantly to your users, automated testing is critical to make the whole process agile. Not just to maintain the quality of the code-base but most importantly to ensure the stability of the web platform throughout its development cycles.

Even though this is well-known in the web development community, it is not always easy to have automated tests as a priority and to convince decision-makers to dedicate time and resources to implement these tests.

We have recently implemented a set of automated tests in a Drupal-based complex platform in close collaboration with our partner Luís Santos from Openquest. As a Senior Drupal Developer working with Drupal for over 11 years, Luís is focused on security, best practices, performance and usability. He was the lead in the implementation of the automated tests for this specific project.

The project was using Headless Drupal 8 for the back-end dealing with all the editorial experience and the front-end we rendered using ReactJS components within Drupal (see our case study). Every time we had to go through a release, we would have to manually test the codebase, which made each release not just time consuming and repetitive but also open to human error. The tester had to check all log files, any external services and confirm if there were any database errors.

Automated tests are very useful to ensure the quality of your projects and are an important part of maintaining high-quality code and software. It is important to highlight to decision-makers that automated tests can reduce or even eradicate these problems, and most importantly it can speed up the testing process.

In general, the objective of the automated tests are the following:

To reduce Testing Cost and Time;
To speed up the Testing Process;
To help improve Quality;
To improve Test coverage;
To reduce Manual Intervention.

There is a lot of discussion on the best way to implement automated tests and there is a variety of frameworks and types of automated tests. If one considers the test automation pyramid strategy by Mike Coen, there are three different levels of test automation: Unit tests; Service tests (also known as automated API tests, automated component tests, or acceptance tests) and finally the User Interface test.

The base of a fully automated test system is the Unit Tests. Implementing Unit Tests means to break your codebase into pieces, and subject each piece to a series of tests. This enables you to have immediate feedback when a failure occurs.

In the case of our project, the first thing we did was to define a strategy. We leaned ourselves into the code-base and listed all the actions and basic functionalities from the web platform we were working on.

It is important to have in mind that in order to have an efficient testing system, we must list all or most important test cases, and test these cases in various scenarios. It is key to have all the unitary actions, everything we were able to break and isolate whether in regards to API connections or even user interaction.

Once we had a general overview of all the testable unitary actions we researched the best approach to use. Since the back-end of our project was in Drupal 8 we started by implementing a test profile based in Drupal Test Traits, an open-source project launched by Mass.gov that enables you to run PHPUnit tests against your Drupal web site, without wiping your database after each test class.

The main objective here was to use the test environment to run the tests with the existing content. We wrote tests for each separate unitary action of the web platform. For you to have an automated testing system covering the whole system, you need to be constantly adding new tests to each new feature added to your site.

Since the platform had many web-services to render and API connections, the test was supposed to validate the web-services inputs and outputs in all the available scenarios.

The result of implementing automated tests for this web platform was immediate. Once we implemented one of the tests, we found a bug that none of the testers spotted before. One of the APIs that was listing articles in different languages per topic was, in a few cases, delivering a wrong listing which resulted in a few cases of wrong indexing and article listing.

As a web development company, we value the quality of the code we deliver and we always make the case for improvements in the platform's structure and deployment pipeline, especially for complex web platforms that are constantly evolving. Implementing automated tests can be a big step and it can mean a lot of effort (and resources), but it will for sure pay off in the short and long term.

Pyramid Image Source: Test Pyramid: the key to good automated test strategy

This blogpost was originally posted on Marzee Labs blog.

Besugo - a boilerplate static site generator with a CMS

Marzee — Mon, 15 Jun 2020 16:45:15 +0000

The challenge!

So, here is one of those challenges, a lightweight project in which (due to budget and/or content restrains) the best option is a static site. Plot twist, it needs a way to edit the content without having to change a single line of code. Drupal or Wordpress being over the top in this case, we started work on an alternative to get this done.

We went fishing...

We found Netlify CMS, an open-source React app with a simple YAML config. It’s quite easy to install, and its editorial workflow goes through a Github repository that holds all the site’s code. It doesn’t use a database, the content is updated directly in the repo through Git commands, e.g.: when saving a draft, the CMS actually commits the changes to a new branch, and opens a pull request.

Once we found this amazing tool, we benchmarked the best fit for it in terms of a static site generator (SSG) and we ended up with Hugo. Written in GO, quite flexible and fast, it is not only a great match for Netlify CMS, but its template system was already familiar, based on simple html markup.

...and we fished out a Besugo.

So we came up with Besugo, born from the combination of those two, a forked framework from Hugo with Netlify CMS integrated, as the result of our research, the full essence of an SSG with a CMS. The name Besugo is a word play on Hugo, coming from the common portuguese name for a fish species red seabream.

Each project is unique and has different goals. Besugo lets us work with as many different content types as the project requires; whether it’s news articles, blog posts, profiles, or just simple About and FAQ pages, Besugo is up to the task.

Because Netlify CMS is a React app, it lets us create our own custom React components for live previewing content as we edit it. With a little magic from our in-house Javascript nerd, we’re using those very same components to also render the content pages themselves after Hugo’s built them. (With server-side rendering done during the build process as a bonus, so that the fully built markup is delivered to the browser! Yay!) With all that, at every point during content creation and edition, you can see exactly how it will look like in the final pages.

Future editors, unfamiliar with any sort of programming, also have their lives made much easier, since they don’t need to worry about any code at all. A Hugo layout can fit any number of entries for that content type. This really lets you focus on your site’s message; in the end that message is always what matters most.

A hard rock fish!

Now, you might ask, isn’t it contradictory, having an SSG with a CMS? Why not go with a full-featured CMS platform in the first place? It all comes down to the practical application within the project. By taking full advantage of the speed and ease of installation and use, Besugo, a simple static site generator, stands as a solid framework for a quick response, while allowing for a highly versatile content management through its CMS that can be turned on at any moment. It supports i18n and responsive images, among other ready-to-use features, which only further solidify it as one of our go-to tools for new projects.

Oh, did we mention that every part of Besugo works seamlessly with Netlify’s own CDN hosting platform, with every change in code and content being published live immediately? :-)

This blog post was originally published at MarzeeLabs blog.

What you need to know about Static Site Generators

Marzee — Mon, 15 Jun 2020 16:40:34 +0000

We've written before about our adventures into assembling our own Static Site Generator tech stack, named Besugo after the Portuguese red seabream. This time we thought we should step even further back, and discuss why we started working on it in the first place.

For nearly two decades, monolithic systems have reigned in the public domain. Powerful PHP platforms, fetching and updating a SQL database multiple times and building the results on the fly with every request, backing everything from simple institutional sites without much interaction, to all kinds of editorial software and online stores.

We ourselves started in the Drupal world, taking advantage of, and contributing to, one of the most versatile Content Management Systems (CMS) out there. And who hasn't heard of WordPress by now?

This was made possible because of the great strides we have seen in hardware development. More powerful machines mean more computational cycles per second. We let our imagination run free with these systems without fearing an immense drop in performance.

Waste!

But we hardly need a full-fledged CMS in every project we work on. Just because Drupal or WordPress can be used, doesn't mean it should.

And if something shouldn't be used, we definitely don't want to waste effort in maintaining it (and money hosting it!). So what else is there?

Back to the Future

For many projects, the obvious solution is static sites, those very same things that existed before all CMSs (GeoCities anyone?).

Great Scott!!Great Scott! It's the future!!

There's no processing done by the server, no complex database calls, no page rendering... A user's browser asks for a page, the server sends it back, that's it. If we want the fastest server response time, there's nothing faster than the server not doing anything. The performance bottleneck then falls on how long that response takes to reach the user.

Then it seems fate that static sites are the ones that can most benefit from an extensive Content Delivery Network (CDN). In simple terms, it means that your site isn't hosted in a single server, but spread over many physical locations across the world. It's not so easy to do this with a full-fledged CMS, but with static assets the user can simply receive them from whatever server is closest and the least busy (have we mentioned we love Netlify recently?).

And because there's no actual server processing, the attack surface on your site is greatly reduced. If security is a concern, as it should be, a static site is as secure as we'll ever get.

Where does the content come from?
We always look for the ability to change the content of a page without having to dive into the code, so we make use of Static Site Generators (SSG). In fact, they become practically mandatory when you scale up the content of your site to beyond two or three simple pages.

An SSG grabs the content of a site, typically in a series of parameters like title, body, lists, image arrays, etc., and renders the full website tree at build time, based on the given set of templates for the different page types. Actually, that's exactly how we build this very same blog you're reading now!

Because SSGs are plenty, we have the freedom to choose the technology that best suits the project. If we want to write HTML directly, and take advantage of some specific features of Go templating, we can use Hugo, a blazing fast SSG written in Go; if we'd rather start from a Javascript/React framework, there's GatsbyJS with a continuously increasing library of ready-to-use plugins to give it a boost; this blog you're reading takes advantage of Harp with Jade templating, yet another SSG tech stack with proven benefits.

All of these can fetch the content from different sources, such as from markdown files committed with the code, managed services like Contentful or Forestry, or even a custom solution from a CMS we can host ourselves.

But a static site is... well, static, right?

Of course not. Even if we're building with HTML directly, we can throw in our own Javascript layer and with that we can do whatever we want. That's what JAMstack is all about, we work with an API mentality from the start, and our static pages can show up-to-date information on statistics, or submit data to a service and show the results as soon as they're ready.

With a full CMS like Drupal or WordPress, we might be using modules for allowing users to comment on pages, for showing interactive maps, or for processing form submissions. We can have all of that and much more in a static site; we can simply use managed services for such things without worrying about their maintenance (many can be free although depending on the usage they can incur on some extra costs), or we can build our own functionality, whether fully custom or based on existing libraries, and integrate it as well. A site is only static if you keep it static.

Besugo slideSomething I put together for my presentation about Besugo at JAMstack OPO last year, demonstrating some of its build dynamics. Great Scott, I suck at diagrams...

We can do all this during the build process even, automating new builds on certain hooks from certain systems to turn a static site into a living organism. Just the benefits of a continuously integrated automated build process merit a blogpost of their own.

SSGs make us happy

A static site gives us speed, security, and freedom from infrastructure concerns. Just the hosting for most static sites may even incur no costs at all.

A Static Site Generator lets us build on the concept of simple static pages, taking full advantage of their benefits, and expand outwards to include functionalities we would never consider before, in particular, manage its content in an intuitive fashion completely separate from implementation.

SSGs can't do everything though (yet!), we still rely on a lot of other strategies for a lot of projects with different goals. But the benefits of SSGs are plenty, and they're becoming more powerful every day. For every new project we ask ourselves "can this be done with a static site?", we smile when the answer is yes!

This blogpost was originally posted at Marzee Labs blog.

Helping local shops survive and thrive during the COVID-19 crisis

Marzee — Mon, 20 Apr 2020 14:37:09 +0000

Marzee Labs has partnered with Menos Hub and has just launched a web platform with a directory of small businesses in Portugal trying to survive the current COVID-19 crisis - compraaospequenos.pt. We've used Gatsby, Google Sheets, Material-UI and Netlify to build it and we are making the code available on GitHub so anyone around the world can use it to create their local network of shops.
No one was expecting this. COVID-19 took us all by surprise. More than posing a threat to the health and life of a hundred thousand people, this health crisis is having a major impact on the global economy.

Social isolation measures imposed by governments all over the world are directly impacting local businesses. Governments are trying to help as much as they can both companies and individuals with specific measures.

In this scenario, small businesses and independent workers are the ones that are suffering the most, struggling to survive during these tough times. This is a reality in many countries but with different scales. In the case of Portugal, "small businesses correspond to 90% of the companies in Portugal" says João Duarte, the founder of Menos Hub.

With this in mind, the initiative Menos Hub - that works directly with small businesses in Portugal - has created the movement #compraaospequenos (#buyfromsmallbusinesses). The movement started as a Google Spreadsheet with a list of small businesses and independent workers that have special offers during the quarantine trying to survive the current crisis.

Menos is doing more than that and has been helping those local small businesses to reinvent their offer and thus make it through the quarantine period.

Looking to support the movement, Marzee Labs reached out to Menos team and offered to build a web platform by making the directory more visually appealing and, most importantly, facilitating its use, so the movement reaches more businesses and people.

It was 2 weeks ago that we had the first meeting with them and we are happy to be launching the first Minimum Viable Product (MVP) version today. The chosen approach was to work with what was already in place (Google Sheets) and make the best use of Open Source technologies to have a website live as fast as possible and within our reduced resources.

From the start, we knew that the best way of building a fast and secure website was by using the JAMstack approach. Since Menos Hub was already using Google Sheets, we've decided to continue using it as the website's database.

We needed a Static Site Generator to build the website pages and we've chosen to use Gatsby. First of all, Gatsby is very easy and quick to set up, and most importantly it allows you to easily fetch data from any source to build the website's content. In addition to that, our in-house team has a liking for React and we thought it made sense to choose a React-based Static Site Generator stack.

Once we had Gatsby ready and the API integration with Google Sheets working we had to implement the filtering for the directory of shops in the platform. After some research, we've decided to use itemsjs, a free search engine in javascript, created to perform a fast search on small JSON datasets.

For the front-end implementation, and having in mind that we had a limited time frame to launch the website, we decided to integrate Material-UI to make it easier to style all the website's components from the buttons to the filters.

Last but not least, we got support from our Netlify that is offering an extension of their free services for people working in solutions for the current COVID-19 crisis. We must say, though, that Netlify was always our choice of CDN even before they confirmed their support. A big part of the websites we build is using Netlify services.

We are very happy with the result - compraaospequenos.pt - and we are ready to start the second phase and further expand this platform. If you are looking to implement something similar or if you are interested in contributing to our current platform, you can find the repository available here.

We are looking forward to seeing this spreading the world.

Stay safe and buy from small businesses.

This blog post was originally published at MarzeeLabs blog.