DEV Community: Webiny

Introducing Webiny Enterprise Headless CMS+

Ben Read — Tue, 18 Oct 2022 12:41:25 +0000

We have struggled for a long time on the question of how to position Webiny as a product. From the start, it was clear that we we were building was a feature-rich product that didn't quite sit firmly in the camp of "Headless CMS", yet that is what the majority of our users focused in on.

As a result, we have doubled down on that particular marketing, building more features and explanation around the concept of Webiny as a Headless CMS. However, as I'm sure you realize, it's an extremely crowded space. And we still didn't quite fit in.

You see, Webiny isn't just one product. It's four fundamental ones:

Content management system
File storage and Management
No-code Web Page Builder
Form builder

In reality, it's actually more than even those products: Not every CMS can scale to tens of thousands of users, serve content for millions of website pages and application interfaces, and be as easily customizable as Webiny is.

We frequently hear about teams using Webiny in unique and different ways: from wealth management portals, to BitCoin exchanges, to online course providers, to customer service desks, to a no-code form builder. It's been amazing to watch different teams do all of this with Webiny and more.

Not every CMS can become all — or any — of these things.

But if we were to look at the terminology around the kind of tools on the market that have those features ... well, that's not what we want to be either.

What if you Need More Than a Headless CMS?

Products that we could compete with that are more than a CMS are varied, but the basic premise is the same: allow large, complex, multinational organizations with content creators in different locations and languages to populate their websites with products, blog posts and other information on a regular basis. This might not just involve writing content, but also creating short-lived product launch pages, gaining feedback using a web-based form, or sharing product literature with vendors and potential contractors.

Popular acronyms in this space are WCM ("Web Content Manager"), and DXP ("Digital Experience Platforms").

These systems are often very clunky, with difficult-to-navigate interfaces. Customization options are very limited, and they often require hiring specialized individuals to manage them. Not to mention the fact that they're nearly always hugely expensive SAAS platforms that exist outside the organization's perimeter.

With the features we have, it would be possible to become a competitor in this space. But Webiny is much more nimble, adaptable and on top of that, is also self-hosted.

By Any Other Name

So we stopped and asked ourselves: do tools to manage content on the web really need more acronyms? CMS is a pretty good fit, and even the "headless" part seems to be on it's way to being generally accepted.

And, we thought, if we adopt that terminology, not only would we be alienating the fact that medium and small businesses can also depend on Webiny for their content, other teams might start to get the impression that we are also a clunky system that's prohibitively expensive, not customizable, and overall difficult to use.

No, we don't want to pigeonhole ourselves like that. Yet, Webiny is more than a CMS.

An All-in-one Solution

So we're sticking with "Headless CMS" as a description of Webiny. But opening a conversation on that front can sometimes present a problem: what if the organization approaching us doesn't have the requirement of a content store. Perhaps instead theyre looking for a way to dynamically populating content from the CMS into a no-code page builder? Or Perhaps they need an advanced publishing workflow for their teams to be able to create, edit, update, approve, and publish their content?

Starting off the conversation "We're a Headless CMS" unfortunately leaves out all of the other cool features we've built, or are planning to build in the next few months.

These new features will enable organizations to build dynamic content quickly, ahead of the competition, without the involvement of technical teams past the initial setup, and do so at a fraction of the cost of other providers. And to do it at the scale that enterprise organizations require.

Yes, for those organizations who need that kind of power, Webiny is an all-in-one solution that might be able to replace several costly products that are strung together with digital duct tape, not designed to work together, and are also individually difficult to work with.

For those people, Webiny isn't just a CMS. It's a CMS Plus.

What Is Webiny Enterprise Headless CMS+?

Plus.

It's succinct, minimalistic, but covers a lot of stuff. Webiny is still a Headless CMS. But it's a Headless CMS Plus all of the other features we've talked about.

Adding the word "Plus" to the business and enterprise products opens the discussion around Webiny's other capabilities. It allows us to showcase other features whilst not detracting from the core use that many of our users see value in.

It allows us to start more conversations around the functionality of the Page Builder and Form Builder tools, tools that are going to get some pretty interesting added features in coming months.

We're really hoping that this new initiative will allows us to open more discussions with organizations looking for, not just a Headless CMS, but something that can solve a lot of the issues around producing content in their business.

If this is you, please reach out to us! We'd love to hear about the difficulties you're facing. For our Enterprise customers we provide SLA support and consultancy services tailored to help your teams to build a solution that'll be designed around your workflow and specific needs. And we can build custom SSO implementations with your IDP of choice.

If you're an agency or a smaller business, Webiny is a great fit for you! Use it as a Headless CMS with multi-tenancy built-in, and pay a reasonable cost for each user seat. And add-on optional benefits such as our advanced publishing workflow and headless pages.

Remember, Webiny is an open-source, self-hosted solution: you can use it at no cost under our MIT license today. We hope this will help you to build smaller projects and thoroughly investigate its capabilities before you make a decision.

Try a live demo today to get more background on the product.

🤔 Should I Use Gatsby or Next.js For My Next Project?

Ben Read — Wed, 05 Oct 2022 10:18:58 +0000

Developers tend to get very passionate about their tools. And rightly so: without the best tools for the job at hand, we couldn't achieve our objectives, make something awesome, or build a better world.

React is still a major player in terms of frontend development. There's currently a competition of sorts going on between the two React heavyweight frameworks Next.js and Gatsby. Which one is better? Which should you learn? Is Gatsby better, or Next.js, for building your next project?

At Webiny, we're interested in providing developers a backend Headless CMS to compliment your frontend, so it's of deep interest to us which tools are popular. Knowing which ones you're using ... even which ones you are preferring, influences our product decisions. We think about it when we talk about marketing. And we consider it when we're planning what we build into our open source CMS.

So we decided to do a bit of critical thinking about whether Next.js or Gatsby is better. We came up with these criteria to match them against. Even so, we're going to leave it up to you as to which one you're going to use!

Server Generation and Configuration

Generating React code, or any JavaScript, when an application is built, is a critical metric when it comes to frontend frameworks like Gatsby and Next.js. But we don't think it's speed that's critical. It's certainly important, but it's one of those metrics that the developers of these frameworks are acutely aware of, so they're going to make sure they keep making advancements in this area.

What's more interesting is what APIs you have access to on server generation. Does the tool make it easier or more difficult to do what you need to? That's what we're assessing here.

Whilst Next.js allows access to customize headers, page initialization, add rewrites and redirects with it's config file and custom _App.js component, Gatsby has a config file, a custom html.js document (which is similar to Next.js). It also has files to directly access the server-rendered application and the hydrated application in the browser.

Gatsby certainly has a wider variety of configuration open to developers here. But that can also be daunting or unnecessarily complex for someone who just wants to build with React but also have static generation.

Also, sometimes it's not clear what the files are for. For example, the gatsby-node.js file isn't specifically for APIs that run in the server environment (which would be using Node.js), instead, this is where the internal node transformation happens, allowing you to create dynamic pages and custom GraphQL resolvers to your application.

Admittedly, once you know that, it's great to have access to all of these APIs to customize your data. But we'll get to that a bit more later.

Documentation

Whether Gatsby or Next.js documentation is good or bad seems highly subjective. But sometimes documentation can inadvertently hide things developers need to know under generalizations or by accidentally missing things out. Sometimes an API badly explained is worse than not having it documented at all.

We all know this, which is why we looked at documentation as a marker on whether you should choose Next.js or Gatsby to build your next project.

Whilst Next.js documentation has a cleaner, focused look, Gatsby's is busier with a 3-column layout. Though sometimes that's a good thing. Developers probably won't get as lost contextually in the Gatsby docs as they might in Next.js.

We generally prefer the darker code blocks and higher contrast on the Next.js website. But it could be because we're just getting old and our eyes aren't as good as they used to be.

Next.js' strengths come out clearly in it's focused approach towards lower-level APIs such as the getStaticProps() and getServerSideProps() functions, and these are explained very well. But there are also some things that are not mentioned except succinctly in an "unsupported features" section. This can be frustrating when you find out you've built your application to use a certain feature, only to realize it's not supported in the environment you intended to deploy to.

Routing

Both frameworks take advantage of file-based routing. Next.js and Gatsby have the ability to define dynamic routes as well, but Gatsby doesn't clearly explain how this works, perhaps assuming you're only looking for this feature if you've already used Next.js.

What is great about Next.js is the ability to create API Routes. This could be really useful in a lot of situations, especially if you have a sprawling REST API endpoint and want a minimal set of data returned from it, you could massage that data down to only the content you need on your frontend. In fact, Next.js has it's own express-like server-side framework called Micro.

This turns Next.js into a truly isomorphic application, which is meant to have components server side and on the client, working in tandem to produce a single application.

On the other hand, Gatsby seems to keep these concerns clearly separated, even with their own Gatsby Functions, the emphasis is on a separate backend that communicates with the hydrated frontend application.

Do you like to mix these concerns? Or keep them separate? It depends slightly on what kind of developer you are, and what kind of project you're building, as to which you'll prefer from this point of view.

Conclusion

Gatsby and Next.js are two very different tools with unique value propositions: either low-level tools to provide fundamental APIs and free developers from unnecessary tooling, or an ecosystem full of plugins of varying quality, together with a data layer to stitch your content together.

As we said at the start, this isn't a full roundup of all of their features. Maybe what you're building needs something more specific like Astro, Nuxt or SvelteKit. We only know what we've encountered when building our starters for Next.js and Gatsby.

Oh we didn't tell you? We've built 1-click deploy starters for Webiny headless CMS.

👉 1-click starter for Next.js and Webiny Headless CMS on Vercel

👉 1-click starter for Gatsby and Webiny Headless CMS on Gatsby Cloud

Make sure you check those out. Maybe it'll help you make your mind up ... should you use Gatsby or Next.js? The choice is ultimately yours.

Alternatives to Heroku’s Free Tier

Ben Read — Sat, 27 Aug 2022 20:23:48 +0000

Heroku announced yesterday that it's extremely popular free tier plans are being phased out starting on November 28 this year (2022). It's the end of an era for a lot of people, particularly independent bloggers who have hobby sites, developers evaluating software for use at their organizations, and non-profits who host their Headless CMS backend. But is there an alternative to Heroku?

The Heroku Story

Heroku started as a platform offering hosting to exclusively Ruby-based websites. It's since expanded that option and now a host of "build packs" for other languages exist which can deploy infrastructure to Heroku in one click. This was likely the inspiration for similar "one click deploy" buttons that have been adopted by Netlify, Vercel and others.

The unique thing about the way Heroku was built is that it leveraged AWS' ability to hibernate EC2 instances. So when you deploy your infrastructure to one of the free-tier "dynos" (a dyno is an environment in which your application can run, similar to containers), after a while of inactivity the dyno would hibernate: it would turn off, or go to sleep. The effect of this was that you couldn't contact the application straight away in that state. It would take around 30 seconds for it to activate, or wake up. This was enough time for Heroku to spin up your dyno again so that it can start accepting requests.

This method did come with some complications like ephermeral file storage, meaning you couldn't store things like SQLLite databases or upload images alongside your application. But there were other Heroku services and 3rd party integrations to allow you to use databases and store images and other files.

This was a great help to many people who's main concern was with small blogs built on an open source blogging platform such as Ghost or Strapi. With Heroku you could deploy your application and build a separate, static frontend (which would be hosted on a separate service, such as Netlify or Vercel). That way, you could write an article, rebuild your static site, and afterwards allow the application to hibernate. And it would be free to use.

Alternatives to Heroku

Looking for an alternative to Heroku depends on what you were using for. Many people had used it to host backend services for small hobby apps that didn't make any money, or for proof-of-concept ideas so they could evaluate tools for their organizations to later use in production.

If this is what you're looking for, there are a plethora of services which deploy containerized applications to a managed environment and that offer free tiers with different pricing models depending on what databases persistent storage and other services you require.

Depending on what you were using, it's feasible that you could move your application stack there without too many code changes. In fact, both Fly and Render offer guides for migrating from Heroku:

But some other popular alternatives include

And there are also the following newer alternatives:

You could also get quite far by using the serverless functions that now come with (formerly) static hosting providers like Vercel, Netlify, and Gatsby Cloud. There are even complex frameworks like RedwoodJs that have been built on top of these providers.

If you were using Heroku for a Headless CMS for a static frontend, most of the open-source Headless CMS platforms offer their own cloud-based infrastructure which will avoid having to re-platform to a different CMS altogether.

There is however potentially another problem looming...

Stay in Control of your Application Stack

If Heroku can suddenly turn around and kill it's free-for-starters offering, who's to say it won't happen for these other services? Even Vercel and Netlify leverage the raw computing power of cloud providers such as AWS, Azure and Google Cloud, repackaging those services and selling them at a profitable margin.

For that reason, we think the best alternative is to embrace these foundational computing platforms, instead of standing again on the unsteady ground of a reseller.

AWS and other providers have a free tier which applies to most of its services. These are specific allowances for each of their services that you can use before you start incurring fees.

With this and similar offerings from other cloud providers like Google Cloud Platform and Microsoft Azure, you could set up an EC2 instance (similar to a virtual machine), configure it to hibernate in a similar fashion to what Heroku was doing, and move your application to it manually.

This would take a bit of work, either to SSH in and upload your application, or set up a deployment pipeline.

But there's also another opportunity here that we would like to highlight.

Embrace Serverless

If you notice, the limits on EC2 instances for containerized applications are time-based. You get a certain number of free hours in traditional "always-on" computing models. After that you need to pay for them. Even though this is quite a generous offering, you could still leave yourself with a substantial bill if you're not careful.

However, other services are based on the number of invocations. Why not take advantage of these?

Functions as a service with AWS Lambda: 1 million free requests per month
File storage You get 5GB of S3 static file storage (think images and static website hosting)
AWS' no-sql Database, DynamoDB, comes at 25GB of storage for free

The next time you need to build a proof of concept, or hobby app, you could build your solution using a combination of these and host it indefinitely. It wouldn't need special configuration to hibernate since that is it's natural state.

Why not try serverless in your next proof of concept or hobby project?

To do so, you could leverage a framework which allows you to write infrastructure-as-code, such as AWS Amplify, Serverless Stack, the Serverless framework, or Pulumi. There is a bit of an overhead to learning these tools so you will need to research which one is best for your needs.

But there's an alternative...

Webiny: Leverage the Power of Serverless

For a little over two years, we have been building Webiny. It's a Headless CMS you can use for your personal blog ... but it's not only a Headless CMS: it's a fully-featured application framework that you can use to scaffold unique applications that will have access to the existing setup we have built via plugins.

You don't need to reinvent the wheel of how to store files, or how to build a GraphQL API, or how to build an admin interface because we've already done that for you.

And what is more, you get all of that for free when you choose the DynamoDB-only option in your setup. You will eventually pay for the amount of requests, storage and database usage ... but only if you exceed the free tier on AWS.

Once your proof-of-concept has satisfied it's requirements, you can destroy your Webiny instance ... or leave it there for future tinkering!

And if you want a place to write blog articles, Webiny is built to do that out-of-the-box.

Visit our GitHub repo and give us a star today. And please let us know if you've built something with Webiny, the world's only serverless Headless CMS.

Learn How to Use Lambda Layers by Building an Image Resize Function Using Sharp

Swapnil M Mane — Wed, 04 May 2022 08:18:45 +0000

Lambda Layer is a true gem in the Lambda function realm. It provides a convenient way to package your libraries and other dependencies that can be used with the Lambda functions. Our open-source project, Webiny which has 50k+ downloads and 5.5k stars on GitHub uses Lambda layers. It helps us reduce our Lambda function size by 12 MB and enable us to share the image processing library sharp across all the Webiny deployments.

By the time you finish reading this article, you will know how to create and use Lambda layers. As an example, we will create a sharp library Layer and use it in a Lambda function to create thumbnail images.
In this tutorial, we will create a Lambda function that will create a thumbnail image using the sharp library. This library will be packaged into the Lambda function using a Lambda Layer.

The workflow will be like this, for each image file uploaded to an S3 bucket (source bucket), the lambda function will be invoked. It will read the image object from the source S3 bucket, create a thumbnail image, and save it to the target S3 bucket.

Prerequisites

AWS account - To use various AWS services, you will need an AWS account. Please follow this link to create an AWS account.
AWS command line - We will use the AWS command line for creating Lambda Function and Lambda Layer. The AWS CLI installation guide is available here.

Step 1 - Create two S3 buckets

Let’s start by creating two buckets, the source and the target. As mentioned earlier, for each image file uploaded to a source bucket, the Lambda function will create a corresponding thumbnail image and save it in the target S3 bucket.

Please follow these steps to create S3 buckets.

1.) Open the Amazon S3 console.
2.) Create two S3 buckets - source and target.

The target bucket name should be source-bucket-name plus suffixed by -resized . For example, if the source bucket is named mybucket then the target bucket should be named as mybucket-resized. Please follow this naming convention because we will be using this naming logic in the Lambda function.

Step 2 - Create the IAM policy

Now let’s create an IAM policy that specifies the permissions for the Lambda function. The Lambda function must have permission for the following operations:

Get the object from the source S3 bucket.
Put the resized object into the target S3 bucket.
Write logs to Amazon CloudWatch Logs.

To create an IAM policy

1.) Open the Policies page in the IAM console.
2.) Choose Create policy.
3.) Choose the JSON tab, and then paste the following policy. Be sure to replace mybucket with the name of the source bucket that you created previously.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:PutLogEvents",
                "logs:CreateLogGroup",
                "logs:CreateLogStream"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::mybucket/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::mybucket-resized/*"
        }
    ]
}

4.) Choose Next: Tags.
5.) Choose Next: Review.
6.) Under Review policy, for Name, enter AWSLambdaS3Policy.
7.) Choose Create policy.

Step 3 - Create the execution role

Create the execution role that gives your Lambda function permissions to access AWS resources.

To create an execution role

1.) Open the Roles page in the IAM console.
2.) Choose Create role.
3.) Create a role with the following properties:
- Trusted entity type – AWS Service (Allow AWS services like EC2, Lambda, or others to perform actions in this account.)
- Use case - Lambda (Allows Lambda functions to call AWS services on your behalf.)
- Permissions policy – AWSLambdaS3Policy
- Role name – lambda-s3-role

Step 4 - Create a sharp Lambda Layer

Here comes the special part of this tutorial. We will create the sharp Lambda Layer in this section.

You will perform this part of the tutorial on your machine. Please ensure you have npm installed (we will need the npm to install the sharp package).

1.) Create a directory with the name nodejs.
2.) Install the sharp dependency in the nodejs directory by running the following commands:

npm install

SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install --arch=x64 --platform=linux sharp

After this step, you will get the following directory structure:

nodejs
|- node_modules
|- package-lock.json

3.) Create a .zip file archive of the nodejs directory.

zip -r nodejs.zip nodejs

4.) Now let’s publish the Layer. Run the following command to create and publish the Layer:

aws lambda publish-layer-version --layer-name sharp --description "Sharp dependency for image transformation" --zip-file fileb://nodejs.zip --compatible-runtimes "nodejs14.x" --region "us-east-1" --output "json"

After successful execution of the command above, you will see an output similar to this one.

Please note the LayerVersionArn, we will use it while creating the Lambda function.

{
    "Content": {
        "Location": "https://prod-04-2014-layers.s3.us-east-1.amazonaws.com/snapshots/400803493251/sharp-539dd937-e29c-4418-bb52-2089f3945afc?versionId=4AMhQSZjY3uw3WI85Jcbf7j7Wj1x1X9i&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEJz%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQDu5lMXWxGc7ydyD6HbmtJk2tnqLLxfRbCYqP7tAwi8wAIgAt3KpyhdECs9I7a2mSGNPjk1n4fq1BKjA7iOmBxOquUq%2BgMIRBAAGgw3NDk2Nzg5MDI4MzkiDOj9vGzyeEXkicvNUirXA9YvAlIvZPE3WdpgIYp51YCl28ISR4eL93paFfqBC0IguHOz%2BctOG2P5MwSc2Vj10UVBUBE4qTiGXYTbB3Gk4jCzTvgDEN90fXdMqQYWXCd8zDpq6OIcZ59qVTQPIFdjo2JZ8U3Tfu68BkIynFNLIiX4Y2WvBqxpJoFUlrmOcU4%2FtuXhSmiW6E7xyQus%2F%2BTq3A2n7JaJ2XgKOKPyINyOOBD3ZsonHjiZw0Djtjk7IgvwMjWVLb6Gn73c%2BG9bLKsHHW7aUnRaNEYSdhWejPpz1QEUlcOKpAkLI87kAgTMPCsljZMujUgWxozAth76hWebjXoE7OHUFx870jscKhpMH2DfJhaOCOsmyme69jpqc4%2FEN6jT2WGjIG42xlfUnxB90DpNBjkPguXFyghOHFlhR07bp9Rom0oW1XOriN9jnbxSZqyQsqDFwbrua4oKtOcJJrzDnUGmhKAlWPMkwE81wDyxXSB%2BGYlYQb%2F%2BXScqvqsieBtIpN5KOfAlB0VuoN17gi5BFXsP5ePaGeJPZ5MJ9COQtm6nXdvclUc8ENc0xPfw0l0485%2BVWe%2BzME%2FIIB0Sclg8TuyNSNcqrSxVxb7VVZMZfFrmMpXpU2FD1chXDGTv1tOP3x%2BthjDLvtWSBjqlATnNWnlBoAfJJE7C6ZqVOgRZU4EvDCpLchTmHT%2F99LoBvqYUU0qospR1VT4pGKtfBBEuNGgg9%2F8Dkj6l0q7p62mlxwt%2BENpVp2QOczzj6CDCjMg1E6jnsVGNzCoPeYiEEx8q6LBM51EOMcm2sSSVjNeVNRhTFrF84ch%2FCteFF5Ilr%2FTHW7pb9pJCaOOuJ%2FMZIpXy2w9CtOXxbO8%2BkG4kHHDarz8M9w%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220412T121324Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=ASIA25DCYHY3S4VTGYGI%2F20220412%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=d84c5f4e44cf6473e0a924f3a5fd57aea32c00b17480af1551a869e41693b030",
        "CodeSha256": "f9IchJrdt01ftKIlcSZ25z6ZRbf780hO8xVYCd6Uxhs=",
        "CodeSize": 8748397
    },
    "LayerArn": "arn:aws:lambda:us-east-1:400803493251:layer:sharp",
    "LayerVersionArn": "arn:aws:lambda:us-east-1:400803493251:layer:sharp:1",
    "Description": "Sharp dependency for image transformation",
    "CreatedDate": "2022-04-12T12:13:27.990+0000",
    "Version": 4,
    "CompatibleRuntimes": [
        "nodejs14.x"
    ]
}

Step 5 - Create Lambda function

Now let’s create a Lambda function that will use the sharp Lambda Layer. As discussed earlier, this Lambda function will be invoked when an object is created in the source bucket. Then it will create the respective thumbnail image and store it in the target bucket. The code example below assumes that you are following the S3 bucket name convention that is mentioned in Step-1.

Step 5.1 Create the function code

Copy the following code example into a file named index.js.

// dependencies
const AWS = require('aws-sdk');
const util = require('util');
const sharp = require('sharp');

// get reference to S3 client
const s3 = new AWS.S3();

exports.handler = async (event, context, callback) => {

  // Read options from the event parameter.
  console.log("Reading options from event:\n", util.inspect(event, {depth: 5}));
  const srcBucket = event.Records[0].s3.bucket.name;
  // Object key may have spaces or unicode non-ASCII characters.
  const srcKey    = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));
  const dstBucket = srcBucket + "-resized";
  const dstKey    = "resized-" + srcKey;

  // Infer the image type from the file suffix.
  const typeMatch = srcKey.match(/\.([^.]*)$/);
  if (!typeMatch) {
      console.log("Could not determine the image type.");
      return;
  }

  // Check that the image type is supported
  const imageType = typeMatch[1].toLowerCase();
  if (imageType != "jpg" && imageType != "png") {
      console.log(`Unsupported image type: ${imageType}`);
      return;
  }

  // Download the image from the S3 source bucket.

  try {
      const params = {
          Bucket: srcBucket,
          Key: srcKey
      };
      var origimage = await s3.getObject(params).promise();

  } catch (error) {
      console.log(error);
      return;
  }

  // set thumbnail width. Resize will set the height automatically to maintain aspect ratio.
  const width  = 200;

  // Use the sharp module to resize the image and save in a buffer.
  try {
      var buffer = await sharp(origimage.Body).resize(width).toBuffer();

  } catch (error) {
      console.log(error);
      return;
  }

  // Upload the thumbnail image to the destination bucket
  try {
      const destparams = {
          Bucket: dstBucket,
          Key: dstKey,
          Body: buffer,
          ContentType: "image"
      };

      const putResult = await s3.putObject(destparams).promise();

  } catch (error) {
      console.log(error);
      return;
  }

  console.log('Successfully resized ' + srcBucket + '/' + srcKey +
      ' and uploaded to ' + dstBucket + '/' + dstKey);
};

Step 5.2 Create the deployment package

The deployment package is a .zip file archive containing your Lambda function code and its dependencies.

1.) Open a command-line terminal or shell in a Linux environment.
2.) Save the function code as index.js in a directory named lambda-s3.
After this step, you have the following directory structure:

lambda-s3
|- index.js

3.) Create a deployment package with the function code and its dependencies. Set the -r (recursive) option for the zip command to compress the subfolders.
Run this command from the lambda-s3 directory.

zip -r function.zip .

Step 5.3 Create a Lambda function

We will use the AWS CLI command (aws lambda create-function) to create the Lambda function.

Please replace the role parameter 123456789012with your AWS account ID and layers ARN with the one we got in Step 4 (LayerVersionArn).

aws lambda create-function --function-name CreateThumbnail \
--zip-file fileb://function.zip --handler index.handler --runtime nodejs14.x \
--timeout 30 --memory-size 1024 \
--role arn:aws:iam::123456789012:role/lambda-s3-role \
--region "us-east-1" \
--layers "arn:aws:lambda:us-east-1:400803493251:layer:sharp:1"

With the successful execution of the above command, you will see an output similar to this:

{
    "FunctionName": "CreateThumbnail",
    "FunctionArn": "arn:aws:lambda:us-east-1:400803493251:function:CreateThumbnail",
    "Runtime": "nodejs14.x",
    "Role": "arn:aws:iam::400803493251:role/lambda-s3-role",
    "Handler": "index.handler",
    "CodeSize": 1061,
    "Description": "",
    "Timeout": 30,
    "MemorySize": 1024,
    "LastModified": "2022-04-12T12:22:02.432+0000",
    "CodeSha256": "y0QVpF6+WN3wH1Zp+sbpbYlQEeAdkBuSf7p5aJV52Sc=",
    "Version": "$LATEST",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "64dc7cab-9adf-4472-b20b-e15237209939",
    "Layers": [
        {
            "Arn": "arn:aws:lambda:us-east-1:400803493251:layer:sharp:4",
            "CodeSize": 8748397
        }
    ],
    "State": "Pending",
    "StateReason": "The function is being created.",
    "StateReasonCode": "Creating",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ],
    "EphemeralStorage": {
        "Size": 512
    }
}

Step 6 - Configure Amazon S3 Trigger

So far we have created all the building blocks. Now let's add the final piece where we will configure the Amazon S3 Trigger to invoke the Lambda function.
When a file is uploaded to S3, it should trigger the Lambda function that will resize, create a thumbnail, and upload it to the target S3 bucket.

To create an Amazon S3 Trigger

1.) Open the Lambda page in the AWS console.
2.) Open the CreateThumbnail function.
3.) Choose Add trigger under the Function overview section
4.) Create a trigger with the following properties:
- Select a trigger – S3
- Bucket - mybucket (your source bucket name)
- Event type – PUT

Step 7 - All set, let’s test it!

Nice, we are all set with our Lambda function, which uses the sharp Layer and creates thumbnail images. Please upload an image (jpg or png) to the source S3 bucket. Once the upload is complete, you should be able to see the thumbnail version of the image in the target S3 bucket.

Lambda Layer is a very efficient and convenient way to package libraries and other dependencies that you can use with your Lambda functions. It offers great benefits in reducing the deployment archives size and faster deployment. layers enable code sharing and separation of responsibilities to create maintainable Lambda functions.

In case you want to explore and learn more about best coding practices and code examples related to Lambda, you can check out the Webiny code repository. Webiny is an open-source enterprise-grade serverless CMS.

If you have any questions related to this tutorial or have any feedback, please feel free to ping us on the Community Slack!

Credits

As this tutorial aims to explain and demonstrate the use of Lambda Layer, we didn't reinvent the wheel and write a new Lambda function to create thumbnail images. The Lambda function code used in this article to create thumbnail images is taken from this official AWS Lambda tutorial. Huge thanks to the authors of this great tutorial on the Lambda function. 🙏🏻

Using Webiny Headless CMS With Our New Starter Kits for Gatsby and NextJS

Ben Read — Fri, 22 Apr 2022 13:59:46 +0000

There's a hidden cost that comes with the majority of self-hosted headless content management systems: the time you must spend initially setting up the project.

Some require you to install Docker and run its cryptic terminal commands. Others ask you to write code to set up content models. With some you have to install and manage MongoDB or SQL servers on your local machine which may keep running in the background, eating valuable resources even when you're not using your CMS.

By contrast, Webiny have always tried to make the experience of getting started with our Headless CMS as easy and quick as possible.

Now it's easier than ever, especially if you're a fan of Gatsby or NextJS, two of the most popular React site builder frameworks.

Official Gatsby Starter for Webiny Headless CMS

Gatsby revolutionised the way frontend development was done. It wasn't the first static site generator, but it was the first to bring React and JavaScript as first-class browser tools along with a unified data layer for generating pages at build time from remote data by using GraphQL schema stitching.

When you partner Gatsby with Webiny, it's an ideal combination! You can spin up a Webiny instance on your own cloud infrastructure in minutes (instead of spending hours wrangling with Docker, MongoDB or other code-specific setup), define your own models and write content in the browser, and pair it with your Gatsby site at the click of a single button.

Read the Documentation or go directly to the Gatsby code for a 1-click deploy button to Gatsby Cloud.

Official NextJS Starter for Webiny Headless CMS

NextJS has revolutionised web development because of it's flexibility. It implements the React framework, along with a custom router, server-side data fetching and other tools, but is unopinionated and allows you to form your code however you wish. For that reason, it's been the tool of choice for organisations and individuals worldwide.

Now it's even easier to get started with Webiny CMS and Next. Install Webiny on your own cloud infrastructure in minutes, which will avoid any issues with setting up MongoDB / Docker or some other tool on your machine, or writing code to generate content models.

Once you're ready, fetch the content on your NextJS site in minutes. Take a look at the documentation or view the starter for a 1-click deploy button to Vercel.

What Starter Should We Build Next?

We don't want to stop there!

Do you like the view from Vue? Do you like to keep your apps svelte with Svelte? Or perhaps it's time to put Remix into the mix.

Please tell us by tweeting to us or via our Community Slack channel what starter you would like to see next!

We hope you enjoy using these tools to get started quickly with Webiny Headless CMS. Please join our community if you get stuck, or to give us feedback you'd like to offer.

Learn About Infrastructure as Code in 5 Minutes and Why You Should Use It

Swapnil M Mane — Fri, 21 Jan 2022 09:15:06 +0000

In the past, IT infrastructure management was challenging. Hardware and software had to be managed and configured manually. Recently, a tool, that allows you to simplify and automate these actions, has appeared. This tool is called infrastructure as code (IaC).

In this article, we will discuss what infrastructure as code is, the best providers that allow you to implement IaC, IaC best practices, examples, and possible challenges.

Infrastructure as code declaration

Infrastructure as Code (IaC) is the process of managing and configuring an infrastructure using configuration files, rather than manually editing configurations on servers or using interactive configuration tools. The IaC model creates the same environment every time it is applied. IaC is used in infrastructure as a service (IaaS), but they should not be confused. The main idea of IaC is to describe the infrastructure using code.

Why it is needed

Without IaC, teams must maintain customization of individual deployment environments. Over time, this leads to each environment having a unique configuration that cannot be reproduced automatically, which in turn leads to problems during deployment. In such a case, infrastructure administration and maintenance involve manual processes that are difficult to track and which can lead to errors. IaC allows you to always set the same configuration for the target environment, regardless of the initial state of the environment.

The best IaC providers

Let’s take a look at the most popular IaC providers and the main features they provide.

Amazon AWS Cloudformation

AWS CloudFormation is a built-in IaC tool in the AWS Cloud Platform that provides the ability to provision and manage a set of multiple related AWS and third-party resources. CloudFormation lets you manage your resource scaling, automate resource management by integrating CloudFormation with other required AWS resources, and create your own resource providers to provide and manage third-party application resources.

Azure Resource Manager

Azure Resource Manager is an IaC tool for deploying and managing Azure resources. It allows you to deploy, maintain, and track resources as a collective action rather than separately. It incorporates role-based access control into the resource control mechanism to enable users to enforce access control over all resources in a resource category.

Google Cloud Deployment Manager

Google Cloud Deployment Manager is an infrastructure deployment service designed to automate the creation, configuration, provision, and management of assets on the Google Cloud Platform. It allows you to easily create a collection of Google cloud assets and then manage them as a whole. Google Cloud Deployment Manager enables multiple resources to be deployed concurrently, controls the order in which resources are created, and creates resource definition dependencies.

Terraform

HashiCorp Terraform is the most popular open-source infrastructure automation tool. terraform allows you to easily plan and build IaC for multiple infrastructure providers with a single workflow. You can easily provision different environments using the same configuration and manage the complete lifecycle of the infrastructure you want; thereby reducing human error and increasing the automation of infrastructure provision and management.

AWS CDK

The AWS CDK is a software development platform for defining cloud infrastructure in code and exposing it using AWS CloudFormation. It allows you to define the framework and application stack in programming languages such as Python, Java, Typescript, .NET, and JavaScript, and deploys a full-featured application stack directly from a supported IDE.

AWS CDK allows you to automatically create a CloudFormation template based on your source code and deploy this template to your AWS account.

Pulumi

Pulumi is an open-source multilingual development platform that lets you manage your cloud infrastructure with code. It provides any cloud deployment functionality, cloud management. Pulumi offers the freedom to use any supported programming language you like. With Pulumi, you can view all existing environments and the changes you make to them. In addition, you can easily customize their configurations to protect your data.

IaC examples

Let’s look at a simple scenario for provisioning an AWS EC2 instance. In the following example, we can see how Terraform code is used for this requirement.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }
}

provider "aws" {
  access_key = "aws_access_key"
  secret_key = "aws_secret_key"
  region = "us-west-1"
}

resource "aws_instance" "web_server" {
  ami                    = "ami-0123456"
  instance_type          = "t3.small"
  subnet_id              = "subnet-a000111x"
  vpc_security_group_ids = "sg-dfdd00011"
  key_name               = "web_server_test_key"

  tags = {
    Name = "Web_Server"
  }
}

IaC best practices

Some of the best IaC practices to consider when developing it are listed below.

Version control

Configuration files should be versioned so that you can easily track, manage, and recover any potential changes. Version control is also very helpful in diagnosing problems.

Document as little as possible

When using IaC, the code is the infrastructure documentation and will always be up-to-date, so your employees don’t need to write a lot of additional instructions.

Modularity and microservices

IaC can be used to develop, manage and maintain applications as many deployable modules. It also allows you to split your infrastructure into multiple modular components and then use automation to combine them in the most appropriate way.

IaC Compliance Regulation

Errors in IaC files can be a serious problem if they are not detected prior to deploying IaC definitions. Therefore, it is recommended to automatically and continuously scan IaC files, ensuring that verification occurs whenever an IaC definition is created or updated. You can do it using such tools as Checkov, TFLint, Accurics.

IaC challenges

While there is a list of IaC advantages, there are certain issues with this model that need to be understood and solved before the IaC implementation process.

Shift of configuration

In the long run, a configuration shift can occur, no matter how consistently or often you configure your servers. Therefore, every time you need to change your infrastructure, you must ensure that it is done in accordance with a predefined maintenance workflow.

Potential duplication of errors

In the process of implementing IaC, there are certain processes that must be done manually. One of these processes is writing the code that generates your IaC architecture, during which there is always the possibility of error.

Need for new skills

The IaC model requires a high level of technical expertise, so teams must continually invest in people and skills. For some teams, this may be too costly.

Conclusion

IaC fully frees developers and other professionals from manual, error-prone tasks. In addition, it lowers costs and increases efficiency throughout the entire software development lifecycle.
If you have questions or comments, please reach out via Twitter or Webiny Community Slack.

Introducing the new Object field 🚀

Swapnil M Mane — Thu, 29 Jul 2021 11:20:41 +0000

⚡️Seamlessly create nested and repeatable data structures with the new Object field.

With the latest Webiny 5.11.0 release, we are glad to introduce the new Object field in Headless CMS.

The nested and repeatable data structures was one of the hot topic discussed in the Webiny community. With the new Object field now, you can create complex data structures with ease. Huge thanks to our community members for their suggestions and contributions.

Let's quickly dive into the highlighting capabilities of the Object field.

Build nested data structures

The new Object field allows you to build nested data structures. You can create an object and add various existing fields (ref, richText field, etc.) to it. Along with this, you can also nest objects within other objects.

Repeatable structure

The icing on the cake is, the data structure created with Object fields can be repeatable (like an array of nested objects).

The Use as a repeatable object property allows creating the repeatable object. E.g. setting the 'Use as a repeatable object' property to true on the Equipment model will make it multi-valued.

Below is the model view for the multi-valued Equipment field.

And the preview of the model will look like this:

GraphQL Schema auto-complete support

Object field respects all the rules and behaviors of all the other existing fields (ref, richText field, etc.). You also get a full GraphQL Schema auto-complete on your nested fields.

Alright, here is the demo in action!

Try the new Headless CMS - Object field and please let us know your valuable feedback!

5 Things to Avoid When Writing Cypress Tests

Swapnil M Mane — Mon, 26 Jul 2021 09:15:05 +0000

When it comes to testing an application, End-to-End (E2E) testing provides the most confidence and the most bang for the buck.

On the contrary, there is no doubt that End-to-End testing is hard, time-consuming, and comes with a bag of issues to solve. But, only if you're using the wrong tool for the job.

Enter Cypress: Fast, easy, and reliable testing for anything that runs in a browser.

Cypress helps in solving most of the pain points of End-to-End testing and makes it fun to write tests.
But, there are certain mistakes to be avoided so that you can get the full benefit of working with Cypress.

In this blog post, we'll cover 5 such common mistakes, which should be avoided when writing Cypress tests.
So, without further ado, let's begin!

Use `id` and `class` For Selecting Element

Using id and class for selecting element is problematic because they are primarily for behavior and styling purposes, due to which they are subject to change frequently.
Doing so result in brittle tests which, you probably don't want.

Instead, you should always try to use data-cy or data-test-id.
Why? Because they are specifically for testing purposes, which makes them decoupled to the behavior or styling, hence more reliable.

For example, let's suppose we have an input element:

<input
  id="main"
  type="text"
  class="input-box"
  name="name"
  data-testid="name"
/>

Instead of using id or class to target this element for test, use data-testid:

// Don't ❌
cy.get("#main").something();
cy.get(".input-box").something();

// Do ☑️
cy.get("[data-testid=name]").something();

What about using text for selecting element?

Sometimes it is necessary to use text such as button label to make an assertion or action.
Although, it's perfectly fine, keep in mind that, your test will fail if the text changes, which is what you might want if the text is critical for the application.

Treating Cypress Commands as Promise

Cypress tests are composed of Cypress commands, for example, cy.get and cy.visit.
Cypress commands are like Promise, but they are not real Promise.

What that means is, we can't use syntax like async-await while working with them. For example:

    // This won't work
    const element = await cy.get("[data-testid=element]");

    // Do something with element

If you need to do something after a command has been completed, you can do so with the help of the cy.then command.
It will guarantee that only after the previous command finishes, the next will run.

    // This works
    cy.get("[data-testid=element]").then($el => {
        // Do something with $el
    });

Note when using a clause like Promise.all with Cypress command, it might not work as you expect because Cypress commands are like Promise, but not real Promise.

Using Arbitrary Waits in Cypress Tests

When writing the Cypress test we want to mimic the behavior of a real user in real-world scenarios.
Real-world applications are asynchronous and slow due to things like network latency and device limitations.

When writing tests for such applications we are tempted to use arbitrary values in the cy.wait command.
The problem with this approach is that, while it works fine in development, it is not guaranteed. Why? Because the underlying system depends upon things like network requests which are asynchronous and nearly impossible to predict.

    // Might work (sometimes) 🤷
    cy.get("[data-testid=element]").performSomeAsyncAction();
    // Wait for 1000 ms
    cy.wait(1000);
    // Do something else after the action is completed

Instead, we should wait for visual elements, for example, completion of loading. Not only does it mimic the real-world use case more closely, but it also gives more reliable results.
Think about it, a user using your application mostly likely wait for a visual clue like loading to determine the completion of an action rather than arbitrary time.

    // The right way ☑️
    cy.get("[data-testid=element]").performSomeAsyncAction();
    // Wait for loading to finish
    cy.get("[data-testid=loader]").should("not.be.visible");
    // Now that we know previous action has been completed; move ahead

Cypress commands, for example, cy.get wait for the element before making the assertion, of course for a predefined timeout value which you can modify.
The cool thing about timeout is that they will only wait until the condition is met rather than waiting for the complete duration like the cy.wait command.

Using Different Domains within a Cypress Test

One limitation of Cypress is that it doesn't allow using more than one domain name in a single test.

If you try using more than one domain in a single test block it(...) or test(...), Cypress will throw a security warning.
This is the way Cypress has been built.

With that being said, sometimes there is a requirement to visit more than one domain in a single test. We can do so by splitting our test logic into multiple test blocks within a single test file. You can think of it as a multi-step test, for example,

describe("Test Page Builder", () => {
    it("Step 1: Visit Admin app and do something", {
        // ...
    });

    it("Step 2: Visit Website app and assert something", {
        // ...
    });
});

We use a similar approach at Webiny for testing the Page Builder application.

Few things to keep in mind when writing tests in such a manner are:

You cannot rely on persistent storage be it variable in test block or even local storage.
Why? Because, when we issue a Cypress command with a domain other than the baseURL defined in the configuration, Cypress performs a tear-down and does a full reload.
Blocks like "before", "after" will be run for each such test block because of the same issue mentioned above.

Be mindful of these issues before adapting this approach and adjust the tests accordingly.

Mixing Async and Sync Code

Cypress commands are asynchronous, and they don't return a value but yield it.

When we run Cypress it won't execute the commands immediately but read them serially and queue them.
Only after it executes them one by one. So, if you write your tests mixing async and sync code, you will get the wrong results.
For example:

it("does not work as we expect", () => {
  cy.visit("your-application") // Nothing happens yet

  cy.get("[data-testid=submit]") // Still nothing happening
    .click() // Nope, nothing

  // Something synchronous
  let el = Cypress.$("title") // evaluates immediately as []

  if (el.length) {
    // It will never run because "el.length" will immediately evaluates as 0
    cy.get(".another-selector")
  } else {
    /*
    * This code block will always run because "el.length" is 0 when the code executes
    */
    cy.get(".optional-selector")
  }
})

Instead, use our good friend cy.then command to run code after the command has been completed. For example,

it("does work as we expect", () => {
  cy.visit("your-application") // Nothing happens yet

  cy.get("[data-testid=submit]") // Still nothing happening
    .click() // Nope, nothing
    .then(() => {
      // placing this code inside the .then() ensures
      // it runs after the cypress commands 'execute'
      let el = Cypress.$(".new-el") // evaluates after .then()

      if (el.length) {
        cy.get(".another-selector")
      } else {
        cy.get(".optional-selector")
      }
    })
})

Conclusion

Cypress is a powerful tool for End-to-End testing, but sometimes we make few mistakes which makes the experience not fun.
By avoiding the common mistakes we can make the journey of End-to-End testing smooth and fun.

Webiny 5.0 Release

Sven A — Fri, 12 Mar 2021 13:22:17 +0000

It took us ~6 months, 179 calls with users and 3676 commits to ship this release. It is a release where all the changes are 100% driven by community feedback.

Here are the main highlights:

🚀 Drastic performance improvements (benchmarks included)
🔐 Default VPC and private VPC deployment are now supported
🗄 DynamoDB and Elasticsearch support
🏗 Page Builder overhaul - way faster and more stable than before
💻 Improved developer experience
🧱 IaC implementation via Pulumi

Check out the blog post for full details.

To get started with Webiny 5.0 (full tutorial)

In Other News

Webiny is proud to announce that we have received funding from Y Combinator and have joined their latest batch of companies.

The Complete Beginners Guide to AWS S3

Albiona — Mon, 25 Jan 2021 18:18:34 +0000

When developing your web application, you must think of a place where to store your data, how to back up them, types of data you want to store, such as images, music, and videos, application hosting, data archiving, disaster recoveries. AWS Simple Storage Service (S3) provides you the solutions for these cases. S3 is one of the core services of AWS cloud infrastructure. It's object storage that acts like a regular file system on your personal computer. S3 scales infinitely, with no limit on the amount of data you store.

In this tutorial, we'll get to learn how to use the AWS S3. First, learn what is S3, the core parts of S3 that are the Buckets, Access Point, and Objects. Then we'll get to the practice, by implementing the AWS SDK for Node.js 💻

Finally, we'll provide a cheat sheet on AWS S3 CMD Commands.

Amazon Web Services (AWS) provides multiple types of cloud computing services, one of them is the AWS Storage Service. There are different storage services, such as Simple Storage Service (S3), AWS Elastic File System (EFS), & Elastic Block Store (EBS). For this tutorial, we'll be focusing on the S3 service. S3 is one of the most favorite cloud computing services among all the other services, based on this poll by acloud.guru on Twitter.

What is S3?

You can use S3 to host your static websites, delivering HTML, JavaScript, images, videos, and other files to your website visitors - that doesn't contain server-side code such as Node.js or PHP. Using S3, you can easily deploy your applications in just two to three clicks via the user interface. S3 provides a simple web services interface you can use to store and retrieve any amount of data from anywhere on the web.

We'll now go through the core concepts of S3, such as buckets, access points, and objects.

Buckets

To upload your data to S3, you must create an S3 bucket in one of the AWS Regions, within one bucket you can upload many objects to the bucket. For implementation, buckets and objects are resources, and S3 provides APIs for you to manage them. There are different methods you can use to create buckets such as.

Amazon S3 Console

Follow the guides here to create your first bucket with the S3 console.

REST API

To create buckets using REST API, you must authenticate your requests — follow the PUT Bucket in the S3 API reference. But it's recommended to use the AWS Management Console or AWS SDKs instead.

AWS SDK

To create buckets with the SDK, you first have to create a client and then use the client to send a request to create a bucket. Note: When creating the client and the bucket, use the same region. Here is a dominant source on creating and using AWS S3 Buckets.

Access points

To access the data that you store on S3, you need the S3 Access Point. These are endpoints that are attached to buckets that you used to perform S3 object operations.

Each access point has distinct permissions and network controls S3 applies for any request that is made through the access point. Access points are used to perform operations on objects, but not on buckets. Go through this source to learn how to manage data access with S3 access points.

Objects

We mentioned that AWS S3 is object storage. Each AWS S3 object has data, a key, and metadata. The object key (or key name) uniquely identifies the object in a bucket. Object metadata is a set of name-value pairs.

You can store objects in one or more buckets, and each object can be up to 5 TB in size. For the real-world solutions, let's say you want to share an image or video stored in AWS S3 bucket on your website, that is possible only if you make the object public or use a pre-signed URL on your website. Follow this source on how to work with S3 objects.

Hands-on S3 with AWS SDK

We'll go with the AWS SDK and Node.js to create S3 buckets, uploading an object to a specified bucket and deleting that bucket afterward; we'll provide a How-To on the S3 section where you can learn more about different use-cases commands to run on S3.

In order to continue, you must:

Install Node.js, in case you don't have it, follow the Node.js website.
Set up your user credentials, follow here for more information.

Configuring the SDK

To use the AWS SDK for JavaScript, we must first initiate a node module for our hands-on project.

To do that, first, create a folder named webiny-hands-on-s3 and cd into that folder.

Run the command npm init — this will ask you to provide the project's name and you can name it as you want, in this case, we'll leave it the same as the folder name with an entry point of the index.js file.

Inside our folder, we'll create a couple of files, such as.

createBucket.js
upload.js
listObjects.js
deleteBucket.js

Before we continue to implement any code, we need to install the AWS SDK package by running this command: npm install aws-sdk --save.

Let's dive in.

Creating an AWS S3 Bucket

In order to use the SDK, we have to configure the SDK package by loading it into our file. Open the createBucket.js file, and start writing the below code in your own file. In the snippet we have comments to explain to you what we're doing:

// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");

// Set the region
AWS.config.update({ region: "us-east-1" });

// Create S3 service object
s3 = new AWS.S3({ apiVersion: "2006-03-01" });

// Create the parameters for calling createBucket -- with this part we'll take the bucket name we'll create
var bucketParams = {
  Bucket: process.argv[2],
};

// Call S3 to create the buckets
s3.createBucket(bucketParams, function(err, data) {
  err ? console.log("Error", err) : console.log("Success", data.Location);
});

To create your S3 bucket, with this script, we must run it and give our bucket a name like so: node createBucket.js webiny-s3-bucket-testing. If it's successful, it will console log the Success message together with the location, which will be the bucket name.

Uploading a file to an AWS S3 Bucket

Open the upload.js file and let's dive in. I will repeat some parts of the snippets, such as the SDK configuration, and the AWS S3 service object as shown in the above snippet. The additional part is that we have two command-line arguments, the first one will be the bucket name where you'll upload your file, and the second argument will be the file itself.

Let's dive into the code.

// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");

// Set the region
AWS.config.update({ region: "us-east-1" });

// Create S3 service object
s3 = new AWS.S3({ apiVersion: "2006-03-01" });

// Call S3 to retrieve upload file to specified bucket
var uploadParams = { Bucket: process.argv[2], Key: "", Body: "" };
var file = process.argv[3];

// Configure the file stream and obtain the upload parameters
// The node.js file system module allows you to work (read, create, update, delete, rename files)
// with the file system on your computer.
var fs = require("fs");
var readingFile = fs.createReadStream(file);
readingFile.on("error", function(err) {
  console.log("File Error", err);
});

uploadParams.Body = readingFile;
// The path module provides utilities for working with file and directory paths.
// We can access by using this:
var path = require("path");
uploadParams.Key = path.basename(file);

// Call S3 to retrieve upload file to specified bucket
s3.upload(uploadParams, function(err, data) {
  err ? console.log("Error", err) : console.log("Upload Success!", data.Location);
});

Now, create an index.txt file in your folder, and add some text to it.

Then, run the script by providing two parameters:

The S3 bucket we create in the first snippet webiny-s3-bucket-testing
The local file you want to upload into that S3 bucket.

The full command node upload.js webiny-s3-bucket-testing index.txt

🎉 You just uploaded a file into your S3 bucket!

Listing objects in an AWS S3 Bucket

Now, open the listObjects.js file where you'll list the content of this bucket. It's again a repetitive task of configuring the SDK and creating the AWS S3 service object. What this script will do, is that we'll provide the bucket name from which we want to read the objects and the result will be a list of objects (files) or a failure message.

Let's dive into the code.

// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");

// Set the region
AWS.config.update({ region: "us-east-1" });

// Create S3 service object
s3 = new AWS.S3({ apiVersion: "2006-03-01" });

// Create the parameters for calling listObjects method
var bucketParams = {
  // in here we'll provide the bucket name we created earlier
  Bucket: "webiny-s3-bucket-testing",
};

// Call S3 to obtain a list of the objects in the bucket

s3.listObjects(bucketParams, function(err, data) {
  err ? console.log("Error", err) : console.log("Success", data);
});

Now, let's run the script by running this command: node listObjects.js Check out the result 😄

Deleting an AWS S3 Bucket

Move to the deleteBucket.js file, and configure the SDK and create the AWS S3 service object. What you'll use in this script, comparing with the above script, is the deleteBucket method. But, this case is different, we previously added objects into our bucket, right? We can't delete the AWS S3 buckets if they're not empty. That means you need to delete the objects inside the bucket first, then delete the bucket.

Let's dive in.

// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");

// Set the region
AWS.config.update({ region: "us-east-1" });

// Create S3 service object
s3 = new AWS.S3({ apiVersion: "2006-03-01" });

// Create params for S3.deleteBucket
var bucketParams = {
  // here you'll provide the name of the bucket you want to delete
  Bucket: "webiny-s3-bucket-testing",
};

// We'll first empty the bucket
async function emptyS3Bucket(bucket) {
  const listParams = {
    Bucket: bucket,
    // Prefix: dir,
  };

  const listedObjects = await s3.listObjectsV2(listParams).promise();

  if (listedObjects.Contents.length === 0) return;

  const deleteParams = {
    Bucket: bucket,
    Delete: { Objects: [] },
  };

  listedObjects.Contents.forEach(({ Key }) => {
    deleteParams.Delete.Objects.push({ Key });
  });

  await s3.deleteObjects(deleteParams).promise();

  if (listedObjects.IsTruncated) await emptyS3Bucket(bucket);
}

emptyS3Directory(bucketParams.Bucket);

// Call S3 to delete the bucket
s3.deleteBucket(bucketParams, function(err, data) {
  err ? console.log("Error", err) : console.log("Success", data);
});

`How-to` on S3 - Cheatsheet

Download an entire AWS S3 bucket

1.1 Install the AWS Command Line Tools

sudo easy_install awscli
// or
sudo pip install awscli
// or
brew install awscli

1.2 Run these commands:

aws s3 sync s3://<source_bucket> <local_destination>

Example: aws s3 sync s3://mybucket. Will download all the objects in mybucket to the current directory. And will output: download: s3://mybucket/test.txt to test.txt

Are AWS S3 buckets region-specific?

The user interface shows all your buckets, in all regions. But buckets exist in a specific region and you need to specify that region when you create a bucket.

How to Configure SSL for AWS S3 bucket?

Example:[https://s3.amazonaws.com/bucket_name/images/logo.gif](https://s3.amazonaws.com/bucket_name/images/logo.gif)

If you use a custom domain for your bucket, you can use S3 and CloudFront together with your own SSL certificate (or generate a free one via Amazon Certificate Manager): Amazon CloudFront Custom SSL

Delete AWS S3 buckets

aws s3 rb s3://bucket-name

By default, the bucket must be empty for the operation to succeed. To remove a bucket that's not empty, you need to include the --force option.

aws s3 rb s3://bucket-name --force

Rename AWS S3 Bucket name

There is no rename bucket functionality for S3 because there are technically no folders in S3, so we have to handle every file within the bucket.

aws s3 mb s3://[new-bucket] // 1. Create a new bucket
aws s3 sync s3://[old-bucket] s3://[new-bucket] // 2. Copy files over
aws s3 rb --force s3://[old-bucket] // 3. Delete the old bucket

Quick way to list all files in AWS S3 bucket

aws s3 ls

AWS S3 copy files and folders between two buckets

aws s3 sync s3://DOC-EXAMPLE-BUCKET-SOURCE s3://DOC-EXAMPLE-BUCKET-TARGET

Is it better to have multiple s3 buckets or one bucket with sub folders?
By default, you can create up to 100 buckets in each of your AWS accounts. If you need additional buckets, you can increase your bucket limit by submitting a service limit increase. Source.
The total volume of data and the number of objects you can store are unlimited. Source.

*Object Storage — Also known as object-based storage, is a strategy that manages and manipulates data storage as distinct units, called objects. There are three key components of an object — the content of the object (data stored in the object such as a file or directory), the unique object identifier (ID), and metadata. It stores the metadata as key-pair values and contains information such as name, size, date, security attributes, content type, and URL. Each object has an access control list (ACL) to configure who may access the object.

Now that you've used AWS SDK for S3, you're able to code the solutions that the AWS S3 Console provides via a few clicks, which is faster but, using the SDK you'll be able to continue developing your applications using the AWS services directly by coding. This is a significant advantage for those interested in building applications using AWS services. In this tutorial, we used the AWS SDK to create buckets, upload data, listing data from the buckets, empty, and afterward deleting buckets via AWS SDK for JavaScript for Node.js.

If you learned something new today and are interested to follow up on our blogs, subscribe to our newsletter and we'll provide you the best content of the serverless world!

Thanks for reading! My name is Albiona and I work as a developer relations engineer at Webiny. I enjoy learning new tech and building communities around them = ) If you have questions or just want to say hi, reach out to me via Twitter.

Get Started with Cloud Computing

Albiona — Tue, 19 Jan 2021 14:21:04 +0000

Nowadays, when working on software systems, you have heard about Cloud Computing technology. Understanding cloud fundamentals is an essential part when getting started with Cloud Computing. Amazon Web Services (AWS) is one of the cloud providers that offer over 300 services. Before starting with AWS, understand cloud concepts such as what is cloud computing, the advantages of using cloud computing, cloud computing types, and cloud deployment models.

In this article, we'll cover the concepts you need to build that solid foundation to implement Cloud Computing technologies.

What is Cloud Computing

Cloud computing gives you the option to transform IT infrastructure into a utility - which means you can get into your IT infrastructure via the internet, using computing resources on the cloud. Without the need to install or maintain these resources on-premises.

What is on-premise you might ask? Going with the on-premise way, you must manage your own servers, hire IT people, payor rent the real-estate for the servers.

The opposite of on-premise is the cloud providers, who provide you these services via the internet. The cloud providers are companies who own the servers, hire IT people, pay and manage the real estate for the servers. Those companies give you the option to manage the configuration of the cloud services you use, and your code and they take care of the rest.

Simply put, Cloud Computing offers you on-demand access, through the internet to cloud computing resources, applications, servers (physical and virtual servers), data storage, development tools, networking capabilities, and more — hosted at remote data centers that are managed by cloud service providers.

Advantages of Using Cloud Computing

We learned what on-premise IT is, the traditional way of managing IT infrastructure. Based on the differences between on-premise and cloud computing provider solutions, there are some obvious benefits you get when using cloud computing resources.

Lower total cost of ownership (TCO)

For all the attributes of having your on-premise infrastructure that we mentioned such as managing your own servers, hiring the IT people, paying or renting the real-estate for the servers, those all come up with a price, and with the cloud computing technology, you eliminate the capital expenses.

Agility

In just a few clicks you're able to get up and running the computing resources you need for your applications — As Webiny's CEO Sven noted on the Serverless Chats podcast by answering Jeremy's question "What are the benefits of serverless for both small and big companies?" - "When using serverless, the cost of managing infrastructure will go way down, by releasing a big chunk of the budget or resources or working hours, that you can now focus on product iterations, so your product can grow faster. And if your product grows faster, you can out-innovate potential competitors, which can't afford that same level of innovation."

Scale and Performance

The two most important benefits you get when switching to cloud computing technologies is when you want your customers to have an amazing user experience using your product, anywhere - that means in the lowest or highest spikes of your product's traffic. Cloud computing services run on a worldwide network, so they're able to provide you the ability to scale and deliver the resources when they're needed.

Types of Cloud Computing Services

The cloud computing services fall into four different categories: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Serverless, and Software as a Service (SaaS). They are all built up on top of one another. We'll go through them to understand what they are and the differences between them.

Infrastructure as a Service (IaaS)

IaaS represent the building blocks for cloud IT by providing access to networking features, computers, and data storage space. You can use those resources to build your own products. These are offered by different cloud provider companies on a pay-as-you-go basis. Some examples of cloud providers are Amazon Web Services (AWS), Google Cloud Platform, and Microsoft Azure.

— Intended for Admins

Platform as a Service (PaaS)

I'm assuming you are a software developer? Well, this cloud computing category is designed for you. PaaS offers cloud computing services with an on-demand environment for development, testing, delivering, and managing software applications. This will make it easier for developers to create web or mobile applications, without worrying about the infrastructure, storage, network, and databases needed for development.

Some examples of PaaS are the AWS Lambda Service, Heroku, etc.

— Intended for Developers

Serverless Computing

I'll mention the definition of serverless computing based on Webiny's guide to serverless ⬇️

"Serverless means there are no servers you have to operate to run a particular service or an app. Usually, in Serverless architectures, you rely on services that are managed for you. Be that function-as-a-service, storage-as-a-service, and others. Using managed services means all the maintenance and operational tasks are abstracted away. Yes, there are still servers there, but they are no longer your responsibility to manage."

Some of the examples that offer serverless services are AWS Lambda Service, Cloudflare Workers, Google Cloud Functions, IBM Cloud Functions, Knative, Microsoft Azure Functions, Oracle Functions. When it comes to building serverless applications and APIs on top of these services, Webiny takes the lead! Webiny is a serverless framework — a complete plugin-based admin interface and a set of ready-made serverless applications. Learn more about Webiny features here.

Software as a Service (SaaS)

SaaS takes place when cloud providers offer complete software products. The infrastructure, product maintenance, security its managed by the cloud provider. Some examples of this cloud computing category are Salesforce, Gmail, Office 336.

— Intended for Customers

Cloud Deployment Models

Public Cloud

With this type of deployment model, you'll have access to anything the cloud provider offers, from SaaS applications, virtual machines (VMs), infrastructure, and development platforms — available to users via the public internet. Different cloud providers offer these services in different pricing models such as subscription-based, or pay-per-use.

One of the known characteristics of the Cloud is the multi-tenant* environment—the data centers infrastructure is shared by all public cloud customers. This leads to one or more disadvantages of the Cloud such as limited resources → Even though they have incredible computing power, they share their resources with multiple tenants.

Some examples of companies using the cloud deployment models are Squarespace, Basecamp, and Dropbox.

Multi-tenant → When a single instance of a software application serves multiple tenants (or user accounts).

Tenant → Can be an individual user, but more often is a group of users that share common access and privileges within that instance of a software application.

On-Premise || Private Cloud

The on-premise cloud refers to the cloud computing resources being used by one organization or customer → sometimes is called a private cloud. One reason is the security that the on-premise offers for sensitive data for customers such as Government, Hospitals, Enterprises with different regulations. The on-premise cloud is hosted in the customer's data center, but it has the option to combine the hosting on an independent cloud provider's infrastructure.

Hybrid Cloud

A combination of the public cloud and on-premise → connecting an organization's on-premise cloud services and public clouds into a single infrastructure for all the organization's applications.

The goal is to have an optimal solution for each of the organization's applications to do the workloads between the two — based on the current requirements.

Using a Hybrid cloud gives organizations the freedom to meet their business and technical objectives with benefits such as lower costs and efficiency rather than using just public or on-premise cloud.

Now that you know the cloud computing concepts, I think you are ready to start with some examples of cloud computing services or you can follow the blog on 5 AWS services you need for web development!

If you learned something new today and are interested to follow up on our blogs, subscribe to our newsletter and we'll provide you the best content of the serverless world!

5 AWS Services you need to know for Web Development

Albiona — Wed, 13 Jan 2021 13:41:49 +0000

If you're getting started with AWS and you're interested to build your web applications using AWS services in a short amount of time, then you're in the right place. AWS offers more than 300 services, and to start developing applications with AWS without the right information might be a huge burden. As for today, we'll provide to you only 5 AWS services you need to know, to develop your web application.

Without further ado, let's get started.

Amazon Web Services

Our top 5 picks of AWS services are:

AWS S3

AWS API Gateway

AWS Lambda

AWS DynamoDB

AWS CloudFront

Extra AWS Service - AWS Cognito

For each service, we will go through what is the service, what are the benefits of using one, the pricing, and how to get started.

Follow the article here to get into the details of the services you need for web development!

Now that you have the tools you need, building your web applications with AWS is an amazing experience considering the benefits it offers for developers. And if you want to go one step further, try Webiny — Webiny is an open-source serverless application framework that brings all those tools together. Start building your serverless applications using Webiny. today 🚀

DEV Community: Webiny

Introducing Webiny Enterprise Headless CMS+

What if you Need More Than a Headless CMS?

By Any Other Name

An All-in-one Solution

What Is Webiny Enterprise Headless CMS+?

🤔 Should I Use Gatsby or Next.js For My Next Project?

Server Generation and Configuration

Documentation

Routing

Conclusion

Alternatives to Heroku’s Free Tier

The Heroku Story

Alternatives to Heroku

Stay in Control of your Application Stack

Embrace Serverless

Webiny: Leverage the Power of Serverless

Learn How to Use Lambda Layers by Building an Image Resize Function Using Sharp

Prerequisites

Step 1 - Create two S3 buckets

Step 2 - Create the IAM policy

To create an IAM policy

Step 3 - Create the execution role

To create an execution role

Step 4 - Create a sharp Lambda Layer

Step 5 - Create Lambda function

Step 5.1 Create the function code

Step 5.2 Create the deployment package

Step 5.3 Create a Lambda function

Step 6 - Configure Amazon S3 Trigger

To create an Amazon S3 Trigger

Step 7 - All set, let’s test it!

Credits

Using Webiny Headless CMS With Our New Starter Kits for Gatsby and NextJS

Official Gatsby Starter for Webiny Headless CMS

Official NextJS Starter for Webiny Headless CMS

What Starter Should We Build Next?

Learn About Infrastructure as Code in 5 Minutes and Why You Should Use It

Infrastructure as code declaration

Why it is needed

The best IaC providers

Amazon AWS Cloudformation

Azure Resource Manager

Google Cloud Deployment Manager

Terraform

AWS CDK

Pulumi

IaC examples

IaC best practices

Version control

Document as little as possible

Modularity and microservices

IaC Compliance Regulation

IaC challenges

Shift of configuration

Potential duplication of errors

Need for new skills

Conclusion

Introducing the new Object field 🚀

Build nested data structures

Repeatable structure

GraphQL Schema auto-complete support

5 Things to Avoid When Writing Cypress Tests

Use id and class For Selecting Element

Treating Cypress Commands as Promise

Using Arbitrary Waits in Cypress Tests

Using Different Domains within a Cypress Test

Mixing Async and Sync Code

Conclusion

Webiny 5.0 Release

In Other News

The Complete Beginners Guide to AWS S3

What is S3?

Buckets

Access points

Objects

Hands-on S3 with AWS SDK

Configuring the SDK

Creating an AWS S3 Bucket

Uploading a file to an AWS S3 Bucket

Use `id` and `class` For Selecting Element

`How-to` on S3 - Cheatsheet