Setting up a S3 File Gateway on a EC2 Windows Server

S3 File Gateway can be used to Store and access objects in Amazon S3 from NFS or SMB file data with local caching.

Typically, the architecture might look like as shown below when you connecting to your s3 file gateway from On-Premises over NFS or SMB.

But in this blog, I am using EC2 hosted Storage Gateway Appliance for demo purpose. So, in this case Storage Gateway Appliance will be placed on the AWS Cloud not On-prem as shown above.If you have your applications on AWS and want to connect through NFS or SMB where files stored in S3,then this will be the ideal setup.

Setting up a EC2 based S3 File Gateway

First, go to Storage Gateway service in the AWS management console and click create gateway. Then you will see a page as shown below.

Select Amazon S3 File Gateway and click Next.

Before select Amazon EC2 and Click next, You can use this CloudFormation template to create the Storage gateway instance and security groups to avoid bit of ClickOps and save time. :-).Otherwise you can click Launch instance to create an EC2 instance.

AWSTemplateFormatVersion: 2010-09-09

Parameters:
  ImageId:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: /aws/service/ami-windows-latest/Windows_Server-2019-English-Full-Base
  InstanceType:
    Type: String
    Default: m6i.8xlarge
    Description: "Select m6i.[xlarge, 2xlarge, 4xlarge, 8xlarge]. Default is m6i.8xlarge for production use."
    AllowedValues:
      - m6i.xlarge   # 4-vCPU, 16G-RAM, 10Gbps-NET
      - m6i.2xlarge  # 8-vCPU, 32G-RAM, 10Gbps-NET
      - m6i.4xlarge  # 16-vCPU,64G-RAM, 10Gbps-NET
      - m6i.8xlarge  # 32-vCPU,128-RAM, 10Gbps-NET
  VpcId:
    Description: VPC IDs
    Type: AWS::EC2::VPC::Id
  SubnetId:
    Description: Subnet ID
    Type: AWS::EC2::Subnet::Id
  KeyName:
    Description: The SSH keypair
    Type: AWS::EC2::KeyPair::KeyName
  VolumeType:
    Type: String
    Default: io2
  RootVolumeType:
    Type: String
    Default: gp3
  VolumeDeleteOnTermination:
    Default: True
    Type: String
  VolumeSize:
    Description: "SGW cache disk size minimum 150 GiB"
    Type: Number
    Default: 150
  RootVolumeIops:
    Description: "Recommended at least 3000 IOPS or more."
    Type: Number
    Default: 3000
  VolumeIops:
    Description: "Recommended at least 3000 IOPS or more for cache disks."
    Type: Number
    Default: 3000
Resources:
  SGWSG01:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      VpcId: !Ref VpcId
      GroupDescription: "EC2 File Storage Gateway Security Group"
      GroupName: !Sub 'ec2-sgw-sg-${AWS::StackName}'
      Tags:
      - Key: "Name"
        Value: SGWSG01
      SecurityGroupEgress:
      - IpProtocol: "-1"
        CidrIp: 0.0.0.0/0
      SecurityGroupIngress:
      - IpProtocol: "-1"
        CidrIp: 10.0.0.0/8

  SGW01:
    Type: 'AWS::EC2::Instance'
    DeletionPolicy: Delete
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${AWS::StackName}-sgw-01'
      PropagateTagsToVolumeOnCreation: True
      KeyName: !Ref KeyName
      ImageId: !Ref ImageId
      InstanceType: !Ref InstanceType
      SecurityGroupIds:
        - !Ref SGWSG01
      SubnetId: !Ref SubnetId
      BlockDeviceMappings:
        - DeviceName: "/dev/xvda"
          Ebs:
            VolumeType: !Ref RootVolumeType
            DeleteOnTermination: !Ref VolumeDeleteOnTermination
            Iops: !Ref RootVolumeIops
        - DeviceName: "/dev/sda1"
          Ebs:
            VolumeType: !Ref VolumeType
            DeleteOnTermination: !Ref VolumeDeleteOnTermination
            VolumeSize: !Ref VolumeSize
            Iops: !Ref VolumeIops
    DependsOn: SGWSG01

So just click Amazon EC2 and click Next.

Then you will be asked to select the Storage Gateway service Endpoint. I am choosing VPC but if you want you can choose Public based on your requirement.

In order to select VPC endpoint, make sure you have created a VPC interface endpoint as shown below.

Next you will need to connect to the storage gateway to get the Activation code. If you choose IP address option make sure your default browser can access storage gateway console. This step will automatically redirect to Storage gateway console and the generated link will be something like this:

http://<Storage GW EC2 instance IP>/?gatewayType=FILE_S3&activationRegion=<region>&no_redirect&vpcEndpoint=<vpc endpoint dns name>

If that link is not accessible from your current browser/network, copy that link and paste somewhere your storage gateway is accessible. Then it will show you the Activation Code. Then you can copy that into the given box.

Then you can enter the gateway name as you prefer and activate your gateway.

Then mention the cache disk size you want to have.

Best practice to allocate at least 150 GiB of cache storage.

Finally there is an optional step. But I would prefer to have Cloudwatch logs but its totally up to you to disable or enable logs. Enabling logging is useful for auditing purposes.