DEV Community

loading...

6/12 TIL: AWS solutions architect cert S3 part 2

David Hwang
・3 min read

S3

  • Encryption
    • Encryption in Transit
      • SSL/TLS (https)
    • Encryption at Rest (data being stored)
      • Server side: Amazon helps you encrypt the object
        • S3 Managed Keys - SSE-S3
        • AWS Key Management Serviced, Managed Keys - SSE-KMS
        • Server Side Encryption With Customer Provided Keys - SSE-C
      • Client side: you encrypt the object and upload it to S3
  • Versioning
    • Stores all versions of an object (including all writes and even if you delete an object)
    • Great backup tool
    • Once enabled, Versioning cannot be disabled, only suspended
    • Integrates with Lifecycle rules
    • Versioning's MFA Delete capability, which uses multi-factor authentication, can be used to provide an additional layer of security
    • Making one version of object public **doesn't* make other (previous or new) objects public; you have to configure them individually
    • *Deleting an object will create Delete Marker over it and hide all the previous versions; deleting the Delete Marker will restore the object
  • Lifecycle Management with S3
    • Go into the Bucket > Management tab > Create Lifecycle rule
    • Automates moving your objects between different storage tiers or permanently delete previous versions (configure the timeline)
    • Can be applied to current and previous versions
  • S3 Object Lock & Glacier Vault Lock
    • S3 Object Lock
      • Stores objects using a write once, read many (WORM) model. It can help you prevent objects from being deleted or modified for a fixed amount of time or indefinitely
      • Used to meet regulatory requirements that require WORM storage, or add an extra layer of protection against object changes and deletion
      • Can be on individual objects or applied across the bucket as a whole
      • Modes
        • Governance mode: users can't overwrite or delete an object version or alter its lock settings; you can still grant some users permission to alter the retention settings or delete an object if necessary
        • Compliance mode: a protected object version can't be overwritten or deleted by any user, including the root user; for the duration of the retention period
          • Retention Periods: protects an object version for a fixed amount of time
          • Legal Holds: works like Retention Periods, only that it remains in effect until the s3:PutObjectLegalHold permission is removed
    • S3 Glacier Vault Lock
      • Allows you to easily deploy and enforce compliance controls for individual S3 Glacier vaults with a Vault Lock policy. You can specify controls such as WORM in a Vault Lock policy and lock the policy from future edits.
  • S3 Performance

    • S3 Prefix
      • mybucketname/folder1/subfolder1/myfile.jpg
      • prefix is the middle bit between the bucket name and the object
      • S3 has extremely low latency. You can get the first byte out of S3 within 100-200ms
      • You can achieve 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEADER requests per second per prefix
      • You can get better performance by spreading your reads across different prefixes. For example, if you are using two prefixes, you can achieve 11,000 requests per second.
      • S3 limitations when using SSE-KMS as your encryption
      • When you upload a file, you will call GenerateDataKey in the KMS API
      • When you download a file, you will call Decrypt in the KMS API
      • Uploading/downloading will count toward the KMS quota (region-specific, it's either 5,500, 10,000, or 30,000 requests per second)
    • Multipart uploads
      • Recommended for uploading files over 100MB, required for files over 5GB
      • Parallelize uploads (increase efficiency)
    • S3 Byte-Range Fetches

      • Parallelize downloads by specifying byte ranges
      • Speed up downloads or download partial amounts of the file
  • S3 Select & Glacier Select

    • S3 Select
      • Enables applications to retrieve only a subset of data from an object by using simple SQL expressions. You can achieve drastic performance increases—up to 400% improvement
      • Without S3 Select, you would need to download, decompress, and process the entire CSV to get the data you needed
    • Glacier Select
      • Some companies in highly regulated industries—e.g. financial services, healthcare, and others—write data directly to Amazon Glacier to satisfy compliance needs like SEC Rule 17a-4 or HIPAA. Many S3 users have lifecycle policies designed to save on storage costs by moving their data into Glacier when they no longer need to access it on a regular basis

Discussion (0)