Overview

• Buckets allow you to persistently store files.
• A Bucket is a collection of "objects". An object is a file and any metadata (headers, tags,...) that describes that file.
• Buckets are easy to use, reliable, durable and highly available. They are powered by AWS S3 object storage service.
• Buckets are serverless. They scale automatically to meet your needs and you are paying only for the files saved (not unused capacity).
• Buckets have a flat structure instead of a hierarchy like you would find in a file system. However, you can simulate a "folder hierarchy" by using a common prefix. For example, all objects stored with a name that starts with photos/ prefix will be shown in the same "folder" in the AWS console.

Under the hood

Buckets are abstraction of AWS S3 Buckets.

When to use

• Buckets can be used to host websites, store user-generated content, store data for Big data analytics, serve as data lakes, backup and restore, archives, etc.

• Because of performance reasons, object storage in general is not optimal for use-cases that require very low read/write latency.

Advantages

• Easy to use - AWS S3 has a simple, HTTP-based API. You can also easily manipulate your objects using an AWS SDK.

• Serverless - Bucket automatically scales to meet your needs. You don't have to worry about scaling your storage size and are paying only for the files saved.

• Reliable & highly available - S3 Buckets are designed for 99.999999999% (11 9’s) of data durability. Files (objects) are stored across multiple physical locations (Availability zones).

• Storage flexibility - You can store your files in multiple storage classes. Different storage classes have different latencies, pricing and durability.

• Access control - You can easily control which compute resources can access your bucket.

• Supports encryption - Supports server-side encryption.

• Integrations - You can easily trigger a function or a batch-job in a reaction to a bucket event.

Disadvantages

• Performance - Compared to using a block storage (physical disk attached to a machine), reading/write operations are significantly slower.

Basic usage

Copy
resources:
  myBucket:
    type: bucket

  myFunction:
    type: function
    properties:
      packaging:
        type: stacktape-lambda-buildpack
        properties:
          entryfilePath: path/to/my/lambda.ts
      environment:
        - name: BUCKET_NAME
          value: $ResourceParam('myBucket', 'arn')
      connectTo:
        - myBucket

Copy
import { S3 } from '@aws-sdk/client-s3';

const s3 = new S3({});

// getObject returns a readable stream, so we need to transform it to string
const streamToString = (stream) => {
  const chunks = [];
  return new Promise((resolve, reject) => {
    stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
    stream.on('error', (err) => reject(err));
    stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
  });
};

const handler = async (event, context) => {
  await s3.putObject({
    Bucket: process.env.BUCKET_NAME,
    Key: 'my-file.json',
    Body: JSON.stringify({ message: 'hello' }) // or fs.createReadStream('my-source-file.json')
  });

  const res = await s3.getObject({
    Bucket: process.env.BUCKET_NAME,
    Key: 'my-file.json'
  });
  const body = await streamToString(res.Body);
};

export default handler;

Directory upload

Copy
resources:
  myBucket:
    type: bucket
    properties:
      directoryUpload:
        directoryPath: ../public

Adding metadata

• You can add metadata to uploaded files. You can configure headers and tags.
• Filters allow you to configure properties of files (objects) that match the filter pattern.
• Configurable properties:
  • headers: when a client makes a request for an object, these headers will be added to the HTTP response. If you are using a CDN, all headers will be forwarded. If you are using the bucket to host a website, this can be for example used to add cache-control headers on a per-file basis (different cache behavior for different file types).
  • tags: can be used to filter specific objects, for example in Lifecycle rules.

Encryption

• If enabled, all objects uploaded to the bucket will be server-side encrypted using the AES256 algorithm.

Copy
resources:
  myBucket:
    type: bucket
    properties:
      encryption: true

Cors

Copy
resources:
  myBucket:
    type: bucket
    properties:
      cors:
        enabled: true

Versioning

Copy
resources:
  myBucket:
    type: bucket
    properties:
      versioning: true

CDN

You can use CDN with bucket.

By putting CDN in front of a bucket, you can distribute content of the bucket to hundreds of edge locations all around the world.

This is useful in cases if you want to for example serve a static website from your bucket.

Stacktape can:

• automatically upload your content (e.g. static website) to the bucket
• configure CDN and deliver your content with minimal latency across the world

For information about using CDN refer to our CDN docs.

Copy
resources:
  myBucket:
    type: bucket
    properties:
      directoryUpload:
        directoryPath: my-web/build
        headersPreset: static-website
      cdn:
        enabled: true

Object lifecycle rules

• Lifecycle rules allow you to configure what happens to your objects after a configured period of time.
• They can be deleted, transitioned to another storage class, etc.
• These rules can be applied to only a subset of objects in the bucket using path prefix and object tags.

Storage class transition

Copy
resources:
  myBucket:
    type: bucket
    properties:
      lifecycleRules:
        - type: class-transition
          properties:
            daysAfterUpload: 90
            storageClass: 'GLACIER'

Expiration

Allows you to delete objects from the bucket after the specified amount of days after upload.

This can be useful in cases if objects become irrelevant for you after some time. These objects can be deleted, thus saving you storage costs.

The following example shows:

• All uploaded objects are transferred to GLACIER storage class 90 days after upload.
• After 365 days, objects are completely deleted.

Copy
resources:
  myBucket:
    type: bucket
    properties:
      lifecycleRules:
        - type: class-transition
          properties:
            daysAfterUpload: 90
            storageClass: 'GLACIER'
        - type: expiration
          properties:
            daysAfterUpload: 365

Non-current version class transition

Allows you to transition versioned objects into a different storage class.

Same as class transition rule but applied to old versions of objects. This can be useful when you want to archive old versions of an object after some time to save you costs.

The following example shows:

• all versioned objects are transferred to GLACIER storage class 10 days after they are versioned (become non-current version).

Copy
resources:
  myBucket:
    type: bucket
    properties:
      versioning: true
      lifecycleRules:
        - type: non-current-version-class-transition
          properties:
            daysAfterVersioned: 10
            storageClass: 'DEEP_ARCHIVE'

Non-current version expiration

Allows you to delete old versions of objects from the bucket after the specified amount of days after they are versioned.

This can be useful if you only need to keep older versions of objects for some time and then they can be deleted thus saving you storage costs.

The following example shows:

• all versioned objects are deleted ten days after being version (become non-current version).

Copy
resources:
  myBucket:
    type: bucket
    properties:
      versioning: true
      lifecycleRules:
        - type: non-current-version-expiration
          properties:
            daysAfterVersioned: 10

Abort incomplete multipart upload

Allows you to stop multipart uploads that do not complete within a specified number of days after being initiated.

When a multipart upload is not completed within the time-frame, it becomes eligible for an abort operation, and Amazon S3 stops the multipart upload (and deletes the parts associated with the multipart upload, thus saving you costs).

The following example shows:

• all incomplete uploads are stopped (and parts deleted) 5 days after initiation of multipart upload.

Copy
resources:
  myBucket:
    type: bucket
    properties:
      lifecycleRules:
        - type: abort-incomplete-multipart-upload
          properties:
            daysAfterInitiation: 5

Accessibility

• Configures who can access the bucket.

Accessibility modes

Access policy statements

• For a more fine-grained access control, you can configure accessPolicyStatements.
• Using these requires knowledge of AWS IAM Docs.
• You can find a list of example bucket policies in AWS Docs.

Copy
resources:
  myBucket:
    type: bucket
    properties:
      accessibility:
        accessibilityMode: private
        accessPolicyStatements:
          - Resource:
              - $ResourceParam('myBucket', 'arn')
            Action:
              - 's3:ListBucket'
            Principal: '*'

Referenceable parameters

API reference