Custom Resources
Custom resources allow you to create resources with custom provisioning logic. This enables you to define the logic for creating, updating, and deleting resources, giving you the ability to provision any type of resource that is not natively supported by Stacktape or Cloudformation. This way, you can manage all your related resources in a single stack.
A common use case for custom resources is provisioning non-AWS resources. While AWS offers a vast array of services, a third-party solution may better fit your needs (e.g., using Auth0 instead of Cognito, or Algolia instead of OpenSearch).
If you are not provisioning a custom resource but are looking for a way to execute custom logic during the deployment process, see the deployment scripts documentation.
How to use custom resources
In Stacktape, a custom resource consists of two parts:
- Definition: A special type of Lambda function that contains the custom logic for creating, updating, and deleting the resource.
- Instance: An instantiation of a custom resource definition.
The following example shows a custom resource that seeds a MongoDB cluster.
resources:myMongoCluster:type: mongo-db-atlas-clusterproperties:clusterTier: M10mongoSeeder:type: custom-resource-definitionproperties:packaging:type: stacktape-lambda-buildpackproperties:entryfilePath: seed-the-mongo-cluster.tsconnectTo:- myMongoClusterseedMongoCluster:type: custom-resource-instanceproperties:definitionName: mongoSeederresourceProperties:mongoConnectionString: $Param('myMongoCluster', 'AtlasMongoCluster::SrvConnectionString')
Custom resource definition
A custom resource definition is similar to a function definition, as it uses a Lambda function to execute the custom logic.
resources:myMongoCluster:type: mongo-db-atlas-clusterproperties:clusterTier: M10mongoSeeder:type: custom-resource-definitionproperties:packaging:type: stacktape-lambda-buildpackproperties:entryfilePath: seed-the-mongo-cluster.tsconnectTo:- myMongoClusterseedMongoCluster:type: custom-resource-instanceproperties:definitionName: mongoSeederresourceProperties:mongoConnectionString: $Param('myMongoCluster', 'AtlasMongoCluster::SrvConnectionString')
Code of custom resource
The packaging
property allows you to specify the path to your code and other packaging-related properties. The code is packaged and executed as a Lambda function. For more information, see the packaging documentation.
resources:myMongoCluster:type: mongo-db-atlas-clusterproperties:clusterTier: M10mongoSeeder:type: custom-resource-definitionproperties:packaging:type: stacktape-lambda-buildpackproperties:entryfilePath: seed-the-mongo-cluster.tsconnectTo:- myMongoClusterseedMongoCluster:type: custom-resource-instanceproperties:definitionName: mongoSeederresourceProperties:mongoConnectionString: $Param('myMongoCluster', 'AtlasMongoCluster::SrvConnectionString')
Code example
The following example shows the code for the MongoDB seeding custom resource. For more information on the types of requests (events) a custom resource receives and the response objects you should return, see the AWS documentation.
import mongoose from 'mongoose';import { SUCCESS, FAILED, send } from 'cfn-response-promise';// incoming event is in following form// {// "RequestType" : "Create" || "Update" || "Delete",// "RequestId" : "9db53695-b0a0-47d6-908a-ea2d8a3ab5d7",// "ResponseURL" : "https://...",// "ResourceType" : "AWS::Cloudformation::CustomResource",// "LogicalResourceId" : "...",// "StackId" : "arn:aws:cloudformation:...",// "ResourceProperties" : {// ... properties of custom-resource-instance// }// }export default async (event, context) => {// custom resource definition codelet success = true;let dataToReturn = {};try {// we are only seeding database if the operation is Createif (event.RequestType === 'Create') {// we are using the "mongoConnectionString" property passed by custom-resource-instance to create connectionconst connection = await mongoose.connect(event.ResourceProperties.mongoConnectionString, {authMechanism: 'MONGODB-AWS',authSource: '$external',useNewUrlParser: true,useUnifiedTopology: true,dbName: 'my-test-database'});// code with seeding the database ...// ...}} catch (err) {success = false;}await send(event, context, success ? SUCCESS : FAILED, dataToReturn, 'customresourceid');};// function must respond to "ResponseURL" with response in following form// we are using "cfn-response-promise" library which formats response for us// {// "Status" : "SUCCESS" || "FAILED",// "RequestId" : "9db53695-b0a0-47d6-908a-ea2d8a3ab5d7",// "LogicalResourceId" : "...",// "StackId" : "arn:aws:cloudformation:...",// "PhysicalResourceId" : "...",// "Data" : {// ... attributes which can be queried in template using $Param// }// }
Accessing other resources
By default, resource-to-resource communication is not allowed in AWS. Access must be explicitly granted using IAM permissions. Stacktape automatically handles IAM permissions for the underlying AWS services it creates.
If your custom resource needs to communicate with other infrastructure components, you must grant the necessary permissions manually. You can do this in two ways:
connectTo
The connectTo
property is a list of resource names or AWS services that the custom resource will be able to access. Stacktape will automatically grant the basic IAM permissions required for communication. This is useful if you do not want to manage IAM permissions yourself.
When you use the connectTo
property, Stacktape also automatically injects information about the connected resource as environment variables into your custom resource's runtime.
By referencing resources (or services) in connectTo
list, Stacktape automatically:
- configures correct compute resource's IAM role permissions if needed
- sets up correct security group rules to allow access if needed
- injects relevant environment variables containing information about resource you are connecting to into the compute resource's runtime
- names of environment variables use upper-snake-case and are in form
STP_[RESOURCE_NAME]_[VARIABLE_NAME]
, - examples:
STP_MY_DATABASE_CONNECTION_STRING
orSTP_MY_EVENT_BUS_ARN
, - list of injected variables for each resource type can be seen below.
- names of environment variables use upper-snake-case and are in form
Granted permissions and injected environment variables are different depending on resource type:
Bucket
- Permissions:
- list objects in a bucket
- create / get / delete / tag object in a bucket
- Injected env variables:
NAME
,ARN
DynamoDB table
- Permissions:
- get / put / update / delete item in a table
- scan / query a table
- describe table stream
- Injected env variables:
NAME
,ARN
,STREAM_ARN
MongoDB Atlas cluster
- Permissions:
- Allows connection to a cluster with
accessibilityMode
set toscoping-workloads-in-vpc
. To learn more about MongoDB Atlas clusters accessibility modes, refer to MongoDB Atlas cluster docs. - Creates access "user" associated with compute resource's role to allow for secure credential-less access to the the cluster
- Allows connection to a cluster with
- Injected env variables:
CONNECTION_STRING
Relational(SQL) database
- Permissions:
- Allows connection to a relational database with
accessibilityMode
set toscoping-workloads-in-vpc
. To learn more about relational database accessibility modes, refer to Relational databases docs.
- Allows connection to a relational database with
- Injected env variables:
CONNECTION_STRING
,JDBC_CONNECTION_STRING
,HOST
,PORT
(in case of aurora multi instance cluster additionally:READER_CONNECTION_STRING
,READER_JDBC_CONNECTION_STRING
,READER_HOST
)
Redis cluster
- Permissions:
- Allows connection to a redis cluster with
accessibilityMode
set toscoping-workloads-in-vpc
. To learn more about redis cluster accessibility modes, refer to Redis clusters docs.
- Allows connection to a redis cluster with
- Injected env variables:
HOST
,READER_HOST
,PORT
Event bus
- Permissions:
- publish events to the specified Event bus
- Injected env variables:
ARN
Function
- Permissions:
- invoke the specified function
- invoke the specified function via url (if lambda has URL enabled)
- Injected env variables:
ARN
Batch job
- Permissions:
- submit batch-job instance into batch-job queue
- list submitted job instances in a batch-job queue
- describe / terminate a batch-job instance
- list executions of state machine which executes the batch-job according to its strategy
- start / terminate execution of a state machine which executes the batch-job according to its strategy
- Injected env variables:
JOB_DEFINITION_ARN
,STATE_MACHINE_ARN
User auth pool
- Permissions:
- full control over the user pool (
cognito-idp:*
) - for more information about allowed methods refer to AWS docs
- full control over the user pool (
- Injected env variables:
ID
,CLIENT_ID
,ARN
SNS Topic
- Permissions:
- confirm/list subscriptions of the topic
- publish/subscribe to the topic
- unsubscribe from the topic
- Injected env variables:
ARN
,NAME
SQS Queue
- Permissions:
- send/receive/delete message
- change visibility of message
- purge queue
- Injected env variables:
ARN
,NAME
,URL
Upstash Kafka topic
- Injected env variables:
TOPIC_NAME
,TOPIC_ID
,USERNAME
,PASSWORD
,TCP_ENDPOINT
,REST_URL
Upstash Redis
- Injected env variables:
HOST
,PORT
,PASSWORD
,REST_TOKEN
,REST_URL
,REDIS_URL
Private service
- Injected env variables:
ADDRESS
aws:ses
(Macro)
- Permissions:
- gives full permissions to aws ses (
ses:*
). - for more information about allowed methods refer to AWS docs
- gives full permissions to aws ses (
In this example, we grant access to a MongoDB cluster. This will inject the necessary credentials into the custom resource's runtime, which are then used in the code example.
resources:myMongoCluster:type: mongo-db-atlas-clusterproperties:clusterTier: M10mongoSeeder:type: custom-resource-definitionproperties:packaging:type: stacktape-lambda-buildpackproperties:entryfilePath: seed-the-mongo-cluster.tsconnectTo:- myMongoClusterseedMongoCluster:type: custom-resource-instanceproperties:definitionName: mongoSeederresourceProperties:mongoConnectionString: $Param('myMongoCluster', 'AtlasMongoCluster::SrvConnectionString')
Environment variables
You can use environment variables to inject information that the resource needs during execution.
Custom resource instance
A custom resource instance creates an instance of a specified custom resource definition. Depending on the action being performed on the stack (create, update, or delete), the instance receives an event and executes the corresponding logic in the custom resource definition.
resources:myMongoCluster:type: mongo-db-atlas-clusterproperties:clusterTier: M10mongoSeeder:type: custom-resource-definitionproperties:packaging:type: stacktape-lambda-buildpackproperties:entryfilePath: seed-the-mongo-cluster.tsconnectTo:- myMongoClusterseedMongoCluster:type: custom-resource-instanceproperties:definitionName: mongoSeederresourceProperties:mongoConnectionString: $Param('myMongoCluster', 'AtlasMongoCluster::SrvConnectionString')