Stacktape

Sign upSign up



Private Services

Overview

Private service is a running container with a private endpoint(only accessible by resources of you stack). Main features of private service are:

  • scalability - automatically add/remove containers based on CPU or memory usage (within your specified thresholds),
  • zero downtime deployments,
  • multiple options to provide container image, including zero-config packaging of code for Typescript/Javascript, Python, Java and Go,
  • no hassle with managing and patching servers or VMs,
  • seamless connectivity with other resources.

Under the hood

Under the hood Stacktape uses AWS ECS(Elastic Container Service) to orchestrate your containers. Containers can be ran using Fargate or EC2 instances:

  • Fargate - technology which allows you to run containers without having to manage servers or clusters. This means you don't have to worry about administration tasks such as scaling, VM security, OS security & much more.
  • EC2 instances - VMs running on AWS

ECS Services are self healing - if your container instance dies for any reason, it is automatically replaced with a healthy instance. They auto-scale from the box(within configured boundaries) leveraging the AWS Application Auto Scaling service.


Private-service provides two load balancing modes. Based on which mode you use, different underlying mechanisms are used underneath:

When to use

If you are unsure which resource type is best suitable for your app, following table provides short comparison of all container-based resource types offered by Stacktape.

Resource typeDescriptionUse-cases
web-servicecontinuously running container with public endpoint and URLpublic APIs, websites
private-servicecontinuously running container with private endpointprivate APIs, services
worker-servicecontinuously running container not accessible from outsidecontinuous processing
multi-container-workloadcustom multi container workload - you can customize accessibility for each containermore complex use-cases requiring customization
batch-jobsimple container job - container is destroyed after job is doneone-off/scheduled processing jobs

Advantages

  • Control over underlying environment - Private service can run any Docker image or image built using your own Dockerfile.
  • Price for compute resources with predictable load - Compared to functions, private services are cheaper if your compute resource has a predictable load.
  • Load-balanced and auto-scalable - Private services can automatically horizontally scale based on the CPU and Memory utilization.
  • High availability - Private services run in multiple Availability Zones.
  • Secure by default - Underlying environment is securely managed by AWS.

Disadvantages

  • Scaling speed - Unlike lambda functions that scale almost instantly, private service requires more time - from several seconds to few minutes to add another container.

  • Not fully serverless - While private services can automatically scale up and down, they can't scale to 0. This means, if your compute resource is unused, you are still paying for at least one instance (minimum ~$8/month)

Basic usage

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/main.ts
resources:
cpu: 2
memory: 2048

Example private service config

Copy

import express from 'express';
const app = express();
app.get('/', async (req, res) => {
res.send({ message: 'Hello' });
});
// for your app use port number stored in PORT environment variable for your application
// this environment variable is automatically injected by Stacktape
app.listen(process.env.PORT, () => {
console.info(`Server running on port ${process.env.PORT}`);
});

Example server container written in Typescript (main.ts)


PrivateService  API reference
type
Required
properties.packaging
Required
properties.resources
Required
properties.port
Default: 3000
properties.protocol
properties.loadBalancing
Default: service-connect
properties.environment
properties.logging
properties.scaling
properties.internalHealthCheck
properties.stopTimeout
properties.connectTo
properties.iamRoleStatements
overrides

Connecting to a Private Service

Private services do not have publicly accessible HTTPS URLs.

Instead, they have service address which is a host:port pair.

  • host is by default lowercased resource name
  • port number is 3000 by default(can be changed by specifying port)

Following example shows privateApi service. host:port pair in this case will be privateapi:3000.

  • You can connect to this address directly from other resources of your stack.

  • You may need to append a URL scheme to the private service address depending on your app requirements. For example, apps that rely on HTTP api endpoint of sorts might require you to specify it, i.e http://privateapi:3000

Copy

resources:
publicService:
type: web-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/main.ts
resources:
cpu: 1
memory: 1024
environment:
# injecting privateApi address
- name: PRIVATE_ADDRESS
value: $ResourceParam('privateApi', 'address')
privateApi:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/private-api/main.ts
resources:
cpu: 1
memory: 1024

Image

Environment variables

Most commonly used types of environment variables:

  • Static - string, number or boolean (will be stringified).
  • Result of a custom directive.
  • Referenced property of another resource (using $ResourceParam directive). To learn more, refer to referencing parameters guide. If you are using environment variables to inject information about resources into your script, see also property connectTo which simplifies this process.
  • Value of a secret (using $Secret directive).

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/main.ts
environment:
- name: STATIC_ENV_VAR
value: my-env-var
- name: DYNAMICALLY_SET_ENV_VAR
value: $MyCustomDirective('input-for-my-directive')
- name: DB_HOST
value: $ResourceParam('myDatabase', 'host')
- name: DB_PASSWORD
value: $Secret('dbSecret.password')
resources:
cpu: 2
memory: 2048

Healthcheck

The purpose of the container health check is to monitor the health of the container from the inside.

Once an essential container of an instance is determined UNHEALTHY, the instance is automatically replaced with a new one.

ContainerHealthCheck  API reference
healthCheckCommand
Required
intervalSeconds
Default: 30
timeoutSeconds
Default: 5
retries
Default: 3
startPeriodSeconds

  • Example: A shell command sends a curl request every 20 seconds to determine if the service is available. If the request fails (or doesn't return in 5 seconds), the command returns with non-zero exit code, and the healthcheck is considered failed.

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/index.ts
internalHealthCheck:
healthCheckCommand: ['CMD-SHELL', 'curl -f http://localhost/ || exit 1']
intervalSeconds: 20
timeoutSeconds: 5
startPeriodSeconds: 150
retries: 2
resources:
cpu: 2
memory: 2048

Shutdown

  • When a running private service instance is deregistered (removed), all running containers receive a SIGTERM signal.
  • By default, you then have 2 seconds to clean up. After 2 seconds, your process receives a SIGKILL signal.
  • You can set the timeout by using stopTimeout property (must be between 2 - 120 seconds).
  • Setting stop timeout can help you give container the time to "finish the job" or to "cleanup" when deploying new version of container or when deleting the service.

Copy

process.on('SIGTERM', () => {
console.info('Received SIGTERM signal. Cleaning up and exiting process...');
// Finish any outstanding requests, or close a database connection...
process.exit(0);
});

Cleaning up before container shutdown.

Logging

  • Every time your code outputs (prints) something to the stdout or stderr, your log will be captured and stored in a AWS CloudWatch log group.
  • You can browse your logs in 2 ways:
    • Browse logs in the AWS CloudWatch console. To get direct link to your logs you have 2 options:
      1. Go to stacktape console. Link is among information about your stack and resource.
      2. You can use stacktape stack-info command.
    • Browse logs using stacktape logs command that will print logs to the console.
  • Please note that storing log data can become costly over time. To avoid excessive charges, you can configure retentionDays.
ContainerWorkloadContainerLogging  API reference
disabled
retentionDays
Default: 90
logForwarding

Forwarding logs

It is possible to forward logs to the third party services/databases. See page Forwarding logs for more information and examples.

Resources

In resources section, you specify amounts of cpu/memory and EC2 instance types available to your service.


There are two ways to run your containers:

  1. using Fargate

    • Fargate is a serverless, pay-as-you-go compute engine for running containers without having to manage underlying instances (servers).
    • With Fargate you only need to specify cpu and memory required for your service.
    • While slightly more expensive, using Fargate is often preferred in strictly regulated industries since running containers on Fargate meets the standards for PCI DSS Level 1, ISO 9001, ISO 27001, ISO 27017, ISO 27018, SOC 1, SOC 2, SOC 3 out of the box.
  2. using EC2 instances

    • EC2 instances are virtual machines (VMs) ran on AWS cloud.
    • Containers are placed on the EC2 instances which are added and removed depending on service needs.
    • You can choose instance type that suit your resource requirements the best.

Whether you chose to use Fargate or EC2 instances, your containers are running securely within your VPC.

Following applies when configuring resources section of your service:

  • If you do not specify instanceTypes, Stacktape uses AWS Fargate to run your containers.
  • If you do specify instanceTypes, Stacktape uses EC2 instances to run your containers (cpu and memory properties are optional in this case).
ContainerWorkloadResourcesConfig  API reference
cpu
memory
instanceTypes

Using Fargate

If you do not specify instanceTypes property in resources section, Fargate is used to run your containers.

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/index.ts
resources:
cpu: 0.25
memory: 512

Example using Fargate

Using EC2 instances

If you specify instanceTypes property in resources section, EC2 instances are used to run your containers.


  • EC2 instances are automatically added or removed to meet the scaling needs of your compute resource(see also scaling property).
  • When using instanceTypes, we recommend to specify only one instance type and to NOT set cpu or memory properties. By doing so, Stacktape will set the cpu and memory to fit the instance precisely - resulting in the optimal resource utilization.
  • Stacktape leverages ECS Managed Scaling with target utilization 100%. This means that there are no unused EC2 instances(unused = not running your workload/service) running. Unused EC2 instances are terminated.
  • Ordering in instanceTypes list matters. Instance types which are higher on the list are preferred over the instance types which are lower on the list. Only when instance type higher on the list is not available, next instance type on the list will be used.
  • For exhaustive list of available EC2 instance types refer to AWS docs.

To ensure that your containers are running on patched and up-to-date EC2 instances, your compute resource gets automatically re-deployed once a week(Sunday 03:00 UTC). Your compute resource stays available throughout this process.

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/index.ts
resources:
instanceTypes:
- c5.large

Example using EC2 instances

Placing containers on EC2

Stacktape aims to achieve 100% utilization of your EC2 instances. However this is not always possible and following behavior can be expected:

  • If you only specify one type in instanceTypes and you do NOT set memory and cpu then Stacktape sets memory and cpu to fit the EC2 instance type precisely(see also Default cpu and memory section).

    This means that if your service scales(new instance of service is added), new EC2 instance is added as well.

  • If you specify cpu and memory properties alongside instanceTypes, they will be respected. If the EC2 instance is larger (has more resources) than specified cpu and memory, AWS places instances of your service on available EC2 instances using binpack strategy. This strategy aims to put as many instances of your service on the available EC2 instances as possible(to maximize utilization).


Default cpu and memory
  • If you do not specify cpu then entire capacity of EC2 instance CPU is shared between containers running on the EC2 instance.

  • If you do not specify memory then memory is set to a maximum possible value so that all EC2 instance types listed in instanceTypes are able provide that amount of memory.

    In other words: Stacktape sets the memory so that the smallest instance type in instanceTypes(in terms of memory) is able to provide that amount of memory.

Scaling

In scaling section, you can configure scaling behavior of your service. You can configure:

  • Minimum and maximum amount of concurrently running instances of your service.
  • Conditions which trigger the scaling (up or down) using a scaling policy.
ContainerWorkloadScaling  API reference
minInstances
Default: 1
maxInstances
Default: 1
scalingPolicy

Scaling policy

  • A scaling policy specifies CPU and memory metric thresholds which trigger the scaling process.

  • Depending on the thresholds, the compute resource can either scale out (add instances) or scale in (remove instances).

  • If both keepAvgCpuUtilizationUnder and keepAvgMemoryUtilizationUnder are used, the compute resource will scale-out if one of the metrics is above the target value. However, to scale in, both of these metrics need to be below their respective target values.

  • Scaling policy is more aggressive in adding capacity then removing capacity. For example, if the policy's specified metric reaches its target value, the policy assumes that your application is already heavily loaded. So it responds by adding capacity proportional to the metric value as fast as it can. The higher the metric, the more capacity is added.

  • When the metric falls below the target value, the policy expects that utilization will eventually increase again. Therefore it slows down the scale-in process by removing capacity only when utilization passes a threshold that is far enough below the target value (usually 20% lower).

ContainerWorkloadScalingPolicy  API reference
keepAvgCpuUtilizationUnder
Default: 80
keepAvgMemoryUtilizationUnder
Default: 80

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/index.ts
resources:
cpu: 0.5
memory: 1024
scaling:
minInstances: 1
maxInstances: 5
scalingPolicy:
keepAvgMemoryUtilizationUnder: 80
keepAvgCpuUtilizationUnder: 80

Example usage of scaling configuration

Storage

  • Each private service instance has access to its own ephemeral storage. It's removed after the private service instances is removed.
  • It has a fixed size of 20GB.
  • If you have 2 concurrently running private service instances, they do not share this storage.
  • To store data persistently, consider using Buckets.

Accessing other resources

  • For most of the AWS resources, resource-to-resource communication is not allowed by default. This helps to enforce security and resource isolation. Access must be explicitly granted using IAM (Identity and Access Management) permissions.

  • Access control of Relational Databases is not managed by IAM. These resources are not "cloud-native" by design and have their own access control mechanism (connection string with username and password). They are accessible by default, and you don't need to grant any extra IAM permissions. You can further restrict the access to your relational databases by configuring their access control mode.

  • Stacktape automatically handles IAM permissions for the underlying AWS services that it creates (i.e. granting web services permission to write logs to Cloudwatch, allowing private services to communicate with their event source and many others).


If your compute resource needs to communicate with other infrastructure components, you need to add permissions manually. You can do this in 2 ways:

Using connectTo

  • List of resource names or AWS services that this service will be able to access (basic IAM permissions will be granted automatically). Granted permissions differ based on the resource.
  • Works only for resources managed by Stacktape in resources section (not arbitrary Cloudformation resources)
  • This is useful if you don't want to deal with IAM permissions yourself. Handling permissions using raw IAM role statements can be cumbersome, time-consuming and error-prone. Moreover, when using connectTo property, Stacktape automatically injects information about resource you are connecting to as environment variables into your workload.

Copy

resources:
photosBucket:
type: bucket
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/index.ts
connectTo:
# access to the bucket
- photosBucket
# access to AWS SES
- aws:ses
resources:
cpu: 0.25
memory: 512


By referencing resources (or services) in connectTo list, Stacktape automatically:

  • configures correct compute resource's IAM role permissions if needed
  • sets up correct security group rules to allow access if needed
  • injects relevant environment variables containing information about resource you are connecting to into the compute resource's runtime
    • names of environment variables use upper-snake-case and are in form STP_[RESOURCE_NAME]_[VARIABLE_NAME],
    • examples: STP_MY_DATABASE_CONNECTION_STRING or STP_MY_EVENT_BUS_ARN,
    • list of injected variables for each resource type can be seen below.

Granted permissions and injected environment variables are different depending on resource type:


Bucket

  • Permissions:
    • list objects in a bucket
    • create / get / delete / tag object in a bucket
  • Injected env variables: NAME, ARN

DynamoDB table

  • Permissions:
    • get / put / update / delete item in a table
    • scan / query a table
    • describe table stream
  • Injected env variables: NAME, ARN, STREAM_ARN

MongoDB Atlas cluster

  • Permissions:
    • Allows connection to a cluster with accessibilityMode set to scoping-workloads-in-vpc. To learn more about MongoDB Atlas clusters accessibility modes, refer to MongoDB Atlas cluster docs.
    • Creates access "user" associated with compute resource's role to allow for secure credential-less access to the the cluster
  • Injected env variables: CONNECTION_STRING

Relational(SQL) database

  • Permissions:
    • Allows connection to a relational database with accessibilityMode set to scoping-workloads-in-vpc. To learn more about relational database accessibility modes, refer to Relational databases docs.
  • Injected env variables: CONNECTION_STRING, JDBC_CONNECTION_STRING, HOST, PORT (in case of aurora multi instance cluster additionally: READER_CONNECTION_STRING, READER_JDBC_CONNECTION_STRING, READER_HOST)

Redis cluster

  • Permissions:
    • Allows connection to a redis cluster with accessibilityMode set to scoping-workloads-in-vpc. To learn more about redis cluster accessibility modes, refer to Redis clusters docs.
  • Injected env variables: HOST, READER_HOST, PORT

Event bus

  • Permissions:
    • publish events to the specified Event bus
  • Injected env variables: ARN

Function

  • Permissions:
    • invoke the specified function
    • invoke the specified function via url (if lambda has URL enabled)
  • Injected env variables: ARN

Batch job

  • Permissions:
    • submit batch-job instance into batch-job queue
    • list submitted job instances in a batch-job queue
    • describe / terminate a batch-job instance
    • list executions of state machine which executes the batch-job according to its strategy
    • start / terminate execution of a state machine which executes the batch-job according to its strategy
  • Injected env variables: JOB_DEFINITION_ARN, STATE_MACHINE_ARN

User auth pool

  • Permissions:
    • full control over the user pool (cognito-idp:*)
    • for more information about allowed methods refer to AWS docs
  • Injected env variables: ID, CLIENT_ID, ARN


SNS Topic

  • Permissions:
    • confirm/list subscriptions of the topic
    • publish/subscribe to the topic
    • unsubscribe from the topic
  • Injected env variables: ARN, NAME


SQS Queue

  • Permissions:
    • send/receive/delete message
    • change visibility of message
    • purge queue
  • Injected env variables: ARN, NAME, URL

Upstash Kafka topic

  • Injected env variables: TOPIC_NAME, TOPIC_ID, USERNAME, PASSWORD, TCP_ENDPOINT, REST_URL

Upstash Redis

  • Injected env variables: HOST, PORT, PASSWORD, REST_TOKEN, REST_URL, REDIS_URL

Private service

  • Injected env variables: ADDRESS

aws:ses(Macro)

  • Permissions:
    • gives full permissions to aws ses (ses:*).
    • for more information about allowed methods refer to AWS docs

Using iamRoleStatements

  • List of raw IAM role statement objects. These will be appended to the private service's role.
  • Allows you to set granular control over your private service's permissions.
  • Can be used to give access to any Cloudformation resource

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: server/index.ts
iamRoleStatements:
- Resource:
- $CfResourceParam('NotificationTopic', 'Arn')
Effect: 'Allow'
Action:
- 'sns:Publish'
resources:
cpu: 2
memory: 2048
cloudformationResources:
NotificationTopic:
Type: 'AWS::SNS::Topic'

StpIamRoleStatement  API reference
Resource
Required
Sid
Effect
Action
Condition

Load balancing

LoadBalancing property is used to configure what type of entry point is used for distributing traffic to containers:

  • Supported types are: service-connect and application-load-balancer.

service-connect

  • distributes traffic to the available containers evenly
  • connections are only possible from other container-based resources of the stack (web-services, worker-services, multi-container-workloads)
  • support any TCP protocol
  • this option is vastly cheaper - you only pay $0.5 a month for background resource (private cloud map dns namespace)

application-load-balancer

  • distributes traffic to the available containers in a round robin fashion
  • supports HTTP protocol only
  • uses pricing which is combination of flat hourly charge(~$0.0252/hour) and used LCUs(Load Balancer Capacity Units)(~$0.08/hour)
  • is eligible for free tier, for better understanding of pricing refer to AWS docs
PrivateServiceLoadBalancing  API reference
type
Required

Copy

resources:
webService:
type: web-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/main.ts
resources:
cpu: 2
memory: 2048
loadBalancing:
type: application-load-balancer # default is http-api-gateway

Default VPC connection

Port

Optionally you can specify your own preferred alias or port

  • Port is injected into service(container) runtime as environment variable PORT
  • By default port 3000 is used.

If loadBalancing property is set to service-connect(default), then connecting to private-service is only possible from other container-based resources (web-services, worker-services, multi-container-workloads). For more information see also loadBalancing property.

Copy

resources:
myPrivateService:
type: private-service
properties:
packaging:
type: stacktape-image-buildpack
properties:
entryfilePath: src/main.ts
resources:
cpu: 2
memory: 2048
port: 8080

Referenceable parameters

The following parameters can be easily referenced using $ResourceParam directive directive.

To learn more about referencing parameters, refer to referencing parameters.

address
  • service host:port pair accessible only to other resources of stack(web-services, multi-container-workloads)

  • Usage: $ResourceParam('<<resource-name>>', 'address')

Pricing

You are charged for:

  • Virtual CPU / hour:

    • depending on the region $0.04048 - $0.0696
  • Memory GB / hour:

    • depending on the region $0.004445 - $0.0076

The duration is rounded to 1 second with a 1 minute minimum. To learn more, refer to AWS Fargate pricing.

API reference

StpIamRoleStatement  API reference
Resource
Required
Sid
Effect
Action
Condition

Need help? Ask a question on Discord or info@stacktape.com.