Private Services
Overview
Private service is a computing resource - it runs your code. Private service runs continuously and scales based on the CPU and RAM usage.
Private service has a private address. This address is only reachable by other resources of the stack (web-services, container-workloads).
Similarly to functions and batch jobs, private services are serverless and fully managed. This means you don't have to worry about administration tasks such as provisioning and managing servers, scaling, VM security, OS security & much more.
The container image can be supplied in 3 different ways:
- built automatically from your source code by Stacktape
- built using a supplied Dockerfile by Stacktape
- pre-built images
Private services run securely within a Virtual Private Cloud (VPC).
When to use
Advantages
- Control over underlying environment - Private service can run any Docker image or image built using your own Dockerfile.
- Price for workloads with predictable load - Compared to functions, private services are cheaper if your workload has a predictable load.
- Load-balanced and auto-scalable - Private services can automatically horizontally scale based on the CPU and Memory utilization.
- High availability - Private services run in multiple Availability Zones.
- Secure by default - Underlying environment is securely managed by AWS.
Disadvantages
Scaling speed - Unlike lambda functions that scale almost instantly, private service requires more time - from several seconds to few minutes to add another container.
Not fully serverless - While private services can automatically scale up and down, they can't scale to 0. This means, if your workload is unused, you are still paying for at least one instance (minimum ~$8/month)
Basic usage
Copy
resources:myPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/main.tsresources:cpu: 2memory: 2048
Example private service config
Copy
import express from 'express';const app = express();app.get('/', async (req, res) => {res.send({ message: 'Hello' });});// for your app use port number stored in PORT environment variable for your application// this environment variable is automatically injected by Stacktapeapp.listen(process.env.PORT, () => {console.info(`Server running on port ${process.env.PORT}`);});
Example server container written in Typescript (main.ts)
Connecting to a Private service
Private services do not have publicly accessible HTTPS URLs.
Instead, they have service address
which is a host:port
pair.
host
is by default lowercased resource name(can be changed by specifying alias)port
number is 3000 by default(can be changed by specifying port)
Following example shows privateApi
service. host:port
pair in this case will be privateapi:3000
.
You can connect to this address directly from other resources (web-services, container-workloads) of your stack.
You may need to append a URL scheme to the private service address depending on your app requirements. For example, apps that rely on HTTP api endpoint of sorts might require you to specify it, i.e
http://privateapi:3000
Copy
resources:publicService:type: web-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/main.tsresources:cpu: 1memory: 1024environment:# injecting privateApi address- name: PRIVATE_ADDRESSvalue: $ResourceParam('privateApi', 'address')privateApi:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/private-api/main.tsresources:cpu: 1memory: 1024
Image
- Private service is a running instance of a Docker image.
- The image of the container can be supplied in 4 different ways:
- images built using stacktape-image-buildpack
- images built using external-buildpack
- images built from the custom-dockerfile
- prebuilt-images
Environment variables
Most commonly used types of environment variables:
- Static - string, number or boolean (will be stringified).
- Result of a custom directive.
- Referenced property of another resource (using $ResourceParam directive). To learn more, refer to referencing parameters guide.
- Value of a secret (using $Secret directive).
Copy
resources:myPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/main.tsenvironment:- name: STATIC_ENV_VARvalue: my-env-var- name: DYNAMICALLY_SET_ENV_VARvalue: $MyCustomDirective('input-for-my-directive')- name: DB_HOSTvalue: $ResourceParam('myDatabase', 'host')- name: DB_PASSWORDvalue: $Secret('dbSecret.password')resources:cpu: 2memory: 2048
Healthcheck
The purpose of the container health check is to monitor the health of the container from the inside.
Once an essential container of an instance is determined UNHEALTHY, the instance is automatically replaced with a new one.
- Example: A shell command sends a
curl
request every 20 seconds to determine if the service is available. If the request fails (or doesn't return in 5 seconds), the command returns with non-zero exit code, and the healtcheck is considered failed.
Copy
resources:mymyPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/index.tsinternalHealthCheck:healthCheckCommand: ['CMD-SHELL', 'curl -f http://localhost/ || exit 1']intervalSeconds: 20timeoutSeconds: 5startPeriodSeconds: 150retries: 2resources:cpu: 2memory: 2048
Shutdown
- When a running private service instance is deregistered (removed), all running containers receive a
SIGTERM
signal. - You then have 30 seconds to clean up. After 30 seconds, your process recives a
SIGKILL
signal.
Copy
process.on('SIGTERM', () => {console.info('Received SIGTERM signal. Cleaning up and exiting process...');// Finish any outstanding requests, or close a database connection...process.exit(0);});
Cleaning up before container shutdown.
Logging
- Every time your code outputs (prints) something to the
stdout
orstderr
, your log will be captured and stored in a AWS CloudWatch log group. - You can browse your logs in 2 ways:
- go to your private service's log-group in the AWS CloudWatch console. You can use
stacktape stack-info
command to get a direct link. - use stacktape logs command that will print logs to the console
- go to your private service's log-group in the AWS CloudWatch console. You can use
Resources
- You must configure computing resources (CPU and RAM) for your private service.
- Configured resources are valid for every container instance in the private service (if there are more than 1 instance running).
Copy
resources:mymyPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/index.tsresources:cpu: 0.25memory: 512
Scaling
- Configures scaling behavior of your private service. You can configure:
- Initial, minimum and maximum amount of concurrently running instances in your workload.
- Conditions which trigger the scaling (up or down) using a scaling policy.
Scaling policy
A scaling policy specifies CPU and memory metric thresholds which trigger the scaling process.
Depending on the thresholds, the workload can either scale out (add instances) or scale in (remove instances).
If both
keepAvgCpuUtilizationUnder
andkeepAvgMemoryUtilizationUnder
are used, the workload will scale-out if one of the metrics is above the target value. However, to scale in, both of these metrics need to be below their respective target values.Scaling policy is more aggressive in adding capacity then removing capacity. For example, if the policy's specified metric reaches its target value, the policy assumes that your application is already heavily loaded. So it responds by adding capacity proportional to the metric value as fast as it can. The higher the metric, the more capacity is added.
When the metric falls below the target value, the policy expects that utilization will eventually increase again. Therefore it slows down the scale-in process by removing capacity only when utilization passes a threshold that is far enough below the target value (usually 20% lower).
Copy
resources:mymyPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/index.tsresources:cpu: 0.5memory: 1024scaling:minInstances: 1maxInstances: 5scalingPolicy:keepAvgMemoryUtilizationUnder: 80keepAvgCpuUtilizationUnder: 80
Example usage of scaling configuration
Storage
- Each private service instance has access to its own ephemeral storage. It's removed after the private service instances is removed.
- It has a fixed size of 20GB.
- If you have 2 concurrently running private service intances, they do not share this storage.
- To store data persistently, consider using Buckets.
Accessing other resources
For most of the AWS resources, resource-to-resource communication is not allowed by default. This helps to enforce security and resource isolation. Access must be explicitly granted using IAM (Identity and Access Management) permissions.
Access control of Relational Databases is not managed by IAM. These resources are not "cloud-native" by design and have their own access control mechanism (connection string with username and password). They are accessible by default, and you don't need to grant any extra IAM permissions. You can further restrict the access to your relational databases by configuring their access control mode.
Stacktape automatically handles IAM permissions for the underlying AWS services that it creates (i.e. granting web services permission to write logs to Cloudwatch, allowing private services to communicate with their event source and many others).
If your workload needs to communicate with other infrastructure components, you need to add permissions manually. You can do this in 2 ways:
Using allowAccessTo
- List of resource names or AWS services that this private service will be able to access (basic IAM permissions will be granted automatically). Granted permissions differ based on the resource.
- Works only for resources managed by Stacktape (not arbitrary Cloudformation resources)
- This is useful if you don't want to deal with IAM permissions yourself. Handling permissions using raw IAM role statements can be cumbersome, time-consuming and error-prone.
Copy
resources:photosBucket:type: bucketmymyPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/index.tsaccessControl:allowAccessTo:# access to the bucket- photosBucket# access to AWS SES- aws:sesresources:cpu: 0.25memory: 512
Granted permissions when using with specific resource:
- Bucket
- list objects in a bucket
- create / get / delete / tag object in a bucket
- DynamoDb Table
- get / put / update / delete item in a table
- scan / query a table
- describe table stream
- MongoDB Atlas Cluster
- Allows connection to a cluster with
accessibilityMode
set toscoping-workloads-in-vpc
. To learn more about MongoDB Atlas clusters accessibility modes, refer to MongoDB Atlas cluster docs.
- Allows connection to a cluster with
- Relational database
- Allows connection to a relational database with
accessibilityMode
set toscoping-workloads-in-vpc
. To learn more about relational database accessibility modes, refer to Relational databases docs.
- Allows connection to a relational database with
- Redis cluster
- Allows connection to a redis cluster with
accessibilityMode
set toscoping-workloads-in-vpc
. To learn more about redis cluster accessibility modes, refer to Redis clusters docs.
- Allows connection to a redis cluster with
- Event bus
- publish events to the specified Event bus
- Function
- invoke the specified function
- Batch job
- submit batch-job instance into batch-job queue
- list submitted job instances in a batch-job queue
- describe / terminate a batch-job instance
- list executions of state machine which executes the batch-job according to its strategy
- start / terminate execution of a state machine which executes the batch-job according to its strategy
- User auth pool
- full control over the user pool (
cognito-idp:*
) - for more information about allowed methods refer to AWS docs
- full control over the user pool (
Granted permissions when using with AWS service:
- aws:ses (Simple Email Service)
- gives full permissions to aws ses (
ses:*
). - for more information about allowed methods refer to AWS docs
- gives full permissions to aws ses (
Using iamRoleStatements
- List of raw IAM role statement objects. These will be appended to the private service's role.
- Allow you to set granular control over your private service's permissions.
- Can be used to give access to any Cloudformation resource
Copy
resources:mymyPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: server/index.tsaccessControl:iamRoleStatements:- Resource:- $CfResourceParam('NotificationTopic', 'Arn')Effect: 'Allow'Action:- 'sns:Publish'resources:cpu: 2memory: 2048cloudformationResources:NotificationTopic:Type: 'AWS::SNS::Topic'
Default VPC connection
- Certain AWS services (such as Relational Databases) must be connected to a
VPC (Virtual private cloud) to be able to run. For stacks that include these resources, Stacktape
does 2 things:
- creates a default VPC
- connects the VPC-requiring resources to the default VPC.
- private services are connected to the default VPC of your stack by default. This means that private services can
communcate with resources that have their accessibility mode set to
vpc
without any extra configuration. - To learn more about VPCs and accessibility modes, refer to VPC docs, accessing relational databases, accessing redis clusters and accessing MongoDb Atlas clusters.
Alias and Port
Optionally you can specify your own prefered alias or port
- Combination of
alias
(host) andport
creates a unique identifier(address). You can then reach service on the address, i.e. using URL in formprotocol://alias:port
for examplehttp://my-service:8080
orgrpc://appserver:8080
. - By default alias is lowercased name of the resource.
- Port is injected into service(container) runtime as environment variable
PORT
- By default port
3000
is used.
Copy
resources:myPrivateService:type: private-serviceproperties:packaging:type: stacktape-image-buildpackproperties:entryfilePath: src/main.tsresources:cpu: 2memory: 2048alias: my-endpointport: 8080
Referenceable parameters
The following parameters can be easily referenced using $ResourceParam directive directive.
To learn more about referencing parameters, refer to referencing parameters.
service
host:port
pair accessible only to other resources of stack(web-services, container-workloads)- Usage:
$ResourceParam('<<resource-name>>', 'address')
Pricing
You are charged for:
Virtual CPU / hour:
- depending on the region $0.04048 - $0.0696
Memory GB / hour:
- depending on the region $0.004445 - $0.0076
The duration is rounded to 1 second with a 1 minute minimum. To learn more, refer to AWS Fargate pricing.