logoStacktape docs


Mongo Db Atlas ClustersWork in progress
Work in progress

Overview and basic concepts

  • MongoDb Atlas cluster is a schemaless, NoSQL database fully managed by MongoDB Inc. Unlike other resources, MongoDb Atlas cluster is not an AWS-native service. However, Stacktape seamlessly integrates it into your stacks.

  • MongoDb Atlas cluster is a secure, scalable, highly available and performant database. It has built-in replication, supports backups and Point-In-Time recovery.

  • Every stack that includes a MongoDb Atlas cluster will additionally create a new MongoDB Atlas project. This ensures isolation between stacks.

  • MongoDb includes an SDK for virtually any programming language. It allows you to easily query your database. Furthermore, you can configure MongoDb clusters to use a BI (Business Intelligence) connector that allows an SQL-based acess.

When to use

Advantages

  • Secure
  • Scalable
  • Performant
  • Supports backups and Point-In-Time recovery
  • High availability
  • Supports Multi-Document ACID Transactions

Disadvantages

  • Separate billing - Eventhough Stacktape seamlessly integrates MongoDb to your stacks, you still need to manage your billing separately.

Provider configuration

  • You must have a MongoDb Atlas account. To create one, head over to MongoDb Atlas registration page.

  • You need following to configure the provider:

    • Atlas Mongo organizationId
    • Atlas Mongo publicKey
    • Atlas Mongo privateKey
  • You can obtain them by following our step-by-step guide.

  • The recommended way to store your credentials is to use secrets.

Copy

serviceName: stack-name
providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2

MongoDbAtlasProviderConfig  API reference
publicKey
Required

MongoDb Atlas public API key

Type: string

privateKey
Required

MongoDb Atlas private API key

Type: string

  • You can get API Keys for you organization by following guide in the docs
  • For security reasons, you should store your credentials as secrets. To learn more about secrets, refer to secrets guide
organizationId
Required

MongoDB Atlas Organization Id

Type: string

accessibility

Configures connectivity settings of the MongoDB Atlas project

Type: MongoDbAtlasAccessibility

  • If your stack contains a MongoDb Atlas cluster, Stacktape automatically creates MongoDb Atlas Project associated with the stack.
  • All of the MongoDb Atlas clusters are deployed within this project
  • Network connectivity to clusters is configured on a per project level. As a result, accessibility settings here are applied to all MongoDb Atlas clusters within your stack.

Basic Usage

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M2
myLambda:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: path/to/my/lambda.ts
environment:
- name: MONGODB_CONNECTION_STRING
value: $ResourceParam('myMongoDbCluster', 'connectionString')
accessControl:
allowAccessTo:
- myMongoDbCluster

Copy

import { MongoClient } from 'mongodb';
const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING);
const handler = async (event, context) => {
await client.connect();
const db = client.db('mydb');
await db.collection('posts').insertOne({
title: 'My first post',
content: 'Hello!'
});
const post = await db.collection('posts').findOne({ title: 'My first post' });
await client.close();
};

Cluster tier

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2

Disk size

  • Each cluster tier comes with a default storage size.
  • All M10+ clusters autoscale the storage without any manual intervention. You can disable this behavior by configuring autoscaling properties
  • All M10+ clusters also provide the ability to customize your storage capacity.

Copy

resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M2
diskSizeGB: 60

Auto-scaling

  • You can configure your cluster to automatically scale its cluster tier, storage capacity, or both based on the cluster usage.
  • To help control the costs, you can select a range of cluster tiers to which your cluster can scale to.
  • Cluster is scaled up (to the next tier), if one the following criteria is met:
    • Average CPU Utilization has exceeded 75% for the past hour
    • Memory Utilization has exceeded 75% for the past hour
  • Cluster is scaled down (to the lower tier), if both of the following criteria are met:
    • The average CPU Utilization and Memory Utilization over the past 24 hours is below 50%
    • The cluster has not been scaled down (manually or automatically) in the past 24 hours

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
autoScaling:
minClusterTier: M10
maxClusterTier: M30
disableDiskScaling: true
disableScaleDown: true

MongoDbAutoScaling  API reference
Parent API reference: MongoDbAtlasCluster
minClusterTier

Minimum cluster tier to scale DOWN to

Type: string ENUM

Possible values: M10M100M140M20M200M200 Low-CPU (R200)M200_NVMEM30M300M300 Low-CPU (R300)M40M40 Low-CPU (R40)M400 Low-CPU (R400)M400_NVMEM40_NVMEM50M50 Low-CPU (R50)M50_NVMEM60M60 Low-CPU (R60)M60_NVMEM700 Low-CPU (R700)M80M80 Low-CPU (R80)M80_NVME

maxClusterTier

Maximum cluster tier to scale UP to

Type: string ENUM

Possible values: M10M100M140M20M200M200 Low-CPU (R200)M200_NVMEM30M300M300 Low-CPU (R300)M40M40 Low-CPU (R40)M400 Low-CPU (R400)M400_NVMEM40_NVMEM50M50 Low-CPU (R50)M50_NVMEM60M60 Low-CPU (R60)M60_NVMEM700 Low-CPU (R700)M80M80 Low-CPU (R80)M80_NVME

disableDiskScaling

Disables disk size scaling

Type: boolean

  • When disk scaling is enabled, cluster storage is automatically increased when disk space usage reaches 90%.
  • Cluster storage scaling aims to achieve 70% disk utilization.
  • Cluster storage is never automatically scaled down.
disableScaleDown

Disables scale down of cluster tier

Type: boolean

  • When scale down is disabled, cluster can only scale Up (to higher tier).

Sharding

  • If you configure more than 1 shard, the cluster will run in a sharded mode.
  • Sharding distributes data across multiple physical machines enabling horizontal scaling.
  • Sharded mode is available only for cluster tiers M30 or higher.
  • To learn more about sharding, refer to MongoDb Docs.

Copy

resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M30
numShards: 3

Backups

  • Backups are copies of your data that encapsulate the state of your cluster at a given time. Backups provide a safety measure in the event of a data loss.
  • The default snapshot time is every day at 18:00 UTC.
  • Available only in M10+ Clusters.
  • Snapshots are automatically taken even for M2/M5 clusters, but have different properties. To learn more, refer to [M2 and M5 backups docs](https://docs.atlas.mongodb.com/backup-restore-cluster/#m2

There are different types of snapshots with different retention period and frequency:

  • Hourly snapshot: every 6 hours, retained for 2 days
  • Daily snapshot: every day, retained for 7 days
  • Weekly snapshot: every Saturday, retained for 4 weeks
  • Monthly snapshot: last day of the month, retained for 12 months

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
enableBackups: true

Point-in-time recovery

  • Enables Continuous Cloud Backups, which replay the oplog (history of ordered logical writes) and enables you to restore a cluster from a particular point in time.
  • You can make a point in time recovery to any point within the last 7 days.
  • Available only in M10+ Clusters.
  • If you enable point-in-time recovery, you must also enable backups.
  • MongoDb Continuous cloud backup includes additional charges. To learn more, refer to MongoDb Continuous cloud backup pricing Docs

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
enablePointInTimeRecovery: true

Accessing clusters from workloads

  • Following example shows how to grant a lambda function myMongoFunction a permission to access your mongo-db-atlas-cluster myMongoCluster.

  • By listing myMongoCluster in allowAccessTo of myMongoFunction, function is injected with credentials needed for accessing the cluster.

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2
myMongoFunction:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: 'lambdas/mongo-lambda.ts'
memory: 512
# by allowing access to cluster, lambda receives permissions for reading and writing into cluster databases
accessControl:
allowAccessTo:
- 'myMongoCluster'
environment:
# injecting the connection string as environment variable
- name: MONGODB_CONNECTION_STRING
value: $ResourceParam('myMongoCluster', 'connectionString')

Accessibility

  • Stacktape allows you to restrict the accessibility of your MongoDb cluster to only certain resources or hosts.
  • In contrary to other Stacktape database resources, accesibility is set globally in your MongoDb Atlas provider configuration, and not on the resource itself. This means every MongoDb cluster in the given stack must have the same accessibility mode.
  • When using shared cluster tiers (M2 and M5), you must set accessibility mode to internet. This is because shared clusters does not support VPC peering. clusters(M10+). and therefore do not allow for same level of network inter-connection. Nevertheless, even when using accessibility set to internet, clusters are still protected as Stacktape utilizes strict identity access management between your stack workloads (functions, container-workloads, batch-jobs) and atlas mongo clusters. See section Accessing clusters from workloads
MongoDbAtlasAccessibility  API reference
Parent API reference: MongoDbAtlasProviderConfig
accessibilityMode
Default: internetRequired

Configures the accessibility mode for this database

Type: string ENUM

Possible values: internetscoping-workloads-in-vpcvpcwhitelisted-ips-only

The following modes are supported:

  • internet - Least restrictive mode. The database can be accessed from anywhere on the internet.
  • vpc - The database can be accessed only from resources within your VPC. This means any function (provided it has joinDefaultVpc set to true), batch job or container workload within your stack can access the cluster. Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).
  • scoping-workloads-in-vpc - similar to vpc mode, but even more restrictive. In addition to being in the same VPC, the resources and hosts accessing your database must also have sufficient IAM permissions (for functions, batch jobs and container workloads, these permissions can be granted using allowsAccessTo or iamRoleStatements in their configuration). Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).
  • whitelisted-ips-only - The database can only be accessed from an IP addresses and CIDR blocks listed in the whitelistedIps list.

To learn more about VPCs, refer to VPC Docs.

whitelistedIps

List of IP addresses or IP ranges (in CIDR format)

Type: Array of string

The behavior of this property varies based on accessibilityMode:

  • in the internet mode, this property has no effect as the database is are already accessible from everywhere.
  • in the vpc mode and scoping-workloads-in-vpc mode, these IP addresses/ranges can be used to allow access from a specific addresses outside of the VPC (i.e IP address of your office).
  • in the whitelisted-ips-only mode, these addresses/ranges are the only addresses that can access the database.

Internet mode

  • Default mode. Least restrictive. The cluster can be accessed from anywhere on the internet.
  • Your MongoDB cluster is still protected, because any host or resource (workload) trying to access your cluster requires IAM (Identity and Access management) permissions (it is also possible to use admin user name and password, but using these credentials is only recommended in cases you cannot use IAM permissions).
  • You can grant IAM permissions to your workloads by configuring allowAccessTo on your (functions, container workloads or batch jobs)

VPC mode

  • In addition to being protected by IAM (Identity and Access management) as described in Internet mode, the mongo cluster is also protected on a network level. This means, that only functions (which have joinDefaultVpc set to true), batch jobs or container workloads within your stack can access the cluster.

  • Additionally, IP addresses configured in whitelistedIps can also access the cluster (even from the internet).

  • When using this mode, the traffic which originates from your stack resources (workloads) to your mongo cluster never leaves AWS network infrastrucutre. This makes the communication cheaper.

Copy

providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
# "accessibility" option is shared between "mongo-db-atlas-cluster" resources of your stack
accessibility:
accessibilityMode: vpc
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10

Scoping workloads in VPC mode

  • Similar to vpc mode, but even more restrictive in terms of network protection. When using this mode allowAccessTo not only grants the required IAM (Identity and Access management) permissions, but also grants access on the network level.

  • Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).

  • When using this mode, the traffic which originates from your stack resources (workloads) to your mongo cluster never leaves AWS network infrastrucutre. This makes the communication cheaper.

Copy

providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
# "accessibility" option is shared between "mongo-db-atlas-cluster" resources of your stack
accessibility:
accessibilityMode: scoping-workloads-in-vpc
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
# functionOne does NOT have access to database eventhough it is joined in vpc
functionOne:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: 'path/to/my-lambda.ts'
joinDefaultVpc: true
# functionTwo does have access to database, because it is scoping the database in allowAccessTo list
functionTwo:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: 'path/to/my-lambda-2.ts'
joinDefaultVpc: true
accessControl:
allowAccessTo:
- 'myMongoCluster'
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10

Whitelisted IPs only mode

  • The database can only be accessed from an IP addresses and CIDR blocks listed in the whitelistedIps list.

Copy

providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
# "accessibility" option is shared between "mongo-db-atlas-cluster" resources of your stack
accessibility:
accessibilityMode: whitelisted-ips-only
whitelistedIps:
- 193.12.16.4
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10

Admin user

  • Optionally, you can create an admin database user user with administrative access privileges.
  • Accessing the cluster from your workloads (batch-jobs, container-workloads or functions), is possible even without creating this user.
  • Creating an admin user can be useful for performing administrative tasks, or when connecting to the cluster from a local machine.
MongoDbAdminUserCredentials  API reference
Parent API reference: MongoDbAtlasCluster
userName
Required

Name of the admin user

Type: string

password
Required

Password for the admin user

Type: string

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2
adminUserCredentials:
userName: my-master-user
password: $Secret('mongo-master-password')

Business Intelligence Connector (SQL)

  • Traditional business intelligence tools are designed to work with tabular, row-and-column data. The MongoDB Connector for BI allows you to query MongoDB data with SQL using tools such as Tableau, Power BI or Excel.
MongoDbBiConnector  API reference
Parent API reference: MongoDbAtlasCluster
enabled
Required

Enables BI connector

Type: boolean

readPreference

Configures the type of node to which the BI connector will connect

Type: string ENUM

Possible values: analyticsprimarysecondary

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: 'M10'
biConnector:
enabled: true

Referenceable parameters

The following parameters can be easily referenced using $ResourceParam directive directive.

To learn more about referencing parameters, refer to referencing parameters.

connectionString
  • Connection string (URL) that allows connecting to the cluster.

  • Usage: $ResourceParam('<<resource-name>>', 'connectionString')

Pricing

You are charged for:

  • Cluster tier:
  • Data transfer:
    • For most customers, data transfer fees amount to less than 10% of their bill.
    • Data transfer fees are not assessed on M0, M2, and M5 instances.
    • Same region: $0.01/GB, Cross-region: $0.02/GB, Internet: $0.09/GB
    • For a more detailed overview, refer to MongoDb billing docs

API reference

MongoDbReplication  API reference
Parent API reference: MongoDbAtlasCluster
numAnalyticsNodes

Number of analytics nodes

Type: number

  • Analytics nodes are read only nodes meant for long-running queries.
  • Executing long-running queries on basic read-only or electable nodes can hinder overall performance of the cluster.
  • By using analytics nodes these queries you avoid a performance impact on your operational workload handled by electable and basic read only nodes.
  • Analytics nodes work well for biConnector with read preference set on analytics
numElectableNodes
Default: 3

Number of electable nodes (can become a primary node)

Type: number ENUM

Possible values: 357

  • Adding electable nodes improves the redundancy and availability of a cluster.
numReadOnlyNodes

Number of read only nodes

Type: number

MongoDbAtlasCluster  API reference
type
Required

No description

Type: string "mongo-db-atlas-cluster"

properties.clusterTier
Required

Configures resources (memory, default storage, IOPS specification) for each data-bearing node in the cluster

Type: string ENUM

Possible values: M10M100M140M2M20M200M200 Low-CPU (R200)M200_NVMEM30M300M300 Low-CPU (R300)M40M40 Low-CPU (R40)M400 Low-CPU (R400)M400_NVMEM40_NVMEM5M50M50 Low-CPU (R50)M50_NVMEM60M60 Low-CPU (R60)M60_NVMEM700 Low-CPU (R700)M80M80 Low-CPU (R80)M80_NVME

properties.diskSizeGB

Size of the disk

Type: number

Setting disk size is NOT available for shared clusters (M2 and M5). For M10+ clusters default disk size depends on the instance size (see MongoDB Atlas docs).

properties.numShards
Default: 1

Amount of shards for the cluster

Type: number

  • If you configure more than 1 shard, the cluster will run in a sharded mode.
  • Sharding distributes data across multiple physical machines enabling horizontal scaling.
  • Sharded mode is available only for cluster tiers M30 or higher.
  • To learn more about sharding, refer to MongoDb Docs.
properties.replication

Configures additional nodes for your cluster

Type: MongoDbReplication

  • Increasing number of nodes can lead to better redundancy and performance.
  • By default, each cluster has 3 data-bearing nodes (electable to master).
properties.enableBackups

Configures backups for the cluster

Type: boolean

  • Backups are copies of your data that encapsulate the state of your cluster at a given time. Backups provide a safety measure in the event of a data loss.
  • The default snapshot time is every day at 18:00 UTC.
  • Available only in M10+ Clusters.
  • Snapshots are automatically taken even for M2/M5 clusters, but have different properties. To learn more, refer to [M2 and M5 backups docs](https://docs.atlas.mongodb.com/backup-restore-cluster/#m2
properties.enablePointInTimeRecovery

Enables point in time recovery

Type: boolean

  • Enables Continuous Cloud Backups, which replay the oplog (history of ordered logical writes) and enables you to restore a cluster from a particular point in time.
  • You can make a point in time recovery to any point within the last 7 days.
  • Available only in M10+ Clusters.
  • If you enable point-in-time recovery, you must also enable backups.
  • MongoDb Continuous cloud backup includes additional charges. To learn more, refer to MongoDb Continuous cloud backup pricing Docs
properties.biConnector

Configures BI (Business Intelligence) connector

Type: MongoDbBiConnector

  • The BI Connector allows SQL-based access.
  • BI Connector performs operations which may be CPU and memory intensive
  • Given the limited hardware resources on M10 and M20 cluster tiers, you may experience performance degradation of the cluster when enabling the BI Connector.
properties.autoScaling

Configures scaling behavior of the cluster

Type: MongoDbAutoScaling

  • You can configure your cluster to automatically scale its cluster tier, storage capacity, or both based on the cluster usage.
  • To help control the costs, you can select a range of cluster tiers to which your cluster can scale to.
  • Cluster is scaled up (to the next tier), if one the following criteria is met:
    • Average CPU Utilization has exceeded 75% for the past hour
    • Memory Utilization has exceeded 75% for the past hour
  • Cluster is scaled down (to the lower tier), if both of the following criteria are met:
    • The average CPU Utilization and Memory Utilization over the past 24 hours is below 50%
    • The cluster has not been scaled down (manually or automatically) in the past 24 hours
properties.adminUserCredentials

Creates and configures MongoDb Atlas user atlasAdmin with specified credentials

Type: MongoDbAdminUserCredentials

  • Optionally, you can create an admin database user user with administrative access privileges.
  • Accessing the cluster from your workloads (batch-jobs, container-workloads or functions), is possible even without creating this user.
  • Creating an admin user can be useful for performing administrative tasks, or when connecting to the cluster from a local machine.
Need help? Ask a question on SlackDiscord or info@stacktape.com.