logoStacktape docs




Mongo Db Atlas Clusters

Overview and basic concepts

  • MongoDb Atlas cluster is a schemaless, NoSQL database fully managed by MongoDB Inc. Unlike other resources, MongoDb Atlas cluster is not an AWS-native service. However, Stacktape seamlessly integrates it into your stacks.

  • MongoDb Atlas cluster is a secure, scalable, highly available and performant database. It has built-in replication, supports backups and Point-In-Time recovery.

  • Every stack that includes a MongoDb Atlas cluster will additionally create a new MongoDB Atlas project. This ensures isolation between stacks.

  • MongoDb includes an SDK for virtually any programming language. It allows you to easily query your database. Furthermore, you can configure MongoDb clusters to use a BI (Business Intelligence) connector that allows an SQL-based acess.

When to use

Advantages

  • Secure
  • Scalable
  • Performant
  • Supports backups and Point-In-Time recovery
  • High availability
  • Supports Multi-Document ACID Transactions

Disadvantages

  • Separate billing - Even tho Stacktape seamlessly integrates MongoDb to your stacks, you still need to manage your billing separately.

Provider configuration

  • You must have a MongoDb Atlas account. To create one, head over to MongoDb Atlas registration page.

  • You need to configure:

    • organizationId: The easiest way to get it is logging to MongoDb Atlas console and copying it from the URL. Example: https://cloud.mongodb.com/v2/<<your-organization-id>>#clusters.
    • API credentials (publicKey and privateKey). Stacktape uses those to interact with your MongoDb atlas account. You can get API Keys for you organization by following MongoDB Atlas tutorial. For security reasons, you shouldn't input these directly. The recommended way is using a .env file together with a File directive or using secrets.

providerConfig:
mongoDbAtlas:
privateKey: xxxxxxxxxxxxxxxxx42543xx
publicKey: xxxxxxx
organizationId: xxxxxxxxxxxxxxxxxcbe63dd

MongoDbAtlasProviderConfig  API reference
publicKey
Required

MongoDb Atlas public API key

Type: string

privateKey
Required

MongoDb Atlas private API key

Type: string

  • You can get API Keys for you organization by following MongoDB Atlas docs tutorial
  • For security reasons, you should store your credentials as secrets. To learn more about secrets, refer to secrets guide

organizationId
Required

MongoDB Atlas Organization Id

Type: string

Basic Usage

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M2
myLambda:
type: function
properties:
packageConfig:
filePath: path/to/my/lambda.ts
environment:
- name: MONGODB_CONNECTION_STRING
value: $Param('myMongoDbCluster', 'AtlasMongoCluster::SrvConnectionString')
accessControl:
allowAccessTo:
- myMongoDbCluster

import { MongoClient } from 'mongodb';
const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING);
const handler = async (event, context) => {
await client.connect();
const db = client.db('mydb');
await db.collection('posts').insertOne({
title: 'My first post',
content: 'Hello!'
});
const post = await db.collection('posts').findOne({ title: 'My first post' });
await client.close();
};

Cluster tier

providerConfiguration:
mongoDbAtlas:
apiCredentials:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2

Disk size

  • Each cluster tier comes with a default storage size.
  • All M10+ clusters autoscale the storage without any manual intervention. You can disable this behavior by configuring autoscaling properties
  • All M10+ clusters also provide the ability to customize your storage capacity.

resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M2
diskSizeGB: 60

Auto-scaling

  • You can configure your cluster to automatically scale its cluster tier, storage capacity, or both based on the cluster usage.
  • To help control the costs, you can select a range of cluster tiers to which your cluster can scale to.
  • Cluster is scaled up (to the next tier), if one the following criteria is met:
    • Average CPU Utilization has exceeded 75% for the past hour
    • Memory Utilization has exceeded 75% for the past hour
  • Cluster is scaled down (to the lower tier), if both of the following criteria are met:
    • The average CPU Utilization and Memory Utilization over the past 24 hours is below 50%
    • The cluster has not been scaled down (manually or automatically) in the past 24 hours

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
autoScaling:
minClusterTier: M10
maxClusterTier: M30
disableDiskScaling: true
disableScaleDown: true

MongoDbAutoScaling  API reference
Parent API reference: MongoDbAtlasCluster
minClusterTier

Minimum cluster tier to scale DOWN to

Type: string ENUM

Possible values: M10M100M140M20M200M200 Low-CPU (R200)M200_NVMEM30M300M300 Low-CPU (R300)M40M40 Low-CPU (R40)M400 Low-CPU (R400)M400_NVMEM40_NVMEM50M50 Low-CPU (R50)M50_NVMEM60M60 Low-CPU (R60)M60_NVMEM700 Low-CPU (R700)M80M80 Low-CPU (R80)M80_NVME

maxClusterTier

Maximum cluster tier to scale UP to

Type: string ENUM

Possible values: M10M100M140M20M200M200 Low-CPU (R200)M200_NVMEM30M300M300 Low-CPU (R300)M40M40 Low-CPU (R40)M400 Low-CPU (R400)M400_NVMEM40_NVMEM50M50 Low-CPU (R50)M50_NVMEM60M60 Low-CPU (R60)M60_NVMEM700 Low-CPU (R700)M80M80 Low-CPU (R80)M80_NVME

disableDiskScaling

Disables disk size scaling

Type: boolean

  • When disk scaling is enabled, cluster storage is automatically increased when disk space usage reaches 90%.
  • Cluster storage scaling aims to achieve 70% disk utilization.
  • Cluster storage is never automatically scaled down.

disableScaleDown

Disables scale down of cluster tier

Type: boolean

  • When scale down is disabled, cluster can only scale Up (to higher tier).

Sharding

  • If you configure more than 1 shard, the cluster will run in a sharded mode.
  • Sharding distributes data across multiple physical machines enabling horizontal scaling.
  • Sharded mode is available only for cluster tiers M30 or higher.
  • To learn more about sharding, refer to MongoDb Docs.

resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M30
numShards: 3

Backups

  • Backups are copies of your data that encapsulate the state of your cluster at a given time. Backups provide a safety measure in the event of a data loss.
  • The default snapshot time is every day at 18:00 UTC.
  • Available only in M10+ Clusters.
  • Snaphots are automatically taken even for M2/M5 clusters, but have different properties. To learn more, refer to [M2 and M5 backups docs](https://docs.atlas.mongodb.com/backup-restore-cluster/#m2

There are different types of snapshots with different retention period and frequency:

  • Hourly snapshot: every 6 hours, retained for 2 days
  • Daily snapshot: every day, retained for 7 days
  • Weekly snapshot: every Saturday, retained for 4 weeks
  • Monthly snapshot: last day of the month, retained for 12 months

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
enableBackups: true

Point-in-time recovery

  • Enables Continuous Cloud Backups, which replay the oplog (history of ordered logical writes) and enables you to restore a cluster from a particular point in time.
  • You can make a point in time recovery to any point within the last 7 days.
  • Available only in M10+ Clusters.
  • If you enable point-in-time recovery, you must also enable backups.
  • MongoDb Continuous cloud backup includes additional charges. To learn more, refer to MongoDb Continuous cloud backup pricing Docs

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
enablePointInTimeRecovery: true

Accessing clusters from workloads

  • Following example shows how to grant a lambda function myMongoFunction a permission to access your mongo-db-atlas-cluster myMongoCluster.

  • By listing myMongoCluster in allowAccessTo of myMongoFunction, function is injected with credentials needed for accessing the cluster.

providerConfiguration:
mongoDbAtlas:
apiCredentials:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2
myMongoFunction:
type: function
properties:
packageConfig:
filePath: 'lambdas/mongo-lambda.ts'
memory: 512
# by allowing access to cluster, lambda receives permissions for reading and writing into cluster databases
accessControl:
allowAccessTo:
- 'myMongoCluster'
environment:
# injecting the connection string as environment variable
- name: MONGODB_CONNECTION_STRING
value: $Param('myMongoCluster', 'AtlasMongoCluster::SrvConnectionString')

Accessibility

  • Stacktape allows you to restrict the accessibility of your MongoDb cluster to only certain resources or hosts.
  • In contrary to other Stacktape database resources, accesibility is set globally in your MongoDb Atlas provider configuration, and not on the resource itself. This means every MongoDb cluster in the given stack must have the same accessibility mode.
  • When using shared cluster tiers (M2 and M5), you must set accessibility mode to internet. This is because shared clusters does not support VPC peering. clusters(M10+). and therefore do not allow for same level of network inter-connection. Nevertheless, even when using accessibility set to internet, clusters are still protected as Stacktape utilizes strict identity access management between your stack workloads (functions, container-workloads, batch-jobs) and atlas mongo clusters. See sectionAccessing clusters from workloads
MongoDbAtlasAccessibility  API reference
Parent API reference: MongoDbAtlasProviderConfig
accessibilityMode
Default: 'internet'Required

Configures the accessibility mode for this database

Type: string ENUM

Possible values: internetscoping-workloads-in-vpcvpcwhitelisted-ips-only

The following modes are supported:

  • internet - Least restrictive mode. The database can be accessed from anywhere on the internet.
  • vpc - The database can be accessed only from resources within your VPC. This means any function (provided it has joinDefaultVpc set to true), batch job or container workload within your stack can access the cluster. Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).
  • scoping-workloads-in-vpc - similar to vpc mode, but even more restrictive. In addition to being in the same VPC, the resources and hosts accessing your database must also have sufficient IAM permissions (for funtions, batch jobs and container workloads, these permissions can be granted using allowsAccessTo or iamRoleStatements in their configuration). Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).
  • whitelisted-ips-only - The database can only be accessed from an IP addresses and CIDR blocks listed in the whitelistedIps list.

To learn more about VPCs, refer to VPC Docs.

whitelistedIps

List of IP addresses or IP ranges (in CIDR format)

Type: Array of string

The behavior of this property varies based on accessibilityMode:

  • in the internet mode, this property has no effect as the database is are already accessible from everywhere.
  • in the vpc mode and scoping-workloads-in-vpc mode, these IP addresses/ranges can be used to allow access from a speicific addresses outside of the VPC (i.e IP address of your office).
  • in the whitelisted-ips-only mode, these addresses/ranges are the only addresses that can access the database.

Internet mode

  • Default mode. Least restrictive. The database can be accessed from anywhere on the internet.
  • Your MongoDb cluster is still protected, because any host or resource trying to access your cluster requires IAM (Identity and Access management) permissions.
  • You can grant these permission to your workloads in 2 ways:

VPC mode

  • The database can be accessed only from resources within the default VPC.
  • Any function (provided it has joinDefaultVpc set to true), batch job or container workload within your stack can access the cluster.
  • Additionally, IP addresses configured in whitelistedIps can also access the cluster (even from the internet).

resources:
auroraSlsPostgres:
type: 'relational-database'
properties:
credentials:
masterUserName: congor
masterUserPassword: mySecretPass
accessibility:
restrictAccess: 'vpc'
additionalWhitelistedIps:
- '147.25.33.12'
engine:
type: aurora-postgresql
properties:
instances:
- dbInstanceSize: db.t3.medium
port: 5432

Scoping workloads in VPC mode

  • Similar to vpc mode, but even more restrictive. In addition to resource being in the VPC, any host or resource trying to access your cluster must also have the sufficient IAM (Identity and Access management) permissions. You can grant these permission to your workloads in 2 ways:
  • Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).

resources:
# functionOne does NOT have access to database eventhough it is joined in vpc
functionOne:
type: function
properties:
packageConfig:
filePath: 'path/to/my-lambda.ts'
joinDefaultVpc: true
# functionTwo does have access to database, because it is scoping the database in allowAccessTo list
functionTwo:
type: function
properties:
packageConfig:
filePath: 'path/to/my-lambda-2.ts'
joinDefaultVpc: true
accessControl:
allowAccessTo:
- 'myPgSql'
myPgSql:
type: 'relation-database'
properties:
accessibility:
restrictAccess: 'scoping-workloads-in-vpc'
credentials:
masterUserName: commander
masterUserPassword: mySecretPassword
engine:
type: postgres
properties:
dbName: appdatabase
port: 5432
storage:
diskSizeGB: 20
instance:
dbInstanceSize: db.t3.micro

Whitelisted IPs only mode

  • The database can only be accessed from an IP addresses and CIDR blocks listed in the whitelistedIps list.

resources:
auroraSlsPostgres:
type: 'relational-database'
properties:
credentials:
masterUserName: congor
masterUserPassword: mySecretPass
accessibility:
restrictAccess: 'whitelisted-ips-only'
additionalWhitelistedIps:
- '147.25.33.12'
engine:
type: aurora-postgresql
properties:
instances:
- dbInstanceSize: db.t3.medium
port: 5432

Admin user

  • Optionally, you can create an admin database user user with administrative access priviliges.
  • Accessing the cluster from your workloads (batch-jobs, container-workloads or functions), is possible even without creating this user.
  • Creating an admin user can be useful for performing administrative tasks, or when connecting to the cluster from a local machine.
MongoDbAdminUserCredentials  API reference
Parent API reference: MongoDbAtlasCluster
userName
Required

Name of the admin user

Type: string

password
Required

Password for the admin user

Type: string

providerConfiguration:
mongoDbAtlas:
apiCredentials:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2
credentials:
masterUserName: 'my-master-user'
masterUserPassword: "$Secret('mongo-master-password')"

Business Intelligence Connector (SQL)

  • Traditional business intelligence tools are designed to work with tabular, row-and-column data. The MongoDB Connector for BI allows you to query MongoDB data with SQL using tools such as Tableau, Power BI or Excel.
MongoDbBiConnector  API reference
Parent API reference: MongoDbAtlasCluster
enabled
Required

Enables BI connector

Type: boolean

readPreference

Configures the type of node to which the BI connector will connect

Type: string ENUM

Possible values: analyticsprimarysecondary

providerConfiguration:
mongoDbAtlas:
apiCredentials:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: 'M10'
biConnector:
enabled: true

Pricing

You are charged for:

  • Cluster tier:
  • Data transfer:
    • For most customers, data transfer fees amount to less than 10% of their bill.
    • Data transfer fees are not assessed on M0, M2, and M5 instances.
    • Same region: $0.01/GB, Cross-region: $0.02/GB, Internet: $0.09/GB
    • For a more detailed overview, refer to MongoDb billing docs

API reference

MongoDbReplication  API reference
Parent API reference: MongoDbAtlasCluster
numAnalyticsNodes

Number of analytics nodes

Type: number

  • Analytics nodes are read only nodes meant for long-running queries.
  • Executing long-running queries on basic read-only or electable nodes can hinder overall performance of the cluster.
  • By using analytics nodes these queries you avoid a performance impact on your operational workload handled by electable and basic read only nodes.
  • Analytics nodes work well for biConnector with read preference set on analytics

numElectableNodes
Default: 3

Number of electable nodes (can become a primary node)

Type: number ENUM

Possible values: 357

  • Adding electable nodes improves the redundancy and availability of a cluster.

numReadOnlyNodes

Number of read only nodes

Type: number