Stacktape

Sign up



Mongo Db Atlas Clusters

Overview and basic concepts

  • MongoDb Atlas cluster is a schemaless, NoSQL database fully managed by MongoDB Inc. Unlike other resources, MongoDb Atlas cluster is not an AWS-native service. However, Stacktape seamlessly integrates it into your stacks.

  • MongoDb Atlas cluster is a secure, scalable, highly available and performant database. It has built-in replication, supports backups and Point-In-Time recovery.

  • Every stack that includes a MongoDb Atlas cluster will additionally create a new MongoDB Atlas project. This ensures isolation between stacks.

  • MongoDb includes an SDK for virtually any programming language. It allows you to easily query your database. Furthermore, you can configure MongoDb clusters to use a BI (Business Intelligence) connector that allows an SQL-based acess.

When to use

Advantages

  • Secure
  • Scalable
  • Performant
  • Supports backups and Point-In-Time recovery
  • High availability
  • Supports Multi-Document ACID Transactions

Disadvantages

  • Separate billing - Eventhough Stacktape seamlessly integrates MongoDb to your stacks, you still need to manage your billing separately.

Provider configuration

  • You must have a MongoDb Atlas account. To create one, head over to MongoDb Atlas registration page.

  • You need following to configure the provider:

    • Atlas Mongo organizationId
    • Atlas Mongo publicKey
    • Atlas Mongo privateKey
  • You can obtain them by following our step-by-step guide.

  • The recommended way to store your credentials is to use secrets.

Copy

serviceName: stack-name
providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2

MongoDbAtlasProviderConfig  API reference
publicKey
Required
privateKey
Required
organizationId
Required
accessibility

Basic Usage

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M2
myLambda:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: path/to/my/lambda.ts
environment:
- name: MONGODB_CONNECTION_STRING
value: $ResourceParam('myMongoDbCluster', 'connectionString')
accessControl:
allowAccessTo:
- myMongoDbCluster

Copy

import { MongoClient } from 'mongodb';
const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING);
const handler = async (event, context) => {
await client.connect();
const db = client.db('mydb');
await db.collection('posts').insertOne({
title: 'My first post',
content: 'Hello!'
});
const post = await db.collection('posts').findOne({ title: 'My first post' });
await client.close();
};

Cluster tier

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2

Disk size

  • Each cluster tier comes with a default storage size.
  • All M10+ clusters autoscale the storage without any manual intervention. You can disable this behavior by configuring autoscaling properties
  • All M10+ clusters also provide the ability to customize your storage capacity.

Copy

resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M2
diskSizeGB: 60

Auto-scaling

  • You can configure your cluster to automatically scale its cluster tier, storage capacity, or both based on the cluster usage.
  • To help control the costs, you can select a range of cluster tiers to which your cluster can scale to.
  • Cluster is scaled up (to the next tier), if one the following criteria is met:
    • Average CPU Utilization has exceeded 75% for the past hour
    • Memory Utilization has exceeded 75% for the past hour
  • Cluster is scaled down (to the lower tier), if both of the following criteria are met:
    • The average CPU Utilization and Memory Utilization over the past 24 hours is below 50%
    • The cluster has not been scaled down (manually or automatically) in the past 24 hours

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
autoScaling:
minClusterTier: M10
maxClusterTier: M30
disableDiskScaling: true
disableScaleDown: true

Sharding

  • If you configure more than 1 shard, the cluster will run in a sharded mode.
  • Sharding distributes data across multiple physical machines enabling horizontal scaling.
  • Sharded mode is available only for cluster tiers M30 or higher.
  • To learn more about sharding, refer to MongoDb Docs.

Copy

resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M30
numShards: 3

Backups

  • Backups are copies of your data that encapsulate the state of your cluster at a given time. Backups provide a safety measure in the event of a data loss.
  • The default snapshot time is every day at 18:00 UTC.
  • Available only in M10+ Clusters.
  • Snapshots are automatically taken even for M2/M5 clusters, but have different properties. To learn more, refer to [M2 and M5 backups docs](https://docs.atlas.mongodb.com/backup-restore-cluster/#m2

There are different types of snapshots with different retention period and frequency:

  • Hourly snapshot: every 6 hours, retained for 2 days
  • Daily snapshot: every day, retained for 7 days
  • Weekly snapshot: every Saturday, retained for 4 weeks
  • Monthly snapshot: last day of the month, retained for 12 months

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
enableBackups: true

Point-in-time recovery

  • Enables Continuous Cloud Backups, which replay the oplog (history of ordered logical writes) and enables you to restore a cluster from a particular point in time.
  • You can make a point in time recovery to any point within the last 7 days.
  • Available only in M10+ Clusters.
  • If you enable point-in-time recovery, you must also enable backups.
  • MongoDb Continuous cloud backup includes additional charges. To learn more, refer to MongoDb Continuous cloud backup pricing Docs

Copy

resources:
myMongoDbCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10
enablePointInTimeRecovery: true

Accessing clusters from workloads

  • Following example shows how to grant a lambda function myMongoFunction a permission to access your mongo-db-atlas-cluster myMongoCluster.

  • By listing myMongoCluster in allowAccessTo of myMongoFunction, function is injected with credentials needed for accessing the cluster.

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2
myMongoFunction:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: 'lambdas/mongo-lambda.ts'
memory: 512
# by allowing access to cluster, lambda receives permissions for reading and writing into cluster databases
accessControl:
allowAccessTo:
- 'myMongoCluster'
environment:
# injecting the connection string as environment variable
- name: MONGODB_CONNECTION_STRING
value: $ResourceParam('myMongoCluster', 'connectionString')

Accessibility

  • Stacktape allows you to restrict the accessibility of your MongoDb cluster to only certain resources or hosts.
  • In contrary to other Stacktape database resources, accesibility is set globally in your MongoDb Atlas provider configuration, and not on the resource itself. This means every MongoDb cluster in the given stack must have the same accessibility mode.
  • When using shared cluster tiers (M2 and M5), you must set accessibility mode to internet. This is because shared clusters does not support VPC peering. clusters(M10+). and therefore do not allow for same level of network inter-connection. Nevertheless, even when using accessibility set to internet, clusters are still protected as Stacktape utilizes strict identity access management between your stack workloads (functions, container-workloads, batch-jobs) and atlas mongo clusters. See section Accessing clusters from workloads
MongoDbAtlasAccessibility  API reference
Parent API reference: MongoDbAtlasProviderConfig
accessibilityMode
Default: internetRequired
whitelistedIps

Internet mode

  • Default mode. Least restrictive. The cluster can be accessed from anywhere on the internet.
  • Your MongoDB cluster is still protected, because any host or resource (workload) trying to access your cluster requires IAM (Identity and Access management) permissions (it is also possible to use admin user name and password, but using these credentials is only recommended in cases you cannot use IAM permissions).
  • You can grant IAM permissions to your workloads by configuring allowAccessTo on your (functions, container workloads or batch jobs)

VPC mode

  • In addition to being protected by IAM (Identity and Access management) as described in Internet mode, the mongo cluster is also protected on a network level. This means, that only functions (which have joinDefaultVpc set to true), batch jobs or container workloads within your stack can access the cluster.

  • Additionally, IP addresses configured in whitelistedIps can also access the cluster (even from the internet).

  • When using this mode, the traffic which originates from your stack resources (workloads) to your mongo cluster never leaves AWS network infrastrucutre. This makes the communication cheaper.

Copy

providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
# "accessibility" option is shared between "mongo-db-atlas-cluster" resources of your stack
accessibility:
accessibilityMode: vpc
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10

Scoping workloads in VPC mode

  • Similar to vpc mode, but even more restrictive in terms of network protection. When using this mode allowAccessTo not only grants the required IAM (Identity and Access management) permissions, but also grants access on the network level.

  • Additionally, IP addresses configured in whitelistedIps can also access the database (even from the internet).

  • When using this mode, the traffic which originates from your stack resources (workloads) to your mongo cluster never leaves AWS network infrastrucutre. This makes the communication cheaper.

Copy

providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
# "accessibility" option is shared between "mongo-db-atlas-cluster" resources of your stack
accessibility:
accessibilityMode: scoping-workloads-in-vpc
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
# functionOne does NOT have access to database eventhough it is joined in vpc
functionOne:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: 'path/to/my-lambda.ts'
joinDefaultVpc: true
# functionTwo does have access to database, because it is scoping the database in allowAccessTo list
functionTwo:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: 'path/to/my-lambda-2.ts'
joinDefaultVpc: true
accessControl:
allowAccessTo:
- 'myMongoCluster'
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10

Whitelisted IPs only mode

  • The database can only be accessed from an IP addresses and CIDR blocks listed in the whitelistedIps list.

Copy

providerConfig:
mongoDbAtlas:
privateKey: 'xxxxfa523543fxxxx42543xx'
publicKey: 'xxxxxxx'
# "accessibility" option is shared between "mongo-db-atlas-cluster" resources of your stack
accessibility:
accessibilityMode: whitelisted-ips-only
whitelistedIps:
- 193.12.16.4
organizationId: 'xxxxxxxxxxx07a593cbe63dd'
resources:
myMongoCluster:
type: mongo-db-atlas-cluster
properties:
clusterTier: M10

Admin user

  • Optionally, you can create an admin database user user with administrative access privileges.
  • Accessing the cluster from your workloads (batch-jobs, container-workloads or functions), is possible even without creating this user.
  • Creating an admin user can be useful for performing administrative tasks, or when connecting to the cluster from a local machine.
MongoDbAdminUserCredentials  API reference
Parent API reference: MongoDbAtlasCluster
userName
Required
password
Required

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: M2
adminUserCredentials:
userName: my-master-user
password: $Secret('mongo-master-password')

Business Intelligence Connector (SQL)

  • Traditional business intelligence tools are designed to work with tabular, row-and-column data. The MongoDB Connector for BI allows you to query MongoDB data with SQL using tools such as Tableau, Power BI or Excel.
MongoDbBiConnector  API reference
Parent API reference: MongoDbAtlasCluster
enabled
Required
readPreference

Copy

resources:
myMongoCluster:
type: 'mongo-db-atlas-cluster'
properties:
clusterTier: 'M10'
biConnector:
enabled: true

Referenceable parameters

The following parameters can be easily referenced using $ResourceParam directive directive.

To learn more about referencing parameters, refer to referencing parameters.

connectionString
  • Connection string (URL) that allows connecting to the cluster.

  • Usage: $ResourceParam('<<resource-name>>', 'connectionString')

Pricing

You are charged for:

  • Cluster tier:
  • Data transfer:
    • For most customers, data transfer fees amount to less than 10% of their bill.
    • Data transfer fees are not assessed on M0, M2, and M5 instances.
    • Same region: $0.01/GB, Cross-region: $0.02/GB, Internet: $0.09/GB
    • For a more detailed overview, refer to MongoDb billing docs

API reference

MongoDbReplication  API reference
Parent API reference: MongoDbAtlasCluster
numAnalyticsNodes
numElectableNodes
Default: 3
numReadOnlyNodes
MongoDbAtlasCluster  API reference
type
Required
properties.clusterTier
Required
properties.diskSizeGB
properties.numShards
Default: 1
properties.replication
properties.enableBackups
properties.enablePointInTimeRecovery
properties.biConnector
properties.autoScaling
properties.adminUserCredentials
Need help? Ask a question on SlackDiscord or info@stacktape.com.