logoStacktape docs




Redis Clusters

Overview

Redis cluster powered by Amazon ElastiCache for Redis is a blazing fast in-memory data store that provides sub-millisecond latency to power internet-scale real-time applications. Built on open-source Redis and compatible with the Redis APIs, ElastiCache for Redis works with your Redis clients and uses the open Redis data format to store your data.

Advantages

  • Extreme performance - Redis works as an in-memory data store to support the most demanding applications requiring sub-millisecond response times. In comparison to disk-based databases where most operations need a roundtrip to disk, in-memory data stores manage data in-memory which is an order of magnitude faster than disks.
  • Fully managed - Redis is a fully managed service. You no longer need to perform management tasks such as hardware provisioning, software patching, setup, configuration, monitoring, failure recovery, and backups. AWS continuously monitors your clusters to keep your Redis up and running so that you can focus on higher-value application development.
  • High availability - Redis supports both cluster and non-cluster modes and provides high availability via support for automatic failover by detecting primary node failures and promoting a replica to be primary with minimal impact.

Definition breakdown

RedisCluster  API reference
type
Required
properties.nodeInstanceSize
Required
properties.defaultUserPassword
Required
properties.sharding
properties.numReplicaNodes
Default: 0
properties.logging
properties.automatedBackupRetention
Default: 0
properties.port
Default: 6379
properties.accessibility
properties.automaticFailover
overrides
  • Following example shows basic usage of redis-cluster resource. Only required properties are nodeInstanceSize and defaultUserPassword.
  • The example shows a single node redis-cluster.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password

Choosing node instance size

Property nodeInstanceSize:

  • Determines the instance size which is used for each node in the cluster (primary and replicas).
  • Different instance sizes have different memory and network capabilities (For more information refer to AWS docs).
  • The instance size can be changed without interuption of availability and data loss.

The following example shows a single node redis-cluster.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password

Accessibility

The following example shows a single node redis-cluster with accessibility set to scoping-workloads-in-vpc.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
accessibility:
restrictAccess: scoping-workloads-in-vpc

Accessing from workload

The following example shows:

  • myRedisCluster - redis-cluster with accessibility set to scoping-workloads-in-vpc.
  • myFunction - function scoping myRedisCluster in allowAccessTo section (if myRedisCluster would have accessibility set to vpc, scoping would not be required).

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
accessibility:
restrictAccess: scoping-workloads-in-vpc
myFunction:
type: function
properties:
packageConfig:
filePath: my-redis-lambda.ts
environment:
- name: REDIS_HOST
value: $Param('redisCluster', 'RedisCluster::PrimaryEndPoint.Address')
- name: REDIS_PORT
value: $Param('redisCluster', 'RedisCluster::PrimaryEndPoint.Port')
- name: REDIS_PASS
value: $This().resources.myRedisCluster.properties.defaultUserPassword
accessControl:
allowAccessTo:
- myRedisCluster

The following snippet shows the code of the myFunction.

  • Example uses ioredis library.

import Redis from 'ioredis';
const client = new Redis({
host: process.env.REDIS_HOST,
port: Number(process.env.REDIS_PORT),
password: process.env.REDIS_PASS,
tls: {}
});
client.on('error', (err) => console.error('Redis Client Error', err));
export default async (event, context) => {
// setting value
await client.set('currentTime', `${Date.now()}`);
// getting value
const value = await client.get('currentTime');
console.log(`result ${value}`);
};

Accessing sharded cluster

When sharding is enabled, the cluster is accessed through a configuration endpoint. See Sharding section to see how the cluster sharding works.

The following example shows:

  • myRedisCluster - sharded redis-cluster (default vpc accessibility is used).
  • myFunction - function which accesses cluster.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
sharding:
enabled: true
numShards: 2
numReplicaNodes: 1
myFunction:
type: function
properties:
packageConfig:
filePath: my-redis-lambda.ts
environment:
- name: REDIS_HOST
value: $Param('redisCluster', 'RedisCluster::ConfigurationEndPoint.Address')
- name: REDIS_PORT
value: $Param('redisCluster', 'RedisCluster::ConfigurationEndPoint.Port')
- name: REDIS_PASS
value: $This().resources.myRedisCluster.properties.defaultUserPassword

The following snippet shows the code of the myFunction.

  • Sharding of a cluster is enabled in this example.
  • Example uses ioredis library.

import { Cluster } from 'ioredis';
const clusterClient = new Cluster([{ host: process.env.REDIS_HOST, port: Number(process.env.REDIS_PORT) }], {
redisOptions: {
tls: {},
password: process.env.REDIS_PASS
},
dnsLookup: (address, callback) => callback(null, address),
slotsRefreshTimeout: 10000
});
clusterClient.on('error', (err) => console.error('Redis Client Error', err));
export default async (event, context) => {
// setting value
await clusterClient.set('currentTime', `${Date.now()}`);
// getting value
const value = await clusterClient.get('currentTime');
console.log(`result ${value}`);
};

Sharded vs Non-sharded clusters

Each redis-cluster can either have sharding enabled or disabled.

Depending on whether the sharding is enabled or not, the topology and number of nodes in your cluster changes.

Single node (non-sharded) cluster

The following example shows a single node non-sharded redis-cluster.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password

Single node (non-sharded) cluster topology
Single node (non-sharded) cluster topology

Multi node (non-sharded) cluster

Adding nodes (read replicas) to non-sharded cluster increases the read throughput of the cluster.

The following example shows a multi node (with read replicas) non-sharded redis-cluster.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
numReplicaNodes: 2

Multi node (non-sharded) cluster topology
Multi node (non-sharded) cluster topology

Sharding the cluster

  • Sharding enables users to scale their redis-cluster horizontally, by splitting the data into shards.
  • Each shard is then served by a shard cluster, which is responsible for given shard. Each shard cluster has a primary instance and numReplicaNodes replicas.
  • Routing to correct shard and shard rebalancing is handeled automatically by AWS.
  • You can update (increase or decrease) number of shards without interupting availability.

Sharding comes with multiple limitations:

  • Sharding can only be enabled during redis-cluster initial creation. After that the cluster cannot be updated from non-sharded to sharded or vice-versa.
  • Changing numReplicaNodes when sharding is enabled is not possible. In this case it is necessary to delete and recreate the cluster.
  • numReplicaNodes must be set to at least 1 when sharding is enabled.
RedisSharding  API reference
Parent API reference: RedisCluster
enabled
Required
numShards

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
sharding:
enabled: true
numShards: 2
numReplicaNodes: 1

Sharded cluster topology
Sharded cluster topology

Automatic failover

When automatic failover is enabled and primary node fails, the replica node is promoted to become the primary node.

  • numReplicaNodes must be set to at least 1 for automaticFailover to be enabled
  • automaticFailover is by default enabled on all sharded clusters.

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
numReplicaNodes: 2
automaticFailover: true

Logging

With logging enabled, the slow-log of the cluster is automatically sent to Cloudwatch log group.

RedisLogging  API reference
Parent API reference: RedisCluster
enabled
Required
format
Default: json
retentionDays
Default: 180

resources:
myRedisCluster:
type: redis-cluster
properties:
nodeInstanceSize: cache.t3.micro
defaultUserPassword: my-ultra-safe-secret-password
logging:
enabled: true