Stacktape

Sign up

Stacktape

Sign up



Log Forwarding

Introduction

Log forwarding is the process of transmitting log data generated by various applications, systems, and services to a centralized destination or third-party log management provider. In cloud environments, like Amazon Web Services (AWS), the primary log management service is Amazon CloudWatch. However, people often choose to forward logs from CloudWatch to third-party providers for several reasons:

  • third-party log management platforms offer advanced features and functionalities beyond what CloudWatch provides, such as real-time log analysis, machine learning-powered insights, and customizable dashboards
  • some organizations have specific compliance requirements or industry standards that necessitate the use of dedicated log management providers.

How it works

Stacktape allows for easy log forwarding to endpoint of your choice.

  • Log forwarding is done using Amazon Kinesis Data Firehose delivery stream.
  • When using log forwarding, you will incur costs based on the amount of data being transferred to the destination (~$0.03 per transferred GB). Refer to AWS Kinesis Firehose Pricing page to see details.
  • Currently supported destinations for logs:

Refer to our docs for more information.

Logs that fail to be delivered to the destination even after multiple retries (time spend on retries can be configured) are put into bucket with name {stackName}-{resourceName}-logs-{generatedHash}

HTTP endpoint forwarding

You can forward logs to any HTTP endpoint that follows Firehose request and response specifications.

Many of the third party vendors are compliant with this specifications out of the box.

Copy

resources:
myFunction:
type: function
properties:
logging:
logForwarding:
type: http-endpoint
properties:
endpointUrl: https://my-endpoint.com
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: lambdas/js-lambda.js

HttpEndpointLogForwarding  API reference
type
Required
properties.endpointUrl
Required
properties.gzipEncodingEnabled
properties.parameters
properties.retryDuration
properties.accessKey

Datadog forwarding

Delivers logs to your Datadog site.

Copy

resources:
myFunction:
type: function
properties:
logging:
logForwarding:
type: datadog
properties:
apiKey: your_datadog_api_key
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: lambdas/js-lambda.js

DatadogLogForwarding  API reference
type
Required
properties.apiKey
Required
properties.endpointUrl
Default: https://aws-kinesis-http-intake.logs.datadoghq.com/v1/input

Highlight forwarding

Delivers logs to your Highlight project.

Copy

resources:
myFunction:
type: function
properties:
logging:
logForwarding:
type: datadog
properties:
apiKey: your_datadog_api_key
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: lambdas/js-lambda.js

HighlightLogForwarding  API reference
type
Required
properties.projectId
Required
properties.endpointUrl
Default: https://pub.highlight.io/v1/logs/firehose

Need help? Ask a question on Discord or info@stacktape.com.