Skip to main content
Datadog is a monitoring and analytics platform for cloud-scale applications, providing monitoring of servers, databases, tools, and services through a SaaS-based data analytics platform. It offers comprehensive logging, metrics, and tracing capabilities to help organizations monitor their infrastructure and applications in real-time.

​
Configuring Datadog as a Source

In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the Datadog option from the list of connectors. Click Next and you’ll be prompted to add your access.

​
1. Add account access

You’ll need to provide API credentials from your Datadog account to authenticate the connection. The following configurations are available:
  • API Key: The API key to authenticate against the API service. You can find this in your Datadog account under Organization Settings > API Keys. Check Datadog’s documentation for more details.
  • App Key: The application key to authenticate against the API service. You can find this in your Datadog account under Organization Settings > Application Keys. For RUM streams, it needs the rum_apps_read permission. Check Datadog’s documentation for more details.
  • Start Date (optional): Starting point for data extraction. Leave empty to backfill the last 30 days on the first run (Datadog RUM retention).
  • Logs query (optional): The query to search for logs (applies to the logs stream only). You can use Datadog’s query syntax to filter by service, host, tags, or any other attributes. If not provided, all logs will be retrieved.
  • Datadog site (optional): Datadog region your account lives in. Check the URL you use to open Datadog: app.datadoghq.com is US1 (default), us5.datadoghq.com is US5, app.datadoghq.eu is EU, and so on. Default is datadoghq.com.
  • RUM query filter (optional): Optional extra RUM search query applied to every RUM stream, e.g., @application.name:my-app. Leave empty to extract all RUM applications.
  • Enable RUM resource events (optional): Adds the rum_resource stream (network requests, assets). This is by far the highest-volume RUM event type and Datadog keeps it for only 15 days, so enable it as soon as you decide you need it.
Once you’re done, click Next.

​
2. Select streams

Choose which data streams you want to sync. The Datadog connector exposes streams for logs and RUM (Real User Monitoring) events (actions, views, sessions, errors, and optionally resources). Select the streams and click Next.

​
3. Configure data streams

Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
  • Layer: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
  • Folder: a folder can be created inside the selected layer to group all tables being created from this new data source.
  • Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix to all tables at once and make this process faster!
  • Sync Type: you can choose between INCREMENTAL and FULL_TABLE.
    • Incremental: every time the extraction happens, we’ll get only the new data - which is good if, for example, you want to keep every record ever fetched.
    • Full table: every time the extraction happens, we’ll get the current state of the data - which is good if, for example, you don’t want to have deleted data in your catalog.
Once you are done configuring, click Next.

​
4. Configure data source

Describe your data source for easy identification within your organization, not exceeding 140 characters. To define your Trigger, consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times). Optionally, you can define some additional settings:
  • Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource here.
  • Determine when to execute an Additional Full Sync. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click Next to finalize the setup.

​
5. Check your new source

You can view your new source on the Sources page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your Catalog, you need at least one successful source run.

​
Streams and Fields

Below you’ll find the available data streams from Datadog and their corresponding fields:
Stream for retrieving log events from Datadog with their associated metadata and attributes.Attributes Object Contains:
  • host - The host where the log originated
  • service - The service name
  • message - The log message content
  • status - Status of the log event
  • tags - Array of tags associated with the log
  • attributes - Array of custom attributes with key-value pairs
  • timestamp - Timestamp within the attributes object
Primary Key: id Replication Key: timestamp
Stream for retrieving Real User Monitoring (RUM) user action events like clicks, taps, and custom actions.Primary Key: id Replication Key: timestamp
Stream for retrieving RUM page view and screen events.Primary Key: id Replication Key: timestamp
Stream for retrieving RUM user session events.Primary Key: id Replication Key: timestamp
Stream for retrieving RUM front-end error events.Primary Key: id Replication Key: timestamp
Stream for retrieving RUM network resources (XHR, fetch, assets). This stream is high volume and requires enabling explicitly via the “Enable RUM resource events” option.Primary Key: id Replication Key: timestamp

​
Data Model

The following diagram shows the structure of the Datadog logs and RUM streams:

​
Implementation Notes

​
Data Quality Considerations

  • The logs and RUM streams both use timestamp as the replication key for incremental syncs.
  • Custom log attributes and tags provide additional context for filtering and analysis.
  • For RUM streams, the attributes column preserves the full nested event payload exactly as delivered by Datadog, serialized as a JSON string. This includes details like action.target.name, view.url, and error.message.
  • Duration fields found inside the RUM attributes payload are delivered in nanoseconds.
  • RUM event timestamps are re-serialized with fixed millisecond precision (.mmmZ). Datadog omits the fractional part when the milliseconds are zero, and mixed precision would break the chronological ordering the incremental bookmark relies on.
  • By default, leaving the Start Date empty will backfill the last 30 days of RUM history (matching Datadog’s standard retention period).

​
API Limits & Performance

  • Use the query and rum_query parameters to filter events at the API level, heavily reducing extraction time and data volume.
  • Consider the volume of events when setting extraction frequency. The rum_resource stream in particular represents an extremely high volume of data (every network request and loaded asset), which is why it requires manual opt-in. Datadog typically retains resource events for only 15 days.
  • Datadog’s API has rate limits that the connector respects automatically, retrying on HTTP 429 (Too Many Requests). If Datadog responds with partial timeout statuses, the connector will retry those pages seamlessly.
  • The connector also retries transient server-side timeouts (HTTP 408) that Datadog’s search endpoints may return when querying large datasets or wide time windows, instead of failing the run.