> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nekt.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Better Stack as a data source
> Bring your uptime monitoring, incident and on-call data from Better Stack to Nekt.
Better Stack is an uptime monitoring, incident management and log management platform. This
connector extracts the operational record behind it: the monitors and heartbeats that watch your
services, every incident they raised and how long it took to acknowledge and resolve it, the
availability figures Better Stack computes, the on-call schedules and escalation policies that
decide who gets paged, the public status pages you publish, and the configuration of your telemetry
sources and collectors.
## 1. Add your Better Stack access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Better Stack option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API token**: required. In Better Stack, go to **API tokens** and copy a token from the **Global API tokens** section. A global token is the right choice for a pipeline: it covers every team and both the Uptime and Telemetry products, so no table is left out.
* **Start date** (advanced): earliest date to read incidents and on-call shifts from. Leave empty to start from the oldest data Better Stack still serves. Every other endpoint returns its full contents on each sync, so this setting does not affect the other tables.
* **Incidents lookback (days)** (advanced): how far before the last synced incident to re-read on each run. 30 by default.
* **Page size** (advanced): records requested per page. 250 by default, which is Better Stack's maximum.
* **Requests per minute** (advanced): how fast the connector is allowed to call the API. 300 by default.
A team-scoped **Uptime API token** also works, but it only reaches one team's Uptime resources. The
four `telemetry_*` tables cannot be read with it and will stay empty. Use a global token unless you
deliberately want to limit the source to a single team.
3. Click **Next**.
## 2. Select your Better Stack streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Four streams cost one request per parent record — `monitor_availability` and
`monitor_response_times` per monitor, `heartbeat_availability` per heartbeat,
`telemetry_source_metrics` per telemetry source. On an account with hundreds of monitors they make
the sync noticeably longer, so select them only if you need them.
2. Click **Next**.
## 3. Configure your Better Stack data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Better Stack data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Before you rely on the data
**`monitor_response_times` cannot be backfilled.** Better Stack keeps individual check timings for
the last 24 hours only and offers no date filter on them, so this table builds up history from your
first sync onwards. Schedule the pipeline to run at least once a day, or the series will have gaps
that nothing can recover.
A few behaviours are worth knowing before you build on these tables:
* **Only `incidents` can sync incrementally.** It is the one Better Stack endpoint that accepts a
date range. Every other table is read in full on each run, which is why almost all of them are
FULL\_TABLE.
* **Incidents keep changing after they start.** An incident's `started_at` never moves, but
`acknowledged_at` and `resolved_at` are filled in later. The connector re-reads the last 30 days
before the bookmark on every run so those updates land; raise **Incidents lookback (days)** if
incidents in your account often stay open for longer than that.
* **The availability tables are rollups, not history.** `monitor_availability` and
`heartbeat_availability` hold lifetime figures that Better Stack recomputes on every call, so each
sync replaces the previous answer. To keep a time series of your SLA, snapshot the table with a
[History](https://docs.nekt.com/histories/overview).
* **Join incidents to monitors through `monitor_id`.** The connector lifts it out of the API's
relationships object into its own column. It is empty for incidents raised by an integration or an
incoming webhook rather than by a monitor.
* **Severities are called urgencies in the API.** Better Stack renamed the concept in the product but
kept the old name on the wire, so `severities` and `severity_groups` come from the `urgencies`
endpoints. The `type` column still reads `urgency`.
* **Nested data arrives as JSON text.** Fields with sub-structure — an incident's `metadata`, a
policy's `steps`, a status page resource's `status_history`, a catalog record's `attributes` — are
stored as JSON strings. Read them with `JSON_VALUE(column, '$.field')` in your queries.
* **`team_members` mixes people and invitations.** A row is an accepted member when `type` is
`team_member` and a pending invitation when it is `team_member_invitation`; invitations carry
`invited_at` instead of `created_at`.
* **Credentials are not extracted.** The heartbeat ping URL, a telemetry source's ingest token, a
collector's secret, an escalation policy's incident token and a monitor's basic-auth password,
environment variables and Playwright script are all write credentials, which have no place in a
warehouse table.
* **Log lines and metric points are out of scope.** Better Stack serves them through a separate
ClickHouse-compatible query API that runs SQL over your retention window. The telemetry tables here
describe where your telemetry comes from and how it is shaped, not the events themselves.
## Available streams
| Stream | Sync | Description |
| ------------------------ | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `monitors` | FULL\_TABLE | Every check configured in the account: the URL or host, the check type (status code, keyword, ping, TCP, SMTP, DNS, Playwright and others), current status, check frequency and timeouts, the regions it runs from, the maintenance window, expiration alerting for SSL certificates and domains, and which channels an incident may use. |
| `monitor_groups` | FULL\_TABLE | Groups monitors are organized into, with their sort order and whether the whole group is paused. |
| `monitor_availability` | FULL\_TABLE | Lifetime availability summary per monitor: uptime percentage, total downtime, number of incidents, and the longest and average incident duration, all in seconds. One request per monitor. |
| `monitor_response_times` | INCREMENTAL | Individual check timings, one row per measurement per region, broken down into DNS lookup, connection, TLS handshake and data transfer time. Covers the last 24 hours only. One request per monitor. |
| `heartbeats` | FULL\_TABLE | Checks that expect a ping from your own job instead of calling out: the expected period and grace, the group they belong to, the maintenance window, and their current status. |
| `heartbeat_groups` | FULL\_TABLE | Groups heartbeats are organized into, with their sort order and whether the whole group is paused. |
| `heartbeat_availability` | FULL\_TABLE | Lifetime availability summary per heartbeat, in the same shape as `monitor_availability`. One request per heartbeat. |
| Stream | Sync | Description |
| -------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `incidents` | INCREMENTAL | Every incident raised, with the cause, the monitor behind it, when it started, when it was acknowledged and by whom, when it was resolved and by whom, the regions that observed the failure, the response captured at the time, the channels that were alerted, the Slack channels opened for it and any custom metadata attached. This is the table time-to-acknowledge and time-to-resolve are computed from. |
| `incident_comments` | FULL\_TABLE | Comments written on an incident, with the author and when they were posted. One request per incident in the sync window. |
| `on_call_calendars` | FULL\_TABLE | On-call schedules, whether each is its team's default, and who is on call right now. |
| `on_call_calendar_events` | FULL\_TABLE | Individual on-call shifts generated by each schedule, past and upcoming, with the people covering them and whether the shift is a manual override. |
| `escalation_policies` | FULL\_TABLE | The ordered steps taken when an incident is not handled — who is alerted, after how long, at which severity, and which policy it falls back to — plus how many times the whole policy repeats. |
| `escalation_policy_groups` | FULL\_TABLE | Groups escalation policies are organized into. |
| `severities` | FULL\_TABLE | Severity levels and which alert channels each is allowed to use: SMS, phone call, e-mail and push. |
| `severity_groups` | FULL\_TABLE | Groups severities are organized into. |
| Stream | Sync | Description |
| ---------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `status_pages` | FULL\_TABLE | Public status pages, with their branding, subdomain or custom domain, the history window they display, whether visitors can subscribe, whether incidents are published automatically, and the state currently shown. |
| `status_page_groups` | FULL\_TABLE | Groups status pages are organized into. |
| `status_page_sections` | FULL\_TABLE | Sections a status page groups its resources into. |
| `status_page_resources` | FULL\_TABLE | Monitors and heartbeats published on a status page, with the public name shown to visitors, the availability over the displayed window, the rules deciding when the resource is marked down or degraded, and the daily status history. |
| `status_page_reports` | FULL\_TABLE | Incident and maintenance reports published on a page, with the title, when the event started and ended, and which resources it affected. |
| `status_page_report_updates` | FULL\_TABLE | Individual updates posted while a report was open, with the message, when it was published, whether subscribers were notified, and the state each affected resource was put in. |
| `status_page_subscribers` | FULL\_TABLE | People subscribed to a page's updates, with the resources their subscription is limited to and when they confirmed their address. |
| Stream | Sync | Description |
| -------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `metadata` | FULL\_TABLE | Custom key/value metadata attached to a monitor, heartbeat, incident or integration — the values incident routing rules match on. |
| `catalog_relations` | FULL\_TABLE | The tables your service catalog is made of, with what each one catalogs. |
| `catalog_attributes` | FULL\_TABLE | Attributes (columns) defined on each catalog relation, and which one identifies the record. |
| `catalog_records` | FULL\_TABLE | Rows held by each catalog relation, with the value of every attribute. |
| `team_members` | FULL\_TABLE | Everyone in the organization plus pending invitations, with the role each holds and the mobile platforms they can receive push alerts on. |
| `organization_roles` | FULL\_TABLE | Roles available in the organization, including custom ones, and the built-in role each maps to. |
| Stream | Sync | Description |
| -------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `telemetry_sources` | FULL\_TABLE | Sources your logs and metrics are ingested into, with the platform they come from, the table they are queryable under, the retention configured for logs and metrics, the data region, and the transformation applied on ingestion. |
| `telemetry_source_groups` | FULL\_TABLE | Groups telemetry sources are organized into. |
| `telemetry_source_metrics` | FULL\_TABLE | Metric definitions extracted from each source's log lines: the SQL expression evaluated, its storage type and the aggregations it can be charted with. One request per source. |
| `telemetry_collectors` | FULL\_TABLE | Better Stack collectors running on your own infrastructure, with their platform, online status, sample rates and batch settings, the source they write into and when they last reported in. |