# Overview
Source: https://docs.nekt.com/build-with-ai
Everything you need to onboard your AI agent to Nekt.
If you're developing with AI, Nekt offers several resources to help your agents work with the platform, from querying data to automating pipelines and managing resources.
* [MCP Server](#mcp-server)
* [Docs for Agents](#docs-for-agents)
* [Platform API](#platform-api)
* [Data API](#data-api)
* [AI Integrations](#ai-integrations)
## Guides
Turn a static HTML report built with Claude or Codex into a live web app powered by Nekt.
## MCP Server
[MCP (Model Context Protocol)](https://modelcontextprotocol.io/) is an open protocol that standardizes how applications provide context to LLMs. Nekt's MCP Server gives AI agents a structured way to discover data, run queries, read files, and learn about the platform, all through a single integration point.
The server supports **OAuth 2.1** for desktop apps like Claude Desktop and Cursor, and **bearer tokens** for CLI tools and automation platforms.
To connect via bearer token, add the server to your MCP client configuration:
```json theme={null}
{
"mcpServers": {
"Nekt": {
"type": "http",
"url": "",
"headers": {
"Authorization": "Bearer "
}
}
}
}
```
Replace `` with the endpoint from the MCP Gateway page in Nekt (click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**), and `` with a token from its **Access tokens** section.
Enable the MCP Server and create tokens.
OAuth 2.1 and bearer token details.
Every tool available to AI agents.
## Docs for Agents
Give your agent up-to-date Nekt documentation for better context when working with the platform.
1. **Markdown docs**
Every doc includes a markdown version (append md to any page):
```
https://docs.nekt.com/build-with-ai.md
```
2. **Full docs file**
Give your agent all of Nekt's documentation in a single file:
```
https://docs.nekt.com/llms-full.txt
```
## Platform API
The Platform API gives agents programmatic control over Nekt resources via REST. Use it to create and trigger sources, manage pipelines, track runs, and operate on the catalog: anything you can do in the UI, agents can automate through the API.
Authenticate with an API key in the `x-api-key` header:
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: "
```
Create an API key in [Workspace Settings](https://app.nekt.ai/settings/api-keys).
Full API reference for managing Nekt resources.
## Data API
The Data API lets agents query your data directly over HTTP using SQL. Submit a query, and download the results as Parquet or CSV.
Query your data using SQL through the Data API.
## AI Integrations
Step-by-step guides for connecting specific AI tools to Nekt.
Connect Claude to your data in Nekt and activate it with flexibility and ease.
Connect Lovable to your data in Nekt.
Connect ChatGPT to your data in Nekt using the MCP Server with OAuth authentication.
Connect OpenClaw to your data in Nekt using the MCP Server.
Connect OpenAI Codex to your data in Nekt using the MCP Server with bearer token authentication.
Connect Hermes Agent to your data in Nekt, running on any LLM such as DeepSeek.
# Expectations
Source: https://docs.nekt.com/catalog/expectations
Make sure your data will arrive as expected.
The Expectations feature allows you to literally set expectations for the data being processed. While processing it, Nekt will validate the records for you and make sure you never get undesired values.
Expectations feature is **only available for Growth or Custom plans**.
Select the table you want to set expectations for. Go to the tab to see what has been defined already and click on **Add Expectations** to add new ones.
This helps you to spot invalid values as soon as possible and avoid error propagation to other tables and data destinations.
# External tables
Source: https://docs.nekt.com/catalog/external-tables
Query a BigQuery table from the Catalog without copying it.
An external table lets you use a BigQuery table that already exists in your own Google Cloud project directly from the Nekt Catalog, without copying it. Nekt reads it in place, so the table shows up alongside your other tables and is available for consumption, queries, and notebooks. Every Nekt permission rule applies to it.
External tables are available only on workspaces that run in your **own Google Cloud project** (customer-hosted GCP), on **Growth plans and above**. They are not available on AWS or on Nekt-hosted workspaces.
## How it works
Nekt does not move your data. It registers a reference to the source table and reads it where it lives. Nothing is copied into Nekt's storage, and you keep control of the source in your own project. Nekt only ever reads it.
## Add an external table
From the Catalog, pick the layer or folder where you want the table, open the **New** menu, and choose **External table**. A wizard walks you through two phases.
Fill in the **Google Cloud project** the table lives in (yours, not Nekt's), the **Dataset**, and the **Table**. Choose a **Name in the Nekt catalog**: use only lowercase letters, numbers, and underscores, and start with a letter.
Datasets you have already shared with Nekt appear in the list. If you haven't shared one yet, type its name and it will appear once the grant in the next phase is in place.
Complete a few steps in your own Google Cloud console. Nekt only checks these; it never changes permissions in your project. Each step is verified before the next one appears.
1. **Give Nekt read access to the source.** Grant the Nekt service account shown in the wizard the **BigQuery Data Viewer** role on the source dataset. (Granting it on the table alone also works, but then Nekt cannot list the dataset's tables and you type the table name by hand.)
2. **Enable queries across regions** *(only if the source is in another region)*. Run the SQL statements the wizard provides in BigQuery. They take a few minutes to take effect.
3. **Authorize the view.** In the BigQuery console, open the source **dataset** (not the table) → **Sharing** → **Authorize views**, and add the view Nekt created in the previous step. BigQuery only authorizes views at the dataset level.
Nekt checks your setup automatically every 5 minutes, and every 10 seconds for a few minutes after you click **Check now**. This is because Google can take a minute or two to apply a grant, so a correct setup often fails the first check and passes shortly after. When everything is in place, the table is added to your catalog and marked with an **External** badge.
If the source table is in a different region from your workspace, queries that read it take noticeably longer, because the data has to cross regions. Nothing is wrong; if it becomes a problem, bringing the data in with a [Source](/sources/overview) is faster.
## Awaiting setup
If you don't finish the grants in one sitting, the registration is saved under **Awaiting setup** in the Catalog. Each entry shows a status tag for what is still missing (for example, *Waiting for access*, *View not authorized*, or *Cross-region off*). Use **Continue** to reopen the wizard where you left off, or **Remove** to discard the registration and free the source so it can be added again.
## Remove an external table
Removing an external table (from the tables list or from **Awaiting setup**) stops Nekt from reading the source. Your source table in BigQuery is never touched. The view Nekt created stays listed under **Authorize views** on the source dataset. It no longer reads anything, but only you can remove it there.
# Folders
Source: https://docs.nekt.com/catalog/folders
Organize tables inside a Layer by source, domain, or project.
A Folder organizes data inside a [Layer](/catalog/layers). Use Folders to group tables by source, business domain, or project, so the Catalog stays understandable as your workspace grows. A Folder can contain both tables and other Folders.
Folders are for organization within a Layer: the Layer remains the top-level storage and governance boundary. Within that boundary, arrange Folders however makes your data easiest to find.
```text theme={null}
bronze/
crm/
finance/
marketing/
```
## Folder permissions
Folders can carry permissions. Access granted on a Folder applies to all of its subfolders, tables, and volumes, including ones created later. This makes Folders a convenient way to grant a team access to a whole domain without listing every table.
If a group of users should see only part of a Layer, put that data in its own Folder and grant access at the Folder level.
## Keep the hierarchy shallow
Deep Folder hierarchies make tables harder to find, not easier. Prefer a flat, predictable structure (one or two levels is usually enough) and let clear [table names](/catalog/tables) do the rest.
## API & MCP
Read the Folder object.
No dedicated folder tool yet.
## Related documentation
The storage and permission boundary Folders live in.
The datasets a Folder groups.
Grant Folder access to Members and groups.
How data flows through the Catalog.
# Layers
Source: https://docs.nekt.com/catalog/layers
The top-level storage and permission boundary in the Catalog.
A Layer is the top-level division of the Catalog. It is both a **storage boundary** and a **permission boundary**: tables live inside a Layer, and access granted on a Layer applies to everything created within it. Use Layers to separate processing stages, clients, or other governance boundaries.
Every table in the Catalog belongs to exactly one Layer. Within a Layer you organize tables further with [Folders](/catalog/folders).
A grant on a Layer also covers Folders, Tables, and Volumes created there later. If two groups of users must not see the same data, separate that data into different Layers and grant each group access only to its own.
## Choose a Layer model
There is no universal Layer structure. Start with the boundary that is hardest to change later (usually data ownership and access) and avoid creating more Layers than your team can maintain.
### Data from your own company
For a workspace that contains internal company data, organize Layers by processing stage:
```text theme={null}
bronze Data as it arrives from Sources
silver Cleaned, standardized, validated, and deduplicated data
gold Business-ready data for analytics, AI agents, and Destinations
```
Use these names consistently so people and AI agents can recognize the stage of each table from its path. Use [Folders](/catalog/folders) inside each Layer to group tables by source or business domain:
```text theme={null}
bronze/
crm/
finance/
marketing/
gold/
revenue/
customer_success/
```
### Data from clients
For agencies, consultancies, and other service providers, client isolation is usually more important than processing stage. Start with one Layer per client:
```text theme={null}
client_acme
client_globex
client_northstar
```
Use Folders inside each client Layer to group Sources, working tables, and business-ready outputs. Grant each client or delivery team access only to the relevant Layer.
If a client has a large or complex data operation, you can create stage-specific Layers such as `acme_bronze`, `acme_silver`, and `acme_gold`. Use this structure only when the additional separation provides a clear governance or operational benefit.
### Internal and client data
For a hybrid workspace, combine the two models. Keep one Layer per client and separate Layers for your internal data lifecycle:
```text theme={null}
client_acme
client_globex
internal_bronze
internal_silver
internal_gold
```
This structure prevents your company's operational data from being mixed with client environments while preserving a clear processing flow for internal analytics.
## API & MCP
List and update layers, and read the Layer object.
`list_layers` and `create_layer`.
## Related documentation
Organize tables inside a Layer.
The datasets that live in a Layer.
Grant Layer access to Members and groups.
How data flows through the Catalog.
# Lineage
Source: https://docs.nekt.com/catalog/lineage
Get an overview of how your data is connected.
Lineage is the module where you can understand your data connections by seeing it. You have sources that connect to transformations that generates a table that will later on be sent to a destination. In scale, that might be a lot to keep in mind, so we drew it for you.
Lineage feature is **only available for Growth or Custom plans**.
From the catalog, select a table. Then, go to the Lineage tab and see its connections to understand how this data evolve through the pipelines.
# Catalog
Source: https://docs.nekt.com/catalog/overview
Get an overview of all your data in one single place, and learn how to structure it.
The Catalog is where you will find all your data. Every extracted table ends up stored there, Queries and Notebooks use data from it as input and store their output tables there, and Destinations and Integrations consume data from it. It's the core of everything.
![]()
As soon as you configure a source, you'll see its tables in the Catalog, although they might be empty. That's expected and only means the tables are ready to receive data. Once the source has a successful run (triggered manually or on its schedule), the tables fill with your data.
## How data flows through the Catalog
* **Sources** write tables to the Catalog as data is extracted.
* **Queries** and **Notebooks** read tables from it and write their outputs back.
* **History** tracks changes to tables over time.
* **Destinations** and **Integrations** consume tables from it to deliver value.
The Catalog gives you an overview of all your data and the control that comes from having everything in a single place.
## The hierarchy
Data in the Catalog is organized into a simple hierarchy. Each building block has its own page:
The top-level storage and permission boundary, such as Bronze, Silver, and Gold.
Organization inside a Layer, grouping tables by source, domain, or project.
The datasets created by Sources, Queries, Notebooks, and History.
A Layer contains Folders and Tables; a Folder can contain more Folders and Tables. The Catalog also stores [Volumes](/catalog/volumes) for non-tabular files and [External tables](/catalog/external-tables) that reference data in place without copying it.
## Organize your Catalog
A clear structure makes data easier to find, protects access boundaries, and gives people and AI agents better context. Define the first version before connecting production sources, then evolve it as real use cases appear.
Choose the top-level structure carefully. Moving tables later can require updates to permissions, Queries, Notebooks, Destinations, and context documents that reference their paths.
For how to pick a Layer model (by processing stage, by client, or both) and how to name tables so people and agents understand them, see [Layers](/catalog/layers) and [Tables](/catalog/tables).
### Start with the minimum structure
Decide whether the workspace will contain internal data, client data, or both.
List which teams or clients must be isolated. Use those boundaries to choose the Layer model.
Create only the Layers required for the first use case. You can add more as the operation grows.
Select only the streams required for the first use case and give their output tables descriptive names.
Revisit the structure after the team has used the data. Recurring questions reveal which Silver or Gold tables should be created next.
### Avoid common mistakes
* Creating a Layer for every Source when there is no access or lifecycle reason for it.
* Mixing source tables and business-ready tables without a naming or Layer convention.
* Connecting every available stream before defining the first use case.
* Creating deep Folder hierarchies that make tables harder to find.
* Renaming Layers and tables without checking dependent pipelines and context documents.
## API & MCP
List layers, then manage tables, fields, folders, and volumes over REST.
Browse and query the catalog with `list_tables`, `get_table_preview`, and the query tools.
## Related documentation
Map the first source streams to tables in your Catalog.
Control which Members can access each Layer and table.
Understand how tables and pipelines depend on each other.
Store files beyond tables, such as PDFs, images, and videos.
# Permissions
Source: https://docs.nekt.com/catalog/permissions
Ensure the right people see the right data.
Permissions will help you to take care of the data governance in your workspace. Using this resource, you can control who can access and edit each layer, folder, or table (individually or for a group of people).
Managing permissions is only available for Growth or Custom plans. On Starter, permissions are not enforced: every Member can access all data, pipelines, and the Context Layer. Creating or deleting Layers remains restricted to Owners and Admins.
Select a table and go to the Permissions tab. Select users or a group of users and define their permissions: Manager, Editor or Viewer.
To understand the effect of permissions and more details about it, please refer to the [Permissions](/workspace/permissions) page.
## API & MCP
List, assign, update, and revoke catalog permissions.
`list_permissions`, `grant_permission`, `update_permission_level`, and `revoke_permission`.
# Tables
Source: https://docs.nekt.com/catalog/tables
The datasets that live in the Catalog, created by Sources, Queries, Notebooks, and History.
A table is a dataset in the Catalog. Tables are created by [Sources](/sources/overview), [Queries](/queries/overview), [Notebooks](/notebooks/overview), and [History](/histories/overview), and every table lives inside a [Layer](/catalog/layers), optionally organized into [Folders](/catalog/folders).
Under the hood, tables are stored as Delta Tables in your Lakehouse, so they are consistent, versioned, and ready for both analytics and processing.
## The table overview
Open a table to see its details:
* Its **columns** and their types.
* **Metadata** such as size, total number of rows, and creation and last-update dates.
* A link to the **source** or pipeline that generates it.
When a source is first configured, its tables appear in the Catalog before any run has filled them. Empty tables are expected at that point: trigger the source or wait for its schedule, and the data will land.
## Name tables for humans and agents
Table names are part of the context your team and AI agents rely on. A person should understand what a table contains without opening it.
* Use lowercase `snake_case` names.
* Name the business entity and, when relevant, its grain: `facebook_ads_campaign_daily` is clearer than `ads_data`.
* Keep source names when they help distinguish similar datasets: `hubspot_deals` and `erp_orders`.
* Avoid unexplained abbreviations, temporary labels, and names such as `table_1`, `new`, or `final`.
* Add descriptions to tables and important fields, especially when a business definition is not obvious.
## API & MCP
Read the Table object and update tables and fields.
`list_tables`, `get_table_preview`, and `get_relevant_tables_ddl`.
## Related kinds of data
* **[External tables](/catalog/external-tables)** reference a BigQuery table in your own project and are read in place, without copying. They appear in the tables list with an **External** badge.
* **[Volumes](/catalog/volumes)** store non-tabular files such as PDFs, images, and videos alongside your tables.
## Related documentation
The storage and permission boundary a table lives in.
Group tables inside a Layer.
See how tables and pipelines depend on each other.
Validate the data in your tables.
# Volumes
Source: https://docs.nekt.com/catalog/volumes
Store more than tables.
If data is everywhere, it's expected that it's not always in a table-shaped format. Beyond the tables and structured data, you can have important information on PDFs, images, videos, and so on. Through Volumes, Nekt supports the storage and manipulation of them too so you can integrate it all in a single place and make sense out of it.
Volumes feature is **only available for Growth or Custom plans**.
From the catalog, select the layer where you want to store your volumes and click on "Add Volume".
You'll have your files on the catalog and can reference them directly in your transformations just as you do with the tables.
## Populated by source connectors
You don't always have to upload files by hand. Some source connectors can push files straight into a volume as part of the extraction, and emit a metadata table that links each row to the stored file. Point the connector at a volume in its configuration and it stays in sync with the source.
A few examples of connectors that support this today:
* [Pipefy](/sources/pipefy): card attachments
* [Gmail](/sources/gmail): email attachments
* [SharePoint](/sources/sharepoint): raw files, when the connector is set to Unstructured mode
Check the individual [Sources](/sources/overview) pages to see whether the connector you need writes to a volume.
## Quick example
Upload a folder of PDFs to a volume, then run a Notebook that reads each file, summarizes it with an LLM, and writes the results to a table. From there you can query the summaries in the [Explorer](/explorer/overview) or push them to Slack, email, or a warehouse through a [Destination](/destinations/overview). Contact support to discuss your use case.
## API & MCP
Read the Volume object.
`list_volumes`, `list_files`, `read_file`, and the upload tools.
# Introduction
Source: https://docs.nekt.com/data-api/introduction
Query your data using SQL through our Data API.
The Data API is available on all **paid plans**.
## Overview
The Data API provides a simplified way to query your data using SQL. Simply submit your query and download the results.
## Output Formats
Data API supports two output formats:
### Parquet Mode
Parquet is a columnar storage format that offers better compression and performance for analytical queries. This is the recommended format for most use cases.
**Advantages:**
* Better compression and smaller file sizes
* Faster query performance
* Preserves data types and schema information
* Ideal for large datasets
**Considerations:**
* May have limitations with certain data types (see [Known Limitations](./known-limitations))
### CSV Mode
CSV (Comma-Separated Values) is a simple text format that's widely supported by most tools and applications.
**Advantages:**
* Universal compatibility
* Easy to read and inspect
* Simple to import into spreadsheet applications
**Considerations:**
* Larger file sizes compared to Parquet
* No schema information preserved
* All values are represented as strings
## How It Works
Go to your Workspace settings and [create an API Key](https://app.nekt.ai/settings/api-keys), it will be necessary to use all Data API endpoints.
Nekt Data API is powered by **AWS Athena** (for AWS organizations) or **GCP BigQuery** (for GCP organizations), using standard SQL syntax.
The most common use case is to query the data of a specific table:
```sql query.sql theme={null}
SELECT
*
FROM
"nekt_raw"."table_name"
LIMIT
200
```
You can also specify the output format (parquet or csv) in your request. If not specified, parquet is used by default.
Use SQL filters, joins, where clauses and many more SQL statements to refine your query.
After submitting your query, the API will automatically process it and provide you with a download link to get the results in your chosen format.
The download link is only valid for 1 hour.
## Pseudo code implementation
```mermaid theme={null}
flowchart TD
A[Start: SQL Query Execution] --> B[Create an API Key]
B --> C[Submit query with SQL statement and format]
C --> D[API processes query automatically]
D --> E[Receive download link for results]
E --> F[Download results file - Parquet or CSV]
F --> G[End: SQL Query Execution]
```
## Next Steps
* Review the [Known Limitations](./known-limitations) for your cloud provider and output format
* Check the [SQL Query endpoint documentation](./sql-query) for detailed API reference
# Known Limitations
Source: https://docs.nekt.com/data-api/known-limitations
Understand the limitations and workarounds for different cloud providers and output formats.
This page outlines known limitations when using Data API with different cloud providers and output formats.
## AWS Limitations
### Parquet Mode (AWS)
#### Timestamp with Timezone Issue
**Issue:** Nekt dates and datetimes are usually stored with timestamp precision, which can cause the following error:
```
NOT_SUPPORTED: Unsupported Hive type: timestamp(3) with time zone
```
**Workaround:** Wrap timestamp columns with CAST in your SELECT query:
```sql theme={null}
SELECT
CAST(created_at AS TIMESTAMP) AS created_at,
CAST(updated_at AS TIMESTAMP) AS updated_at,
other_column
FROM
"nekt_raw"."table_name"
```
#### Duplicate Column Names
**Issue:** Parquet format does not support duplicate column names in the result set.
**Error Example:**
```
Column name 'column_name' is duplicated
```
**Workaround:** Use column aliases to ensure all columns have unique names:
```sql theme={null}
SELECT
table1.id AS table1_id,
table2.id AS table2_id,
table1.name
FROM
"nekt_raw"."table1"
JOIN
"nekt_raw"."table2" ON table1.id = table2.ref_id
```
#### All NULL Values (Unknown Type)
**Issue:** When a column contains only NULL values, Parquet cannot infer the data type, which may cause errors.
**Error Example:**
```
Unable to infer schema for Parquet. It must be specified manually
```
**Workaround:** Explicitly cast NULL columns to a specific type:
```sql theme={null}
SELECT
column_with_data,
CAST(column_with_all_nulls AS STRING) AS column_with_all_nulls
FROM
"nekt_raw"."table_name"
```
#### Complex Nested Types
**Issue:** Deeply nested structures (arrays of structs, maps with complex types) may not serialize correctly to Parquet.
**Workaround:**
* Flatten nested structures using lateral views and explode functions
* Convert complex types to JSON strings:
```sql theme={null}
SELECT
id,
TO_JSON(complex_column) AS complex_column_json
FROM
"nekt_raw"."table_name"
```
#### Large String Values
**Issue:** Extremely large string values (> 2GB) may cause memory issues during Parquet serialization.
**Workaround:**
* Filter or truncate large text fields
* Use CSV mode instead for tables with very large text columns
### CSV Mode (AWS)
#### Special Characters in Data
**Issue:** Data containing special characters like commas, quotes, or newlines may not be properly escaped.
**Workaround:** The CSV mode uses standard escaping, but verify your data after download if you have complex text fields.
#### Data Type Loss
**Issue:** All values are represented as strings in CSV format, losing type information.
**Impact:** You'll need to manually convert data types when importing the CSV into your target system.
#### Decimal Precision
**Issue:** Very large decimal values may lose precision when represented as strings.
**Workaround:** Review decimal columns after import to ensure precision is maintained.
## GCP Limitations
### Parquet Mode (GCP)
#### Timestamp Formatting
**Issue:** GCP may handle timestamp formats slightly differently than AWS.
**Workaround:** Similar to AWS, use CAST for timestamp columns:
```sql theme={null}
SELECT
CAST(timestamp_column AS TIMESTAMP) AS timestamp_column
FROM
"nekt_raw"."table_name"
```
#### BigDecimal Precision
**Issue:** Very high precision decimal numbers may be rounded in Parquet format.
**Workaround:** If exact precision is critical, consider using CSV mode or casting to string:
```sql theme={null}
SELECT
CAST(high_precision_decimal AS STRING) AS high_precision_decimal
FROM
"nekt_raw"."table_name"
```
### CSV Mode (GCP)
The CSV mode limitations for GCP are similar to AWS. Refer to the AWS CSV Mode section above.
## General Limitations
### Query Timeout
**Issue:** Queries that take longer than 30 minutes will timeout.
**Workaround:**
* Add filters to reduce the data volume
* Break large queries into smaller chunks
* Use LIMIT clause for testing
### Result Set Size
**Issue:** Very large result sets (> 100GB) may fail or take a long time to generate.
**Workaround:**
* Use pagination with LIMIT and OFFSET
* Filter data to only include necessary columns and rows
* Consider using data snapshots for very large datasets
### Memory-Intensive Operations
**Issue:** Operations like large JOINs, window functions on huge datasets, or complex aggregations may fail due to memory constraints.
**Workaround:**
* Optimize queries by filtering early in the query
* Use smaller time windows for analysis
* Pre-aggregate data in transformations before querying
## Best Practices
To avoid common limitations:
1. **Always test queries in the Explore module first** to validate they work correctly
2. **Start with a LIMIT clause** to test the query structure before processing large datasets
3. **Use explicit type casting** for timestamp and decimal columns when using Parquet mode
4. **Monitor query execution time** and optimize queries that approach the timeout limit
5. **Choose the right format**: Use Parquet for analytical queries and CSV for maximum compatibility
6. **Handle NULL values explicitly** by casting them to appropriate types
## Need Help?
If you encounter an issue not listed here, please contact support with:
* Your SQL query
* The error message you received
* Your cloud provider (AWS/GCP)
* The output format you're using (Parquet/CSV)
# SQL Query
Source: https://docs.nekt.com/data-api/sql-query
POST /api/v1/sql-query/
Execute a SQL query on the organization's data warehouse and return presigned URLs to download results.
This endpoint executes SQL queries on the organization's data warehouse (AWS Athena or GCP BigQuery)
and returns presigned URLs for downloading the results directly from cloud storage (S3 or GCS).
**Important**: Customer data never transits through Nekt servers. Results are stored in the
organization's cloud storage and accessed via time-limited presigned URLs (1 hour expiration).
**Requirements**:
- Organization must have completed cloud setup (`cloud_setup_completed = True`)
**Cloud Provider Support**:
- AWS: Executes queries on Athena
- GCP: Executes queries on BigQuery
**Output Formats**:
- `csv`: CSV format (default) - supports simple data types only
- `parquet`: Parquet format - supports all data types including complex types (arrays, structs, etc.)
**Limitations**:
- BigQuery CSV export does not support complex data types (ARRAY, STRUCT, JSON). Use Parquet mode instead.
- Athena UNLOAD for Parquet may not support certain timestamp types. Cast problematic columns in the SQL query.
**Example Request**:
```json
{
"sql": "SELECT * FROM my_table LIMIT 100",
"mode": "parquet",
"mode_options": {"compression": "SNAPPY"}
}
```
**Example Response**:
```json
{
"state": "SUCCEEDED",
"presigned_urls": [
"https://storage.googleapis.com/bucket/query-20251119-154427-123456-000000000000.parquet?..."
],
"data_scanned_in_bytes": 11250,
"execution_time_in_millis": 408
}
```
**Security**: Your data never transits through Nekt servers. Results are stored in your organization's cloud storage and accessed via time-limited presigned URLs (1 hour expiration).
# Amazon SQS as a destination
Source: https://docs.nekt.com/destinations/amazon-sqs
Send your data from Nekt to Amazon SQS.
Amazon SQS (Simple Queue Service) is a fully managed message queuing service provided by AWS that enables decoupling and scaling of microservices, distributed systems, and serverless applications. It provides reliable, scalable message delivery for asynchronous communication between application components.
## 1. Setting up permissions for sending messages to your SQS queue
You need to define some permissions to allow Nekt to send messages to your Amazon SQS queue. Check the instructions below:
**Create role and add custom policy**
* Open the AWS Console using the account that hosts the SQS queue you'd like to send data to.
* Enter the IAM (Identity and Access Management) service page.
* In the left panel under **Access management**, select **Roles**.
* Click **Create role**.
* In **Trusted entity type**, select **Custom trust policy**.
* Change the **Principal** value to:
```json theme={null}
{
"AWS": "arn:aws:iam::{AWS_ACCOUNT_ID}:role/nekt-ecs-task-role"
}
```
The variable must be replaced by the ID of the AWS account where the Nekt workspace is deployed. In order to find this information, click the dropdown in the top-right corner of the AWS console page after you log in with your account, then click on the "copy" icon to the right of your **Account ID**.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::{AWS_ACCOUNT_ID}:role/nekt-ecs-task-role"
},
"Action": "sts:AssumeRole"
}
]
}
```
* Click **Next**.
* Click **Next** again without adding any permission. We'll create an inline policy further on.
* Under **Role details**, you can type any **Role name** as you want. Example: `nekt-sqs-destination`.
* Optionally, type a **Description**. Example: `Used by Nekt to send data to Amazon SQS`.
* Click **Create role** .
You should get redirected back to the IAM roles page and a message of success should appear at the top.
**Add permissions to role**
* Open the role you just created by clicking **View role** in the right end of the green banner that appeared at the top of the page. If you dismissed it already, you can type the given **Role name** in the search field and then select it when it appears in the list.
* Within **Permission policies (0)**, click the **Add permissions**.
* Select **Create inline policy**.
* Within **Policy editor**, select `JSON`.
* Paste the following policy in the **Policy editor**:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"sqs:SendMessage",
"sqs:GetQueueUrl",
"sqs:GetQueueAttributes"
],
"Resource": "{SQS_QUEUE_ARN}"
}
]
}
```
* Replace the `{SQS_QUEUE_ARN}` part with the ARN of the SQS queue you'd like to send data to.
* If you don't know it, open a new tab in your browser and enter the Amazon SQS service page.
* In the left panel, click **Queues**.
* Open the desired queue by clicking on its name in the list.
* The **ARN** is displayed under **Details**. Click the "copy" icon next to it.
* Go back to the IAM policy editor that you left open in the previous tab and paste the ARN you just copied over `{SQS_QUEUE_ARN}`.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"sqs:SendMessage",
"sqs:GetQueueUrl",
"sqs:GetQueueAttributes"
],
"Resource": "arn:aws:sqs:{AWS_REGION}:{AWS_ACCOUNT_ID}:{QUEUE_NAME}"
}
]
}
```
* Click **Next**.
* Under **Policy details** type any Policy name as you want. Example: `nekt-sqs-destination-policy`.
* Click **Create policy**.
You should get redirected back to the role you just created and a message of success should appear at the top.
When setting up the connector at Nekt, you will use the ARN of the role you've just created.
Get in touch with us if you face any problem setting up this permission.
## 2. Add your Amazon SQS access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Amazon SQS option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Queue name:** Specify the queue name you want to send data to.
* **AWS region name:** The region name from the respective Amazon SQS account.
* **Assume IAM Role ARN:** the ARN of the role you've just created from [Step 1: Setting up permissions](#1-setting-up-permissions-for-sending-messages-to-your-sqs-queue).
* **Convert payload keys:** Set this property if you want to convert the payload keys to a specific format (snake\_case, camelCase or PascalCase). If set to 'None', the keys will be sent as is.
3. Click **Next**.
## 3. Select your catalog data to send
1. The next step is letting us know which data you want to send to the queue. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 4. Configure your Amazon SQS data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Amazon SQS, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the Amazon SQS.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| The load stops on a specific row (FIFO queue) | The **Message group ID column** is empty on that row. | Make sure the column is filled on every row, or set a **Message group ID fallback**. |
| Re-running re-enqueues every message (FIFO queue) | Without a deduplication column, each message gets a new identifier. | Set **Message deduplication ID column**. |
# BigQuery as a destination
Source: https://docs.nekt.com/destinations/bigquery
Send your data from Nekt to BigQuery.
BigQuery is Google Cloud's fully managed, serverless data warehouse that enables super-fast SQL queries using the processing power of Google's infrastructure. It's designed for large-scale data analytics and provides tools for data analysis, machine learning, and business intelligence.
## 0. Required pre work
Make sure the account you are using on the integration has the following permissions:
\_ BigQuery Data Editor
\_ BigQuery Job User
## 1. Add your BigQuery access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the BigQuery option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Project:** The ID of the GCP project your BigQuery workspace is linked to.
* **Dataset name:** The target dataset where data will be sent to. For detailed instructions, [click here](https://cloud.google.com/bigquery/docs/datasets-intro).
* **Location:** The location and the different regions where data can be stored and processed. For detailed instructions, [click here](https://cloud.google.com/bigquery/docs/locations#regions).
* **Credentials file (JSON):** The credentials file for the service account linked to your BigQuery project. Make sure the account has access to perform write operations on BigQuery. For detailed instructions, [click here](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account).
3. Decide about:
* **Write method:** The method to use for writing to BigQuery.
* **Denormalize data:** Determines whether to denormalize the data before writing to BigQuery. A false value will write data using a fixed JSON column based schema, while a true value will write data using a dynamic schema derived from the input table.
* **Upsert mode:** A value of true will write to a temporary table and then merge into the target table (upsert). This requires the target table to have unique key properties. A value of false will write to the target table directly (append).
* **Overwrite mode:** Determines if the target table should be overwritten on load. This means replacing the actual data with the incoming one. If set to false, data will be appended. IMPORTANT: if Upsert is set to true, this setting will be ignored.
4. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to BigQuery. You can select a whole layer or specific tables.
> **Tip:** The tables can be easily found by typing its name.
2. Click **Next**.
## 3. Map your data to send
1. Give a name to be given to each table in the destination.
2. For each table, select the column that will work as a primary key.
3. By clicking on the fields mapping button, you will see a list of columns and their correspondent names in the destination. You can modify it if you want.
4. Click **Next**.
## 4. Configure your BigQuery data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
Once completed, you'll receive confirmation that your new destination is set up!
You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your BigQuery account, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon.
Once executed, your data should be seen on BigQuery.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `Too many total leaf fields` | The table exceeded BigQuery's field limit. | Reduce the table's width at the source, or store the properties object as a JSON string, then run a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). |
| `Incompatible type change for column` | A column's type changed incompatibly between runs. | Run a full sync to recreate the table with the new type. |
| `Provided Schema does not match Table - Cannot add fields` | The table structure at the destination cannot accommodate the change. | Run a full sync to recreate the table. |
| `Array specified for non-repeated field` | A field went from a single value to multiple values at the source. | Run a full sync of the affected stream. |
# BotConversa as a destination
Source: https://docs.nekt.com/destinations/botconversa
Send your data from Nekt to BotConversa.
BotConversa is a chatbot platform that helps businesses automate customer interactions and support. It provides tools for building, deploying, and managing chatbots across various messaging channels, improving customer engagement and operational efficiency.
## 1. Add your BotConversa configuration
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the BotConversa option from the list of connectors.
2. Click **Next** and you'll be prompted to add your configurations:
* **API key:** The token to authenticate against the API service, obtained from your BotConversa account.
* **Message template:** The message template to send via WhatsApp. Each row of your table will be translated to a WhatsApp message. You can use variables such as for inserting the current date in the format YYYY-MM-DD or reference a column value by using .
* **Subscriber ID column name:** The name of the column on your input table where the `subscriber_id` for each user on BotConversa can be found. This is necessary in order for the connector to know who to send the messages to.
* **Enable partial success:** Whether the connector should continue trying to sync records after one fails.
> ### Short example
>
> Imagine you have the following table:
>
> | name | balance | subscriber\_id |
> | - | - | - |
> | John Doe
| 1200.00
| 123514
|
> | Peter Krauss
| 554.50
| 155699
|
> | Mary Jane
| 2000.0
| 877623
|
>
> For each row in the table, a WhatsApp message will be sent to the correspondent `subscriber_id`.
>
> If you define your `message` config as:
>
> ```
> Hello {name},
> Your balance for {datestamp} is $ {balance}.
> ```
>
> The final message for the first row of the table would look like:
>
> ```
> Hello John Doe,
> Your balance for 2025-02-06 is $ 1200.00.
> ```
>
> This process repeats for each row in the table.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which table you want to use as input for the BotConversa destination.
> **Tip:** The tables can be easily found by typing its name.
2. Click **Next**.
## 3. Configure your BotConversa data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
Once completed, you'll receive confirmation that your new destination is set up!
You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, in order for the messages to be sent, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the play icon.
Once executed, your messages should be sent through BotConversa.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# ClickHouse as a destination
Source: https://docs.nekt.com/destinations/clickhouse
Send your data from Nekt to a ClickHouse database.
ClickHouse is an open-source, column-oriented database management system built for real-time analytics on large volumes of data. The same connection works for both ClickHouse Cloud and self-managed ClickHouse servers.
![]()
## 1. Add your ClickHouse access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the ClickHouse option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Host:** The host address of your ClickHouse database, without the protocol (`http://` or `https://`). For ClickHouse Cloud, copy it from your service's **Connect** screen.
* **Port:** The connection port. The default secure (HTTPS) port is `8443`; your port might be different.
* **Database:** The name of a database in your ClickHouse account.
* **Schema:** (Optional) The database to write to. Defaults to the connection database.
* **User:** The user name for accessing your ClickHouse database.
* **Password:** The password used for accessing your ClickHouse database.
3. Decide about the **Load method**:
* **Append:** Always inserts new rows (uses a `MergeTree` table engine).
* **Upsert:** Inserts or updates based on the table's primary keys, keeping the latest row per key (uses a `ReplacingMergeTree` engine ordered by the primary keys).
* **Overwrite:** Clears the table before loading.
4. Click **Next**.
You must ensure your database accepts incoming connections from Nekt, and the database user has write permissions in order to write data.
In ClickHouse, deduplication for the **Upsert** method is handled by the table engine and applied during background merges. Querying with `SELECT ... FINAL` (or running `OPTIMIZE TABLE ... FINAL`) returns the deduplicated result. The table engine is set when the table is created, so changing the load method on an existing table requires recreating it.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data to send
1. Give a name to be given to each table in the destination.
2. For each table, select the column that will work as a primary key. This is required for the **Upsert** load method, as it defines the ordering key used for deduplication.
3. By clicking on the fields mapping button, you will see a list of columns and their correspondent names in the destination. You can modify it if you want.
4. Click **Next**.
## 4. Configure your ClickHouse data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your ClickHouse database, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the defined ClickHouse location.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `ACCESS_DENIED` mentioning `SHOW TABLES` | The destination user lacks the required grants. | Grant `SHOW`, `SELECT`, `INSERT`, `CREATE TABLE`, and `ALTER TABLE` on the destination database. |
# ClickUp as a destination
Source: https://docs.nekt.com/destinations/clickup
Send your data from Nekt to ClickUp.
ClickUp is a project management and productivity platform that helps teams organize work into tasks, lists, and projects. With this destination you can create and update ClickUp **tasks** directly from your Nekt tables.
## 1. Add your ClickUp access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the ClickUp option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **API Token**: Your ClickUp personal API token (starts with `pk_`). Generate it in ClickUp under **Settings → Apps → API Token**.
* **List ID**: The default ClickUp List ID where new tasks are created. Required unless you map a `list_id` field per record. It is ignored when updating an existing task.
* **Enable partial success**: Whether the connector should continue trying to sync records after one fails.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data fields
Configure how the columns from your source table should be mapped to ClickUp task fields. Map a column to `name` to create new tasks, and map a column to `id` to update existing ones (matched by their ClickUp task ID).
Below are the most common ClickUp task fields you can map your data to. See the [ClickUp Create Task reference](https://developer.clickup.com/reference/createtask) for the full list.
| Field | Type | Required | Description |
| :- | :- | :- | :- |
| `name` | String | Yes\* | The task title. Required to create a task. |
| `id` | String | No | The ClickUp task ID. When mapped, the task is **updated** instead of created. |
| `list_id` | String | No | Per-record override of the List a new task is created in. |
| `description` | String | No | The task description (plain text). |
| `markdown_content` | String | No | The task description in Markdown (overrides `description`). |
| `status` | String | No | The task status. Must match a status configured on the destination List. |
| `priority` | Int | No | `1` Urgent, `2` High, `3` Normal, `4` Low. |
| `due_date` | Int (ms) | No | Due date as an epoch timestamp in milliseconds. Use the `to_epoch_ms` transform for date strings. |
| `start_date` | Int (ms) | No | Start date as an epoch timestamp in milliseconds. |
| `assignees` | Array of Ints | No | ClickUp user IDs to assign. A single value is wrapped into a list automatically. |
| `tags` | Array of Strings | No | Tag names to apply to the task. |
| `parent` | String | No | The parent task ID, to create a subtask. |
\* Required only when creating tasks (i.e. when no `id` is mapped).
### Transforms
Each mapping rule can apply ordered value transforms before sending: `uppercase`, `lowercase`, `trim`, `to_string`, `to_int`, `round_int`, `to_epoch_ms` (date string → ClickUp millisecond timestamp), `json_parse` (parse a JSON string into a list/object), `yesno_to_bool`, `null_if:`, and `only_if:`.
## 4. Configure your ClickUp data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on ClickUp, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on ClickUp.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# CRM Educacional as a destination
Source: https://docs.nekt.com/destinations/crm-educacional
Send leads and enrollments from Nekt to CRM Educacional.
CRM Educacional is the admissions CRM used by Brazilian educational institutions to run their selection processes end to end: capturing leads, taking applications (*inscrições*) and calling candidates for enrolment. The CRM Educacional destination writes data from your Lakehouse back into the CRM: it creates and updates leads, registers candidates in a selection process (*concurso*), and updates custom fields of existing enrollments.
Every institution runs on its own address, so the connector asks for the URL of your CRM in addition to the credentials of an integration user. It uses the same access as the [CRM Educacional source](https://docs.nekt.com/sources/crm-educacional).
CRM Educacional sells its software under the **Wakeme** name. This connector writes through the integration API of **Wakeme PRO** and **Wakeme Essencial**. **Wakeme One** is a different system, with its own application and API, and is not covered by this connector.
The connector performs one operation per destination, selected through the **Mode** setting:
* **Create/Update Leads** - creates a lead, or updates it when the CRM finds one with the same CPF or e-mail.
* **Create/Update Enrollments (Lead + Inscrição)** - creates or updates a lead **and** its enrollment in a concurso, optionally creating the offer (*oferta*) and category it points to.
* **Update Enrollment Custom Fields** - updates fields your institution created on an existing enrollment, found by its enrollment number.
To run more than one operation, create one destination per mode.
## 1. Add your CRM Educacional access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the CRM Educacional option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **API URL**: the address of your institution's CRM Educacional API, for example `https://yourschoolapi.crmeducacional.com`. You can find it in the CRM under **Configurações > CRM Educacional > aba Captação > "Endereço da Api Ficha de Inscrição"**.
* **Username** and **Password**: the credentials of an integration user (*Usuário de Integração*), created in the CRM under **Configurações > Usuário de integração > Novo**, with type **API CRM EDUCACIONAL**.
* **Mode**: the operation to perform (see above).
* **Concurso code** (only for the **Enrollments** mode): the code of the concurso every row is enrolled in, used when no column of your table carries it.
* **Dry run**: test mode. The connector signs in, maps and checks every row, and shows the first rows it would send in the run log, **without writing anything to the CRM**. Use it before the first real load.
* **Enable partial success**: whether the connector should keep loading after the CRM refuses a row. When off, the load stops at the first refused row.
* **Authentication type** (advanced): **Integration user** (recommended) or **Dynamics CRM user**.
* **Requests per minute** (advanced): how fast the connector calls the API. Defaults to 50, the limit CRM Educacional documents per user.
* **Form fields location** (advanced): keep **Automatic** unless Nekt support asks you to change it.
CRM Educacional allows **50 requests per minute per user**, and every row is one request, about 3,000 rows per hour. The quota is shared with every integration signed in with the same user, including the CRM Educacional source. Create an integration user for this destination alone, and send only the rows that changed instead of the full table.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your columns
Map the columns of your input table to CRM Educacional fields. Use the field's **logical name** in the CRM (for example `cad_cpf`, `emailaddress1`, `firstname`, `mobilephone`). You can find logical names in the CRM under **Configurações avançadas > Personalizações > Personalizar o sistema > Entidades**, by opening the entity and its **Campos** list.
* Option set fields take the option's **integer value**, not its label (for example, a situação of 2).
* Empty and null values are never sent, so existing data in the CRM is preserved when a column has no value.
### Create/Update Leads
The CRM finds an existing lead by **CPF** (`cad_cpf`) or **e-mail** (`emailaddress1`) and updates it; when neither matches, it creates a new lead. Map at least one of the two, or every row becomes a new lead. Any lead field can be sent by its logical name.
To place the lead in a specific business unit, map a column to `usuario` (a CRM user name) or `equipe` (a CRM team name).
**Example input table:**
| cpf | email | first\_name | last\_name | campaign |
| :- | :- | :- | :- | :- |
| 52998224725 | [ana@example.com](mailto:ana@example.com) | Ana | Souza | vestibular-2027 |
Column mapping: `cpf` → `cad_cpf` (transform `cpf_mask`), `email` → `emailaddress1`, `first_name` → `firstname`, `last_name` → `lastname`, `campaign` → `cad_campaignname`.
### Create/Update Enrollments (Lead + Inscrição)
Prefix every destination field with the entity it belongs to:
| Prefix | Required field | Notes |
| :- | :- | :- |
| `concurso.` | `concurso.cad_codigo` | The code of the concurso. It can also be set once in the **Concurso code** setting. |
| `lead.` | `lead.cad_cpf` or `lead.emailaddress1` | A new lead also needs `lead.firstname` and `lead.lastname`. |
| `inscricao.` | None | Enrollment fields, for example `inscricao.cad_situacao`. |
| `oferta.` | `oferta.cad_identificadoroferta` | Optional. The offer is created when it does not exist (it then needs `oferta.cad_nome`), and never updated. |
| `categoria.` | `categoria.cad_identificadorcategoria` | Optional. Same behavior as the offer (a new category needs `categoria.cad_name`). |
CRM Educacional processes enrollments **after** accepting them. The run log confirms each row was accepted; problems found while the CRM processes it are reported by the CRM to the address configured in its own **"URL POST Cadastro Inscrição"** setting, not to Nekt.
### Update Enrollment Custom Fields
Map the column holding the enrollment number (for example `CAN-000000-CRM`) to `cad_numeroinscricao`, and the columns to update to their fields. Only fields **created by your institution** can be updated this way; CRM Educacional refuses product fields.
### Transforms
Each mapping accepts an optional ordered list of value transforms, applied before sending:
| Transform | Description |
| :- | :- |
| `trim` | Remove surrounding whitespace. |
| `uppercase` / `lowercase` | Change the text case. |
| `to_string` | Cast the value to text. |
| `to_int` | Cast the value to an integer. Use for option sets whose values are stored as text. |
| `round_int` | Round a decimal to the nearest integer. |
| `null_if:` | Drop the value when it equals `` (e.g. `null_if:-`). |
| `yesno_to_bool` | Convert `Sim`/`Não` (or `Yes`/`No`) to `true`/`false`. |
| `digits_only` | Keep only the digits, for example of a phone number. |
| `cpf_mask` | Format a CPF as `000.000.000-00`, restoring leading zeros lost in numeric columns. |
| `date_br` | Format a date as `DD/MM/YYYY`. |
## 4. Configure your CRM Educacional data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on CRM Educacional, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on CRM Educacional.
Start with **Dry run** enabled and a small table. Check the rows shown in the run log, then turn **Dry run** off to write for real.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# CSV as a destination
Source: https://docs.nekt.com/destinations/csv
Send your data from Nekt to CSV.
CSV (Comma-Separated Values) is a simple file format used to store tabular data, such as a spreadsheet or database. Each line represents a row of data, with values separated by commas, making it a widely supported format for data exchange between different applications and systems.
![]()
## 1. Add your CSV (Remote SSH) access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the CSV (Remote SSH) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access: host, port, user and password.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the spreadsheet. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your CSV (Remote SSH) data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your CSV, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the CSV.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# CSV AWS S3 as a destination
Source: https://docs.nekt.com/destinations/csv-aws-s3
Send your data from Nekt to CSV AWS S3.
CSV AWS S3 refers to CSV (Comma-Separated Values) files stored in Amazon Simple Storage Service (S3), which is a scalable cloud storage service. This allows you to store and access CSV data files in the cloud, providing reliable and secure data storage with high availability.
## 1. Add your CSV AWS S3 access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the CSV (AWS S3) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
### Required Configuration
* **Bucket name:** The AWS S3 bucket name where you want to store the CSV files.
* If using a Nekt-managed bucket (pattern: `nekt-lakehouse-{company-name}`), no additional authentication is required.
* If using your own AWS S3 bucket, you must provide AWS credentials (see below).
### AWS Authentication (Required for Custom Buckets)
If you're using your own AWS S3 bucket (not a Nekt-managed bucket), you need to provide AWS credentials to authenticate.
* **Access Key ID:** Your AWS access key ID for authentication. You can create this in the AWS IAM console.
* **Secret Access Key:** Your AWS secret access key for authentication. Keep this secure and never share it publicly.
To create AWS credentials for accessing your S3 bucket:
1. Log in to your [AWS IAM Console](https://console.aws.amazon.com/iam/)
2. Navigate to **Users** and select your user (or create a new one)
3. Go to the **Security credentials** tab
4. Click **Create access key**
5. Select **Application running outside AWS** or **Other** as the use case
6. Copy the **Access Key ID** and **Secret Access Key** (you won't be able to see the secret again)
**Required IAM Permissions:**
Your AWS user/role needs the following S3 permissions:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::your-bucket-name",
"arn:aws:s3:::your-bucket-name/*"
]
}
]
}
```
### File Configuration
* **Output folder path:** Folder path where to store output files inside the S3 bucket. Leave empty if the file should be stored in the bucket's root folder.
* Example: `data/exports` or `reports/daily`
* **Stream name:** A custom name for the stream that can be used to compose the file name.
* **File naming format:** The format to be used to generate the output file names. Available variables:
* `{stream_name}` - The custom stream name you defined
* `{datestamp}` - Date in YYYY-MM-DD format
* `{timestamp}` - Timestamp in YYYY-MM-DDTHHMMSS format
* `{batch_id}` - Unique identifier for the batch
* `{batch_index}` - Sequential index for the batch
* Example: `{stream_name}_{timestamp}_{batch_index}.csv`
* **Maximum CSV file size (MB):** (Optional, default: 1000 MB) Define a maximum size for each CSV file. If your input data exceeds this size, multiple files will be generated respecting the maximum size defined.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to export to S3. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your CSV (AWS S3) data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it exports, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated in S3 (every day, once a week, only at specific times, etc.).
3. Click **Done**.
## 4. Check your new destination
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. The pipeline will run according to your configured trigger schedule. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion.
3. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your CSV files will be available in your S3 bucket at the path you specified.
You can access your exported CSV files directly in your S3 bucket using the AWS S3 Console or any S3-compatible tool.
Make sure your S3 bucket has the appropriate permissions and policies configured to prevent unauthorized access to your data.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Dayforce as a destination
Source: https://docs.nekt.com/destinations/dayforce
Send your data from Nekt to Dayforce.
Dayforce is a human capital management platform that provides comprehensive solutions for workforce management, payroll, benefits, and talent management. It helps organizations streamline HR processes, manage employee data, and optimize workforce operations.
![]()
## 1. Add your Dayforce access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Dayforce option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access credentials and endpoints. The following configurations are available:
* **Username**: Username to authenticate against the Dayforce API.
* **Password**: Password to authenticate against the Dayforce API.
* **Company ID**: The company ID associated with your Dayforce workspace.
* **Authentication endpoint URL**: The auth endpoint URL to generate the access token (e.g., `https://your-company-url.dayforcehcm.com/connect/token`).
* **API endpoint URL**: The API endpoint URL to send data to (e.g., `https://your-company-url.dayforcehcm.com/api/cbb/V1/EmployeeRawPunches`).
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your Dayforce data to send
1. Select the fields from your selected table that you want to send.
2. For each field selected from the catalog, configure the mapping rules that dictate how incoming data translates to Dayforce fields:
* **Lakehouse Column**: The search path to get the value from the incoming stream (e.g., `$.last_name`, `$.email[*].value`).
* **Dayforce Field**: The field name on Dayforce's API where the data should be sent. Please note it is case sensitive (e.g., `LASTNAME`, `EMAIL`).
* **Default**: An optional default value to be used for the field if the lakehouse field is empty or missing. If no default value is provided and the field is not found, an empty value is used.
3. Click **Next**.
## 4. Configure your Dayforce data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Dayforce account, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Dayforce.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Implementation Notes
* **Authentication Verification**: The connector verifies your authentication credentials before sending any records. If the credentials or company ID are incorrect, the sync immediately fails with a clear error, ensuring bad credential issues are caught early instead of being treated as per-record failures.
* **Mapping Defaults**: When a mapping rule lacks a default value and the source field is missing, the connector sends an empty value instead of a spurious string (such as `"None"`) to the Dayforce API.
# Every Action as a destination
Source: https://docs.nekt.com/destinations/every-action
Send your data from Nekt to Every Action.
Every Action is a digital organizing and advocacy platform designed for nonprofits and political campaigns. It provides tools for donor management, email marketing, advocacy campaigns, and constituent relationship management to help organizations engage supporters and drive social impact.
![]()
## 1. Add your Every Action access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the EveryAction option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access. The following configurations are available:
* **Application name**: The application name associated with the API key.
* **API Key**: The API key used to authenticate against EveryAction's API service.
* **Should apply tag**: Whether to associate tags with newly created contacts or not.
* **Tag column name**: If you chose to apply tags, inform the name of the column that holds tag information on the input table. The tag value should be a string corresponding to the EveryAction tag ID.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your Every Action data to send
1. Select the fields from your selected table that you want to send.
2. For each field selected from the catalog, configure the mapping rules that describe how the incoming stream fields should be mapped to EveryAction fields:
* **Lakehouse Column**: The search path to get the value on the incoming stream (e.g., `$.last_name`, `$.email[*].value`).
* **EveryAction Field**: The destination field name on EveryAction's API (e.g., `LASTNAME`, `EMAIL`). Please note that this is case sensitive.
* **Default**: An optional default value to be used for the field if the mapped column is empty or cannot be found on the source data.
3. Click **Next**.
## 4. Configure your Every Action data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Every Action account, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Every Action.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Facebook Campaign Management as a destination
Source: https://docs.nekt.com/destinations/facebook-campaign-management
Update the status and budgets of your Meta campaigns, ad sets, and ads from Nekt.
Facebook Campaign Management lets you control your Meta advertising objects, campaigns, ad sets, and ads, directly from your data. Instead of manually editing each object in Ads Manager, you keep the desired state in a table and let Nekt apply it.
The Facebook Campaign Management connector reads a table where each row describes how a single Meta object should look, and applies those changes through Meta's Marketing API. This is ideal for automating routine ad operations: pausing under-performing campaigns, activating seasonal ones, or adjusting budgets in bulk based on the metrics already living in your catalog. By driving these changes from data, marketing and operations teams can react faster, keep spend under control, and avoid the errors that come with editing dozens of objects by hand.
## 1. Overview
Each row of your input table represents the **desired state of one Meta object**, identified by its ID. For every row, the connector sends an update to that object with the fields you mapped.
* One column must map to `id`, the Meta object you want to update.
* Every other mapped column (for example `status` or `daily_budget`) is a value to apply to that object.
* The same connector works for **campaigns, ad sets, and ads**, since they are all updated the same way by their ID.
The most common use cases are pausing/activating objects and changing budgets.
## 2. How the input table works
Think of your input table as "what each object should look like after the sync". Here is a simple example:
| id | status | daily\_budget |
| :- | :- | :- |
| 120247332527910413 | PAUSED | 3000 |
| 120247332527910999 | ACTIVE | |
When this table is synced, the connector will:
* **Row 1**: pause campaign `120247332527910413` and set its daily budget to `3000` (which is R\$30,00, see [Important considerations](#3-important-considerations) about units).
* **Row 2**: activate campaign `120247332527910999` and leave its budget unchanged.
A few rules that make the table easy to work with:
* **Empty cells are ignored.** In row 2 the `daily_budget` cell is empty, so the budget is not touched. This means you only need to fill the columns you actually want to change.
* **A row needs at least the `id` plus one field to change.** A row with only an `id` and no other values is skipped.
* **You can mix object types.** Some rows can be campaigns and others ad sets or ads, as long as each `id` is correct.
You usually build this table with a Query that selects the objects you want to manage and computes the new `status` or budget from your metrics, for example "pause every campaign whose ROAS in the last 7 days is below 1".
### Example: pause under-performing campaigns and reward the good ones
This Query joins campaign spend from the [Facebook Ads source](/sources/facebook-ads) with revenue from your own data, then decides what each campaign's state should be. Campaigns performing within the acceptable range are left out of the result entirely, so nothing is touched.
```sql theme={null}
WITH performance AS (
SELECT
i.campaign_id,
sum(i.spend) AS spend,
sum(coalesce(r.revenue, 0)) AS revenue,
sum(coalesce(r.revenue, 0)) / nullif(sum(i.spend), 0) AS roas
FROM silver.facebook_ads_insights i
LEFT JOIN silver.attributed_revenue r ON r.campaign_id = i.campaign_id
WHERE i.date >= current_date() - INTERVAL 7 DAYS
GROUP BY i.campaign_id
)
SELECT
campaign_id AS id,
CASE WHEN roas < 1 THEN 'PAUSED' END AS status,
CASE WHEN roas >= 3 THEN round(spend / 7 * 1.2, 0) END AS daily_budget
FROM performance
WHERE spend > 0
AND (roas < 1 OR roas >= 3)
```
Which produces:
| id | status | daily\_budget |
| :- | :- | :- |
| `120247332527910413` | `PAUSED` | |
| `120247332527910999` | | `43` |
The first campaign is paused and its budget untouched, the second gets a 20% budget increase and keeps its current status. Map `id` to `id`, `status` to `status`, and `daily_budget` to `daily_budget`.
Facebook Ads reports spend in the account currency major unit, so `43` here means R\$43,00, not 43 cents. Enable **"Enter budgets in your account currency?"** for this table, and Nekt converts to cents before sending.
## 3. Important considerations
* **Budgets are in cents by default.** Meta expects budget values in the account currency minor unit. For Brazilian Reais, `3000` means R$30,00 and `2000` means R$20,00. If you would rather enter values in your account currency (for example `30` for R\$30,00), enable **"Enter budgets in your account currency?"** in the configuration, and Nekt will multiply your value by 100 before sending it. This applies to `daily_budget` and `lifetime_budget`.
* **Activating objects can spend money.** Setting `status` to `ACTIVE` makes the object eligible to deliver and spend, exactly as if you flipped it on in Ads Manager. Make sure the budgets and the objects are correct before activating.
* **Status accepts `ACTIVE` or `PAUSED`.** These are applied as-is to the object.
The access token used by this connector can modify real campaigns, ad sets, and ads. We strongly recommend testing on a paused campaign first to confirm your table and mapping behave as expected before managing live objects.
## 4. Add your Facebook Campaign Management access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Facebook Campaign Management option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **Facebook authorization**: Complete Facebook authorization with an account that has the `ads_management` permission on the Facebook Ad Account whose objects you want to manage.
* **Account ID**: Select the specific Facebook Ad Account that owns the objects you want to update.
* **Enter budgets in your account currency?**: When enabled, you can provide budgets in your account currency major unit (for example `30` for R\$30,00) and Nekt multiplies the value by 100 to send it in cents, as Meta requires. Applies to `daily_budget` and `lifetime_budget`. Leave it disabled if your table already stores budgets in cents.
* **Continue after a failed update?**: When enabled, if one object fails to update (for example an ID that no longer exists), the error is logged and the sync continues with the remaining rows, instead of stopping the whole run on the first error.
* **Log request payload**: When enabled, the fields sent to the Marketing API for each updated object are written to the run logs. Useful for auditing budget and status changes.
3. Click **Next**.
## 5. Select your data to send
1. The next step is letting us know which data you want to send to the Facebook Campaign Management destination. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 6. Map your data to the destination
In this step, you map the columns from your input table to the Meta object fields. One column must be mapped to `id`; the others are the values that will be applied to each object.
### Available fields
| Field name | Description |
| :- | :- |
| `id` | **(required)** The ID of the Meta object to update: a campaign, ad set, or ad. You can find these IDs in Ads Manager or bring them into your catalog with the Facebook Ads source. |
| `status` | The delivery status to apply to the object. Accepted values are `ACTIVE` and `PAUSED`. |
| `daily_budget` | The daily budget of the object, in the account currency minor unit (cents), e.g. `3000` = R\$30,00, unless "Enter budgets in your account currency?" is enabled. Applies to campaigns using Campaign Budget Optimization or to ad sets. |
| `lifetime_budget` | The total budget for the whole lifetime of the object, with the same unit rules as `daily_budget`. The object must have a defined end date for a lifetime budget to be valid. |
The only required field is `id`. Map at least one other field (`status`, `daily_budget`, or `lifetime_budget`) so each row has something to apply. For the exact behavior of each field, see Meta's [Marketing API reference](https://developers.facebook.com/docs/marketing-api/reference/ad-campaign-group#Updating).
Once the fields are configured, click **Next**.
## 7. Configure your Facebook Campaign Management data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need the changes applied on Facebook (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. For the changes to reach Facebook, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your updated statuses and budgets should be reflected in Facebook Ads Manager.
## Related destinations
Send offline and server-side conversion events to Meta.
Build targeting audiences from your customer lists.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Facebook Conversions as a destination
Source: https://docs.nekt.com/destinations/facebook-conversions
Send offline and server-side conversion events from Nekt to Meta's Conversions API so your campaigns optimize on outcomes that actually happened.
Meta's Conversions API is a server-to-server channel for conversion events. Instead of relying on the browser Pixel, which loses events to ad blockers, cookie restrictions, and conversions that never happen on your site, you send the event straight from your data. A deal closed by a sales rep, an invoice paid in your ERP, a lead qualified in your CRM: all of it can go back to Meta and feed campaign optimization and attribution.
![]()
## 1. How it works
You already have the outcome in Nekt: a table of paid orders, won deals, or qualified leads. This destination turns each row into an event and posts it to your Pixel or Dataset in Meta Events Manager.
```
Your table in Nekt → Nekt destination → Pixel / Dataset → attribution and campaign optimization
```
Meta needs to know **who** the event belongs to. It never receives raw personal data: you send identifiers in the `user_data` object, Nekt hashes the sensitive ones with SHA-256, and Meta matches those hashes against its own users. The more identifiers you send per row, the higher your match rate.
| Signal you have | Field to send | Match quality |
| :- | :- | :- |
| Meta Lead Ads lead ID | `user_data.lead_id` | Highest, it is an exact Meta identifier |
| `_fbc` / `_fbp` cookies captured on your site | `user_data.fbc`, `user_data.fbp` | Very high |
| Email, phone | `user_data.em`, `user_data.ph` | High |
| Name, city, state, zip, country, birthdate | `user_data.fn`, `ln`, `ct`, `st`, `zp`, `country`, `db` | Moderate, use to complement the above |
| Your own customer ID | `user_data.external_id` | Only matches if the same ID reaches Meta elsewhere |
Send every identifier you have, not just one. Match rate is the single biggest driver of value from this destination, and each extra field improves it.
## 2. Before you start
Open [Meta Events Manager](https://business.facebook.com/events_manager2/) and go to the **Data sources** tab. The ID is shown under the name of the data source. See [Meta's article](https://www.facebook.com/business/help/952192354843755?id=1205376682832142) for details.
![]()
Still in Events Manager, open your data source, go to **Settings**, and in the **Set up direct integration** section click **Generate access token**.
![]()
![]()
Copy the token immediately, it is shown only once. The token must belong to a user with the `ads_management` permission on the Business account that owns the data source.
Use a [standard event name](https://www.facebook.com/business/help/402791146561655) such as `Purchase`, `Lead`, `CompleteRegistration`, or `Subscribe` when one fits. Standard events plug directly into campaign optimization and reporting. A custom name such as `QualifiedLead` also works, but you have to register it as a custom conversion in Events Manager before campaigns can optimize for it.
**Meta rejects events older than 7 days.** If a single row in a batch has an `event_time` more than 7 days in the past, Meta returns an error for the **entire batch of 100 rows**, and the Nekt run fails. Always filter your table by date, as shown in the examples below.
## 3. Prepare your input table
This is where most of the work is. Build a table in the Catalog with **one row per conversion event**, then map its columns to Conversions API fields.
### How the mapping builds the payload
The Conversions API payload is nested, so destination field names use dots. `user_data.em` means "the `em` key inside the `user_data` object". Nekt assembles the nesting for you.
A table like this:
| event\_name | event\_time | action\_source | email | order\_total | order\_id |
| :- | :- | :- | :- | :- | :- |
| `Purchase` | `1756742400` | `physical_store` | `maria.silva@example.com` | `1250.00` | `8842` |
mapped like this:
| Input table column | Conversions field |
| :- | :- |
| `event_name` | `event_name` |
| `event_time` | `event_time` |
| `action_source` | `action_source` |
| `email` | `user_data.em` |
| `order_total` | `custom_data.value` |
| `order_id` | `custom_data.order_id` |
produces this request:
```json theme={null}
{
"data": [
{
"event_name": "Purchase",
"event_time": 1756742400,
"action_source": "physical_store",
"user_data": {
"em": "5504a61f3ca7140339278c6dabcc059695c852b2aba44c4d00b7ca8c5716106a"
},
"custom_data": {
"value": 1250.0,
"order_id": "8842"
}
}
]
}
```
Note that the email left Nekt as a SHA-256 hash. Rows are sent in batches of up to 100 events per request.
### Normalize your values before they are hashed
Nekt hashes the value **exactly as it appears in your table**. It does not lowercase, trim, or reformat anything first. `Maria.Silva@Example.com` and `maria.silva@example.com` produce completely different hashes, and only the second one matches. Normalizing in your Query is therefore not optional.
| Field | Rule | Example |
| :- | :- | :- |
| `user_data.em` | Lowercase, trimmed | `Maria.Silva@Example.com ` → `maria.silva@example.com` |
| `user_data.ph` | Digits only, with country code, no `+`, no punctuation, no leading zeros | `+55 (11) 98888-7777` → `5511988887777` |
| `user_data.fn` / `ln` | Lowercase, no punctuation. Accented characters are accepted | `Silva-Souza` → `silvasouza` |
| `user_data.ge` | Single lowercase letter, `f` or `m` | `Female` → `f` |
| `user_data.db` | `YYYYMMDD` | `1991-02-14` → `19910214` |
| `user_data.ct` | Lowercase, no spaces or punctuation | `São Paulo` → `sãopaulo` |
| `user_data.st` | Lowercase two-letter code | `SP` → `sp` |
| `user_data.zp` | Lowercase, no spaces or dashes | `01310-100` → `01310100` |
| `user_data.country` | Lowercase ISO 3166-1 alpha-2 code | `Brazil` → `br` |
Do not pre-hash these columns while **"Would you like Nekt to automatically hash fields required by Facebook?"** is enabled. The value would be hashed a second time and could never match. If your table already stores hashed values, disable that option, and then every field in the list above must arrive already hashed.
### Example 1: offline purchases from your ERP
The most common case. Orders paid in a store or by bank transfer, sent back so Meta can attribute them to the ads that generated the visit.
```sql theme={null}
SELECT
'Purchase' AS event_name,
unix_timestamp(o.paid_at) AS event_time,
'physical_store' AS action_source,
concat('erp-order-', o.order_id) AS event_id,
-- Identifiers. Normalized here, hashed by Nekt before sending.
lower(trim(c.email)) AS em,
CASE
WHEN length(regexp_replace(c.phone, '[^0-9]', '')) BETWEEN 10 AND 11
THEN concat('55', regexp_replace(c.phone, '[^0-9]', ''))
ELSE regexp_replace(c.phone, '[^0-9]', '')
END AS ph,
lower(trim(c.first_name)) AS fn,
lower(trim(c.last_name)) AS ln,
regexp_replace(lower(trim(c.city)), '[^\\p{L}]', '') AS ct,
lower(c.state) AS st,
lower(regexp_replace(c.zip_code, '[^0-9a-zA-Z]', '')) AS zp,
'br' AS country,
cast(c.customer_id AS string) AS external_id,
-- Event value
o.total_amount AS value,
'BRL' AS currency,
cast(o.order_id AS string) AS order_id
FROM silver.erp_orders o
JOIN silver.erp_customers c ON c.customer_id = o.customer_id
WHERE o.status = 'paid'
AND o.paid_at >= current_timestamp() - INTERVAL 6 DAYS
```
Mapping:
| Column | Conversions field |
| :- | :- |
| `event_name`, `event_time`, `action_source`, `event_id` | same name |
| `em`, `ph`, `fn`, `ln`, `ct`, `st`, `zp`, `country`, `external_id` | `user_data.` |
| `value`, `currency`, `order_id` | `custom_data.` |
The `INTERVAL 6 DAYS` filter is deliberate. The limit is 7 days, and a run that starts near the boundary would otherwise pick up rows that expire mid-run and fail the whole batch. Keeping a day of margin makes the pipeline stable.
### Example 2: qualified leads from Meta Lead Ads
If your leads come from Instant Forms, your CRM has the Meta lead ID. Sending it back when the lead becomes qualified is the most accurate event you can produce, and it is what powers the **Conversion Leads** optimization goal.
```sql theme={null}
SELECT
'QualifiedLead' AS event_name,
unix_timestamp(d.qualified_at) AS event_time,
'system_generated' AS action_source,
concat('crm-deal-', d.deal_id) AS event_id,
cast(l.lead_id AS string) AS lead_id,
lower(trim(l.email)) AS em,
d.expected_revenue AS value,
'BRL' AS currency
FROM silver.crm_deals d
JOIN silver.facebook_leads l ON l.id = d.facebook_lead_id
WHERE d.stage = 'qualified'
AND d.qualified_at >= current_timestamp() - INTERVAL 6 DAYS
```
Map `lead_id` to `user_data.lead_id`, `em` to `user_data.em`, and `value` / `currency` to `custom_data.value` / `custom_data.currency`.
Bring the leads into the Catalog with the [Facebook Leads source](/sources/facebook-leads), and join on the lead ID your CRM stored when the form was submitted. Send the event to the same Pixel or Dataset that the lead form is attached to.
### Example 3: web purchases deduplicated with the Pixel
If the Pixel already fires `Purchase` on your site, you can send the same event server-side to recover what the browser lost, as long as both carry the same `event_id` and `event_name`. Meta keeps the first copy it receives and discards the duplicate, within a 48-hour window.
```sql theme={null}
SELECT
'Purchase' AS event_name,
unix_timestamp(o.created_at) AS event_time,
'website' AS action_source,
o.checkout_id AS event_id, -- same value the Pixel sent as eventID
o.landing_url AS event_source_url,
lower(trim(o.customer_email)) AS em,
o.fbc_cookie AS fbc, -- captured from _fbc, not hashed
o.fbp_cookie AS fbp, -- captured from _fbp, not hashed
o.client_ip AS client_ip_address,
o.user_agent AS client_user_agent,
o.total_amount AS value,
'BRL' AS currency
FROM silver.ecommerce_orders o
WHERE o.created_at >= current_timestamp() - INTERVAL 6 DAYS
```
Deduplication only works when the `event_id` is **the same string** the browser sent as `eventID`. If your site does not emit a stable ID, do not send the same event from both sides, or your conversions will be double counted.
### Sending a list of products
`custom_data.contents` expects an array of objects. Build it in your Query with `collect_list` and `named_struct`, and map the resulting column to `custom_data.contents`.
```sql theme={null}
SELECT
o.order_id,
collect_list(named_struct(
'id', cast(i.sku AS string),
'quantity', i.quantity,
'item_price', i.unit_price
)) AS contents,
sum(i.quantity) AS num_items
FROM silver.erp_order_items i
JOIN silver.erp_orders o ON o.order_id = i.order_id
GROUP BY o.order_id
```
The same applies to `user_data.em` and `user_data.ph`: if a customer has more than one email or phone, an array column is accepted and every item is hashed individually.
## 4. Configure the destination
Go to [Destinations](https://app.nekt.ai/destinations), click **Add destination**, and select **Facebook Conversions**.
* **Pixel ID or Dataset ID**: the ID from Events Manager.
* **Access token**: the token you generated.
* **Would you like Nekt to automatically hash fields required by Facebook?**: leave enabled unless your table already stores hashed values. It covers `user_data.em`, `ph`, `fn`, `ln`, `ge`, `db`, `ct`, `st`, `zp`, and `country`.
* **Log request payload**: writes the full payload of every batch to the run logs. Useful while validating a new pipeline, but leave it off in steady state.
Choose the layer and the table you prepared, then click **Next**.
Map each column to a Conversions field, using the reference below. `event_name` and `event_time` are required by Nekt, and `action_source` plus at least one `user_data` identifier are required by Meta.
Describe the destination, set a [trigger](https://docs.nekt.com/runs/scheduling-and-triggers), and click **Done**. Daily is a good default. Optimization does not need events in real time, but do not let the schedule drift close to the 7-day limit.
## 5. Field reference
### Event fields
| Field | Required | Description |
| :- | :- | :- |
| `event_name` | Yes | Standard event (`Purchase`, `Lead`, `CompleteRegistration`, …) or your own custom event name. |
| `event_time` | Yes | Unix timestamp in **seconds**, as an integer. Must be within the last 7 days. |
| `action_source` | Yes, by Meta | Where the conversion happened. One of `website`, `email`, `app`, `phone_call`, `chat`, `physical_store`, `system_generated`, `business_messaging`, `other`. |
| `event_id` | No | Your deduplication key, sent as text. Required if the same event also comes from the Pixel. |
| `event_source_url` | No | URL where the event happened. Required by Meta for `website` events, and must be on a verified domain. |
| `referrer_url` | No | The referring URL of the page. |
| `opt_out` | No | `true` uses the event for attribution only, never for optimization. |
| `data_processing_options` | No | Array of processing options, for example `array('LDU')` for Limited Data Use. Send an empty array for no restriction. |
| `data_processing_options_country` | No | Country code for Limited Data Use, as an integer. `0` lets Meta infer it. |
| `data_processing_options_state` | No | State code for Limited Data Use, as an integer. `0` lets Meta infer it. |
| `advanced_measurement_table` | No | Used only in advanced measurement setups agreed with Meta. |
### Customer identifiers (`user_data`)
At least one identifier is required per row. Everything in the first table is hashed by Nekt before leaving your environment.
**Hashed automatically**
| Field | Description |
| :- | :- |
| `user_data.em` | Email address. |
| `user_data.ph` | Phone number with country code. |
| `user_data.fn` / `user_data.ln` | First and last name. |
| `user_data.ge` | Gender, `f` or `m`. |
| `user_data.db` | Date of birth, `YYYYMMDD`. |
| `user_data.ct` / `user_data.st` / `user_data.zp` / `user_data.country` | City, state, zip, and country. |
**Sent as-is**
| Field | Description |
| :- | :- |
| `user_data.lead_id` | Meta Lead Ads lead ID, sent as text. |
| `user_data.fbc` | Click ID from the `_fbc` cookie, format `fb.1..`. |
| `user_data.fbp` | Browser ID from the `_fbp` cookie. |
| `user_data.external_id` | Your own identifier for the person, such as a customer or loyalty ID. Use the same value everywhere you send it to Meta. |
| `user_data.client_ip_address` | IPv4 or IPv6 address of the user. |
| `user_data.client_user_agent` | Browser user agent string. Meta requires it, together with the IP, for `website` events. |
| `user_data.subscription_id` | Subscription identifier associated with the event. |
| `user_data.fb_login_id` | App-scoped ID issued by Meta on Facebook Login. |
| `user_data.madid` | Mobile advertiser ID (IDFA or AAID). |
| `user_data.anon_id` | App installation ID, for app events. |
### Event details (`custom_data`)
| Field | Description |
| :- | :- |
| `custom_data.value` | Monetary value of the event, as a number. Required for value-based optimization. |
| `custom_data.currency` | ISO 4217 code, such as `BRL` or `USD`. |
| `custom_data.order_id` | Your order or transaction ID. |
| `custom_data.content_ids` | Array of product IDs. |
| `custom_data.contents` | Array of objects with `id`, `quantity`, `item_price`, and optionally `delivery_category`. |
| `custom_data.content_name` / `content_category` / `content_type` | Product name, category, and type (`product` or `product_group`). |
| `custom_data.num_items` | Number of items in the event. |
| `custom_data.predicted_ltv` | Predicted lifetime value of the customer. |
| `custom_data.search_string` | Search term, for `Search` events. |
| `custom_data.status` | Registration status, for `CompleteRegistration` events. |
| `custom_data.delivery_category` | `in_store`, `curbside`, or `home_delivery`. |
| `custom_data.event_source` / `custom_data.lead_event_source` | Origin of the event, for example the name of your CRM. Recommended for CRM lead events. |
### App events (`app_data`)
Only relevant when `action_source` is `app`: `app_data.advertiser_tracking_enabled`, `app_data.extinfo`, `app_data.install_referrer`, `app_data.installer_package`, `app_data.url_schemes`, and `app_data.windows_attribution_id`. See Meta's [app events reference](https://developers.facebook.com/docs/marketing-api/conversions-api/app-events).
The field list also shows `app_data.application_traccking_enabled`, whose name is misspelled and which Meta therefore ignores. Use `app_data.advertiser_tracking_enabled`.
### Correcting a previous event (`original_event_data`)
Used when the event you are sending refers to an event already reported, for example a refund pointing back to the original purchase: `original_event_data.event_name`, `original_event_data.event_time`, `original_event_data.event_id`, and `original_event_data.order_id`.
For the complete definition of every parameter, see Meta's [Conversions API parameters reference](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters).
## 6. After the run
A successful run means Meta **accepted** your events. Processing and attribution happen afterwards.
* Open [Events Manager](https://business.facebook.com/events_manager2/), select your data source, and check the **Overview** tab. Events usually appear within minutes, and can take up to 20 minutes.
* The **Event Match Quality** score, per event, tells you how well your identifiers are matching. Below "Good", add more `user_data` fields or double-check your normalization.
* The **Diagnostics** tab reports issues Meta found in your payloads, including rows dropped after the request was accepted.
For the first run, add a test event code in Events Manager under **Test events** and validate with a handful of rows before pointing the destination at the full table.
## 7. Troubleshooting
The connector stops the run as soon as Meta rejects a batch, and the log contains Meta's raw response. The `message` and `error_user_title` fields in that response name the problem. Enable **Log request payload** and run again to see exactly what was sent.
One or more rows have an `event_time` older than 7 days. Meta rejects the whole batch of 100 rows, not just the offending one. Add a date filter to your Query, keeping a day of margin: `WHERE event_date >= current_timestamp() - INTERVAL 6 DAYS`.
It must be a Unix timestamp in seconds, as an integer. A timestamp column or a string date is rejected. Use `unix_timestamp(your_column)`. If your source stores milliseconds, divide by 1000.
Meta accepted the events but matched few of them to people. The usual causes are values that were not normalized before hashing (an uppercase email, a phone without the country code), only one identifier per row, or pre-hashed values being hashed a second time. Compare a known customer's hash against the value Meta expects, and add more identifiers.
A custom `event_name` has to be registered as a custom conversion in Events Manager and selected in the campaign before it can drive optimization. Standard event names avoid this step. Also confirm the events landed on the Pixel or Dataset the campaign is actually using.
The Pixel and this destination are both reporting the same event without a shared `event_id`, or with an `event_id` that changes between the two. Deduplication requires the same `event_id` **and** the same `event_name` on both sides, and only applies within 48 hours.
The token expired, was revoked, or belongs to a user who lost access to the data source. Generate a new one in Events Manager and update the destination configuration.
## Related destinations
Build targeting audiences from your customer lists.
Pause, activate, and rebudget campaigns from a table.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Facebook Custom Audiences as a destination
Source: https://docs.nekt.com/destinations/facebook-custom-audiences
Turn a table of customers in Nekt into a Meta Custom Audience you can target, exclude, or use as a lookalike seed.
A Meta Custom Audience is a list of people built from your own data. Once the list exists in Ads Manager you can target it, exclude it from prospecting campaigns, or use it as the seed for a Lookalike Audience. This destination keeps that list in sync with a table in your Catalog, so the audience always reflects the segment your data defines instead of a CSV someone exported last quarter.
![]()
## 1. How it works
Each row of your input table is one person. Nekt hashes the personal fields with SHA-256, uploads them in batches of 100, and Meta matches those hashes against its users.
```
Your table in Nekt → Nekt destination → Custom Audience in Ads Manager
```
A few behaviors worth knowing before you build the table:
* **Each run adds the people in your table to the audience.** People already in the audience stay there, and uploading the same person again does not create a duplicate.
* **The connector does not remove people.** If your segment is meant to shrink, for example a "trial users who have not converted" audience, either create a fresh audience or manage removals in Ads Manager.
* **Meta needs at least 100 matched people** before an audience can be used for targeting. Small segments will be created but stay unusable.
* **Matching is not instant.** After a successful run the audience shows as "Populating" in Ads Manager and takes from a few minutes to a few hours to be ready.
Send more than just the email. Each additional identifier gives Meta another chance to match the person, and match rate is what decides whether the audience is large enough to target.
## 2. Prepare your input table
### The table must contain exactly the columns you map
Every column of the input table is announced to Meta as part of the audience schema. A leftover `created_at` or `customer_id` column that you do not map causes the upload to be rejected. Build a Query whose `SELECT` lists only the fields you are going to send.
### Normalize your values before they are hashed
Nekt hashes the value **exactly as it appears in your table**, with no lowercasing, trimming, or reformatting. `Maria.Silva@Example.com` and `maria.silva@example.com` produce different hashes, and only the second one matches anyone.
| Field | Meaning | Rule | Example |
| :- | :- | :- | :- |
| `EMAIL` | Email address | Lowercase, trimmed | `Maria.Silva@Example.com ` → `maria.silva@example.com` |
| `PHONE` | Phone number | Digits only, with country code, no `+` or punctuation | `+55 (11) 98888-7777` → `5511988887777` |
| `FN` / `LN` | First and last name | Lowercase, no punctuation | `Silva-Souza` → `silvasouza` |
| `FI` | First name initial | Single lowercase letter | `Maria` → `m` |
| `GEN` | Gender | `f` or `m`, lowercase | `Female` → `f` |
| `DOBY` | Birth year | `YYYY` | `1991` |
| `DOBM` | Birth month | `MM`, zero padded | `02` |
| `DOBD` | Birth day | `DD`, zero padded | `14` |
| `CT` | City | Lowercase, no spaces or punctuation | `São Paulo` → `sãopaulo` |
| `ST` | State | Lowercase two-letter code | `SP` → `sp` |
| `ZIP` | Postal code | Lowercase, no spaces or dashes | `01310-100` → `01310100` |
| `COUNTRY` | Country | Lowercase ISO 3166-1 alpha-2 code | `Brazil` → `br` |
Do not pre-hash these columns while **"Would you like Nekt to automatically hash fields required by Facebook?"** is enabled, or the values are hashed twice and match nobody. If your table already stores hashed values, disable the option, and then every field above must arrive already hashed.
### Example: high-value customers from your CRM
```sql theme={null}
SELECT
lower(trim(c.email)) AS email,
CASE
WHEN length(regexp_replace(c.phone, '[^0-9]', '')) BETWEEN 10 AND 11
THEN concat('55', regexp_replace(c.phone, '[^0-9]', ''))
ELSE regexp_replace(c.phone, '[^0-9]', '')
END AS phone,
lower(trim(c.first_name)) AS fn,
lower(trim(c.last_name)) AS ln,
regexp_replace(lower(trim(c.city)), '[^\\p{L}]', '') AS ct,
lower(c.state) AS st,
lower(regexp_replace(c.zip_code, '[^0-9a-zA-Z]', '')) AS zip,
'br' AS country,
date_format(c.birth_date, 'yyyy') AS doby,
date_format(c.birth_date, 'MM') AS dobm,
date_format(c.birth_date, 'dd') AS dobd
FROM silver.crm_customers c
WHERE c.lifetime_value >= 5000
AND c.email IS NOT NULL
```
The resulting table:
| email | phone | fn | ln | ct | st | zip | country | doby | dobm | dobd |
| :- | :- | :- | :- | :- | :- | :- | :- | :- | :- | :- |
| `maria.silva@example.com` | `5511988887777` | `maria` | `silva` | `sãopaulo` | `sp` | `01310100` | `br` | `1991` | `02` | `14` |
Mapping is one to one: `email` → `EMAIL`, `phone` → `PHONE`, `fn` → `FN`, and so on.
A person is matched if **any** identifier matches, so rows with missing fields are still useful. Do not drop a customer just because you lack their address.
## 3. Add your Facebook Custom Audiences access
Go to [Destinations](https://app.nekt.ai/destinations), click **Add destination**, and select **Facebook Custom Audiences**.
* **Facebook authorization**: authorize an account with access to the Facebook Ad Account you want to send data to.
* **Account ID**: the specific Facebook Ad Account that will own the audience.
* **Create new audience**: fill in **Audience name** and **Audience description**. Nekt creates the audience on the first run and reuses it on every run after that.
* **Use existing audience**: fill in **Audience ID**, found in [Ads Manager → Audiences](https://adsmanager.facebook.com/adsmanager/audiences).
**Would you like Nekt to automatically hash fields required by Facebook?** should stay enabled unless your table already stores hashed values.
Choose the layer and the table you prepared, then map each column to its audience field.
Describe the destination, set a [trigger](https://docs.nekt.com/runs/scheduling-and-triggers), and click **Done**. Daily is a good default for an audience fed by a segment that changes slowly.
Accept Meta's [Custom Audiences terms](https://business.facebook.com/ads/manage/customaudiences/tos/) in the Business account before the first run. Uploads from an account that has not accepted them are rejected.
## 4. Available fields
| Field | Description |
| :- | :- |
| `EMAIL` | Email address. The strongest single identifier. |
| `PHONE` | Phone number including the country code. |
| `FN` / `LN` | First and last name. |
| `FI` | First name initial. |
| `GEN` | Gender, `f` or `m`. |
| `DOBY` / `DOBM` / `DOBD` | Birth year, month, and day, as separate fields. |
| `CT` / `ST` / `ZIP` / `COUNTRY` | City, state, postal code, and country. |
All of them are hashed by Nekt when automatic hashing is enabled. See Meta's [customer list reference](https://developers.facebook.com/docs/marketing-api/audiences/guides/custom-audiences#hash) for the full definition of each field.
## 5. After the run
1. Open [Ads Manager → Audiences](https://adsmanager.facebook.com/adsmanager/audiences) and find your audience by name.
2. **Availability** shows `Populating` right after the run, then `Ready` once Meta finishes matching.
3. **Size** is the number of matched people, not the number of rows you uploaded. A large gap between the two means your identifiers are not matching, usually because values were not normalized before hashing.
## 6. Troubleshooting
Your input table has columns that were not mapped. Rewrite the Query so the `SELECT` returns only the columns you are sending.
Meta is still processing a previous upload of the same audience. Wait for the availability to go back to `Normal` in Ads Manager and run again. Overlapping schedules on the same audience are the usual cause.
Meta matched few of the rows. Check that values were normalized before hashing, that phone numbers carry the country code, and that automatic hashing is not being applied on top of already-hashed values. Adding more identifiers per person also helps.
Meta requires at least 100 matched people. Widen the segment that feeds the table, or improve the match rate by sending more identifiers.
Runs only add people. To shrink an audience, remove them in Ads Manager or point the destination at a newly created audience.
## Related destinations
Send offline and server-side conversion events to Meta.
Pause, activate, and rebudget campaigns from a table.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Gmail as a destination
Source: https://docs.nekt.com/destinations/gmail
Send your data from Nekt to Gmail.
Gmail is Google's email service that provides secure, reliable email communication for personal and business use. It offers features like spam filtering, search capabilities, and integration with other Google services, making it a popular choice for email management and communication.
![]()
## Configuring Gmail as a Destination
In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Gmail option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to configure your Gmail destination settings:
* **Gmail Authorization**: Authorize nekt with your Gmail account. Make sure you are doing it with the email address you want to use to send the email.
* **Recipients**: The list of email addresses that should receive the email. It should be a comma-separated list such as `john.doe@example.com, jane.doe@gmail.com` and so on.
* **Email subject**: The subject of the email. If you want to add a datestamp to the subject, please include the parameter `{datestamp}` in brackets. With the **Single record input table** option enabled, you can also reference the record's columns as `{column_name}`.
* **Email body**: The email body message to send. There are variables you can include in the email body that will be replaced by dynamic information during execution:
* **Data preview**: This will include a preview of your data (max of 100 records) in the email body in a tabular form. This is useful to give the recipients a quick glimpse on the data. In order to use data, include the variable `{data_preview}` in your email body.
* **Datestamp**: This will include the current date timestamp in the format YYYY-MM-DD in the email body. Useful if you want to add a date reference to the email body. To use that include the variable `{datestamp}` in your email body.
* **Column reference**: With the **Single record input table** option enabled, you can reference the record's columns as `{column_name}` in your email body.
* **Maximum preview size**: The total number of rows to preview in the message body. Maximum is 100.
* **Send email with empty data**: Whether the email should be sent in case the input table is empty (due to filtering or other conditions). If this option is True, then you have the option to specify a different email body to be used in such cases.
* **Single record input table**: (Advanced) In case your input table has a single record, you can reference its columns in the subject, the body message and the CSV filename using this notation `{column_name}`.
**CSV Attachment Options:**
* **Attach data as CSV**: When enabled, the full dataset will be attached to the email as a CSV file. This is useful when you need to share the complete data with recipients, not just a preview.
* **CSV filename**: The filename for the CSV attachment. You can use the `{datestamp}` placeholder to include the current date in the filename. For example: `report_{datestamp}.csv` would generate `report_2024-01-15.csv`. With the **Single record input table** option enabled, you can also reference the record's columns as `{column_name}`.
When CSV attachment is enabled, all records from your data source will be included in the attachment, regardless of the preview size setting. The data preview in the email body and the CSV attachment work independently.
**Example Email:**
This is an example of how the email would look like when using `{datestamp}` in the subject and `{data_preview}` in the email body:
![]()
Once you're done, click **Next**.
### 2. Select your data to send
The next step is letting us know which data you want to attach to the email. Select the layer and then the desired table.
The table can be found more easily by typing its name.
Click **Next**.
### 3. Configure data destination
Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want this email to be sent to the list of recipients. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
Click **Done**.
### 4. Check your new destination
Once completed, you'll receive confirmation that your new destination is set up!
You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for the email to be actually sent, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your email should be sent.
If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Implementation Notes
### Email Delivery
* The connector uses Gmail's SMTP service to send emails
* Emails with attachments use multipart MIME formatting to properly handle both text content and file attachments
* All email content is properly encoded to ensure compatibility across different email clients
### Data Handling
* The data preview in the email body is limited to the configured maximum preview size
* CSV attachments contain the complete dataset regardless of preview size settings
* Dynamic variables (`{datestamp}`, `{data_preview}`) are processed at runtime during email generation
* System variables (`{datestamp}`, `{data_preview}`) take precedence over column names if there is a naming collision
* When the **Single record input table** option is enabled:
* The pipeline will fail and stop execution if the input table contains more than one record
* If the input table is empty, any `{column_name}` reference is left unresolved and sent exactly as written
* Line breaks in column values are collapsed into spaces to ensure email headers (like the Subject) remain valid and serialize correctly
* Any slashes (`/` or `\`) in column values used for the CSV filename are converted to dashes (`-`) to prevent directory path errors
### Authentication & Security
* OAuth2 authentication ensures secure access to Gmail services
* Email credentials and recipient lists are stored securely
* All data transmission uses encrypted connections
# Google Ads Call Conversions as a destination
Source: https://docs.nekt.com/destinations/google-ads-call-conversions
Send your data from Nekt to Google Ads Call Conversions.
Google Ads Call Conversions is Google's platform for tracking and measuring phone call conversions that result from advertising campaigns. It helps businesses attribute phone calls to specific ads and campaigns, enabling better optimization of advertising spend and campaign performance.
## 1. Overview
You can use the Google Ads Call Conversions connector to import offline call conversions into Google Ads in order to track when ads leads to phone calls and when those phone calls lead to valuable customer actions.
## 2. Important considerations
It should take up to 3 hours for imported conversion statistics to appear in your Google Ads account.
## 3. Configuring the destination
1. For configuring this destination, go to the [Destinations](https://app.nekt.ai/destinations) tab and click on the "Add destination" button located on the top right of your screen. Then, select the Google Ads - Call Conversions option from the list of connectors.
2. Click **Next** and you'll be prompted to add the connector configuration:
* **Authorization:** you should authenticate this connector with a Google Account with enough permissions to access the Google Ads account to which you want to send the Call Conversions data.
* **Customer ID**: the Google Ads customer ID from the account you want to extract data from. You can obtain this information from the top-right corner of your Google Ads account. For more info, please check [this article](https://support.google.com/google-ads/answer/1704344?hl=en).
* **Use fixed conversion action**: if you're using the same conversion action for all conversion entries in the input table, set this config to `True`. If you have different conversion actions for each entry, then you should set this information in the column mapping part.
* **Conversion action name**: in case you're using a fixed conversion action, you should specify its name on this config and the connector will automatically assign this conversion action for each entry in the input table. Please note the conversion action must have a `type` of `UPLOAD_CALLS`, and must exist in your Google Ads account.
* **Ad user data consent**: specify the consent of the uploaded users for sending user data to Google for advertising purposes.
> Important: if you are uploading data for users in the European Economic Area (EEA), you need to ensure you are passing the required consent signals to Google on the Ad User Consent field.
3. Click **Next**.
## 4. Select your data to send
1. The next step is letting us know which data you want to send to the Google Ads - Call Conversions destination. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 5. Map your data to the destination
In this step, you should map the fields from the input table to the destination. It's important that your table contains the structured data needed to sync with Google Ads - Call Conversions. This is the list of fields available for syncing with Google Ads.
### Available fields
| Field name | Description |
| :- | :- |
| `caller_id` | The caller id from which the call was placed. Caller id is expected to be in E.164 format with preceding '+' sign, for example, `+16502531234`. |
| `call_start_date_time` | The date time at which the call occurred. The timezone must be specified. The format is `yyyy-mm-dd hh:mm:ss+-hh:mm`, for example, `2019-01-01 12:32:45-08:00`. |
| `conversion_date_time` | The date and time of the conversion. The value must have a timezone specified, and the format must be `yyyy-mm-dd HH:mm:ss+-hh:mm`, for example: `2022-01-01 19:32:45-05:00`. |
| `conversion_action_name` | The resource name of the conversion action created in Google Ads for the conversion. The conversion action must have a `type` of `UPLOAD_CLICKS`, and must exist in your Google Ads account. |
| `conversion_value` | The value of the conversion. It must be a `double` type. |
| `currency_code` | The currency code of the conversion value. Accepted codes are available [here](https://developers.google.com/google-ads/api/data/codes-formats#currency_codes) |
| `ad_user_data_consent` | Whether the user has given consent for sending user data to Google for advertising purposes. The accepted values are: `UNSPECIFIED`, `UNKNOWN`, `GRANTED`, `DENIED`. |
The required fields are:
* `caller_id`
* `call_start_date_time`
* `conversion_date_time`
`conversion_action_name` is required if you didn't set a fixed conversion action in the configs.
### Custom fields
Besides the pre-defined fields available from Google Ads, you can also send custom fields associated with your conversions. You can map custom variables when syncing conversion events to Google Ads. Please check [this article](https://support.google.com/google-ads/answer/9962082) if you need info on setting up custom conversion variables in Google Ads.
Once the fields are configured, you can move forward by clicking the 'Next' button.
## 6. Configure your Google Ads Call Conversions data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers),consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Your data appears in Google Ads only after the pipeline runs. You can monitor execution and completion on the [Destinations](https://app.nekt.ai/destinations) page, and trigger a run manually with the refresh icon.
> Important: once the destination runs successfully, it can take as little as 10 minutes or up to 24 hours for the upload job to complete. You should check the status directly on Google Ads portal what's the status of the job.
# Google Ads Click Conversions as a destination
Source: https://docs.nekt.com/destinations/google-ads-click-conversions
Send offline conversions from Nekt to Google Ads so your campaigns can optimize on what actually closed.
Google Ads Click Conversions lets you import conversions that happened outside your website, such as a deal closed by a sales rep, a subscription activated in your billing system, or an opportunity marked as won in your CRM. Google attributes each conversion back to the ad click that produced it, so Smart Bidding optimizes for revenue you actually earned instead of form fills.
## 1. How it works
You already have the outcome data in Nekt: a table of closed deals, paid invoices, or qualified leads. This destination sends each row to Google Ads as a conversion, matched to the original click.
```
Your table in Nekt → Nekt destination → Google Ads conversion action → campaign optimization
```
Matching happens in one of two ways:
* **By click ID.** Your table carries the `gclid` (or `gbraid`/`wbraid`) captured when the user first landed on your site. This is the most accurate method.
* **By customer data (enhanced conversions).** Your table carries the customer's email or phone number. Google matches it against signed-in users. Use this when you never captured a click ID.
If you can capture the GCLID at lead creation and carry it through to your CRM, do that. Click ID matching is more accurate than customer data matching and is not affected by signed-out traffic.
## 2. Before you start
In Google Ads, go to **Goals → Conversions → Summary**, click **New conversion action**, and choose **Import → CRM, files, or other data sources → Track conversions from clicks**.
Note the exact name you give it. You will need it later, and it must match character for character. The conversion action must have a type of `UPLOAD_CLICKS`; other types are rejected.
The Google account you authorize must have Standard or Admin access to the Google Ads account you are sending to. Read-only access is not enough to import conversions.
Google only accepts conversions whose click happened within the conversion window of the action (90 days by default, up to 540). Conversions older than the window are rejected during processing, so if you are backfilling historical data, check this first.
**Reconnect your Google authorization if it was created before September 2026.** Google moved conversion imports to its Data Manager API, which needs a permission that older Nekt connections do not have. If your runs fail with a message asking you to reconnect the Google account, open the destination settings and re-authorize. Existing connections keep working for every other Google Ads destination, so this is easy to miss until a run fails.
## 3. Prepare your data
Build a table in the Catalog with one row per conversion. A typical table looks like this:
| gclid | conversion\_action\_name | conversion\_date\_time | conversion\_value | currency\_code | order\_id |
| :- | :- | :- | :- | :- | :- |
| `Cj0KCQjw...` | `HubSpot - Opportunity` | `2026-08-30 15:30:00+00:00` | `1250.00` | `BRL` | `DEAL-8842` |
| `Cj0KCQjw...` | `HubSpot - Opportunity` | `2026-08-31 09:12:00+00:00` | `430.50` | `BRL` | `DEAL-8843` |
A Query that shapes CRM deals into that form:
```sql theme={null}
SELECT
d.google_click_id AS gclid,
'HubSpot - Opportunity' AS conversion_action_name,
DATE_FORMAT(d.closed_at, 'yyyy-MM-dd HH:mm:ssXXX') AS conversion_date_time,
d.amount AS conversion_value,
'BRL' AS currency_code,
d.deal_id AS order_id
FROM silver.crm_deals d
WHERE d.stage = 'closed_won'
AND d.google_click_id IS NOT NULL
AND d.closed_at >= CURRENT_DATE - INTERVAL 60 DAYS
```
The `XXX` in the timestamp format produces the `+00:00` offset that Google requires. A timestamp without a timezone offset is rejected.
## 4. Configure the destination
Go to [Destinations](https://app.nekt.ai/destinations), click **Add destination**, and select **Google Ads - Click Conversions**.
* **Authorization**: authorize a Google account with access to the target Google Ads account.
* **Account ID**: your Google Ads customer ID, shown in the top-right corner of Google Ads. Hyphens are optional, `793-830-3574` and `7938303574` both work. See [Google's article](https://support.google.com/google-ads/answer/1704344?hl=en).
* **Manager Account ID**: only if you authorized through a manager (MCC) account. Leave blank otherwise.
* **Use fixed conversion action**: set to `True` when every row goes to the same conversion action. Set to `False` when each row names its own.
* **Conversion action name**: the exact name of the conversion action, used only when the option above is `True`.
* **Ad user data consent** and **Ad personalization consent**: the consent signals for the users in this upload.
Choose the layer and table you prepared. Click **Next**.
Map each column in your table to a Google Ads field, using the reference below.
Describe the destination, set a [trigger](https://docs.nekt.com/runs/scheduling-and-triggers), and click **Done**. Daily is a good default; conversions do not need to arrive in real time for Smart Bidding to use them.
If you upload data for users in the European Economic Area, you must pass consent signals in **Ad user data consent** and **Ad personalization consent**, or per row via the mapped fields. Google discards EEA conversions without consent.
## 5. Field reference
### Identifiers (at least one required)
The connector picks the **first** identifier present, in this exact order:
| Priority | Field | Description |
| :- | :- | :- |
| 1 | `gclid` | Google Click ID, captured from the `gclid` URL parameter when the user landed on your site. |
| 2 | `gbraid` | Click identifier for app conversions from iOS 14+ devices. |
| 3 | `wbraid` | Click identifier for web conversions from iOS 14+ devices. |
| 4 | `email` / `phone_number` | Customer data for enhanced conversions, used only when no click ID is present. |
Because the first match wins, mapping both `gclid` and `email` does **not** send both. The `gclid` is used and the email is ignored. To get enhanced conversion matching for rows without a click ID, leave `gclid` empty on those rows.
### Conversion details
| Field | Required | Description |
| :- | :- | :- |
| `conversion_date_time` | Yes | When the conversion happened. Format `yyyy-mm-dd HH:mm:ss+hh:mm`, timezone required. Example: `2026-08-30 15:30:00-03:00`. |
| `conversion_action_name` | Yes, unless **Use fixed conversion action** is `True` | Name of the conversion action in Google Ads. Must exist and be of type `UPLOAD_CLICKS`. |
| `conversion_value` | No | The value of the conversion, as a number. |
| `currency_code` | No | ISO 4217 code, exactly three uppercase letters, such as `BRL` or `USD`. See [the list](https://developers.google.com/google-ads/api/data/codes-formats#currency_codes). |
| `order_id` | No | Your transaction ID. Strongly recommended: it is the key you need later to adjust or retract a conversion with the Conversion Adjustments destination. |
| `ad_user_data_consent` | No | `GRANTED` or `DENIED`. |
| `ad_personalization_consent` | No | `GRANTED` or `DENIED`. |
Consent accepts `UNSPECIFIED` and `UNKNOWN` as well, but those are treated as "not stated" and are not sent to Google. Only `GRANTED` and `DENIED` carry a signal.
### Personal data is hashed for you
Map `email` and `phone_number` as **plain text**. The connector normalizes and SHA-256 hashes them before anything leaves Nekt, exactly as Google requires. Phone numbers are converted to E.164 format automatically.
Do not pre-hash these columns. A pre-hashed value gets hashed a second time and will never match.
### Fields no longer supported
`conversion_environment`, `external_attribution_credit`, and `external_attribution_model` still appear in the column mapping, but Google's Data Manager API does not accept them. If you map them, the run logs a warning listing which columns were ignored and uploads everything else normally.
### Custom fields
You can send custom variables alongside each conversion, for example product category or payment method. Create the custom variable in Google Ads first, following [this article](https://support.google.com/google-ads/answer/9962082), then map your columns to it in the **Custom fields** section.
## 6. After the run
A successful run means Google **accepted** your conversions, not that it finished processing them. Processing is asynchronous.
* The run log reports how many records were sent and whether Google flagged any of them.
* Conversions typically appear in Google Ads within a few hours, and can take up to 24 hours.
* Confirm the numbers moved in **Goals → Conversions → Summary**, on the row for your conversion action.
If Google reports problems with individual records, the run log names the reason and the count, and the remaining records still upload. Partial problems do not fail the run.
**Do not use Goals → Conversions → Uploads to check these imports.** That page only lists conversions imported the legacy way, through file uploads or the older API. Nekt now sends Click Conversions through Google's Data Manager API, so this destination's runs never appear there, no matter how many conversions succeeded. Google shows a "New data connections can be made in Data Manager" message on that page instead. Use the Nekt run log and the Conversions Summary page.
## 7. Troubleshooting
Your authorization predates Google's Data Manager API and lacks the required permission. Open the destination settings, re-authorize the Google account, accept all requested permissions, and run again. This affects Click Conversions and Customer Match; other Google Ads destinations are unaffected.
The name must match the conversion action in Google Ads exactly, including spaces, capitalization, and punctuation. Check **Goals → Conversions → Summary**. Also confirm you are pointing at the right **Account ID**: a conversion action in one Google Ads account is invisible to another.
The row has none of `gclid`, `gbraid`, `wbraid`, `email`, or `phone_number`. Filter these rows out in your Query, since they can never be matched. Note the field is `phone_number`, not `phone`.
The timestamp is missing its timezone offset or is not in `yyyy-mm-dd HH:mm:ss+hh:mm` format. `2026-08-30 15:30:00` fails; `2026-08-30 15:30:00+00:00` works.
`currency_code` must be exactly three uppercase letters. `brl` and `R$` are rejected; `BRL` is accepted.
First confirm you are looking in the right place: **Goals → Conversions → Uploads** does not show Data Manager imports, so an empty page there means nothing. Check **Goals → Conversions → Summary** instead, and confirm you are viewing the same Google Ads account as the **Account ID** in the destination settings.
If the numbers still have not moved, the usual causes are a click outside the conversion window, a GCLID that expired or belongs to a different account, or a conversion action not set to **Include in "Conversions"**. The Nekt run log reports per-reason error counts returned by Google, which is the most direct signal.
## Related destinations
Restate or retract a conversion you already uploaded, matched by `order_id`.
Import conversions that happened over the phone.
Build targeting audiences from your customer lists.
# Google Ads Conversion Adjustments as a destination
Source: https://docs.nekt.com/destinations/google-ads-conversion-adjustments
Send your data from Nekt to Google Ads Conversion Adjustments.
Google Ads Conversion Adjustments is Google's platform for modifying and correcting conversion data in advertising campaigns. It allows businesses to adjust conversion values, remove invalid conversions, and ensure accurate campaign performance measurement for better optimization decisions.
## 1. Overview
Once a conversion has already been reported to Google Ads, you can adjust the conversion at a later point in time in the Google Ads API. In order to adjust a conversion, you must first have a conversion action set up, and you must also have recorded the conversions that you are about to adjust.
## 2. Configuring the destination
1. For configuring this destination, go to the [Destinations](https://app.nekt.ai/destinations) tab and click on the "Add destination" button located on the top right of your screen. Then, select the Google Ads Conversion Adjustments option from the list of connectors.
2. Click **Next** and you'll be prompted to add the connector configuration:
* **Authorization:** you should authenticate this connector with a Google Account with enough permissions to access the Google Ads account to which you want to send the Conversion Adjustments data.
* **Customer ID**: the Google Ads customer ID from the account you want to extract data from. You can obtain this information from the top-right corner of your Google Ads account. For more info, please check [this article](https://support.google.com/google-ads/answer/1704344?hl=en).
* **Use fixed conversion action**: if you're using the same conversion action for all conversion entries in the input table, set this config to `True`. If you have different conversion actions for each entry, then you should set this information in the column mapping part.
* **Conversion action name**: in case you're using a fixed conversion action, you should specify its name on this config and the connector will automatically assign this conversion action for each entry in the input table.
* **Adjustment type**: the type of adjustment you want to make to already reported conversions. These are the available types:
* `RETRACTION`: Negates a conversion so that its total value and count are both zero.
* `RESTATEMENT`: Changes the value of a conversion.
* `ENHANCEMENT`: Supplements an existing conversion with provided user identifiers and user agent, which can be used by Google to enhance the conversion count.
3. Click **Next**.
## 3. Select your data to send
1. The next step is letting us know which data you want to send to the Google Ads Conversion Adjustments destination. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 4. Map your data to the destination
In this step, you should map the fields from the input table to the destination. The fields to be mapped depend on the adjustment type chosen.
### Adjustment type: RETRACTION
| Field name | Description |
| :- | :- |
| `order_id` | The order ID of the conversion to be adjusted. If the conversion was reported with an order ID specified, that order ID must be used as the identifier here. |
| `gclid` | The restated conversion value. This is the value of the conversion after restatement. For example, to change the value of a conversion from 100 to 70, an adjusted value of 70 should be reported. |
| `conversion_date_time` | The date and time of the original conversion. The value must have a timezone specified, and the format must be `yyyy-mm-dd HH:mm:ss+-hh:mm`, for example: `2022-01-01 19:32:45-05:00`. |
You should send either an `order_id` or a pair `conversion_date_time` + `gclid` so the conversion to be retracted can be identified.
### Adjustment type: RESTATEMENT
| Field name | Description |
| :- | :- |
| `restatement_value` | The restated conversion value. This is the value of the conversion after restatement. For example, to change the value of a conversion from 100 to 70, an adjusted value of 70 should be reported. |
| `restatement_currency` | The currency of the restated value. If not provided, then the default currency from the conversion action is used, and if that is not set then the account currency is used. This is the ISO 4217 3-character currency code for example, USD or EUR. |
| `adjustment_date_time` | The date time at which the adjustment occurred. Must be after the conversion\_date\_time. The value must have a timezone specified, and the format must be `yyyy-mm-dd HH:mm:ss+-hh:mm`, for example: `2022-01-01 19:32:45-05:00`. |
| `order_id` | The order ID of the conversion to be adjusted. If the conversion was reported with an order ID specified, that order ID must be used as the identifier here. |
| `gclid` | The restated conversion value. This is the value of the conversion after restatement. For example, to change the value of a conversion from 100 to 70, an adjusted value of 70 should be reported. |
| `conversion_date_time` | The date and time of the original conversion. The value must have a timezone specified, and the format must be `yyyy-mm-dd HH:mm:ss+-hh:mm`, for example: `2022-01-01 19:32:45-05:00`. |
| `conversion_action_name` | The resource name of the conversion action created in Google Ads for the conversion. The conversion action must have a `type` of `UPLOAD_CLICKS`, and must exist in your Google Ads account. |
You should send either an `order_id` or a pair `conversion_date_time` + `gclid` so the conversion to be retracted can be identified. In both cases, `adjustment_date_time` is mandatory.
### Adjustment type: ENHANCEMENT
| Field name | Description |
| :- | :- |
| `restatement_value` | The restated conversion value. This is the value of the conversion after restatement. For example, to change the value of a conversion from 100 to 70, an adjusted value of 70 should be reported. |
| `restatement_currency` | The currency of the restated value. If not provided, then the default currency from the conversion action is used, and if that is not set then the account currency is used. This is the ISO 4217 3-character currency code for example, USD or EUR. |
| `adjustment_date_time` | The date time at which the adjustment occurred. Must be after the conversion\_date\_time. The value must have a timezone specified, and the format must be `yyyy-mm-dd HH:mm:ss+-hh:mm`, for example: `2022-01-01 19:32:45-05:00`. |
| `order_id` | The order ID of the conversion to be adjusted. If the conversion was reported with an order ID specified, that order ID must be used as the identifier here. |
| `gclid` | The restated conversion value. This is the value of the conversion after restatement. For example, to change the value of a conversion from 100 to 70, an adjusted value of 70 should be reported. |
| `conversion_date_time` | The date and time of the original conversion. The value must have a timezone specified, and the format must be `yyyy-mm-dd HH:mm:ss+-hh:mm`, for example: `2022-01-01 19:32:45-05:00`. |
| `conversion_action_name` | The resource name of the conversion action created in Google Ads for the conversion. The conversion action must have a `type` of `UPLOAD_CLICKS`, and must exist in your Google Ads account. |
| `user_agent` | The user agent to enhance the original conversion. This can be found in your user's HTTP request header when they convert on your web page. Example, `Mozilla/5.0 (iPhone; CPU iPhone OS 12_2 like Mac OS X)`. User agent can only be specified in enhancements with user identifiers. |
| `email` | User identifier of type `email` indicating the email address of the user. |
| `phone_number` | User identifier indicating the phone number of the user. |
| `first_name` | User identifier of type `address` indicating the first name of the user. |
| `last_name` | User identifier of type `address` indicating the last name of the user. |
| `country_code` | User identifier of type `address` indicating the country code of the user. It should match a country code provided by [this list](https://developers.google.com/google-ads/api/data/codes-formats#country_codes). |
| `postal_code` | User identifier of type `address` indicating the postal code of the user. |
You should send either an `order_id` or a pair `conversion_date_time` + `gclid` so the conversion to be enhanced can be identified. The other fields are optional, depending on which data you want to enhance on your existing conversion.
If you use an user identifier of type `address`, you should send all the following fields: `first_name`, `last_name`, `country_code` and `postal_code`.
## 5. Configure your Google Ads Conversion Adjustments data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers),consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Your data appears in Google Ads only after the pipeline runs. You can monitor execution and completion on the [Destinations](https://app.nekt.ai/destinations) page, and trigger a run manually with the refresh icon.
> Important: once the destination runs successfully, it can take as little as 10 minutes or up to 24 hours for the upload job to complete. You should check the status directly on Google Ads portal what's the status of the job.
# Google Ads Customer Match as a destination
Source: https://docs.nekt.com/destinations/google-ads-customer-match
Send your data from Nekt to Google Ads Customer Match.
Google Ads Customer Match is Google's platform for creating targeted advertising audiences based on your customer data. It allows businesses to upload customer lists and create custom audiences for retargeting campaigns, enabling more precise audience targeting and improved ad performance.
## 1. Overview
Imagine you have customer data coming from other sources such as CRMs, sales and financials systems and others. You want to be able to feed Google Ads with these customers' information, so ads are targeted to a very specific audience. Customer Match lets you use your online and offline data to reach and re-engage with your customers across Search, the Shopping tab, Gmail, YouTube, and Display. Using information that your customers have shared with you, Customer Match targets ads to those customers and other customers like them.
## 2. Important considerations
The three primary match keys are email address, mailing address, and phone number. You can use user ID and mobile device ID as match keys, but these solutions are less future-proof given their reliance on cookies and device ID. We recommend uploading user contact information, such as email address, mailing address, and phone number, when possible, instead of CRM or mobile IDs.
Google Ads only uses a Customer Match user list for targeting if it has met a minimum threshold of active users at the time the ad is served. Active users is the number of users on your list who are active on Gmail, Search, YouTube, or Display. Upload at least 5,000 members to increase the chance of having enough matched, active users for targeting.
**Accept the Customer Match terms in Google Ads first.** Uploads are rejected until the customer data terms of service have been accepted in the Google Ads account you are sending to. This is a one-time action per account, done in Google Ads under **Tools → Audience manager**.
**Reconnect your Google authorization if it was created before September 2026.** Google moved Customer Match uploads to its Data Manager API, which needs a permission that older Nekt connections do not have. If your runs fail with a message asking you to reconnect the Google account, open the destination settings and re-authorize. This affects Customer Match and Click Conversions; other Google Ads destinations are unaffected.
## 3. Configuring the destination
1. For configuring this destination, go to the [Destinations](https://app.nekt.ai/destinations) tab and click on the "Add destination" button located on the top right of your screen. Then, select the Google Ads - Customer Match List option from the list of connectors.
2. Click **Next** and you'll be prompted to add the connector configuration:
* **Authorization:** you should authenticate this connector with a Google Account with enough permissions to access the Google Ads account to which you want to send the Customer Match List data.
* **Customer ID**: the Google Ads customer ID from the account you want to extract data from. You can obtain this information from the top-right corner of your Google Ads account. For more info, please check [this article](https://support.google.com/google-ads/answer/1704344?hl=en).
* **List name**: defines the list to which you want to upload your customer data. If a list with the specified name doesn't exist on your Google Ads account, a new one will be created.
* **Membership lifespan**: number of days a user stays on your list since its most recent addition. The maximum allowed lifespan is 540 days.
* **Ad user data consent**: specify the consent of the uploaded users for sending user data to Google for advertising purposes.
* **Ad user personalization consent**: specify the consent of the uploaded users for ad personalization.
* **Should hash PII information**: indicates whether you want the connector to hash PII information like email, phone number, first name, last name and street address. If this is set to False, you need to provide the hashed data to the connector, as it's a Google Ads requirements.
* **Upload key type:** there are three types of list identifiers that can be used.
* **CONTACT\_INFO**: this allows you to upload customer information such as email addresses, mailing addresses and phone numbers. Note that contact information must be associated with a Google Account in order to be matched, and corporate accounts, such as Google Workspace, cannot be targeted.
* **CRM\_ID**: CRM IDs are matched from a user ID generated and assigned by the advertiser.
* **MOBILE\_ADVERTISING\_ID**: you can perform customer matching using Identifier for Advertising (IDFA) or Google Advertising ID (AAID) mobile device IDs
While customer lists can contain only one list identifier, multiple types of contact information can be uploaded for an upload\_key\_type of CONTACT\_INFO. This is recommended for increased match rates. You should choose the upload key that aligns with the data fields you are sending.
> Important: if you are uploading data for users in the European Economic Area (EEA), you need to ensure you are passing the required consent signals to Google on the Ad User Consent and Ad Personalization Consent fields.
3. Click **Next**.
## 4. Select your data to send
1. The next step is letting us know which data you want to send to the Google Ads - Customer Match List destination. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 5. Map your data to the destination
In this step, you should map the fields from the input table to the destination. It's important that your table contains the structured data needed to sync with Google Ads - Customer Match List. This is the list of fields available at the destination for each upload key type.
### CONTACT\_INFO
| Field name | Description |
| :- | :- |
| first\_name | The first name of the user. |
| last\_name | The last name of the user. |
| email | The email address of the user. |
| phone\_number | The phone number of the user. The connector automatically normalizes and format to E164. |
| country\_code | The country code of the user. It should match a country code provided by [this list](https://developers.google.com/google-ads/api/data/codes-formats#country_codes). |
| postal\_code | The postal code of the user. |
The unique identifiers allowed by Google Ads are:
* `email`
* `phone_number`
* A combination of `first_name`, `last_name`, `country` and `postal_code`
You should send **at least one** of the above options for each user in the list. The
more you send, the higher are the chances of a successful match by Google.
### CRM\_ID
| Field name | Description | Required |
| :- | :- | - |
| crm\_id | The CRM identifier from the advertiser platform. | Yes |
| first\_name | The first name of the user. | No |
| last\_name | The last name of the user. | No |
| email | The email address of the user. | No |
| phone\_number | The phone number of the user. The connector automatically normalizes and format to E164. | No |
| country\_code | The country code of the user. It should match a country code provided by [this list](https://developers.google.com/google-ads/api/data/codes-formats#country_codes). | No |
| postal\_code | The postal code of the user. | No |
Even though the user identifiers are not required when the upload key is CRM\_ID,
it's a good practice to add them to increase the chances of matching.
### MOBILE\_ADVERTISING\_ID
| Field name | Description | Required |
| :- | :- | - |
| mobile\_advertising\_id | The mobile advertising ID of the user. | Yes |
In addition to that, for all upload types you can add specific consent information for each user in a separate field.
| Field name | Description | Required |
| :- | :- | - |
| ad\_user\_data\_consent | Whether the user has given consent for sending user data to Google for advertising purposes. | No |
| ad\_personalization\_consent | Whether the user has given consent for ad personalization. | No |
For both `ad_user_data_consent` and `ad_personalization_consent` the accepted
values are: UNSPECIFIED, UNKNOWN, GRANTED, DENIED. Only `GRANTED` and `DENIED`
carry a signal to Google; `UNSPECIFIED` and `UNKNOWN` are treated as "not stated"
and are not sent.
Once the fields are configured, you can move forward by clicking the 'Next' button.
## 6. Configure your Google Ads Customer Match List data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers),consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Your list appears in Google Ads only after the pipeline runs. You can monitor execution and completion on the [Destinations](https://app.nekt.ai/destinations) page, and trigger a run manually with the refresh icon. Once it completes, the list is visible in Google Ads under **Tools → Audience manager**.
> Important: once the destination runs successfully, it can take as little as 10 minutes or up to 24 hours for the upload job to complete. You should check the status directly on Google Ads portal what's the status of the job.
# Google Chat as a destination
Source: https://docs.nekt.com/destinations/google-chat
Send your data from Nekt to a Google Chat space.
Google Chat is Google Workspace's team communication platform, organizing conversations into spaces and direct messages. It is where many teams follow up on day-to-day operations, which makes it a natural place to receive automated reports from your data.
## 1. Introduction
The Google Chat destination allows you to send the contents of a table directly into a Google Chat space.
You write the message yourself and choose what goes in it: free text with Google Chat formatting, the current date, and a preview of your data rendered as a table. This makes it easy to share a recurring report (daily sales, weekly KPIs, a list of pending items) with the team that needs it, without anyone having to open a dashboard.
In order to properly use the connector, there are a few pre-requisites that need to be fulfilled:
1. You need a **Google Workspace** account. Incoming webhooks are not available for personal Google accounts.
2. You need permission to create a webhook in the space that will receive the messages.
Google Chat webhooks cannot upload files, so this destination does not attach a CSV or an image. Your data is sent as a preview inside the message itself. If you need the full dataset as a file, consider using the Google Drive or Gmail destinations alongside this one.
## 2. Create the webhook in your Google Chat space
1. Open the Google Chat space that should receive the messages.
2. Click the space name at the top to open the menu, then choose **Apps & integrations**.
3. Click **Webhooks** (or **Add webhooks**).
4. Give the webhook a name (for example `Nekt`), optionally set an avatar URL, and click **Save**.
5. Copy the generated URL. It looks like this:
```
https://chat.googleapis.com/v1/spaces/SPACE_ID/messages?key=KEY&token=TOKEN
```
This URL is a credential: anyone holding it can post messages to your space. Keep it secret and paste it only into the destination configuration, where Nekt stores it encrypted.
## 3. Configure the Google Chat destination
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Google Chat option from the list of connectors.
2. Click **Next** and you'll be prompted to add the connector configuration.
* **Webhook URL**: the URL you copied in the previous section.
* **Message body**: the message to post in the space. You can use the text formatting supported by Google Chat (`*bold*`, `_italic_`, `~strikethrough~` and monospace), as described in the [official Google Chat documentation](https://developers.google.com/workspace/chat/format-messages). Two variables are available:
| Variable | What it renders |
| - | - |
| `{datestamp}` | The current date, using the format set in **Date format** |
| `{data_preview}` | The first rows of your table, as a text table |
* **Maximum preview size** (optional): how many rows `{data_preview}` renders. Defaults to 20, and the maximum is 100.
Google Chat limits a message to 32,000 characters. If your message goes over that limit, Nekt automatically reduces the number of preview rows so the message can still be delivered, and warns you in the run log.
* **Date format** (optional): controls the format of the `{datestamp}` variable. Defaults to `%Y-%m-%d`. Uses Python `strftime` syntax. For the full list of format codes, see the [Python documentation](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes).
Common examples:
| Format string | Result |
| - | - |
| `%Y-%m-%d` | 2026-03-12 |
| `%d/%m/%Y` | 12/03/2026 |
| `%B %d, %Y` | March 12, 2026 |
| `%m-%d-%Y` | 03-12-2026 |
* **Single record input table** (optional): when your table has a single record, enable this option to reference its columns in the message using the `{column_name}` notation.
* **Send message with empty data** (optional): by default, no message is sent when the table has no records. Enable this option to receive a message anyway.
* **Message body (empty data)** (optional): the message to post when the table is empty. If left blank, the regular message body is used.
* **Thread key** (optional): an arbitrary identifier (for example `daily-metrics-report`) that groups every message sent by this destination into the same Google Chat thread, keeping the space organized. Leave it empty to post each run as a new message.
Assuming you have a daily metrics table like this:
| date | revenue | total\_clients | new\_clients |
| - | - | - | - |
| 2026-08-10 | 135000.00 | 475 | 10 |
| 2026-08-11 | 141200.50 | 481 | 6 |
Your message body looks like this:
```
*Metrics report - {datestamp}*
Hi everyone, below is a preview of the latest data:
{data_preview}
Any questions, please let us know!
```
The resulting Google Chat message will be:
```
*Metrics report - 2026-08-12*
Hi everyone, below is a preview of the latest data:
date revenue total_clients new_clients
---------- --------- ------------- -----------
2026-08-10 135000.00 475 10
2026-08-11 141200.50 481 6
Any questions, please let us know!
```
Assuming you have a sales KPIs table with a single record:
| total\_revenue | total\_orders | average\_order\_value | conversion\_rate |
| - | - | - | - |
| 12500 | 250 | 50 | 3.2 |
Enable **Single record input table** and write the message body like this:
```
*Sales report for {datestamp}*
Total revenue: ${total_revenue}
Total orders: {total_orders}
Average order value: ${average_order_value}
Conversion rate: {conversion_rate}%
Great job team!
```
The field references are dynamically replaced with the actual values from your table.
3. Click **Next**.
## 4. Select your data to send
1. The next step is letting us know which data you want to send in the Google Chat message. Select the layer and then the desired table.
Only the first rows of the table are rendered in the message, according to the **Maximum preview size** setting. If you need to communicate a large dataset, filter the table down to the rows that matter, or use a destination that writes files.
2. Click **Next**.
## 5. Define the trigger
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want this message to be sent to the space. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for the message to actually be sent, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your message should appear in the space.
## Troubleshooting
| What you see in the run log | What it means |
| - | - |
| The webhook URL is not a valid Google Chat webhook | The value pasted is not the full webhook URL. Copy it again from **Apps & integrations** > **Webhooks** in the space. |
| Google Chat refused the message for this space | The webhook was deleted or the space no longer exists. Recreate the webhook and paste the new URL in the destination configuration. |
| Google Chat is rate limiting the messages for this space | Google allows one message per second per space. Nekt waits and retries automatically, so the run may take a little longer. |
| The data preview was reduced to N rows | The message exceeded Google Chat's 32,000 character limit. Reduce the message body or the **Maximum preview size**. |
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Google Drive CSV as a destination
Source: https://docs.nekt.com/destinations/google-drive-csv
Send your data from Nekt to Google Drive as CSV files.
Google Drive is Google's cloud storage service that allows users to store, share, and collaborate on files from anywhere. By using Google Drive as a destination, you can automatically export your processed data as CSV files, making them easily accessible and shareable across your organization.
![]()
## 1. Add your Google Drive access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Google Drive (CSV) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* First, grant the Google Authorization using an account that has access to the destination folder.
* Then, input the **Folder ID** of the Google Drive folder where you want your files to be uploaded.
To find your Folder ID, open the desired folder in [Google Drive](https://drive.google.com). The Folder ID is the last segment of the URL in your browser's address bar:
`https://drive.google.com/drive/folders/`**`1aBcDeFgHiJkLmNoPqRsTuVwXyZ`**
Copy that string and paste it into the **Destination folder** field.
* Optionally, input a custom filename. If left blank, the file will be named after the data table being sent.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to Google Drive. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Google Drive CSV data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the CSV file updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Google Drive, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your CSV file should appear in the selected Google Drive folder.
If a file with the same name already exists in the destination folder, it will be updated with the latest data instead of creating a duplicate.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Google Merchant Center XML as a destination
Source: https://docs.nekt.com/destinations/google-merchant-center-xml
Send your data from Nekt to Google Merchant Center XML.
Google Merchant Center XML is Google's platform for managing product data feeds in XML format for e-commerce businesses. It enables businesses to upload and manage product information for Google Shopping, allowing products to appear in search results and shopping campaigns.
## 1. Add your Google Merchant Center XML access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Google Merchant Center (XML) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Title:** The title to be inserted in the `title` tag from the template.
* **Description:** The description to be inserted in the `description` tag from the template.
* **Link:** The link to be inserted in the `link` tag from the template.
* **Jinja template file:** The jinja template file used to map the input table data to the target element structure. For detailed instructions, [click here](https://docs.nekt.com/destinations/xml#2-how-to-generate-a-jinja-template-file).
* **Bucket name:** The name of the S3 bucket where the generated XML file will be stored.
* **S3 folder path** (optional): Optionally define the name of folder inside the S3 bucket where the generated XML file will be stored.
* **Output file name:** The static name of the generated XML file that will be uploaded to S3.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the spreadsheet. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Google Merchant Center (XML) data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers),consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Google Merchant Center, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Google Merchant Center.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Google Sheets as a destination
Source: https://docs.nekt.com/destinations/google-sheets
Send your data from Nekt to Google Sheets.
Google Sheets is Google's cloud-based spreadsheet application that allows users to create, edit, and collaborate on spreadsheets in real-time. It's part of Google Workspace and provides tools for data organization, analysis, and sharing, making it popular for business data management and team collaboration.
![]()
Google Sheets is a powerful spreadsheet application that allows teams to collaborate on data analysis, reporting, and data sharing. By setting up Google Sheets as a destination at Nekt, you can automatically export your processed data to spreadsheets, making it easily accessible for stakeholders, analysts, and team members who are used to working with spreadsheet tools.
This guide will walk you through the process of configuring Google Sheets as a destination, including authentication setup, data selection, and configuration options. You'll learn how to map your data to spreadsheet tabs, configure data ranges, and set up automated data synchronization to keep your spreadsheets up to date.
## Configuring Google Sheets as a Destination
In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the **Google Sheets** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You will be prompted to add your access configuration:
* First, inform the spreadsheet link.
* Then, make sure you grant 'Editor' permission on your spreadsheet for the provided email.
* You can enable record flattening to turn nested data into separate columns, so each value gets its own cell instead of being grouped together. If enabled, you'll need to specify how many levels deep we should go in the nested structure to create new columns.
Click **Next**.
### 2. Select your data to send
The next step is letting us know which data you want to send to the spreadsheet. Select the layer and then the desired tables. You can add multiple tables on this step. Each table will be sent to a different tab on your spreadsheet.
If you select an empty table, it will be ignored.
Select the tables and click **Next**.
### 3. Configure data streams
In this step, you will be able to configure the following parameters for each one of your input tables:
* **Destination tab name**: you can define the name of the tab your input table should be sent to. If left blank or the tab doesn't exist, a new one will be created using the input table name.
* **Cell range to clear**: If there's a specific range you want to clear before writing data, you can define it here. This is useful when you have additional columns or formulas in the same sheet you're writing data to.
You can only limit the range horizontally, which means all rows before the initial offset will be cleared.
The notation to be used is:
`{start_column}{start_row}:{end_column}`
For example:
`A5:F` would clear the range from column A to F, starting at row 5.
* **Fields mapping**: using the field mapping, you can rename each column from your input table before writing it to the sheet.
Once you are done configuring, click **Next**.
### 4. Configure data destination
Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc., not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
Once you are ready, click **Done** to finalize the setup.
### 5. Check your new destination
Once completed, you'll receive confirmation that your new destination is set up!
You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Spreadsheet, you have to wait for the pipeline to run. You can monitor it on the Destinations page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the Spreadsheet.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# HubSpot as a destination
Source: https://docs.nekt.com/destinations/hubspot
Send your data from Nekt to HubSpot.
HubSpot is a comprehensive customer relationship management (CRM) and marketing automation platform that helps businesses attract, engage, and delight customers. It provides tools for marketing, sales, customer service, and content management, all integrated into a single platform.
![]()
## 1. Add your HubSpot access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the **HubSpot** option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Authorization:** Login with a Hubspot account that has **Super Admin** permissions in order to install the Nekt app into the workspace.
* **Date Format:** The format for dates included in the input table.
* **Import Mode:** Used to indicate whether the import should create and update, only create, or only update records for a certain object or activity.
* **Fail on rejected rows:** When enabled, the run is marked as failed if HubSpot rejects any row of the import (for example, an invalid value or a missing associated record). The accepted rows are still imported. Keep it disabled to only get a warning, which is the default behavior. Enable it when other pipelines are triggered by this destination and must not run on incomplete data.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be easily found by typing its name.
2. Click **Next**.
## 3. Map your HubSpot data to send
1. Select the columns from your selected table that you want to send.
2. For each column selected:
* Select the column from your input table;
* Inform the internal name of the field on Hubspot. You can find the internal name by accessing Settings -> Properties -> Click the `Property Name (ex. Company name)` of interest -> Finding the value next to the `Internal name` field;
* The name of the target entity. If it's a custom object, please inform the ObjectType of your Custom Object (see more [here](https://developers.hubspot.com/docs/guides/api/crm/understanding-the-crm)). The ID of a Custom Object usually has the following structure `2-XXX`;
* Inform whether this field should be considered an object or alternate ID. If not, leave this field empty. In HubSpot, Object ID (or `hs_object_id`) is a system-generated ID used to identify and reference records within HubSpot's API and database. An Alternate ID, on the other hand, is a custom property you create and define to represent a unique identifier for a record, often used for external systems or to map records from other platforms.
When upserting, at least one field should be selected as ID for proper merging.
3. Click **Next**.
## 4. Configure your HubSpot data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your HubSpot account, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on HubSpot.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# IndeCx as a destination
Source: https://docs.nekt.com/destinations/indecx
Send your data from Nekt to IndeCx.
IndeCx is a customer engagement platform that helps businesses manage and optimize their customer interactions and data. It provides tools for customer data integration, enabling companies to streamline their customer relationship management processes.
![]()
## Configuring IndeCx as a Destination
In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the IndeCx option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your IndeCx API credentials for authentication with their v3 integration API.
The following configurations are available:
* **Company Key**: Your Company Key for authenticating with the IndeCx API. This key is used to obtain access tokens for secure API communication.
* **Action Identifier**: The Action Identifier used to specify the integration endpoint in the IndeCx API.
If you don't know where to find your Company Key and Action Identifier, check the [IndeCx API documentation](https://github.com/indecx/doc-indecx/blob/main/api.md#autenticação) or reach out to your IndeCx account manager.
Once you're done, click **Next**.
### 2. Select your data to send
Choose which data you want to send to IndeCx. Select the layer and then the desired table.
The table can be found more easily by typing its name.
Select your data and click **Next**.
### 3. Map your data fields
Configure how the fields from your source table should be mapped to the fields in the IndeCx API payload. Below are the available fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `nome` | String | The customer's name. |
| `email` | String | The customer's email address. |
| `telefone` | String | The customer's phone number. |
At least one of `telefone` or `email` must be provided.
Once you've mapped your fields, click **Next**.
### 4. Configure data destination
Describe your destination for easy identification within your organization, not exceeding 140 characters. Include details like what data it contains and which team it belongs to.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need the data updated in IndeCx. Common choices include:
* **Daily**: For regular customer data updates
* **Weekly**: For less frequent synchronization needs
* **Real-time**: For immediate data availability requirements
Optionally, you can configure additional settings:
* Configure Delta Log Retention to determine how long to store old states of synchronized data. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new destination
Once completed, you'll receive confirmation that your new destination is set up successfully!
You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. To see the data in IndeCx, wait for the pipeline to run or manually trigger it by clicking the refresh icon. Monitor execution status on the Destinations page.
If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Implementation Notes
### Authentication & Security
* The connector uses IndeCx's v3 API with token-based authentication
* Access tokens are automatically refreshed every 30 minutes to maintain secure connections
* Authentication tokens are cached and reused until near expiration to optimize API calls
### Data Processing
* Data is sent to IndeCx in batches of up to 1,000 records at a time for optimal performance
* The connector automatically handles rate limiting and retries failed requests
* All customer data is validated before transmission to ensure API compatibility
### API Limitations
* **Rate Limiting**: The IndeCx API may apply rate limits during high-volume operations
* **Batch Size**: Maximum of 1,000 records per API request
* **Required Fields**: At least one contact method (`email` or `telefone`) must be provided for each record
* **Authentication**: Access tokens expire every 30 minutes and are automatically renewed
# Linear as a destination
Source: https://docs.nekt.com/destinations/linear
Send your data from Nekt to Linear.
Linear is an issue tracking and project management tool that helps software teams streamline their workflows. It provides features for managing tasks, tracking bugs, and planning sprints to improve productivity and collaboration.
## 1. Add your Linear access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Linear option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Linear Authentication**: Log in with your Linear account to authorize the Nekt connector.
* **Integration endpoint**: The endpoint you want to integrate with this destination. Currently, only `CustomerUpsert` is supported.
* **Continue after failure:** Whether the destination should continue trying to sync records even after one fails.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data fields
Configure how the fields from your source table should be mapped to the fields in the selected endpoint. Below you can find the supported fields for each endpoint.
This endpoint creates a new customer or updates an existing one. Below are the fields you can map your data to:
| Field | Type | Required | Description |
| :- | :- | :- | :- |
| `name` | String | Yes | The name of the customer. |
| `id` | String | No | The unique identifier for the customer. Only used for upsert. |
| `externalId` | String | No | The customer's ID in an external system. |
| `ownerId` | String | No | The ID of the user who owns the customer relationship. |
| `logoUrl` | String | No | A URL for the customer's logo. |
| `revenue` | Int | No | The customer's annual revenue. |
| `size` | Int | No | The number of employees at the company. |
| `slackChannelId` | String | No | The ID of a Slack channel associated with the customer. |
| `statusId` | String | No | The ID of the customer's status. |
| `tierId` | String | No | The ID of the customer's tier. |
| `tierName` | String | No | The name of the customer's tier. |
## 4. Configure your Linear data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Linear, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Linear.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Meta Catalog XML as a destination
Source: https://docs.nekt.com/destinations/meta-catalog-xml
Send your data from Nekt to Meta Catalog XML.
Meta Catalog XML is Meta's platform for managing product catalogs in XML format for advertising campaigns. It enables businesses to upload and manage product information for Facebook and Instagram shopping features, allowing products to appear in ads and shopping experiences.
## 1. Add your Meta Catalog XML access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Meta Catalog option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Title:** The title to be inserted in the `title` tag from the template.
* **Description:** The description to be inserted in the `description` tag from the template.
* **Link:** The link to be inserted in the `link` tag from the template.
* **Jinja template file:** The jinja template file used to map the input table data to the target element structure. For detailed instructions, [click here](https://docs.nekt.com/destinations/xml#2-how-to-generate-a-jinja-template-file).
* **Bucket name:** The name of the S3 bucket where the generated XML file will be stored.
* **S3 folder path** (optional): Optionally define the name of folder inside the S3 bucket where the generated XML file will be stored.
* **Output file name:** The static name of the generated XML file that will be uploaded to S3.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the spreadsheet. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Meta Catalog data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Meta Catalog, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Meta Catalog.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Microsoft SQL Server as a destination
Source: https://docs.nekt.com/destinations/microsoft-sql-server
Send your data from Nekt to a Microsoft SQL Server database.
SQL Server is Microsoft's relational database management system designed for enterprise applications and data warehousing. It provides robust data storage, advanced analytics, and business intelligence capabilities with features for high availability, security, and performance optimization.
![]()
## 1. Add your Microsoft SQL Server access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the SQL option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access: host, port, database name, password, user and schema.
3. Click **Next**.
You must ensure your database accepts incoming connections from Nekt, and the database user has write permissions in order to write data.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Microsoft SQL Server data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Microsoft SQL Server database, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the defined Microsoft SQL Server location.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Notion as a destination
Source: https://docs.nekt.com/destinations/notion
Send your data from Nekt to Notion.
Notion is a workspace app that combines documents, wikis and databases. With this destination you can create and update **pages in a Notion database** directly from your Nekt tables: each row becomes a page, with its columns written into the database properties.
## 1. Add your Notion access
Before configuring the destination, create the integration Notion uses to authenticate:
1. Go to [notion.so/profile/integrations](https://www.notion.so/profile/integrations) and click **New integration**. Give it a name (e.g. "Nekt"), select the workspace, and make sure it has the **Insert content** and **Update content** capabilities.
2. Copy the **Internal Integration Token** (it starts with `ntn_`).
3. Open the Notion database that will receive the data, click the **`...`** menu at the top right, choose **Connections** and add your integration. **Without this step Notion cannot see the database** and every request fails with "Could not find database".
4. Copy the database URL from your browser. You can paste the whole URL into Nekt.
Then, in Nekt:
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Notion option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Integration Token**: The internal integration token you copied above.
* **Database ID or URL**: The Notion database that receives the records. Paste the database ID or the full database URL. The ID is extracted automatically.
* **Match property**: Optional. The Notion property used to find an existing page to update (e.g. `Email` or an "External ID" property). Leave it empty to always create new pages.
* **Create when no match**: When a match property is set and no page matches, create a new page. Turn it off to only update pages that already exist.
* **Replace content on update**: When updating a page and the mapping writes to `page_content`, replace the page body. Off by default, so updates never delete content.
* **Requests per second**: Maximum request pace. Notion allows an average of 3 requests per second per integration.
* **Dry run**: Validation mode that performs the lookups and logs what would be created or updated, without writing anything to Notion.
* **Enable partial success**: Whether the connector should continue trying to sync records after one fails.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data fields
Configure how the columns from your source table should be mapped to the Notion database properties. Use the **property name exactly as it appears in Notion** (matching is case-insensitive). The connector reads the database schema from Notion, so each value is automatically converted to the format the property type expects. You never need to write Notion's typed JSON.
Below is what each Notion property type accepts as input. See the [Notion page property values reference](https://developers.notion.com/reference/page-property-values) for the full specification.
| Notion type | Accepted input |
| :- | :- |
| `title` | Any value, written as text. Every Notion database has exactly one title property. |
| `rich_text` | Any value, written as text. |
| `number` | A number or a numeric string. |
| `select` | Text: the option name. Notion creates the option if it does not exist yet. |
| `status` | Text: the status name. Must already exist in Notion. |
| `multi_select` | A list, or comma-separated text: "vip, brasil" becomes two options. |
| `date` | An ISO 8601 date or date-time (e.g. "2026-07-24" or "2026-07-24T10:00:00Z"). |
| `checkbox` | A boolean, or `true`/`false`/`yes`/`no`/`1`/`0`/`sim`/`não`. |
| `url` | Text: the URL. |
| `email` | Text: the email address. |
| `phone_number` | Text: the phone number. |
| `people` | A Notion user ID, or a comma-separated list of IDs. |
| `relation` | A Notion page ID, or a comma-separated list of IDs. |
| `files` | An external URL, or a comma-separated list of URLs. |
Two destination names address the page itself instead of a database property:
| Name | Description |
| :- | :- |
| `page_id` | The Notion page ID to update. When mapped, the record always **updates** that page. |
| `page_content` | Text written as the body of the page, as paragraph blocks. Blank lines separate paragraphs. |
Properties that Notion computes itself (formulas, rollups, unique IDs, created/last edited time and created/last edited by) cannot be written and are skipped with a warning. Empty values are never sent, so a blank column never overwrites data already in Notion.
### Creating vs. updating pages
* With no **Match property** and no `page_id` mapped, every record **creates** a new page.
* When `page_id` is mapped, the record **updates** that specific page.
* When a **Match property** is set, the connector looks the page up by that property: a match is updated, and a record with no match creates a new page (or is skipped when "Create when no match" is off). The match property must also be present in the mapping, so the connector knows which value to look up.
### Transforms
Each mapping rule can apply ordered value transforms before sending: `uppercase`, `lowercase`, `trim`, `round_int`, `to_string`, `yesno_to_bool`, `null_if:` (drop a placeholder such as a dash) and `only_if:` (keep the value only when it equals a given value).
## 4. Configure your Notion data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Notion, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Notion.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Omie as a destination
Source: https://docs.nekt.com/destinations/omie
Send your data from Nekt to Omie.
Omie is a cloud-based ERP platform for small and medium-sized businesses in Brazil. It offers a comprehensive suite of tools for managing finances, sales, inventory, and services, helping companies streamline their operations and ensure tax compliance.
## 1. Add your Omie access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Omie option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **App Key**: Your App Key for the Omie API.
* **App Secret**: Your App Secret for the Omie API.
* **Endpoint**: Select to which endpoint you want to send your data: `Upsert Clientes`, `Upsert Contas a Receber`, `Upsert Contas a Pagar`, `Excluir Contas a Receber`, `Excluir Contas a Pagar`, `Upsert Projetos`, `Upsert Vendedores`, `Upsert Categorias`, `Associar Código Integração Cliente` and `Upsert Contratos`. Please pay attention to the available fields on the `Map your data fields` section and whether they are mandatory or not.
* **Enable partial success:** Whether the connector should continue trying to sync records after one fails.
If you don't know where to find your App Key & Secret, check [this doc](https://ajuda.omie.com.br/pt-BR/articles/499061-obtendo-a-chave-de-acesso-para-integracoes-de-api).
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data fields
Configure how the fields from your source table should be mapped to the fields in the selected endpoint. Below you can find the supported fields for each endpoint.
This endpoint creates or updates clients (customers). Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `codigo_cliente_omie` | Integer | Unique identifier for the client in Omie. |
| `codigo_cliente_integracao` | String(60) | External unique identifier for the client. |
| `razao_social` | String(60) | The legal name of the company. |
| `cnpj_cpf` | String(20) | The client's tax identifier (CNPJ for companies, CPF for individuals). |
| `nome_fantasia` | String(100) | The trade name of the company. |
| `telefone1_ddd` | String(5) | The area code for the primary phone number. |
| `telefone1_numero` | String(15) | The primary phone number. |
| `contato` | String(100) | The name of the primary contact person. |
| `endereco` | String(60) | The street name. |
| `endereco_numero` | String(60) | The street number. |
| `bairro` | String(60) | The neighborhood. |
| `complemento` | String(60) | Additional address information. |
| `estado` | String(2) | The state abbreviation (e.g., SP). |
| `cidade` | String(40) | The city code. |
| `cep` | String(10) | The postal code. |
| `codigo_pais` | String(4) | The country code (e.g., `1058` for Brazil). |
| `separar_endereco` | String(1) | Separate address (`S` or `N`). |
| `pesquisar_cep` | String(1) | Search for address by postal code (`S` or `N`). |
| `telefone2_ddd` | String(5) | The area code for the secondary phone number. |
| `telefone2_numero` | String(15) | The secondary phone number. |
| `fax_ddd` | String(5) | The area code for the fax number. |
| `fax_numero` | String(15) | The fax number. |
| `email` | String(500) | The client's email address. |
| `homepage` | String(100) | The client's website URL. |
| `inscricao_estadual` | String(20) | The state registration number. |
| `inscricao_municipal` | String(20) | The municipal registration number. |
| `inscricao_suframa` | String(20) | The SUFRAMA registration number. |
| `optante_simples_nacional` | String(1) | Indicates if the client is part of the "Simples Nacional" tax regime (`S` or `N`). |
| `tipo_atividade` | String(1) | The type of activity. |
| `cnae` | String(7) | The CNAE code (National Classification of Economic Activities). |
| `produtor_rural` | String(1) | Indicates if the client is a rural producer (`S` or `N`). |
| `contribuinte` | String(1) | Indicates if the client is a taxpayer (`S` or `N`). |
| `observacao` | Text | Internal observations. |
| `obs_detalhadas` | Text | Detailed observations. |
| `recomendacao_atraso` | String(2) | The protest instruction code. |
| `tags` | Array | An array of tags associated with the client. |
| `pessoa_fisica` | String(1) | Indicates if the client is an individual (`S`) or a company (`N`). |
| `exterior` | String(1) | Indicates if the client is located abroad (`S` or `N`). |
| `cidade_ibge` | String(7) | The IBGE city code. |
| `valor_limite_credito` | Decimal | The total credit limit value. |
| `bloquear_faturamento` | String(1) | Block invoicing for the client (`S` or `N`). |
| `nif` | String(100) | The NIF (Tax Identification Number). |
| `enviar_anexos` | String(1) | Send attachments by email (`S` or `N`). |
| `logradouro` | String(6) | Deprecated. |
| `importado_api` | String(1) | Indicates if the record was imported via API (`S` or `N`). |
| `bloqueado` | String(1) | Deprecated. |
| `recomendacoes` | Object | An object containing client recommendations. |
| `enderecoEntrega` | Object | An object containing the delivery address details. |
| `documento_exterior` | String(20) | The client's foreign document number. |
| `inativo` | String(1) | Indicates if the client is inactive (`S` or `N`). |
| `dadosBancarios` | Object | An object containing the client's bank details. |
| `caracteristicas` | Array | An array of client characteristics. |
| `info` | Object | An object containing metadata about the record (read-only). |
| `bloquear_exclusao` | String(1) | Blocks the deletion of the record (`S` or `N`). |
This endpoint creates or updates accounts payable records. Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `codigo_lancamento_omie` | Integer | Unique identifier for the accounts payable entry in Omie. |
| `codigo_lancamento_integracao` | String(60) | External unique identifier for the accounts payable entry. |
| `codigo_cliente_fornecedor` | Integer | The supplier's unique identifier. |
| `codigo_cliente_fornecedor_integracao` | String(60) | The supplier's external unique identifier. |
| `data_vencimento` | String(10) | The due date of the bill (YYYY-MM-DD). |
| `valor_documento` | Decimal | The total amount of the bill. |
| `codigo_categoria` | String(20) | The category code. |
| `categorias` | Array | An array for category allocation. |
| `data_previsao` | String(10) | The expected payment date (YYYY-MM-DD). |
| `id_conta_corrente` | Integer | The bank account ID. |
| `numero_documento_fiscal` | String(20) | The fiscal document number (e.g., invoice number). |
| `data_emissao` | String(10) | The emission date of the document (YYYY-MM-DD). |
| `data_entrada` | String(10) | The registration date of the document (YYYY-MM-DD). |
| `codigo_projeto` | Integer | The project code. |
| `observacao` | Text | Observations for the payment entry. |
| `valor_pis` | Decimal | The PIS tax amount. |
| `retem_pis` | String(1) | Indicates if PIS tax should be withheld (`S` or `N`). |
| `valor_cofins` | Decimal | The COFINS tax amount. |
| `retem_cofins` | String(1) | Indicates if COFINS tax should be withheld (`S` or `N`). |
| `valor_csll` | Decimal | The CSLL tax amount. |
| `retem_csll` | String(1) | Indicates if CSLL tax should be withheld (`S` or `N`). |
| `valor_ir` | Decimal | The IR tax amount. |
| `retem_ir` | String(1) | Indicates if IR tax should be withheld (`S` or `N`). |
| `valor_iss` | Decimal | The ISS tax amount. |
| `retem_iss` | String(1) | Indicates if ISS tax should be withheld (`S` or `N`). |
| `valor_inss` | Decimal | The INSS tax amount. |
| `retem_inss` | String(1) | Indicates if INSS tax should be withheld (`S` or `N`). |
| `distribuicao` | Array | An array for department allocation. |
| `numero_pedido` | String(15) | The purchase order number. |
| `codigo_tipo_documento` | String(5) | The document type code. |
| `numero_documento` | String(20) | The document number. |
| `numero_parcela` | String(7) | The installment number. |
| `chave_nfe` | String(44) | The electronic invoice key (NF-e). |
| `codigo_barras_ficha_compensacao` | String(70) | The barcode for the payment slip. |
| `codigo_vendedor` | Integer | The salesperson code. |
| `id_origem` | String(4) | The origin code. |
| `info` | Object | An object containing metadata about the record (read-only). |
| `operacao` | String(2) | The operation code. |
| `status_titulo` | String(3) | The status of the bill. |
| `importado_api` | String(1) | Indicates if the record was imported via API (`S` or `N`). |
| `bloquear_exclusao` | String(1) | Blocks the deletion of the record (`S` or `N`). |
| `cnab_integracao_bancaria` | Object | An object containing CNAB and bank integration data. |
| `servico_tomado` | Object | An object with information from the "Serviço Tomado" tab of the Service Order. |
| `valor_pag` | Decimal | The amount to be paid. |
| `aprendizado_rateio` | String(1) | Department allocation learning (`S` or `N`). |
| `baixar_documento` | String(1) | Automatically settles the document (`S` or `N`). |
| `conciliar_documento` | String(1) | Automatically reconciles the document (`S` or `N`). |
| `pagamento` | Object | An object containing payment (settlement) details. |
This endpoint creates or updates accounts receivable records. Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `codigo_lancamento_omie` | Integer | Unique identifier for the accounts receivable entry in Omie. |
| `codigo_lancamento_integracao` | String(60) | External unique identifier for the accounts receivable entry. |
| `codigo_cliente_fornecedor` | Integer | The client's unique identifier. |
| `codigo_cliente_fornecedor_integracao` | String(60) | The client's external unique identifier. |
| `data_vencimento` | String(10) | The due date of the invoice (YYYY-MM-DD). |
| `valor_documento` | Decimal | The total amount of the invoice. |
| `codigo_categoria` | String(20) | The category code. |
| `data_previsao` | String(10) | The expected receipt date (YYYY-MM-DD). |
| `categorias` | Array | An array for category allocation. |
| `id_conta_corrente` | Integer | The bank account ID. |
| `numero_documento` | String(20) | The document number. |
| `numero_parcela` | String(7) | The installment number (e.g., "001/003"). |
| `codigo_tipo_documento` | String(5) | The document type code. |
| `numero_documento_fiscal` | String(20) | The fiscal document number (e.g., invoice number). |
| `numero_pedido` | String(15) | The sales order number. |
| `chave_nfe` | String(44) | The electronic invoice key (NF-e). |
| `observacao` | Text | Observations for the receipt entry. |
| `codigo_barras_ficha_compensacao` | String(70) | The barcode for the payment slip. |
| `codigo_cmc7_cheque` | String(40) | The CMC7 code of the check. |
| `data_emissao` | String(10) | The emission date of the document (YYYY-MM-DD). |
| `id_origem` | String(4) | The origin code. |
| `operacao` | String(2) | The operation code. |
| `valor_pis` | Decimal | The PIS tax amount. |
| `retem_pis` | String(1) | Indicates if PIS tax should be withheld (`S` or `N`). |
| `valor_cofins` | Decimal | The COFINS tax amount. |
| `retem_cofins` | String(1) | Indicates if COFINS tax should be withheld (`S` or `N`). |
| `valor_csll` | Decimal | The CSLL tax amount. |
| `retem_csll` | String(1) | Indicates if CSLL tax should be withheld (`S` or `N`). |
| `valor_ir` | Decimal | The IR tax amount. |
| `retem_ir` | String(1) | Indicates if IR tax should be withheld (`S` or `N`). |
| `valor_iss` | Decimal | The ISS tax amount. |
| `retem_iss` | String(1) | Indicates if ISS tax should be withheld (`S` or `N`). |
| `valor_inss` | Decimal | The INSS tax amount. |
| `retem_inss` | String(1) | Indicates if INSS tax should be withheld (`S` or `N`). |
| `bloqueado` | String(1) | Blocks the entry (`S` or `N`). |
| `bloquear_baixa` | String(1) | Blocks the settlement of the entry (`S` or `N`). |
| `importado_api` | String(1) | Indicates if the record was imported via API (`S` or `N`). |
| `baixar_documento` | String(1) | Automatically settles the document (`S` or `N`). |
| `conciliar_documento` | String(1) | Automatically reconciles the document (`S` or `N`). |
| `acao` | Text | The action to be performed (`A`-Add, `S`-Subtract, `L`-Launch). |
| `lancamento_detalhe` | Array | An array of objects with entry details. |
| `distribuicao` | Array | An array for department allocation. |
| `status_titulo` | String(100) | The status of the invoice. |
| `codigo_vendedor` | Integer | The salesperson code. |
| `codigo_projeto` | Integer | The project code. |
| `nsu` | String(100) | The Unique Sequential Number (payment receipt). |
| `data_registro` | String(10) | The registration date (YYYY-MM-DD). |
| `tipo_agrupamento` | String(1) | The type of grouping (`C`-Consolidated, `I`-Individual). |
| `info` | Object | An object containing metadata about the record (read-only). |
| `boleto` | Object | An object containing bank slip (boleto) information. |
| `nCodPedido` | Integer | The Sales Order ID. |
| `bloquear_exclusao` | String(1) | Postpones the due date (`S` or `N`). |
| `nCodOS` | Integer | The Service Order ID. |
| `cPedidoCliente` | String(30) | The customer's purchase order number. |
| `cNumeroContrato` | String(20) | The customer's contract number. |
| `recebimento` | Object | An object containing receipt (settlement) details. |
| `repeticao` | Object | An object to configure recurring entries. |
| `aprendizado_rateio` | String(1) | Department allocation learning (`S` or `N`). |
This endpoint creates or updates projects. Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | Integer | Unique identifier for the project in Omie. |
| `codInt` | String(20) | External unique identifier for the project. |
| `nome` | String(70) | Name of the project. |
| `inativo` | String(1) | Indicates if the project is inactive (`S` or `N`). |
This endpoint creates or updates sellers. Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | Integer | Código do Vendedor |
| `codInt` | String(30) | Código de Integração do Vendedor |
| `nome` | String(70) | Nome do Vendedor |
| `inativo` | String(1) | Indica se o vendedor está inativo \[S/N] |
| `email` | String(100) | E-mail do vendedor. |
| `fatura_pedido` | String(1) | O vendedor pode faturar um pedido? |
| `visualiza_pedido` | String(1) | Visualiza apenas os pedidos em que é o vendedor. Informar "S" ou "N". |
| `comissao` | Decimal | Percentual de Comissão. |
This endpoint creates categories. Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `categoria_superior` | String | Code of the category group. |
| `descricao` | String(50) | Description of the category. |
| `natureza` | String(50) | Description of the account nature, equivalent to the "Observation" field of the category in the ERP. |
| `tipo_categoria` | String(3) | Type of expense/revenue made with this category. |
| `codigo_dre` | String(10) | Code in the Income Statement (DRE). |
This endpoint associates an external integration code with an existing Omie client. Below are the fields you can map your data to:
| Field | Type | Description |
| :- | :- | :- |
| `codigo_cliente_omie` | Integer | Unique identifier for the client in Omie. |
| `codigo_cliente_integracao` | String(60) | External integration code for legacy systems. |
This endpoint creates or updates service contracts. The contract payload is structured in sections. Below are the fields you can map your data to:
**cabecalho** (Contract Header)
| Field | Type | Description |
| :- | :- | :- |
| `nCodCtr` | Integer | Unique contract code in Omie. |
| `cCodIntCtr` | String(20) | External integration code for the contract. |
| `nCodCli` | Integer | Customer code. **Required.** |
| `cNumCtr` | String(60) | Contract number. **Required.** |
| `cCodSit` | String(2) | Contract status (`00` - Active, `10` - Suspended, `90` - Cancelled, `99` - Finished). |
| `dVigInicial` | String(10) | Start date of the contract. **Required.** |
| `dVigFinal` | String(10) | End date of the contract. **Required.** |
| `nDiaFat` | Integer | Billing day of the month. **Required.** |
| `nValTotMes` | Decimal | Monthly total value. |
| `cTipoFat` | String(2) | Billing frequency (`01` - Monthly, `02` - Bimonthly, `03` - Quarterly, `06` - Semiannual, `12` - Annual). **Required.** |
**itensContrato** (Contract Items)
Each item contains the following sub-sections:
*itemCabecalho*
| Field | Type | Description |
| :- | :- | :- |
| `codItem` | Integer | Item ID in Omie. |
| `codIntItem` | String(30) | Item integration code. |
| `seq` | Integer | Item sequence number. |
| `codServico` | Integer | Service code. |
| `natOperacao` | String(2) | Operation nature/tax type. **Required.** |
| `codServMunic` | String(20) | Municipal service code. **Required.** |
| `codLC116` | String(10) | LC 116 service code. **Required.** |
| `quant` | Decimal | Quantity. **Required.** |
| `valorUnit` | Decimal | Unit value. **Required.** |
| `cTpDesconto` | String(1) | Discount type (`P` - Percentage, `V` - Value). |
| `valorDesconto` | Decimal | Discount value. |
| `aliqDesconto` | Decimal | Discount rate. |
| `valorOutrasRetencoes` | Decimal | Other retentions value. |
| `valorAcrescimo` | Decimal | Additions/fees value. |
| `codNBS` | String(9) | NBS code. |
| `cNaoGerarFinanceiro` | String(1) | Skip financial generation (`S` or `N`). |
| `cCodCategItem` | String(20) | Item category code. |
| `valorDed` | Decimal | Deduction value. |
| `valorTotal` | Decimal | Total item value. |
*itemDescrServ*
| Field | Type | Description |
| :- | :- | :- |
| `descrCompleta` | Text | Complete service description. **Required.** |
*itemImpostos* (Item Taxes)
| Field | Type | Description |
| :- | :- | :- |
| `aliqISS` | Decimal | ISS rate. |
| `valorISS` | Decimal | ISS value. |
| `retISS` | String(1) | Retain ISS (`S` or `N`). |
| `aliqIR` | Decimal | IR rate. |
| `valorIR` | Decimal | IR value. |
| `retIR` | String(1) | Retain IR (`S` or `N`). |
| `aliqPIS` | Decimal | PIS rate. |
| `valorPIS` | Decimal | PIS value. |
| `retPIS` | String(1) | Retain PIS (`S` or `N`). |
| `aliqCOFINS` | Decimal | COFINS rate. |
| `valorCOFINS` | Decimal | COFINS value. |
| `retCOFINS` | String(1) | Retain COFINS (`S` or `N`). |
| `aliqCSLL` | Decimal | CSLL rate. |
| `valorCSLL` | Decimal | CSLL value. |
| `retCSLL` | String(1) | Retain CSLL (`S` or `N`). |
| `aliqINSS` | Decimal | INSS rate. |
| `valorINSS` | Decimal | INSS value. |
| `retINSS` | String(1) | Retain INSS (`S` or `N`). |
**infAdic** (Additional Information)
| Field | Type | Description |
| :- | :- | :- |
| `cCodCateg` | String(20) | Category code. |
| `nCodVend` | Integer | Vendor code. |
| `nCodProj` | Integer | Project code. |
| `nCodCC` | Integer | Cost center code. |
| `cContato` | String(100) | Contact name. |
| `cCidPrestServ` | String(40) | City/state where the service is provided. |
| `cCodObra` | String(15) | Work/construction code. |
| `cCodART` | String(20) | ART code. |
| `cDadosAdicNF` | Text | Additional NF data. |
| `nIdDestinatario` | Integer | Recipient client ID. |
**vencTextos** (Maturity & NFS-e Texts)
| Field | Type | Description |
| :- | :- | :- |
| `cTpVenc` | String(3) | Maturity type (`001` - Days after billing, `002` - Fixed day). **Required.** |
| `nDias` | Integer | Days to maturity. |
| `nDiaFixo` | Integer | Fixed maturity day (1-31). |
| `cProxMes` | String(1) | Next month only (`S` or `N`). |
| `cAntecipar` | String(1) | Anticipate business days (`S` or `N`). |
| `cPostergar` | String(1) | Postpone weekends/holidays (`S` or `N`). |
| `cAdPeriodo` | String(1) | Add reference period to description (`S` or `N`). |
| `cAdContrato` | String(1) | Add contract number to description (`S` or `N`). |
| `cAdVenc` | String(1) | Add maturity to description (`S` or `N`). |
**departamentos** (Department Distribution)
| Field | Type | Description |
| :- | :- | :- |
| `cCodDep` | String(40) | Department code. |
| `cDesDep` | String(50) | Department description. |
| `nValDep` | Decimal | Department allocation value. |
| `nPerDep` | Decimal | Department allocation percentage. |
**emailCliente** (Client Email Settings)
| Field | Type | Description |
| :- | :- | :- |
| `cEnviarRecibo` | String(1) | Send service receipt (`S` or `N`). |
| `cEnviarLinkNfse` | String(1) | Send NFS-e link (`S` or `N`). |
| `cEnviarBoleto` | String(1) | Send billing slip (`S` or `N`). |
| `cEnviarPix` | String(1) | Send PIX payment (`S` or `N`). |
| `cUtilizarEmails` | Text | Override email addresses. |
**observacoes** (Observations)
| Field | Type | Description |
| :- | :- | :- |
| `cObsContrato` | Text | Contract observations/notes. |
For the `Excluir` endpoints, you should pass the Omie unique key associated with the record:
* `Contas a Pagar` -> `conta_pagar_cadastro_chave`
* `Contas a Receber` -> `conta_receber_cadastro_chave`
## 4. Configure your Omie data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Omie, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Omie.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# OneDrive XLSX as a destination
Source: https://docs.nekt.com/destinations/onedrive-xlsx
Send your data from Nekt to OneDrive XLSX.
OneDrive XLSX refers to Microsoft Excel files stored in OneDrive, which is Microsoft's cloud storage service. This allows you to store and access Excel spreadsheet data in the cloud, providing secure and collaborative access to spreadsheet files from anywhere.
![]()
## 1. Add your OneDrive XLSX access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the OneDrive Excel (xlsx) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* First of all, grant the Microsoft Authorization using an account that has access to the destination file.
* Then, select the OneDrive Folder
* Input the name of the file you want to send data to
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your OneDrive Excel data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Excel, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the OneDrive Excel.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# OpenAI Ads Conversions as a destination
Source: https://docs.nekt.com/destinations/openai-ads-conversions
Send your conversions from Nekt to the OpenAI Ads Conversions API.
OpenAI Ads is OpenAI's advertising platform, where ads appear inside ChatGPT conversations and are managed in OpenAI Ads Manager. The Conversions API is its server-to-server measurement endpoint: instead of relying only on the browser pixel, you tell OpenAI directly which conversions happened, so attribution and reporting reflect what actually closed.
This destination sends conversions straight from your Lakehouse. That makes it the natural place for everything the pixel cannot see: a deal marked won in the CRM, a subscription that survived the trial, a sale closed over the phone or in a physical store.
If you also want to read your OpenAI Ads campaign and delivery data into the Lakehouse, see [OpenAI Ads as a data source](/sources/openai-ads).
## 1. Overview
Each row of your input table becomes one conversion event. Events travel in batches of up to 1,000, and personal identifiers (email addresses, phone numbers, customer IDs, names) are normalized and hashed with SHA-256 inside Nekt, so the raw values never leave your Lakehouse.
## 2. Important considerations
OpenAI only accepts conversions from the **last 7 days**, and no more than 10 minutes in the future. Rows outside that window are skipped and reported in the run log. This is a limit of the OpenAI Ads API: historical conversions cannot be backfilled, so schedule this destination to run at least once a week (daily is better).
* **Deduplication.** If the same conversion also reaches OpenAI through the pixel, send the same value in the pixel's `event_id` and in this destination's `event_id`, with the same Pixel ID. OpenAI keeps the first event it receives and ignores the later duplicate.
* **Match quality.** Every identifier you map raises the chance OpenAI can attribute the conversion. Mapping the email address alone already works; adding phone number, customer ID, name and location improves it.
* **Batch behavior.** OpenAI rejects an entire batch when a single event in it is invalid. With **Continue on error** enabled, Nekt splits a rejected batch in half and retries until the invalid rows are isolated, so a single bad row never costs you the 999 good ones travelling with it.
## 3. Configuring the destination
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click the "Add destination" button at the top right of your screen. Then select the OpenAI Ads - Conversions option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
Sign in at [ads.openai.com](https://ads.openai.com) with the ad account the conversions belong to.
Open the **Conversions** tab.
Take the Pixel ID of the data source you are measuring, then create a Conversions API key and copy its value. Both must belong to the same ad account.
This is not the same credential as the Ads API key used by the OpenAI Ads source and by the Custom Audiences destination. The Conversions API has its own key, issued from the Conversions tab.
The following configurations are available:
* **Conversions API key**: the key you copied from the Conversions tab.
* **Pixel ID**: the Pixel ID the conversions are attributed to.
* **Default action source**: where the conversion happened, used for rows whose table has no column for it. Choose **Offline** for conversions closed in a CRM, **Physical store** for in-person sales, **Phone call** for telesales.
* **Default source URL** (optional): the URL recorded for web conversions when the table has no column for it. OpenAI requires it whenever the action source is **Website**.
* **Validate only** (optional): send the events for validation without storing them, to confirm the column mapping before the first real run.
* **Continue on error**: keep syncing the remaining rows after OpenAI refuses one, isolating the invalid rows instead of failing the run.
* **Dry run**: log what would be sent without sending anything.
3. Click **Next**.
## 4. Select your data to send
1. Select the layer and then the table holding the conversions you want to send.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 5. Map your data to the destination
### Required fields
| Field name | Description |
| :- | :- |
| event\_id | Your identifier for the conversion, such as the order ID. Reuse the same value when the same conversion is resent, so OpenAI counts it once. |
| event\_type | What happened. See the supported event types below. |
| event\_time | When the conversion happened. A timestamp column, an ISO 8601 string or a Unix epoch all work. Must be within the last 7 days. |
The supported event types are `order_created`, `checkout_started`, `items_added`, `contents_viewed`, `page_viewed`, `lead_created`, `appointment_scheduled`, `registration_completed`, `subscription_created`, `trial_started` and `custom`. Native app events also support `app_installed` and `app_opened`.
### Conversion value
| Field name | Description | Required |
| :- | :- | - |
| amount | The value of the conversion, as an integer in the currency's minor unit: `4290` means R\$ 42,90. If your column holds a decimal, add the `to_minor_units` transform to the mapping and Nekt converts it. | No |
| currency | The three-letter ISO 4217 currency code, such as `BRL` or `USD`. Required whenever an amount is sent. | No |
| contents | The line items of the conversion, as a JSON array of objects. Each item accepts `id`, `group_id`, `name`, `content_type`, `quantity`, `amount`, `currency` and `variant_dict`. | No |
| plan\_id | Your identifier for the subscription or trial plan, for `subscription_created` and `trial_started`. | No |
### Matching the conversion to a person
Every field below is optional, and every one you map raises the chance OpenAI attributes the conversion. The first five are hashed by Nekt before being sent.
| Field name | Description |
| :- | :- |
| email | The email address of the customer. |
| phone\_number | The phone number of the customer, in any format. Nekt normalizes it to the international digits-only form OpenAI expects. |
| external\_id | Your own stable, pseudonymous customer identifier, such as a CRM ID. |
| first\_name | The first name of the customer. |
| last\_name | The last name of the customer. |
| city | The city of the customer, sent as a raw value. |
| region | The state or region of the customer, sent as a raw value. |
| postal\_code | The postal code of the customer, sent as a raw value. |
| country | The two-letter country code of the customer, such as `BR`. |
| ip\_address | The IP address recorded for the conversion. |
| user\_agent | The user agent recorded for the conversion. |
| obref | The browser reference from the pixel's `__obref` first-party cookie, passed through unchanged. |
### Other fields
| Field name | Description |
| :- | :- |
| action\_source | Where the conversion happened, overriding the default action source for this row: `web`, `mobile_app`, `offline`, `physical_store`, `phone_call`, `email` or `other`. |
| source\_url | The page the web conversion happened on, overriding the default source URL for this row. |
| custom\_event\_name | The name of your own event, required when the event type is `custom`. Use 1 to 64 letters, digits, underscores or hyphens. |
| data\_type | Overrides the data shape OpenAI expects for the event type. Leave it unmapped unless OpenAI support asks you to set it. |
| oppref | The attribution identifier OpenAI provided for the click, when you captured it. |
| android\_advertising\_id | The Google Advertising ID of the device, for app conversions. |
| opt\_out | Set to true to opt this conversion out of future user-level personalization. |
Once the fields are mapped, click **Next**.
## 6. Configure your OpenAI Ads Conversions destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), remember the 7-day window: a daily trigger keeps every conversion inside it comfortably.
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up.
2. You can view it on the [Destinations](https://app.nekt.ai/destinations) page and trigger the pipeline manually with the refresh icon. Once it has run, the conversions appear in the reporting of OpenAI Ads Manager, attributed to the campaigns that earned them.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# OpenAI Ads Custom Audiences as a destination
Source: https://docs.nekt.com/destinations/openai-ads-custom-audiences
Send your customer lists from Nekt to OpenAI Ads Custom Audiences.
OpenAI Ads is OpenAI's advertising platform, where ads appear inside ChatGPT conversations and are managed in OpenAI Ads Manager. Custom audiences let you use a customer or prospect list to control who can see those ads: you send the list of people, wait for OpenAI to match them, and then include or exclude the matched customers in a campaign, or raise your bid for them in an ad group.
This destination keeps an audience in step with a table in your Lakehouse. Whatever the table holds after each run (the customers who bought last quarter, the trials that never converted, the accounts a campaign should never reach) is what the audience contains.
If you also want to read your OpenAI Ads campaign and delivery data into the Lakehouse, see [OpenAI Ads as a data source](/sources/openai-ads).
## 1. Overview
The destination addresses the audience **by name**. On the first run it looks the name up on your ad account and creates the audience if it is not there yet, so there is no ID to copy between OpenAI and Nekt. Every run after that updates the same audience.
Each row of your input table contributes one identifier: an email address or a phone number. Rows with a blank value are skipped, duplicates within a run are sent once, and a value that is already a SHA-256 hash is detected and never hashed twice.
## 2. Important considerations
Custom audiences are not supported for campaigns targeting the European Economic Area (EEA) or Switzerland, where personalized ads are not yet available.
* **Minimum size.** An audience generally needs about **25,000 matched users** before OpenAI lets you use it for targeting or bid adjustments. Sending 25,000 identifiers does not guarantee that many matches. The audience's `ready` status in Ads Manager is the authoritative signal.
* **Processing time.** OpenAI takes roughly **20 to 30 minutes** to process a membership change before the audience is usable again.
* **Enabled per account.** Using an audience in campaign targeting, and applying ad-group bid multipliers, are switched on by OpenAI for each ad account. If either is unavailable, ask your OpenAI representative to enable it for the account.
* **First-party data only.** Only upload audience data you have the right to use for advertising, and never broker-sourced data. Confirm your use complies with the required rights, notices, consents and legal bases, and with OpenAI's [Ad Tools Terms](https://openai.com/policies/ad-tools-terms/), before sending anything.
## 3. Configuring the destination
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click the "Add destination" button at the top right of your screen. Then select the OpenAI Ads - Custom Audiences option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
Sign in at [ads.openai.com](https://ads.openai.com) with the ad account whose audience you want to keep in sync.
Open the **Settings** tab.
Create a new API key and copy the value.
This is not the same credential as an OpenAI API key from `platform.openai.com`. The Ads API uses its own key, issued from Ads Manager, and each key is tied to a single ad account.
The following configurations are available:
* **Ads API key**: the key you copied from Ads Manager.
* **Audience name**: the name of the audience to keep in sync. It is created on the first run if the ad account does not have it yet.
* **Identifier type**: the kind of identifier every row of your table carries. An audience holds a single type.
| Option | What Nekt sends |
| :- | :- |
| Email address | The address, trimmed and lowercased. OpenAI hashes it on arrival. |
| Phone number (E.164) | The number in E.164 form, such as `+5511987654321`. OpenAI hashes it on arrival. |
| Email address (hashed with SHA-256) | Only the hash of the normalized address. The raw address never leaves your Lakehouse. |
| Phone number (hashed with SHA-256) | Only the hash of the normalized number. The raw number never leaves your Lakehouse. |
* **Write mode**: how the table changes the audience.
| Option | Effect |
| :- | :- |
| Add the rows to the audience | The rows join the current membership. Anyone already in the audience stays. |
| Remove the rows from the audience | The rows are taken out of the audience. |
| Replace the whole audience with the rows | The audience ends up holding exactly the rows of this run, and nothing else. |
* **Audience description** (optional): applied to the audience when Nekt creates it.
* **Audience ID** (optional): the ID of an existing audience, used instead of the name lookup. Leave it empty unless two audiences share the same name.
* **Continue on error**: keep syncing the remaining rows after OpenAI refuses one. When off, the run stops at the first refused row.
* **Dry run**: log what would be written without writing anything, to validate a new column mapping.
Choosing an unhashed identifier type and letting OpenAI normalize the value gives the highest match rate, because OpenAI applies its own normalization rules before matching. Choose a hashed type when your policy is that raw personal data must not leave the Lakehouse.
3. Click **Next**.
## 4. Select your data to send
1. Select the layer and then the table holding the customer list you want to send.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 5. Map your data to the destination
This destination has a single field. Map it to the column of your table that holds the identifier.
| Field name | Description | Required |
| :- | :- | - |
| identifier | The email address or phone number of one person, matching the identifier type selected in the configuration. | Yes |
Once the field is mapped, click **Next**.
## 6. Configure your OpenAI Ads Custom Audiences destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently the audience needs to reflect changes in the source table.
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up.
2. You can view it on the [Destinations](https://app.nekt.ai/destinations) page. For the audience to appear in Ads Manager, wait for the pipeline to run (you can trigger it manually with the refresh icon). Remember that OpenAI then takes 20 to 30 minutes to finish processing the membership before the audience can be used.
A full replacement is never applied from an empty result. If the input table produces no usable identifiers, the audience is left untouched, so a failed upstream run cannot empty an audience your campaigns are targeting.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# OpenAI Ads Product Feed as a destination
Source: https://docs.nekt.com/destinations/openai-ads-product-feed
Keep prices and availability in your OpenAI Ads product feed up to date from Nekt.
OpenAI Ads is OpenAI's advertising platform, where ads appear inside ChatGPT conversations and are managed in OpenAI Ads Manager. Product-feed campaigns advertise items from a merchant catalog linked to your ad account, and the Delta Feeds API lets you update that catalog incrementally, sending only the products that changed instead of re-uploading everything.
This destination points that endpoint at a table in your Lakehouse, so the price and stock level your ads show follow the same numbers your business runs on.
If you also want to read your OpenAI Ads campaign and delivery data into the Lakehouse, see [OpenAI Ads as a data source](/sources/openai-ads).
## 1. Overview
Each row of your input table is one product variant to update. Rows are grouped by parent product before being sent, and a variant that appears twice in the same run is sent once, with the last row winning.
## 2. Important considerations
This destination **updates existing variants only**. It never creates products and never adds items that are not already in the feed. The initial catalog has to be uploaded to OpenAI first. See OpenAI's [Product Feeds](https://developers.openai.com/ads/product-feeds) guide.
* **Enabled per account.** The product feed API is switched on by OpenAI for each ad account. If the destination reports that the feed API is unavailable, ask your OpenAI representative to enable it for the account.
* **Acceptance is not delivery.** OpenAI acknowledges the update as accepted by feed processing; the change then propagates to serving asynchronously. Marking a product in stock does not by itself make it serve: the product, campaign, ad group and ad still have to meet the normal serving requirements.
* **Prices are integers.** Every monetary value in OpenAI Ads is an integer in the currency's minor unit: `8999` means R\$ 89,99. If your column holds a decimal, add the `to_minor_units` transform to the mapping and Nekt converts it.
## 3. Configuring the destination
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click the "Add destination" button at the top right of your screen. Then select the OpenAI Ads - Product Feed option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
Sign in at [ads.openai.com](https://ads.openai.com) with the ad account the feed is linked to.
Open the **Settings** tab and issue an API key, then copy the value.
Take the ID of the product feed you want to keep up to date.
This is not the same credential as an OpenAI API key from `platform.openai.com`. The Ads API uses its own key, issued from Ads Manager, and each key is tied to a single ad account.
The following configurations are available:
* **Ads API key**: the key you copied from Ads Manager.
* **Feed ID**: the product feed to update. It must be linked to the same ad account as the key.
* **Default currency**: the ISO 4217 currency code used for prices when your table has no currency column, such as `BRL`.
* **Continue on error**: keep syncing the remaining rows after OpenAI refuses one. When off, the run stops at the first refused row.
* **Dry run**: log what would be written without writing anything.
3. Click **Next**.
## 4. Select your data to send
1. Select the layer and then the table holding the products you want to keep up to date.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 5. Map your data to the destination
| Field name | Description | Required |
| :- | :- | - |
| product\_id | The parent product identifier, exactly as it exists in the feed. | Yes |
| variant\_id | The variant or item identifier, exactly as it exists in the feed. | Yes |
| title | The updated title of the variant. | No |
| price\_amount | The updated price, as an integer in the currency's minor unit. Use the `to_minor_units` transform if your column holds a decimal. | No |
| price\_currency | The three-letter ISO 4217 currency code for the price. Falls back to the default currency when not mapped. | No |
| availability | Whether the variant is in stock. Accepts `in_stock` and `out_of_stock`, as well as the usual spellings a warehouse column carries: true/false, yes/no, 1/0, available/unavailable, sold out. | No |
Map at least one of `title`, `price_amount` or `availability`. A row carrying only the identifiers has nothing for OpenAI to change and is skipped.
Once the fields are mapped, click **Next**.
## 6. Configure your OpenAI Ads Product Feed destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how quickly a price change or a stockout has to reach your ads.
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up.
2. You can view it on the [Destinations](https://app.nekt.ai/destinations) page and trigger the pipeline manually with the refresh icon. Use your normal feed and campaign monitoring in Ads Manager to confirm the change reached serving: the acceptance response carries no completion timestamp.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# OpenAI Vector Store as a destination
Source: https://docs.nekt.com/destinations/openai-vector-store
Send your data from Nekt to OpenAI Vector Store.
OpenAI Vector Store is OpenAI's platform for storing and searching vector embeddings used in AI applications. It provides tools for semantic search, similarity matching, and AI-powered data retrieval, enabling advanced machine learning and natural language processing capabilities.
## 1. Add your OpenAI Vector Store access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the OpenAI Vector Store option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access: API key, Vector store ID. Also, adjust any configurations as needed.
> Tip: Make sure the API key and Vector store ID are under the same OpenAI project.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the OpenAI Vector Store. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your OpenAI Vector Store data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Spreadsheet, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the Spreadsheet.
> If you encounter any issues, reach out to us via Slack or WhatsApp, and we'll gladly assist you!
# Outlook as a destination
Source: https://docs.nekt.com/destinations/outlook
Send your data from Nekt to Outlook.
Outlook is Microsoft's email and personal information management application that provides email, calendar, contact, and task management capabilities. It's part of Microsoft Office and offers integration with other Microsoft services for comprehensive business communication and organization.
## 1. Add your Outlook access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Outlook option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **Microsoft Authentication**: Authorize Nekt with your Outlook account. Make sure you are doing it with the email address you want to use to send the email.
* **Recipients**: A list of recipient emails separated by comma (,).
* **Email subject**: The subject of the email. If you want to add a datestamp to the subject, include the parameter `{datestamp}` in brackets.
* **Email body**: The email body message. If you want to include a preview of the data in the body, use the variable `{data_preview}` where you want to place it.
* **Maximum preview size**: The total number of rows to preview in the message body. Maximum is 100.
* **Attach table data**: Whether to send data attached in the email as .csv or .xlsx files. Maximum file size is 150MB.
* **Format of attached file**: In case you want to send data attached, decide the file format: CSV or Excel (.xlsx).
* **Filename**: The name of the file to be attached in the email. If you want to add a datestamp to the file name, include the parameter `{datestamp}` in brackets.
* **Apply filter to data preview**: Filter the rows that should be sent in the email body.
* **Send email with empty data**: Whether the email should be sent in case the input table is empty (due to filtering or other conditions).
* **Email body (empty data)**: The body message to be used when the input table is empty.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to attach to the email. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Outlook data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want this email to be sent to the list of recipients. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for the email to be actually sent, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your email should be sent.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Destinations
Source: https://docs.nekt.com/destinations/overview
Take your data wherever you need it.
Nekt helps you build your catalog with the data you need, in the format you need. After that, we help you to take that wherever you need to easily play with it.
## What is a destination?
A destination is the system where you aim to export data. This can be a third-party API, SaaS application, database, data warehouse, or data lake.
For example, you can configure a destination to export data from your Nekt catalog to a Google Spreadsheet or your CRM, such as HubSpot.
## What can I do with the Destinations feature at Nekt?
On this page, you'll find a comprehensive list of your destinations along with associated metadata. The Add destination button allows you to create new destinations and send table data to different systems.
Currently, you can export data to Google Sheets through our application. If you want to work with other destinations, please let us know, and we'll accommodate your needs!
## API & MCP
List, create, update, trigger, and delete destinations.
Browse with `list_resources`, and build destinations with the `create_draft` to `complete_pipeline` flow.
# Pinecone as a destination
Source: https://docs.nekt.com/destinations/pinecone
Send your data from Nekt to Pinecone.
Pinecone is a vector database designed for AI and machine learning applications. It provides tools for storing, searching, and managing vector embeddings, enabling semantic search, recommendation systems, and other AI-powered features that require similarity matching and vector operations.
## 1. Add your Pinecone access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Pinecone option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **API Key:** Your Pinecone API key, generated on the Pinecone platform.
![]()
* **Index name:** The Pinecone index where your data will be written. Your index is created on the Pinecone platform, and its dimension must match the embed model you select below (each model option indicates its dimension). For more details, refer to [Pinecone's documentation](https://docs.pinecone.io/guides/indexes/create-an-index).
![]()
* **Mode:** The sync mode to use when writing data:
* **Append:** Adds the incoming records to the existing data in the namespace.
* **Overwrite:** Clears the namespace before loading, so it will contain only the incoming records after the run.
* **Embed model:** The model used to generate the vector embeddings. You can choose between Pinecone-hosted models (e.g., `llama-text-embed-v2`, the default, and `multilingual-e5-large`), OpenAI models (e.g., `text-embedding-3-small`, `text-embedding-3-large`), and open-source models (e.g., `BAAI/bge-small-en-v1.5`, `sentence-transformers/all-MiniLM-L6-v2`). Each option indicates the embedding dimension it produces, which must match your index's dimension.
* **OpenAI API key:** Required only if you select an OpenAI embed model. This key is used to generate the embeddings on OpenAI's API.
3. Click **Next**.
Records are written to a namespace named after the input table, inside the index you configured.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to Pinecone. Select the tables or even entire layers, sources or transformations to send more than one table.
> Tip: The resource can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data fields
Configure how the fields from your source table should be mapped to the Pinecone record fields:
| Field | Required | Description |
| :- | :- | :- |
| `id` | Yes | The unique identifier of the vector. |
| `text` | Yes | The text content that will be embedded. |
| `metadata` | No | Additional metadata stored alongside the vector, useful for filtering. |
Once you are done configuring, click **Next**.
## 4. Configure your Pinecone data destination
1. Describe your destination for easy identification within your workspace. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Pinecone platform, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Pinecone.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Pipefy as a destination
Source: https://docs.nekt.com/destinations/pipefy
Send your data from Nekt to Pipefy.
Pipefy is a workflow management platform that helps teams automate and optimize business processes using customizable pipes, cards, and phases. With the Pipefy destination, you can create new cards, update existing ones, or perform bulk field updates directly from your Nekt data.
## 1. Add your Pipefy access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Pipefy option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Access Token**: Your Pipefy authentication token. You can use either a Personal Access Token (PAT) or a Service Account token.
* **Endpoint**: The operation to perform. See the details for each endpoint below:
* `CreateCard` creates new cards in the pipe.
* `UpdateCard` updates existing cards (metadata and custom fields).
* `BatchUpdateCardField` performs bulk updates on custom fields across many cards in a single request. Recommended for high-volume scenarios (e.g. updating a lead score on 100k+ cards).
* **Pipe ID**: The ID of the Pipefy pipe where cards will be created or updated. You can find this in the pipe URL (e.g. `https://app.pipefy.com/pipes/305737084` means the Pipe ID is `305737084`).
* **Batch Size** *(advanced, BatchUpdateCardField only)*: The number of field updates to include in each API request. Default is `100`. Higher values reduce the total number of requests but increase payload size.
* **Enable partial success**: Whether the connector should continue trying to sync records after one fails. Recommended for large batch operations.
To generate a Personal Access Token, go to your Pipefy account settings, navigate to **Personal access tokens**, and create a new token. For production integrations, Pipefy recommends using Service Accounts instead.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your data fields
Configure how the fields from your source table should be mapped to the fields in the selected endpoint. Below you can find the supported fields for each endpoint.
### Understanding Pipefy field types
Pipefy cards have two categories of fields:
* **Card-level fields**: Built-in fields like `title`, `due_date`, `assignee_ids`, `label_ids`, and `phase_id`. These are common to all pipes.
* **Custom fields**: Fields defined in the pipe's configuration (e.g. "Nome completo", "CPF", "Renda mensal"). Each custom field has an internal **field ID** (e.g. `nome_completo`, `valor_do_cr_dito`) that you must use as the destination field name in the mapping.
To find a custom field's internal ID, go to the pipe settings in Pipefy and check the field configuration. The ID is displayed under the field name. Alternatively, you can use the Pipefy GraphQL API to query the pipe's `start_form_fields` or phase `fields`.
Creates a new card in the specified pipe. Card-level fields are set directly on the card, and custom fields are set via the `fields_attributes` parameter in a single API call.
**Card-level fields**
| Field | Type | Required | Description |
| :- | :- | :- | :- |
| `title` | String | No | The title of the card. |
| `due_date` | DateTime | No | The due date for the card. |
| `phase_id` | String | No | The ID of the phase to place the card in. If omitted, the card is placed in the first phase. |
| `assignee_ids` | Array of Strings | No | The IDs of users to assign to the card. |
| `label_ids` | Array of Strings | No | The IDs of labels to apply to the card. |
**Custom fields**
Any destination field name that is not a card-level field is treated as a custom field ID. For example, if your pipe has a field called "Nome completo" with internal ID `nome_completo`, map your source column to destination field `nome_completo`.
| Destination field | Type | Description |
| :- | :- | :- |
| *Any field ID* | String | Maps to a custom pipe field. The value is sent as a string. |
The pipe may have required fields configured in Pipefy. If you omit a required field, the API will reject the card creation. Check your pipe's form settings to identify which fields are mandatory.
**Connector fields** (fields that link to cards in another pipe) require special formatting. The value must reference valid connected card IDs or labels as expected by the Pipefy API. Check the connected pipe for valid values.
Updates an existing card. Card-level fields (title, due date, assignees, labels) are updated in a single API request via the `updateCard` mutation. Custom fields are updated individually via the `updateCardField` mutation (one API call per field).
**Card-level fields**
| Field | Type | Required | Description |
| :- | :- | :- | :- |
| `id` | String | **Yes** | The Pipefy card ID to update. You can obtain this from the Pipefy source connector or the Pipefy UI. |
| `title` | String | No | The new title of the card. |
| `due_date` | DateTime | No | The new due date for the card. |
| `assignee_ids` | Array of Strings | No | The new list of assignee IDs. |
| `label_ids` | Array of Strings | No | The new list of label IDs. |
**Custom fields**
Any destination field name that is not a card-level field (and not `id`) is treated as a custom field ID and updated via the `updateCardField` mutation.
| Destination field | Type | Description |
| :- | :- | :- |
| *Any field ID* | String | Maps to a custom pipe field. Updated one at a time via the API. |
Since each custom field requires a separate API call, updating many fields on many cards can be slow. If you only need to update a single field across many cards, consider using `BatchUpdateCardField` instead.
Performs bulk updates on a single custom field across many cards. Instead of sending one API call per card, this endpoint batches multiple `updateCardField` mutations into a single GraphQL request using aliases.
This is the recommended endpoint for high-volume operations like updating a lead score, status, or any computed field across thousands of cards.
**Performance comparison**
| Scenario (100,000 cards) | UpdateCard | BatchUpdateCardField |
| :- | :- | :- |
| API requests | 100,000 | \~1,000 (at batch\_size=100) |
| Estimated time | Hours | Minutes |
**Required fields**
| Field | Type | Required | Description |
| :- | :- | :- | :- |
| `card_id` | String | **Yes** | The Pipefy card ID to update. |
| `field_id` | String | **Yes** | The internal ID of the custom field to update (e.g. `renda_mensal`). |
| `new_value` | String | **Yes** | The new value for the field. |
**Source table format**
Your source table should have **one row per field update**. For example, to update the `lead_score` field on three cards:
| card\_id | field\_id | new\_value |
| :- | :- | :- |
| 301234 | lead\_score | 85 |
| 301235 | lead\_score | 42 |
| 301236 | lead\_score | 97 |
If all rows update the same field, you can use a **default value** in the mapping to avoid storing the field ID in your source table. Map the `field_id` destination to an empty origin field with `lead_score` as the default value. Then your source table only needs `card_id` and the score value.
To update **multiple fields per card**, include one row per card-field combination:
| card\_id | field\_id | new\_value |
| :- | :- | :- |
| 301234 | lead\_score | 85 |
| 301234 | renda\_mensal | 25.000,00 |
| 301235 | lead\_score | 42 |
| 301235 | renda\_mensal | 10.000,00 |
### Common Pipefy field types and value formats
When mapping values to Pipefy custom fields, use the format that Pipefy expects for each field type:
| Pipefy field type | Expected value format | Example |
| :- | :- | :- |
| `short_text` | Plain string | `John Doe` |
| `email` | Valid email string | `john@example.com` |
| `phone` | Digits only (no formatting) | `11999999999` |
| `cpf` | Formatted CPF | `529.982.247-25` |
| `number` | Numeric string | `42` |
| `currency` | Brazilian format with comma as decimal separator | `25.000,00` |
| `date` | `DD/MM/YYYY` | `15/03/1990` |
| `select` | One of the allowed option values | `Sim` |
| `radio_vertical` | One of the allowed option values | `Sim` |
| `checklist_vertical` | Comma-separated values | `Banco A, Banco B` |
## 4. Configure your Pipefy data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Pipefy, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Pipefy.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# PostgreSQL as a destination
Source: https://docs.nekt.com/destinations/postgres
Send your data from Nekt to PostgreSQL.
PostgreSQL is a powerful open-source object-relational database system that extends the SQL language with advanced features. It's known for its reliability, data integrity, and support for complex queries, making it ideal for enterprise applications, data warehousing, and web services.
![]()
## Configuring PostgreSQL as a Destination
In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Postgres option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll be prompted to provide your connection details:
* **Host**: The hostname or IP address of your PostgreSQL server.
* **Port**: The port number your PostgreSQL server is listening on (default is usually 5432).
* **User**: The username for authenticating with the database.
* **Password**: The password for the specified user.
* **Database Name**: The name of the database you want to write data to.
* **Target Schema**: The schema within the database where tables will be created.
Define your load method: append, upsert, or overwrite.
* **Append:** This method will simply add all records from the incoming dataset to the existing table at the destination, without removing or updating any existing data. You can only use this method if a primary key was not defined for the input table.
* **Upsert:** This method will check for records with matching IDs in the destination table. If a match is found, the existing record will be updated with the new data. If no match is found, the new record will be inserted. In other words, it merges the incoming dataset with the existing table based on the record ID.
* **Overwrite:** This method will completely replace all existing data in the destination table with the incoming dataset. Any data that was previously in the table will be deleted.
Decide on the following advanced settings:
* **Enable SSH Tunnel**: When this is turned on, the system will route the connection through the configured SSH tunnel to reach the PostgreSQL database. The SSH tunnel is an intermediary machine required when the PostgreSQL is not directly accessible over the internet. You can authenticate using either an **SSH password** or an **SSH private key**, both of which are treated as sensitive credentials and stored securely.
* **Enable Interactive Tunnel Authentication**: In some environments, logging into the SSH tunnel may require interactive authentication steps. Enabling this option allows the connection process to simulate such interactive behavior, which is sometimes necessary for the tunnel to be established successfully.
We do not recommend enabling it initially. If your pipeline fails, edit the destination and activate this option to try again.
* **Enable SSL**: Activates SSL encryption for the database connection. This is required when the PostgreSQL server enforces secure connections using SSL certificates. It ensures encrypted communication between your system and the database, improving security.
* **Add record metadata**: If active, it will add metadata columns with timestamps related to the loading process. Read more about each column below.
* `_sdc_extracted_at`: Timestamp indicating when record was extracted from the source
* `_sdc_received_at`: Timestamp indicating when the destination received the record for loading
* `_sdc_batched_at`: Timestamp indicating when the destination loaded the batch the record was a part of into the destination
* `_sdc_sequence`: A Unix epoch (in nanoseconds) that indicates the order in which data points were considered for loading
* `_sdc_table_version`: Indicates the version of the table. Used to determine when to issue TRUNCATE commands during loading, where applicable.
Click **Next**.
### 2. Select your data to send
1. Let us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Define which column from the selected table should be used as the primary key on the destination database.
3. Click **Next**.
### 3. Configure your data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### 4. Check your new destination
Once completed, you'll receive confirmation that your new destination is set up!
You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. To see the data in your PostgreSQL database, wait for the pipeline to run. You can monitor its execution and completion on the Destinations page. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed successfully, your data will appear in PostgreSQL.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Implementation Notes
### Table and Schema Naming
The connector fully supports mixed-case identifiers for schema and table names. Identifiers are properly quoted using dialect-specific formatting during all database operations, including `TRUNCATE` commands executed during **Overwrite** syncs or table version updates. This ensures PostgreSQL maintains the exact casing provided (e.g., `SA3010` does not get folded to lowercase) and prevents "relation not found" errors.
# RD Station CRM as a destination
Source: https://docs.nekt.com/destinations/rd-station-crm
Send your data from Nekt to RD Station CRM.
RD Station CRM is a sales management platform that helps businesses organize and track their sales pipeline. It provides tools for managing contacts, deals, organizations, and sales activities to improve conversion rates and revenue.
The RD Station CRM destination supports three operating modes:
* **Contacts** - creates or updates contacts in the CRM.
* **Deals** - creates or updates deals (sales opportunities) in the CRM.
* **Organizations** - creates or updates organizations (companies) in the CRM.
## 1. Add your RD Station CRM access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the RD Station CRM option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Authorization**: Complete RD Station authorization to grant permissions for Nekt to load data into the CRM.
* **Mode**: Select the operating mode for the connector:
* **Create/Update Contacts** - maps your input table columns to contact fields and creates or updates contacts.
* **Create/Update Deals** - maps your input table columns to deal fields and creates or updates deals.
* **Create/Update Organizations** - maps your input table columns to organization fields and creates or updates organizations.
* **Enable partial success**: Whether the connector should continue trying to sync records after one fails.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your columns
Map the columns from your input table to the corresponding RD Station CRM fields. The available destination fields depend on the mode you selected.
### Create vs. update behavior
If you map a column to the `_id` destination field, the connector performs an update (PUT) on the existing record with that ID. If no `_id` is mapped, the connector creates a new record (POST).
### Custom fields
RD Station CRM supports custom fields on contacts, deals, and organizations. Any destination field that is not in the standard fields list is automatically sent inside the `custom_fields` object in the API payload. Simply use the custom field slug as the destination field name (e.g., `cpf`, `erp_id`).
### Contacts mode
In contacts mode, each row in your input table represents a contact to create or update.
Available standard destination fields:
| Destination field | Description |
| :- | :- |
| `_id` | CRM contact ID (for updates only) |
| `name` | Contact full name |
| `job_title` | Contact job title |
| `birthday` | Contact birthday (YYYY-MM-DD format) |
| `emails` | Array of email objects (e.g., `[{"email": "...", "type": "work"}]`) |
| `phones` | Array of phone objects (e.g., `[{"phone": "...", "type": "work"}]`) |
| `social_profiles` | Array of social profile objects |
| `organization_id` | ID of the associated organization |
Any other field (e.g., `cpf`, `erp_id`) is sent as a custom field.
**Example input table (contacts mode):**
| full\_name | contact\_email | phone\_number | cpf |
| :- | :- | :- | :- |
| Maria Silva | [maria@example.com](mailto:maria@example.com) | +55 11 91234-5678 | 111.111.111-11 |
| Joao Santos | [joao@example.com](mailto:joao@example.com) | +55 21 99876-5432 | 222.222.222-22 |
Column mapping: `full_name` -> `name`, `contact_email` -> `emails`, `phone_number` -> `phones`, `cpf` -> `cpf`.
In this example, `cpf` is automatically sent inside `custom_fields` in the API payload.
### Deals mode
In deals mode, each row in your input table represents a deal to create or update.
Available standard destination fields:
| Destination field | Description |
| :- | :- |
| `_id` | CRM deal ID (for updates only) |
| `name` | Deal name **(required for creation)** |
| `status` | Deal status (must be `ongoing` for creation) |
| `contact_id` | ID of the associated contact |
| `organization_id` | ID of the associated organization |
| `stage_id` | ID of the deal stage |
| `pipeline_id` | ID of the sales pipeline |
| `owner_id` | ID of the deal owner (user) |
| `campaign_id` | ID of the associated campaign |
| `expected_close_date` | Expected close date (ISO 8601 format) |
| `rating` | Numeric rating/classification |
| `lost_reason_id` | ID of the loss reason |
Any other field is sent as a custom field.
When creating deals, the `status` field must be set to `ongoing`. The RD Station CRM API only allows creating deals with this status.
**Example input table (deals mode, create):**
| deal\_name | org\_id | stage | amount |
| :- | :- | :- | :- |
| Credito PJ 001 | 6621a1b2c3d4e5f678905678 | 6621a1b2c3d4e5f678900001 | 50000 |
| Credito PF 002 | 6621a1b2c3d4e5f678905679 | 6621a1b2c3d4e5f678900001 | 15000 |
Column mapping: `deal_name` -> `name`, `org_id` -> `organization_id`, `stage` -> `stage_id`, `amount` -> `valor_credito`.
In this example, `valor_credito` is automatically sent inside `custom_fields` in the API payload.
### Organizations mode
In organizations mode, each row in your input table represents an organization to create or update.
Available standard destination fields:
| Destination field | Description |
| :- | :- |
| `_id` | CRM organization ID (for updates only) |
| `name` | Organization name |
| `owner_id` | ID of the organization owner (user) |
| `segment_ids` | Segment IDs to associate |
Any other field is sent as a custom field.
**Example input table (organizations mode):**
| company\_name | company\_website | cnpj |
| :- | :- | :- |
| Empresa ABC Ltda | [https://empresaabc.com.br](https://empresaabc.com.br) | 11.111.111/0001-11 |
| Tech Corp SA | [https://techcorp.com.br](https://techcorp.com.br) | 22.222.222/0001-22 |
Column mapping: `company_name` -> `name`, `company_website` -> `website`, `cnpj` -> `cnpj`.
In this example, both `website` and `cnpj` are automatically sent inside `custom_fields`.
## 4. Configure your RD Station CRM data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on RD Station CRM, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on RD Station CRM.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# RD Station Marketing as a destination
Source: https://docs.nekt.com/destinations/rd-station-marketing
Send your data from Nekt to RD Station Marketing.
RD Station Marketing is a marketing automation and lead generation platform that helps businesses attract, convert, and nurture leads. It provides tools for email marketing, landing page creation, lead scoring, and marketing analytics to improve conversion rates and customer acquisition.
The RD Station destination supports two operating modes:
* **Contacts**: creates or updates contacts via the RD Station Contacts API.
* **Qualification Events**: sends contact qualification events (opportunity, sale, or lost) via the RD Station Events API.
## 1. Add your RD Station Marketing access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the RD Station option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Authorization**: Complete RD Station authorization to grant permissions for Nekt to load data.
* **Mode**: Select the operating mode for the connector:
* **Create/Update Contacts**: maps your input table columns to RD Station contact fields and creates or updates contacts.
* **Qualification Events**: maps your input table columns to event fields and sends qualification events (opportunity, sale, or lost) to RD Station.
* **Enable partial success**: Whether the connector should continue trying to sync records after one fails.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your columns
Map the columns from your input table to the corresponding RD Station fields. The available destination fields depend on the mode you selected.
### Contacts mode
In contacts mode, each row in your input table represents a contact to create or update. The `email` field is required: it's used as the unique identifier for upserting contacts.
Available destination fields:
| Destination field | Description |
| :- | :- |
| `email` | Contact email address **(required)** |
| `name` | Contact full name |
| `job_title` | Contact job title |
| `birthdate` | Contact birthdate |
| `bio` | Contact biography |
| `website` | Contact website URL |
| `personal_phone` | Personal phone number |
| `mobile_phone` | Mobile phone number |
| `city` | City |
| `state` | State |
| `country` | Country |
| `twitter` | Twitter handle |
| `facebook` | Facebook profile |
| `linkedin` | LinkedIn profile |
| `tags` | Tags to apply to the contact |
| `legal_bases` | Legal bases for LGPD/GDPR compliance |
You must ensure the records in your input table have a column mapped to `email`, as it's required by the RD Station API to identify contacts.
Deduplicate records with the same email in your input table before sending to the destination. RD Station has a limit of 24 updates per contact in a 24-hour period. Exceeding this limit will cause the API to block further requests to that contact.
**Example input table (contacts mode):**
| email | full\_name | phone | company\_city |
| :- | :- | :- | :- |
| [alice@example.com](mailto:alice@example.com) | Alice Santos | +55 11 91234-5678 | São Paulo |
| [bob@example.com](mailto:bob@example.com) | Bob Silva | +55 21 99876-5432 | Rio de Janeiro |
| [carol@example.com](mailto:carol@example.com) | Carol Mendes | +55 31 98765-4321 | Belo Horizonte |
In this example, you would map: `email` → `email`, `full_name` → `name`, `phone` → `mobile_phone`, `company_city` → `city`.
### Qualification Events mode
In qualification events mode, each row in your input table represents a contact qualification event to send to RD Station. This is used to move contacts through your sales funnel by marking them as opportunities, won sales, or lost opportunities.
Available destination fields:
| Destination field | Description |
| :- | :- |
| `email` | Contact email address **(required)** |
| `event_type` | Type of qualification event **(required)**. Must be one of: `OPPORTUNITY`, `SALE`, `OPPORTUNITY_LOST` |
| `funnel_name` | Name of the sales funnel (optional, defaults to `default`) |
| `reason` | Reason for losing the opportunity (optional, only applied to `OPPORTUNITY_LOST` events) |
| `sale_amount` | Monetary value of the sale (optional, only applied to `SALE` events) |
Both `email` and `event_type` are required for every record. Records missing either field will cause the pipeline to fail (unless partial success is enabled).
**Example input table (qualification events: opportunity):**
Send contacts as new opportunities in the "Inbound" funnel:
| contact\_email | type | funnel |
| :- | :- | :- |
| [alice@example.com](mailto:alice@example.com) | OPPORTUNITY | Inbound |
| [bob@example.com](mailto:bob@example.com) | OPPORTUNITY | Inbound |
| [carol@example.com](mailto:carol@example.com) | OPPORTUNITY | Outbound |
Column mapping: `contact_email` → `email`, `type` → `event_type`, `funnel` → `funnel_name`.
**Example input table (qualification events: sale):**
Mark contacts as won deals with their sale amounts:
| contact\_email | type | funnel | amount |
| :- | :- | :- | :- |
| [alice@example.com](mailto:alice@example.com) | SALE | Inbound | 4500.00 |
| [bob@example.com](mailto:bob@example.com) | SALE | Inbound | 12000.00 |
Column mapping: `contact_email` → `email`, `type` → `event_type`, `funnel` → `funnel_name`, `amount` → `sale_amount`.
**Example input table (qualification events: lost):**
Mark contacts as lost opportunities with a reason:
| contact\_email | type | funnel | lost\_reason |
| :- | :- | :- | :- |
| [carol@example.com](mailto:carol@example.com) | OPPORTUNITY\_LOST | Outbound | Chose a competitor |
| [dave@example.com](mailto:dave@example.com) | OPPORTUNITY\_LOST | Inbound | Budget constraints |
Column mapping: `contact_email` → `email`, `type` → `event_type`, `funnel` → `funnel_name`, `lost_reason` → `reason`.
**Example input table (qualification events: mixed):**
You can also send mixed event types in a single table:
| contact\_email | type | funnel | lost\_reason | amount |
| :- | :- | :- | :- | :- |
| [alice@example.com](mailto:alice@example.com) | OPPORTUNITY | Inbound | | |
| [bob@example.com](mailto:bob@example.com) | SALE | Inbound | | 8500.00 |
| [carol@example.com](mailto:carol@example.com) | OPPORTUNITY\_LOST | Outbound | No budget | |
Column mapping: `contact_email` → `email`, `type` → `event_type`, `funnel` → `funnel_name`, `lost_reason` → `reason`, `amount` → `sale_amount`. Fields that don't apply to a given event type (e.g. `reason` for a `SALE` row) are ignored.
## 4. Configure your RD Station data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on RD Station, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on RD Station.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Redshift as a destination
Source: https://docs.nekt.com/destinations/redshift
Send your data from Nekt to Redshift.
Redshift is AWS's fully managed, petabyte-scale data warehouse service designed for analytics and business intelligence. It provides fast query performance, columnar storage, and parallel processing capabilities for large-scale data analysis and reporting.
![]()
## 1. Add your Redshift access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Redshift option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access credentials and destination details. The following configurations are available:
* **Host**: The host name of your Redshift instance (e.g., `my-workgroup.127631541.us-east-1.redshift-serverless.amazonaws.com`).
* **Port**: The port used for connecting to your Redshift instance. The default port for Redshift is 5439, but your port might be different.
* **Database**: The name of the database you want to send data to inside your Redshift instance.
* **User**: User name used to authenticate against your Redshift database.
* **Password**: Password used to authenticate against your Redshift database.
* **Schema name**: The schema name inside your database where you want to send your data.
* **Table name**: Redshift table to send data to. If you don't define one, data will be sent to a table with the same name as the input table.
* **Copy role ARN**: Redshift copy role ARN to use for the COPY command from S3. For detailed instructions on creating and getting this ARN, [check the AWS documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/default-iam-role.html#create-iam).
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to Redshift. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Redshift data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Redshift account, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Redshift.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Parquet AWS S3 as a destination
Source: https://docs.nekt.com/destinations/s3-parquet
Send your data from Nekt to AWS S3 as Parquet files.
Parquet is a columnar storage format optimized for analytics, offering efficient compression and fast reads. This destination writes your Lakehouse tables to an Amazon S3 bucket as compressed Parquet datasets, ready to be consumed by Athena, Spark, pandas, or any Parquet-compatible tool.
## 1. Add your AWS S3 access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Parquet (AWS S3) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
### Required Configuration
* **Bucket name:** The AWS S3 bucket name where you want to store the Parquet files.
### AWS Authentication
Three authentication methods are supported:
1. **Access keys**: provide an **Access key ID** and **Secret access key** created in the AWS IAM console.
2. **IAM role**: provide an **Assume IAM Role ARN** (e.g. `arn:aws:iam::123456789012:role/MyRole`). Nekt assumes the role via STS with automatically refreshed credentials, so you don't need to share access keys.
3. **None**: leave all authentication fields empty to use the AWS account where your pipelines run (e.g. for Nekt-managed buckets, pattern: `nekt-lakehouse-{company-name}`).
To create AWS credentials for accessing your S3 bucket:
1. Log in to your [AWS IAM Console](https://console.aws.amazon.com/iam/)
2. Navigate to **Users** and select your user (or create a new one)
3. Go to the **Security credentials** tab
4. Click **Create access key**
5. Select **Application running outside AWS** or **Other** as the use case
6. Copy the **Access Key ID** and **Secret Access Key** (you won't be able to see the secret again)
**Required IAM Permissions:**
Your AWS user/role needs the following S3 permissions:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::your-bucket-name",
"arn:aws:s3:::your-bucket-name/*"
]
}
]
}
```
`s3:DeleteObject` is required because each sync replaces the table folder contents.
### File Configuration
* **Output folder path:** Folder path where to store the output tables inside the S3 bucket. Leave empty if the tables should be stored in the bucket's root folder.
* Example: `data/exports` or `reports/daily`
* **Table folder name:** A custom folder name for the output table. If empty, the name of the table being sent is used. When several tables are sent, use the per-table folder name instead.
* **Compression:** (Advanced, default: `snappy`) Compression codec used when writing the Parquet files (`snappy` or `gzip`).
3. Click **Next**.
### Output layout
Each table is written as a Parquet dataset:
```
s3://{bucket}/{output folder path}/{table folder}/_nekt_sync_at={sync timestamp}/*.parquet
```
When sending several tables to the same destination, each one is written to its own folder (named after the stream unless overridden by the per-table folder name setting).
Each sync replaces the table folder contents, so the destination always mirrors the current state of the table in Nekt. The `_nekt_sync_at` partition carries the UTC timestamp of the sync that wrote the data.
Two tables can never share the same folder. Since every sync wipes the table folder before writing, sharing a folder would cause the tables to erase each other. The pipeline will refuse to start if multiple tables are configured to use the same output folder.
## 2. Select your data to send
1. The next step is letting us know which data you want to export to S3. Select the layer and then the desired table(s).
> Tip: Tables can be found more easily by typing their names.
2. Click **Next**.
## 3. Configure your Parquet (AWS S3) data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it exports, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated in S3 (every day, once a week, only at specific times, etc.).
3. Click **Done**.
## 4. Check your new destination
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. The pipeline will run according to your configured trigger schedule. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion.
3. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your Parquet files will be available in your S3 bucket at the path you specified.
You can query the exported datasets directly with Athena, Spark, DuckDB, or pandas. They all read the Hive-style `_nekt_sync_at` partitioning natively.
Make sure your S3 bucket has the appropriate permissions and policies configured to prevent unauthorized access to your data.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Implementation Notes
### Error Logging
When a batch of records fails to be processed, the destination logs diagnostic information to help identify the issue. To ensure that error logs are reliably captured without exceeding size limits (which could otherwise cause the entire error traceback to be dropped by the logging system), the connector bounds the diagnostic dump:
* **Sample records**: Logs a sample of up to 5 records from the failed batch.
* **Record truncation**: Each logged record is truncated if it exceeds 4,000 characters.
* **Schema truncation**: The table schema is truncated if it exceeds 64,000 characters.
This guarantees that the core exception reaches the logs and provides actionable context for debugging.
# Slack as a destination
Source: https://docs.nekt.com/destinations/slack
Send your data from Nekt to Slack.
Slack is a team collaboration and communication platform that enables real-time messaging, file sharing, and team coordination. It organizes conversations into channels and provides tools for workplace communication, project management, and team collaboration.
![]()
## 1. Introduction
The Slack destination allows you to send your data directly into a Slack channel or direct message of your choice.
It allows for automated workflows to share data with your team in a flexible and efficient way. We allow you to customize a message using Slack's markup language, and sending the input data attached as a CSV file or an image.
The result would look like this:
![]()
In order to propertly use the connector, there are a few pre-requisites that need to be fullfilled:
1. You should install the Nekt bot and grant permission for accessing and sending messages to your channels.
2. You should add the Nekt bot to the channel you want to send messages to (or obtain the person's Slack member ID if you want to send a direct message).
We explain it in more detail in the following section.
## 2. Configure the Slack destination
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Slack option from the list of connectors.
2. Click **Next** and you'll be prompted to add the connector configuration.
* **Authorization**: first of all, authorize Nekt through the **Slack Authorization** button. You must be an App Manager to authorize the installation of our app into your workspace.
You'll see the following warning when installing the app:
`This app is not approved by Slack. Apps are reviewed to ensure a quality experience.`
This occurs because our app is private and not yet available in the Slack marketplace.
* **Channel or person ID**: The ID of the channel you want to send the message to. To send a direct message to one person instead, use their Slack member ID (it starts with `U` or `W`): open their profile in Slack, click the three dots and choose 'Copy member ID'. Do not use the `D` ID taken from your own conversation with them: that conversation belongs to you, and the bot cannot post in it.
The channel ID can be found by right-clicking the channel name and selecting 'View details'. The channel ID will be available at the bottom of the modal.
![]()
> **Note that the Slack app must be a member of the channel to be able to send messages. This is how to do it:**
>
> 1. Type the command `/add` in the message input box of the channel you want to add the bot.
> 2. Choose `Add apps to your channel`.
> 3. Select the `Nekt Destination` app.
> 4. You're all set!
* **Message body**: The message to send along with the attached file. You can use Slack markup language to customize and style your message using emojis, different font styles, breaking lines, and so on. Please check the [official Slack documentation](https://slack.com/help/articles/202288908-Format-your-messages) for more information. Use `{datestamp}` to inject the current date (format configurable via the Date format field).
* **Thread reply** (optional): When filled, this text is posted as a reply in the thread of the main message above. Supports the same markup and the same `{datestamp}` and `{field_name}` variables as the message body. Leave it empty to send a single message, as before.
* **Where to share the file** (optional): Choose whether the file is shared as its own new message in the conversation, or as a reply in the thread of the message above. Defaults to **As a new message**.
* **Date format** (optional): Controls the format of the `{datestamp}` variable used in the message body. Defaults to `%Y-%m-%d`. Uses Python `strftime` syntax. For the full list of format codes, see the [Python documentation](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes).
Common examples:
| Format string | Result |
| - | - |
| `%Y-%m-%d` | 2026-03-12 |
| `%d/%m/%Y` | 12/03/2026 |
| `%B %d, %Y` | March 12, 2026 |
| `%m-%d-%Y` | 03-12-2026 |
This field is located under **Advanced Settings** in the destination configuration.
* **Single record mode** (optional): Whether to send a single record message body. If enabled, it will only accept tables composed of a single record, where the columns from the input table can be referenced in the message using `{field_name}`.
Assuming you have a sales KPIs table like this:
| total\_revenue | total\_orders | average\_order\_value | conversion\_rate |
| - | - | - | - |
| 12500 | 250 | 50 | 3.2 |
Your message body looks like this:
```
:bar_chart: *Sales Report for {datestamp}*
:moneybag: Total Revenue: ${total_revenue}
:package: Total Orders: {total_orders}
:dart: Average Order Value: ${average_order_value}
:chart_with_upwards_trend: Conversion Rate: {conversion_rate}%
Great job team! :rocket:
```
The resulting Slack message will be:
```
📊 *Sales Report for 2026-03-12*
💰 Total Revenue: $12500
📦 Total Orders: 250
🎯 Average Order Value: $50
📈 Conversion Rate: 3.2%
Great job team! 🚀
```
This will dynamically replace the field references with actual values from your table.
3. Click **Next**.
## 3. Select your data to send
1. The next step is letting us know which data you want to send on the Slack message. Select the layer and then the desired table.
The maximum allowed file size is 1GB for Slack. In case of large datasets, please limit the number of records from your input table to avoid hitting this constraint.
2. Click **Next**.
## 4. Define the trigger
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want this email to be sent to the list of recipients. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for the email to be actually sent, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your email should be sent.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Snowflake as a destination
Source: https://docs.nekt.com/destinations/snowflake
Send your data from Nekt to Snowflake.
Snowflake is a cloud-based data warehouse platform that provides scalable, secure, and high-performance data storage and analytics capabilities. It offers features like automatic scaling, data sharing, and multi-cloud support for enterprise data management and analytics.
![]()
## 1. Add your Snowflake access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Snowflake option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access. The following configurations are available:
* **Account**: Your Snowflake account identifier (e.g., `xy12345.us-east-1`).
* **User**: The username used to authenticate against Snowflake.
* **Password**: The password for the specified user account. You can use either password or private key authentication.
* **Database**: The name of the database where your data will be sent.
* **Schema**: The schema to use within the specified database.
* **Warehouse**: The Snowflake warehouse to be used for executing operations.
The following advanced settings are also available:
* **Private Key**: The private key for key-pair authentication, used as an alternative to password authentication. This value is treated as a secret and stored securely.
* **Private Key Passphrase**: The passphrase for the private key, if the key is encrypted.
* **Batch Size (MB)**: The target size in megabytes for each batch of records to be uploaded. Larger batches are more efficient but use more memory.
* **Max Batch Records**: Maximum number of records per batch. Leave empty for no limit.
* **Float Precision**: The number of decimal places for floating-point numbers.
* **Network Timeout (seconds)**: Maximum time in seconds to wait for Snowflake operations. Increase it if you experience timeouts on large MERGE operations. Default: 900 (15 minutes).
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Snowflake data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your Snowflake account, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Snowflake.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Implementation Notes
### Sync Modes and Schema Evolution
* **Append Mode**: The connector automatically handles schema evolution. If new columns appear in your source data, they are automatically added to the existing Snowflake table.
* **Overwrite Mode**: The destination table is dropped and completely recreated. Any rows already in the table will be replaced by the current run.
### Data Type Conversion
* During date, time, and datetime processing, any values that cannot be parsed into a Snowflake-compatible format are loaded as `NULL`. The number of failed conversions per column is registered as a warning in your logs.
### Connection and Warehouse Management
* **Authentication Resilience**: The connector validates connections before reusing them and automatically reconnects if an authentication token expires during a load, ensuring long-running syncs don't fail due to session timeouts or dropped sockets.
* **Warehouse Status Check**: The connector checks your warehouse's state once per table load and logs a warning if it is not in a `STARTED` or `RESUMING` state. A suspended warehouse must resume before loading, which may cause a slight delay before the first table insertion begins.
# Track CXM as a destination
Source: https://docs.nekt.com/destinations/track-cxm
Use your data to create distributions on Track CXM.
Track.co is a CX Management ecosystem that offers a large set of tools from monitoring key Voice of the Customer indicators to AI-powered CX data analysis, all seamlessly integrated. It enables users to monitor live dashboards, import external feedback, and configure surveys with skip logic to the customer's preferred channels.
## 1. Add your Track CXM access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Track CXM option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Organization ID**: Your organization identifier within Track CXM.
* **Authentication token**: The token used to authenticate requests to the Track CXM API.
* **Distribution type**: The type of the distributions to be created. Available options are **Regular** and **LinkList**.
You need an [API Integration](https://app.track.co/organization/integrations/) in order to obtain your authentication token.
3. Click **Next**.
## 2. Select your tables
1. The next step is letting us know which tables contain the data that can be used to create the distributions on Track CXM.
> Tip: The table can be found more easily by typing its name.
The selected tables should contain the following fields:
Check more details in the [Track CXM API docs](https://documenter.getpostman.com/view/14932352/Tz5v1upm#f2236bcb-b45c-4b76-94d7-0d6f5f11e0cb).
| Field | Required | Type | Notes |
| :- | :- | :- | :- |
| `survey_uuid` | Yes | UUID string | |
| `distribution_channel` | Yes | String | Allowed values: `email`, `smslink`, `whatsapp` |
| `reminder_channel` | No | String | Allowed values: `email`, `smslink`, `whatsapp` |
| `distribution_name` | No | String | |
| `schedule_at` | No | Integer | Unix timestamp |
| `validity_at` | No | Integer | Unix timestamp |
| `reminder_at` | No | Integer | Unix timestamp |
| `is_anonymous` | No | Boolean | False by default |
| `template_invite_hash` | No | String | Hash of template type invite. |
| `template_reminder_hash` | No | String | Hash of template type reminder. |
| `template_expired_hash` | No | String | Hash of template type expired. |
| `template_thanks_hash` | No | String | Hash of template type thanks. |
| `import_lines` | Yes | Array of objects | Each objects should contain interaction data, such as `name`, `email`, `phone` and others. |
| `whatsapp_template` | Conditional | Object | Required for WhatsApp distribution. |
| `whatsapp_template.name` | Yes | String | |
| `whatsapp_template.language` | Yes | String | Example: `en_US` |
| `whatsapp_template.variables` | Yes | Array of strings | |
| `whatsapp_template.id_provider` | Yes | String | Template ID, ask Track support team. |
| `whatsapp_integration_uuid` | No | UUID string | |
Check more details in the [Track CXM API docs](https://documenter.getpostman.com/view/14932352/Tz5v1upm#520acf36-0666-4af5-bad8-b676a347dee1).
| Field | Required | Type | Notes |
| :- | :- | :- | :- |
| `survey_uuid` | Yes | UUID string | |
| `distribution_name` | No | String | |
| `validity_at` | No | Integer | Unix timestamp |
| `async` | No | Boolean | |
| `shortened_link` | Yes | Boolean | |
| `callback_url` | Conditional | URL string | Required if `async` is true. |
| `template_survey_expired_hash` | No | String | Hash of template type expired. |
| `template_survey_thanks_hash` | No | String | Hash of template type thanks. |
| `import_lines` | Yes | Array of objects | Each objects should contain interaction data, such as `name`, `email`, `phone` and others. |
2. Click **Next**.
## 3. Configure your Track CXM destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see the distributions on Track CXM, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your distributions should be seen on Track CXM.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Twenty as a destination
Source: https://docs.nekt.com/destinations/twenty
Send your data from Nekt to Twenty CRM.
Twenty is an open-source CRM for managing companies, people, opportunities, and custom objects. The Twenty destination creates or updates records in your Twenty workspace through its REST API.
The connector writes to a single Twenty object per destination, selected through the **Mode** setting:
* **People** - creates or updates people (contacts).
* **Companies** - creates or updates companies.
* **Opportunities** - creates or updates opportunities.
* **Notes** - creates or updates notes.
* **Tasks** - creates or updates tasks.
* **Custom object** - creates or updates records of a custom object you define.
To send data to more than one object, create one destination per object.
## 1. Add your Twenty access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Twenty option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **API Key**: Your workspace-scoped Twenty API key (Bearer token). Generate it in Twenty under **Settings → API & Webhooks**.
* **Mode**: Select the object you want to write to (People, Companies, Opportunities, Notes, Tasks, or a custom object).
* **API URL** (advanced): Base URL of your Twenty instance, without the trailing `/rest`. Defaults to `https://api.twenty.com` for Twenty Cloud. Set your own host when self-hosting.
* **Custom object** (advanced): The REST path segment of your custom object (plural camelCase, e.g. `myCustomObjects`). Only used when Mode is set to **Custom object**.
* **Upsert** (advanced): When enabled, the connector creates a record or updates it when a matching record already exists, matched on a unique field such as email. Recommended to avoid duplicates on recurring syncs.
* **Enable partial success** (advanced): Whether the connector should continue trying to sync records after one fails.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Map your columns
Map the columns from your input table to the corresponding Twenty fields. Use the exact API name of each Twenty field as the destination.
### Create vs. update behavior
With **Upsert** enabled, the connector sends each record to `POST /rest/{object}?upsert=true`: Twenty creates the record if it does not exist, or updates it when a matching record is found on a unique field (for example, email). This is the recommended behavior for recurring syncs.
If you map a column to the `id` destination field, the connector instead performs an update (`PATCH`) on the existing record with that Twenty ID.
### Composite fields
Several Twenty fields are composite (objects with sub-fields). Map to a sub-field using **dot-notation** in the destination field name:
| Composite field | Example destination |
| :- | :- |
| Full name | `name.firstName`, `name.lastName` |
| Emails | `emails.primaryEmail` |
| Links | `linkedinLink.primaryLinkUrl` |
| Phones | `phones.primaryPhoneNumber` |
| Currency | `annualRecurringRevenue.amountMicros` |
Currency fields are stored in **micros** (the amount multiplied by 1,000,000). Map your monetary column to the `amountMicros` sub-field (e.g. `totalRevenue.amountMicros`) and the connector converts the value for you, so `49.99` is sent as `49990000`.
### Transforms
Each mapping accepts an optional ordered list of value transforms, applied before sending:
| Transform | Description |
| :- | :- |
| `uppercase` | Convert to upper case. Use for Select fields whose options are upper case (e.g. `active` → `ACTIVE`). |
| `lowercase` | Convert to lower case. Useful for normalizing email keys. |
| `trim` | Remove surrounding whitespace. |
| `round_int` | Round a decimal to the nearest integer (e.g. `7.22` → `7`). |
| `to_string` | Cast the value to text. Use for an integer id going to a text field. |
| `yesno_to_bool` | Convert `Yes`/`No` to `true`/`false`. |
| `null_if:` | Drop the value when it equals `` (e.g. `null_if:-`). |
| `only_if:` | Keep the value only when it equals ``, otherwise leave the field blank. |
Empty and null values are skipped, so existing data in Twenty is preserved when a column has no value.
**Example input table (People):**
| email | first\_name | company\_size | revenue |
| :- | :- | :- | :- |
| [ada@example.com](mailto:ada@example.com) | Ada | 50 | 1250000.00 |
| [alan@example.com](mailto:alan@example.com) | Alan | 12 | 300000.00 |
Column mapping: `email` → `emails.primaryEmail` (transform `lowercase`), `first_name` → `name.firstName`, `company_size` → `employees`, `revenue` → `annualRecurringRevenue.amountMicros`.
## 4. Configure your Twenty data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Twenty, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Twenty.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Wake as a destination
Source: https://docs.nekt.com/destinations/wake
Send your data from Nekt to Wake.
Wake is a Brazilian digital commerce and customer experience platform, offering tools for online stores, customer engagement, and marketing automation.
## 1. Add your Wake access
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the Wake option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **API Key:** Token for accessing Wake's API.
* **Map Field Rules:** Describe how the incoming stream fields should be mapped to Wake fields. For each rule, you must specify:
* **Search path**: Search path to get the value on the incoming stream (e.g., `$.last_name` or `$.email[*].value`).
* **Default value**: Default value to be used for the field, if applicable.
* **Wake field name**: Field name on Wake's platform (e.g., `LASTNAME` or `EMAIL`).
* **Custom Attributes** (optional): Describe how the incoming stream fields should be mapped to Wake's custom attribute fields. For each attribute, you must specify:
* **Search path**: Search path to find the desired information on the incoming stream (e.g., `$.last_name` or `$.email[*].value`).
* **Default value**: Default value to be used for the field, if applicable.
* **Wake attribute name**: The name of the custom attribute on Wake's platform (e.g., `CorFavorita` or `TemCachorro`).
* **Wake attribute type**: The type of the custom attribute on Wake's platform. Choose between: `string`, `boolean`, `float`, `integer`, or `datetime`.
3. Click **Next**.
## 2. Select your data to send
1. The next step is letting us know which data you want to send to the Wake platform. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Wake data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need to have the data updated on the destination (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on Wake, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on Wake.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# XML as a destination
Source: https://docs.nekt.com/destinations/xml
Send your data from Nekt to XML.
XML (eXtensible Markup Language) is a markup language designed to store and transport data in a structured format. It uses tags to define elements and attributes, making it self-descriptive and widely used for data exchange between different systems and applications.
![]()
## 0. Pre-work needed to configure this connector
### 1. Generate a Jinja template file
We use [Jinja templates](https://www.devdoc.net/python/jinja-2.10.1-doc/) to map the data from the input table to your XML file. You can use conditionals, for loops and other features available on Jinja to make your template dynamic and flexible to fit your needs. You can also add attributes to tags, if needed.
**Example**
Imagine the following input table:
| title | description | sale\_price | total\_area | features |
| - | - | - | - | - |
| Cozy Cottage | A cozy cottage in the countryside. | \$150,000 | 1,200 sqft | \["Fireplace", "Garden"] |
| Modern Apartment | A sleek apartment in the city center. | \$300,000 | 850 sqft | \["Gym", "Rooftop"] |
| Family House | Spacious family home with a large backyard. | \$400,000 | 2,500 sqft | \["Backyard", "Garage"] |
| Luxurious Villa | A luxury villa with ocean views. | \$1,500,000 | 4,500 sqft | \["Pool", "Home Theater"] |
| Studio Apartment | Compact studio with modern amenities. | \$200,000 | 600 sqft | \["Central Heating", "Balcony"] |
If you want to map these values to an XML file, here's how your template would look like:
```XML theme={null}
{% for listing in records %}
{{ listing.title }}
{{ listing.description }}
{{ listing.sale_price }}
{{ listing.total_area }}
{% for feature in listing.features %}
{{ feature }}
{% endfor %}
{% endfor %}
```
Where:
* **records** correspond to the rows of incoming data from a table in your Data Lake that you will select while configuring the destination
* `` is the name of the target tag
This template file combined with the input table from the section above would render the following list of target tags.
```XML theme={null}
Cozy Cottage
A cozy cottage in the countryside.
$150,000
1,200 sqft
Fireplace
Garden
Modern Apartment
A sleek apartment in the city center.
$300,000
850 sqft
Gym
Rooftop
Family House
Spacious family home with a large backyard.
$400,000
2,500 sqft
Backyard
Garage
Luxurious Villa
A luxury villa with ocean views.
$1,500,000
4,500 sqft
Pool
Home Theater
Studio Apartment
Compact studio with modern amenities.
$200,000
600 sqft
Central Heating
Balcony
```
### 2. Add write permissions to your S3 bucket
The XML file generated by this destination will be uploaded to an S3 bucket specified when configuring the connector. For that, you have to grant write permissions to Nekt, so the connector can write files to this bucket.
Here's the step-by-step:
* Add the following policy to your S3 bucket:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::720174903362:role/nekt-backend-production-execution-role", // If the bucket is in the same AWS account as your Nekt workspace, you don't need this line
"arn:aws:iam:::role/nekt-ecs-task-role"
]
},
"Action": ["s3:ListBucket", "s3:GetObject", "s3:PutObject"],
"Resource": [
"arn:aws:s3:::",
"arn:aws:s3:::/*" // Here you can limit access to a specific folder
]
}
]
}
```
* Add the following permission block to `nekt-ecs-task-role` (be careful to not modify existing permissions):
```json theme={null}
{
"Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
"Resource": ["arn:aws:s3:::", "arn:aws:s3:::/*"],
"Effect": "Allow",
"Sid": ""
}
```
Now you're ready to jump to the next step and configure your XML destination.
## 1. Configure the XML connector
1. In the [Destinations](https://app.nekt.ai/destinations) tab, click on the "Add destination" button located on the top right of your screen. Then, select the XML option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **XML format:** The format of the XML file to use pre-defined names for the static part of the XML file, including root tags, header and footer. Supported formats include: `VrSync`, `Chaves na Mão`, `Imóvel Web`, and `SP Imóvel`.
* **Jinja template file:** The jinja template file used to map the input table data to the target element structure. Don't forget to validate your jinja template to ensure your destination will work as expected.
* **Bucket name:** The name of the S3 bucket where the generated XML file will be stored.
* **S3 folder path:** Optionally define the name of folder inside the S3 bucket where the generated XML file will be stored.
* **Output file name:** The static name of the generated XML file that will be uploaded to S3.
Looking for Meta or Google product feeds? These formats are available as their own connectors: [Meta Catalog XML](https://docs.nekt.com/destinations/meta-catalog-xml) and [Google Merchant Center XML](https://docs.nekt.com/destinations/google-merchant-center-xml).
3. Click **Next**.
## 2. Select the data from Catalog to send
1. The next step is letting us know which data you want to send to the XML. Select the layer and then the desired table.
> Tip: The table can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your XML data destination
1. Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the data updated on your XML (every day, once a week, only at specific times, etc.).
3. Click **Done**.
### Check your new destination!
1. Once completed, you'll receive confirmation that your new destination is set up!
2. You can view your new destination on the [Destinations](https://app.nekt.ai/destinations) page. Now, for you to be able to see it on your XML, you have to wait for the pipeline to run. You can monitor it on the [Destinations](https://app.nekt.ai/destinations) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon.
Once executed, your data should be seen on the defined XML location.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Explorer
Source: https://docs.nekt.com/explorer/overview
Explore your data with Nekt
This is one of the most used features in Nekt. The Explorer module offers a simple way to learn more about your data and easily extract information.
With it, you can quickly get answers using the AI assistant and have the necessary infrastructure to advance your queries with SQL.
***
### **Nekt AI in Action**
Curious to see it in action? Watch this video to learn how the AI Assistant works.
***
## Step by step to successfully explore your data
### **1. Know your data**
Before deciding which questions to answer, it makes sense to understand what data you have in Nekt, where it comes from, and how they relate to each other.
For this, the first step is to check whether the data source you're looking for is already registered in [`Sources`](https://app.nekt.ai/sources) and verify in the [`Catalog`](https://app.nekt.ai/catalog) which data is being extracted into the tables.
Use the `Preview` feature within each table in the Catalog to understand the data format, as this can be important for more effective exploration.
### **2. Explore your data with SQL**
SQL is the most widely used solution for data exploration. In Nekt, it's the language that allows you to modify and perform operations on the data to answer specific questions.
You can use the AI assistant to generate SQL code from natural language commands, or you can create the SQL code yourself if you're familiar with the language. Let's take a closer look at both options:
The introduction video on this page shows the AI assistant being used and generating results in seconds. You can ask questions in English or Portuguese, whichever you prefer.
Our assistant will use the tables in your workspace and the current SQL query (if there is one) as context. This allows the assistant to generate a new SQL query that answers your questions based on this context. It can even change the table being queried if it determines that the answer to your question is elsewhere (or in a combination of different tables).
Generally, you don't need to worry about the code! Just interact with the assistant until you get the answer you're looking for. Of course, it's worth remembering that like any AI assistant, the answers may not always be correct. In this case, you can continue iterating with the AI to correct it, try your hand at SQL to adjust the query, or seek help from someone who knows SQL.
If you're already familiar with SQL or want to try it out, it's useful to know that Nekt uses Trino as its execution engine. This will help when writing code and knowing which syntax to use and which features (like *Common Table Expressions*) can be utilized.
If needed, you can consult the [Trino](https://trino.io/docs/current/functions.html) documentation to help you write specific queries using the correct syntax.
### **3. Save your queries or create a transformation**
After creating the SQL query that produces the expected results, you have several options:
**3.1 Save to repeat the query later**
Use the `Save query` function to save this query for future use. This will allow you to quickly access this code as many times as you want. Each time you run it, it will be executed on the most up-to-date data. It's useful when you need an answer frequently.
**3.2 Create a transformation and generate a new table**
Use the `Create transformation` function when you want the result of your query to become a new table in your Catalog.
It's quite common, for example, for a table to come from the data source with more data than necessary for a specific use case or with unformatted data, which makes it harder to use. For this, your SQL code can handle these cases and generate the ideal table, which will only be created and updated in the Catalog (and become available for destinations and visualizations, for example) through a transformation.
**3.3 Discard**
If you've made a query that probably won't be repeated and whose result doesn't need to be stored in a table, you can simply close the query and do nothing with it. The Explorer is like a playground: feel free to explore your data and test what results you can achieve.
***
### **Need help?**
If you run into any issues along the way, feel free to contact our support team. We'll be happy to help!
## API & MCP
Explorer queries run through the Data API.
`generate_sql`, `execute_sql`, and the table-discovery tools.
# Bring your AI agent
Source: https://docs.nekt.com/get-started/bring-your-agent
Plug an AI agent into Nekt with the MCP Server so it can explore and act on your data
Now that you have data in your Catalog, you can connect an AI agent to Nekt so it can work with that data on your behalf. No SQL or dashboards required.
Nekt's [MCP Server](/mcp-server/overview) gives agents like Claude, Cursor, ChatGPT, and any MCP-compatible client native access to your data through natural language. Once connected, your agent can:
* **Explore your data**: discover tables, preview their contents, and understand schemas.
* **Query in plain English**: ask questions and let the agent generate and run SQL against your warehouse.
* **Read files**: browse layers, volumes, and files stored in Nekt.
* **Build and run pipelines**: create Queries and Notebooks, schedule them, and trigger runs.
* **Learn about Nekt**: ask about platform features and capabilities without switching context.
The MCP Server is available on all **paid plans**.
### Enable the MCP Server
In order to connect an agent, first enable the server and create a token:
1. In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway** (or go straight to [app.nekt.ai/mcp](https://app.nekt.ai/mcp)).
2. Click **Enable now**. Setup varies slightly by cloud; see the [setup guide](/mcp-server/setup) for GCP and AWS specifics.
3. In the **Access tokens** section, create a token. A token can be scoped per resource type (tables, volumes, secrets, context layer, and live connections) and per tool, so you control what your agent can see and do.
If you connect via OAuth and don't have a token yet, one is created automatically with the same permissions as your user account.
### Connect your AI agent
The MCP Server supports two authentication methods, depending on how your agent connects:
* **OAuth 2.1**: for desktop apps like Claude Desktop and Cursor. Point the app to your MCP server URL and log in with your Nekt credentials.
* **Bearer token**: for CLI tools and automation platforms like N8N or Make. Add the server to your MCP client configuration:
```json theme={null}
{
"mcpServers": {
"Nekt": {
"type": "http",
"url": "",
"headers": {
"Authorization": "Bearer "
}
}
}
}
```
Replace `` with the endpoint from the MCP Gateway page in Nekt, and `` with the token you created. See [Authentication](/mcp-server/authentication) for full details.
### Start asking
With the agent connected, ask questions in natural language and let it pick the right [tools](/mcp-server/tools):
> *"Which tables should I use to analyze customer churn?"*
> *"How many leads were created each month?"*
> *"Create a daily Query that aggregates revenue by customer into the trusted layer."*
### Connect a specific tool
Step-by-step guides for the most common AI agents:
Connect Claude to your data in Nekt.
Connect ChatGPT using OAuth authentication.
Connect any MCP-compatible client.
***
**Nice! Your AI agent can now work directly with your data in Nekt 🤖**
➡️ Next, let's explore your data with Nekt's built-in [AI assistant](./explore-your-data).
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Queries & Notebooks
Source: https://docs.nekt.com/get-started/build-your-first-transformation
Find the insights you need with the power of SQL and Python
Processing your data is like cooking: you take raw ingredients (your source data) and turn them into a finished dish (clean, organized data ready for analysis).
In Nekt, there are three modules for processing your data:
* **Queries**: Write SQL to process and aggregate your data
* **Notebooks**: Run Python or PySpark for complex logic, API enrichment, or web scraping
* **History**: Track changes in your data over time with a no-code SCD Type 2 template
This guide walks you through your first **Query** (SQL) and your first **Notebook** (Python/PySpark).
SQL is a good choice for filtering, aggregating, and joining data. You can use the AI assistant to generate the SQL for you, or write it yourself.
Python is a good choice for complex logic, data enrichment, or AI. You can use popular libraries like pandas, numpy, scikit-learn, and more.
In the sections below you'll find examples on how to build your first Query (SQL) and Notebook (Python/PySpark).
## **Use case**
Using the data from the source configured in the [previous section](./explore-your-data), let's build a query that creates a lead scoring and segmentation engine.
Here's a sample of the source data:
| Name | Creation Date | Email | Source | Age Range | Main Interest |
| - | - | - | - | - | - |
| Bonnie Jones | 2023-10-22 17:24:17 | [erikakramer@example.org](mailto:erikakramer@example.org) | Email Campaign | 65+ | Healthcare |
| Victoria Schaefer | 2023-10-23 17:24:17 | [fisherjohn@example.net](mailto:fisherjohn@example.net) | Trade Show | 25-34 | Technology |
| Jonathon Lucas | 2023-10-23 17:24:17 | [arichardson@example.com](mailto:arichardson@example.com) | LinkedIn | 25-34 | Travel |
| Rebecca Buchanan | 2023-10-26 17:24:17 | [kblankenship@example.net](mailto:kblankenship@example.net) | Referral | 65+ | Food & Beverage |
| Jennifer Rodriguez | 2023-10-27 17:24:17 | [garycampos@example.org](mailto:garycampos@example.org) | Google Ads | 18-24 | Travel |
With our data transformation, we will segment the leads, ordering them by highest likelihood to convert.
Let's put this into practice 🚀
## **SQL**
We can use the AI assistant to help us build the SQL query:
1. Go to the [Explorer](/explorer/overview) page and click on the **AI Assistant** button on the right side of the screen.
2. Select the table you want to use for the query (in our case `outbound_leads`).
3. Enter the following prompt:
```markdown theme={null}
Create a SQL query that builds a lead scoring and segmentation engine.
1. Calculate a `lead_score` (0-100) based on weighted factors:
- Source quality: Referral/Direct (30pts), LinkedIn/Trade Show (25pts), Email Campaign (20pts), Google/Facebook Ads (15pts), Social Media (10pts), Organic Search (5pts)
- Recency: Leads from last 30 days (30pts), 31-90 days (20pts), 91-180 days (10pts), older (5pts)
- Interest value: Technology/Healthcare/Finance (25pts), Real Estate/Consulting (20pts), others (15pts)
- Age range fit: 25-44 (15pts), 35-54 (12pts), others (8pts)
2. Assign a `lead_tier` based on score:
- 'Hot' for scores 80-100
- 'Warm' for scores 50-79
- 'Cold' for scores 0-49
Output all original columns plus the new calculated fields.
```
4. The AI assistant will generate the SQL query.
5. You'll see the results in the section below the query editor.
Once the results are available, we can save it as a Query. This ensures the results are saved in your Lakehouse and can be accessed later via Explorer, Destinations, or Integrations (BI tools, AI agents, APIs):
1. In the action bar below the query editor, click on **Save as Query**.
2. Select the layer and define a name for the table that will be saved based on the query results.
3. Click **Next**.
4. Create a description and define the trigger.
5. Click **Done**.
Now that you created the query, you can run it manually or wait for the automated trigger to see the output table available in your Catalog.
The difference between running a query in Explorer and saving it as a Query is that a saved Query can be orchestrated and generates an output table in your Catalog, which can be used downstream for further processing and activation.
**Video Tutorial**
For a visual walkthrough of how to build your first SQL query, watch this video:
## **Python Notebooks**
We have a set of resources available to help you build Python notebooks, including:
* Ready-to-use notebook templates
* Data access tokens to securely access your data
* Nekt SDK to easily access tables from your Lakehouse
Notebook templates are pre-configured with the necessary imports and setup to access data from your Lakehouse. They work like a playground where you can explore the data and validate your transformation logic before running it on Nekt.
### Working with notebook templates
Here's the step by step guide for using a notebook template:
1. Go to the [Notebooks](/notebooks/overview) module and click on the **Add Notebook** button.
2. Choose **PySpark** as the transformation type.
3. Click on **Tokens** to create a data access token.
4. In the modal that pops up, click on **Create token**.
5. Select the tables you want to use in your transformation.
6. Click **Create**.
7. Close the modal, we'll come back to it later to copy the token and input tables.
8. Click on **Notebooks** and select Google Colab (feel free to use any other notebook provider you prefer).
9. This will open the Nekt template on Google Colab.
10. Click on **File** > **Save a copy in Drive** to save a copy of this template on your own Google account.
11. Run the cell right after the **Default installations** section to install the necessary dependencies. This ensures you have the latest version of the Nekt SDK and the necessary libraries.
12. Copy the token you created earlier and replace the `ADD_YOUR_TOKEN_HERE` placeholder in the cell below.
13. In the **Example** section you have an example of transformation. You can keep the imports and remove the rest of the code.
14. Copy the input tables and paste them right below the imports.
Now you have everything you need to work on your transformation: the setup is done and data is already loaded into the notebook.
Below you'll find the PySpark equivalent code from the previous use case to validate your transformation logic before running it on Nekt.
```python theme={null}
# Custom imports
import nekt
from pyspark.sql import DataFrame
from pyspark.sql import functions as F
outbound_leads_df = nekt.load_table(layer_name="Raw", table_name="google_sheets_outbound_leads")
creation_date_col = F.to_date(F.substring(F.col("creation_date"), 1, 10))
# Calculate scores
outbound_leads_df = outbound_leads_df \
.withColumn("source_score",
F.when(F.col("source").isin("Referral", "Direct"), 30)
.when(F.col("source").isin("LinkedIn", "Trade Show"), 25)
.when(F.col("source") == "Email Campaign", 20)
.when(F.col("source").isin("Google Ads", "Facebook Ads"), 15)
.when(F.col("source") == "Social Media", 10)
.when(F.col("source") == "Organic Search", 5)
.otherwise(0)
) \
.withColumn("recency_score",
F.when(creation_date_col >= F.date_sub(F.current_date(), 30), 30)
.when(creation_date_col >= F.date_sub(F.current_date(), 90), 20)
.when(creation_date_col >= F.date_sub(F.current_date(), 180), 10)
.otherwise(5)
) \
.withColumn("interest_score",
F.when(F.col("main_interest").isin("Technology", "Healthcare", "Finance"), 25)
.when(F.col("main_interest").isin("Real Estate", "Consulting"), 20)
.otherwise(15)
) \
.withColumn("age_score",
F.when(F.col("age_range") == "25-44", 15)
.when(F.col("age_range") == "35-54", 12)
.otherwise(8)
)
# Calculate total lead score
outbound_leads_with_score_df = outbound_leads_df \
.withColumn("lead_score", F.col("source_score") + F.col("recency_score") + F.col("interest_score") + F.col("age_score"))
outbound_leads_with_score_df = outbound_leads_with_score_df \
.withColumn("lead_tier", F.when(F.col("lead_score").between(80, 100), "Hot")
.when(F.col("lead_score").between(50, 79), "Warm")
.otherwise("Cold")
)
# Create the final dataframe with all columns
final_df = outbound_leads_with_score_df.select(
"name",
"creation_date",
"email",
"source",
"age_range",
"main_interest",
"lead_score",
"lead_tier",
"_nekt_sync_at"
)
```
When working with notebooks, you can split the code into multiple cells to make it easier to debug and test, as we can run them separately. You can make extensive use of the `printSchema()` function to check the schema of the dataframe and the `show()` function to print the first few rows of a dataframe.
### Creating a PySpark Notebook
Now that you have validated your logic, you can create a notebook in Nekt.
1. Go to the [Notebooks](/notebooks/overview) module and click on the **Add Notebook** button.
2. Choose **PySpark** as the notebook type.
3. Copy and paste the code from your local notebook into the editor.
4. Add the `nekt.save_table()` call to ensure the final dataframe is saved to your Lakehouse as a new table.
5. Click **Next**.
6. Create a description and define the trigger.
7. Click **Done**.
Here's the full code to create a PySpark notebook in Nekt.
```python theme={null}
# Custom imports
import nekt
from pyspark.sql import DataFrame
from pyspark.sql import functions as F
outbound_leads_df = nekt.load_table(layer_name="Raw", table_name="google_sheets_outbound_leads")
creation_date_col = F.to_date(F.substring(F.col("creation_date"), 1, 10))
# Calculate scores
outbound_leads_df = outbound_leads_df \
.withColumn("source_score",
F.when(F.col("source").isin("Referral", "Direct"), 30)
.when(F.col("source").isin("LinkedIn", "Trade Show"), 25)
.when(F.col("source") == "Email Campaign", 20)
.when(F.col("source").isin("Google Ads", "Facebook Ads"), 15)
.when(F.col("source") == "Social Media", 10)
.when(F.col("source") == "Organic Search", 5)
.otherwise(0)
) \
.withColumn("recency_score",
F.when(creation_date_col >= F.date_sub(F.current_date(), 30), 30)
.when(creation_date_col >= F.date_sub(F.current_date(), 90), 20)
.when(creation_date_col >= F.date_sub(F.current_date(), 180), 10)
.otherwise(5)
) \
.withColumn("interest_score",
F.when(F.col("main_interest").isin("Technology", "Healthcare", "Finance"), 25)
.when(F.col("main_interest").isin("Real Estate", "Consulting"), 20)
.otherwise(15)
) \
.withColumn("age_score",
F.when(F.col("age_range") == "25-44", 15)
.when(F.col("age_range") == "35-54", 12)
.otherwise(8)
)
# Calculate total lead score
outbound_leads_with_score_df = outbound_leads_df \
.withColumn("lead_score", F.col("source_score") + F.col("recency_score") + F.col("interest_score") + F.col("age_score"))
outbound_leads_with_score_df = outbound_leads_with_score_df \
.withColumn("lead_tier", F.when(F.col("lead_score").between(80, 100), "Hot")
.when(F.col("lead_score").between(50, 79), "Warm")
.otherwise("Cold")
)
# Create the final dataframe with all columns
final_df = outbound_leads_with_score_df.select(
"name",
"creation_date",
"email",
"source",
"age_range",
"main_interest",
"lead_score",
"lead_tier",
"_nekt_sync_at"
)
# Save the final dataframe to your Lakehouse as a new table - you're free to choose the layer and table name you prefer
nekt.save_table(dataframe=final_df, layer_name="Trusted", table_name="outbound_leads_with_score_pyspark")
```
The main difference is the addition of the `nekt.save_table()` call at the end to ensure the final dataframe is saved to your Lakehouse as a new table.
Now that you created the notebook, you can run it manually or wait for the automated trigger to see the output table available in your Catalog.
**Video Tutorial**
For a visual walkthrough of how to build your first PySpark notebook, watch this video:
**It's been an exciting journey so far! But processing the data is just the beginning.**
➡️ Let's move on to the next page to understand how we can put the transformed data in the hands of the right people.
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Connect a data source
Source: https://docs.nekt.com/get-started/connect-first-source
Connect the first data source and start exploring your data
In this guide we'll show you how to connect and run your first data source.
Data sources can be anything from databases, files, spreadsheets, websites, or even third-party APIs.
For simplicity, we'll use a Google Sheets as our first data source. However, a similar process applies to any other data source from our **list of 120+ available connectors**.
### Connecting a data source
In order to connect a data source, follow the steps below:
1. Go to [Sources](https://app.nekt.ai/sources) and click the **+ Add Source** button.
2. Select the data source you want to connect (e.g. Google Sheets).
3. Add the connection details.
4. Configure how each stream should be mapped to a table in the Catalog (each stream represents a dataset from the data source).
5. Define a trigger and description.
6. Click the **Save** button.
Once the data source is created, you need to wait until the first extraction is completed to see the data in your Catalog. This may take a few minutes (or even hours), depending on the amount of data being extracted.
### Running your data source
The creation of the data source does not trigger the extraction of the data. You need to trigger it manually or wait for it to run on a schedule.
Manual triggers don't affect scheduled or event-based triggers.
Once the data source finishes the first extraction, the data will be available in the Catalog.
Subsequent runs will extract fresh data from the source based on the trigger configuration (schedule, event-based or manual), updating the data in the Lakehouse according to the sync type (full sync, incremental or log-based).
This ensures your data is always up-to-date and you always have the latest version of your data available for exploration and use.
**Video Tutorial**
For a visual walkthrough of how to connect your first data source, watch this video:
***
**And there you go! You've completed your first data extraction ✨**
➡️ Now, let's move on to the next page of this guide to learn [how to bring your AI agent](./bring-your-agent).
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Connect to a BI tool
Source: https://docs.nekt.com/get-started/connect-to-bi-tool
Connect your data to a BI tool for reporting and visualization
Dataviz and BI tools are a great way to visualize and share your data with your team and stakeholders.
In Nekt, you can connect your data to a BI tool using the **Integrations** module.
We support the most popular BI tools, including Looker Studio, Power BI, Tableau, Metabase, Grafana, Qlik Sense, and more.
Nekt has custom connectors for Looker Studio (AWS) and Power BI (AWS). For Nekt Express and GCP deployments, connect directly via BigQuery. See the [Integrations](/integrations/overview) section for per-tool setup guides.
In this guide, we'll use Looker Studio as an example. It's free and easy to get started with.
### Generating an integration token
For AWS deployments, Nekt's custom connectors require a token:
1. Go to [Integrations](https://app.nekt.ai/visualizations) and click **+ Add Integration**.
2. Select the BI tool you want to connect (e.g. Looker Studio).
3. Select the tables you want to make available in the BI tool.
4. Copy the generated token. You'll need it to authenticate the connection.
### Connecting to Looker Studio
1. Use [this link](https://lookerstudio.google.com/datasources/create?connectorId=AKfycbwFfHPxHQzcxVuHeIbCoMywA1UjGCv0YY2gD9N1pnQ) to open Nekt's connector in Looker Studio.
2. Enter your integration token and click **Next**.
3. Select the table you want to connect.
4. Optionally set a date range column and row limit (use `-1` to return all rows).
5. Click **Connect**.
Your data is now connected in Looker Studio.
**Video Tutorial**
For a visual walkthrough of how to connect your data to a BI tool, watch this video:
***
**Great job! You've gone through the process of setting up an entire data pipeline 🚀🚀**
Now you're ready to explore further and work on more advanced use cases.
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Explore your data
Source: https://docs.nekt.com/get-started/explore-your-data
Explore your data using our AI assistant and SQL
Now that we've connected our first data source, let's explore the data.
If you are not familiar with SQL, python or other data exploration languages and frameworks available in popular data warehouses, worry not!
**Our AI assistant is here to help you.**
### Using the AI assistant
In order to use the AI assistant, follow the steps below:
1. Go to [Explorer](https://app.nekt.ai/explorer) and click the **AI Assistant** button on the right side of the screen.
2. Enter a question about your data.
3. The AI assistant will generate a SQL query to answer your question.
4. Review the query and generated results.
The AI assistant learns from your interactions and feedback, so the more reinforce positive results using the like button, the better it gets.
If you want to make adjustments to the query or even create your own queries from scratch, be aware that the language used in the SQL query editor is Presto for AWS-based workspaces or BigQuery for GCP-based workspaces. See more details about the syntax in the [Presto](https://prestodb.io/docs/current/functions.html) and [BigQuery](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax) official docs.
### Example
Consider a table named `outbound_leads` with the following schema and sample data:
| Name | Creation Date | Email | Source | Age Range | Main Interest |
| - | - | - | - | - | - |
| Bonnie Jones | 2023-10-22 17:24:17 | [erikakramer@example.org](mailto:erikakramer@example.org) | Email Campaign | 65+ | Healthcare |
| Victoria Schaefer | 2023-10-23 17:24:17 | [fisherjohn@example.net](mailto:fisherjohn@example.net) | Trade Show | 25-34 | Technology |
| Jonathon Lucas | 2023-10-23 17:24:17 | [arichardson@example.com](mailto:arichardson@example.com) | LinkedIn | 25-34 | Travel |
Suppose you want to know how many leads were created each month to understand your marketing performance.
You can simply ask the AI assistant in natural language:
> *"How many leads were created each month?"*
The AI assistant will generate a SQL query to answer your question. It will create the query based on the context of the tables, columns, and descriptions available in your workspace.
You can keep iterating on the query until you get the data you need, or even explore the data further with follow-up questions.
**Video Tutorial**
For a visual walkthrough of how to explore your data with the AI assistant, watch this video:
***
**Great! You've just learned how to use our AI Assistant to help build your SQL queries.**
➡️ Now, let's move on to the next page to understand [how to transform your data](./build-your-first-transformation).
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Invite team members
Source: https://docs.nekt.com/get-started/invite-team-members
Invite team members to your Nekt workspace
We believe data can only be useful when shared with the right people. Thus, in order to use Nekt to its full potential, make sure to invite the relevant team members to help build your data platform together.
We don't charge for number of users, so you can invite as many people as you want.
In order to invite a team member, follow the steps below:
1. Go to [Settings > Members](https://app.nekt.ai/settings/members) and click the **+ Invite Member** button.
2. Enter the name and email of the team member you want to invite.
3. Define their access level:
* **Owner:** Full control over the workspace, including billing and all administrative settings.
* **Admin:** Same as Owner, except billing and credits, cloud setup, creating workspaces, and managing Owners and other Admins.
* **Member:** Can access workspace resources based on assigned permissions. Cannot invite new team members.
4. Click the **Invite** button.
5. The team member will receive an email with a link to accept the invitation and join your workspace.
You can also resend or revoke an invitation by selecting the `Invited` filter on the [Members' page](https://app.nekt.ai/settings/members), clicking the three dots on the right side of the member's row and selecting the desired action.
**Video Tutorial**
For a visual walkthrough of how to invite team members, watch this video:
***
**Inviting members is as simple as that!**
➡️ Move on to the next page of this guide to learn [how to connect your first data source](./connect-first-source).
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Quickstart
Source: https://docs.nekt.com/get-started/quickstart
Get to know our core features with a practical guide
Are you new to Nekt? This guide is for you!
In this guide, you'll learn how to:
* [Setup your account and workspace](./setup): choose **Nekt Express** for a workspace ready in minutes, or connect your own AWS/GCP accounts
* [Invite your team members](./invite-team-members)
* [Connect your first data source](./connect-first-source)
* [Explore it using our AI assistant](./explore-your-data)
* [Transform it using SQL or Python](./build-your-first-transformation)
* [Send it to a destination (e.g. a spreadsheet or an email)](./send-to-destination)
* [Plug your data in a BI tool (e.g. Looker Studio)](./connect-to-bi-tool)
All of this in a few minutes of short instructions and how-to videos.
Sit tight and let's get started!
➡️ Go to [Workspace Setup](./setup).
# Send to a destination
Source: https://docs.nekt.com/get-started/send-to-destination
Send data from your Lakehouse to a destination
So far we've learned how to transform data and explore it using the AI assistant and SQL. However, the insights you get from the data are only valuable if shared with the right people at the right time.
In Nekt, you can send data to a destination using the Destinations module, so users can easily access the data in their favorite tools and systems.
Destinations can be anything from databases, files, spreadsheets, emails, or even third-party APIs to integrate with popular tools, systems, agents and workflows.
We currently have over **30 destinations**. The most popular ones are: Gmail, Google Sheets, Postgres, Sharepoint and Slack.
### Configuring a destination
In order to configure a destination, follow the steps below:
1. Go to [Destinations](https://app.nekt.ai/destinations) and click the **+ Add Destination** button.
2. Select the destination you want to connect (e.g. Gmail).
3. Add the connection details.
4. Configure the data to be sent to the destination.
5. Define a trigger and description.
6. Click the **Save** button.
Choosing an event-based trigger is usually the best option, as it will send the data to the destination whenever the input table gets updated.
Once the destination is created, you need to wait for the trigger or manually run it.
**Video Tutorial**
For a visual walkthrough of how to send data to a destination, watch this video:
***
**Nice! You've sent your data to a destination for the first time ✨**
➡️ Now, let's move on to the next page of this guide to learn how to connect your data to a BI tool.
***
**Need Help?**
If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
# Workspace setup
Source: https://docs.nekt.com/get-started/setup
Create an account and setup your Nekt workspace in just a few minutes
The setup process consists of two main steps:
1. Create your Nekt account
2. Set up your workspace (Nekt Express or your own cloud: AWS/GCP)
The entire setup process should take around 5-10 minutes. With **Nekt Express**, your workspace can be ready in just a few minutes. No cloud account or infrastructure setup required.
Feel free to watch the video tutorial below for a visual walkthrough, or follow the step-by-step instructions right after it.
### **Step 1: Create your account**
1. Go to `https://app.nekt.ai/signup` and enter your first name, last name, and email (preferably your company email). Click **Continue**.
2. Follow the instructions on your email to verify your account. You'll need to define your password at this point.
3. Log in with the email and password you just created.
### **Step 2: Create the company workspace**
1. Add your company logo.
2. Enter your company name, size, country, and timezone. Click **Continue**.
### **Step 3: Set up your workspace**
You can run your Nekt workspace in two ways:
* **Nekt Express** (recommended): A cloud environment hosted and managed by Nekt. Set up in a few minutes with no AWS or GCP account required. Ideal for teams that want to get started quickly without involving their infrastructure team.
* **Your own cloud (AWS or GCP)**: Nekt runs inside your existing cloud account. You create the stack (AWS) or link your project (GCP) and retain full control and ownership of data and resources.
Nekt Express is the fastest way to get a workspace running. No cloud account, no stack deployment, no infrastructure to maintain.
1. In the setup flow, select **Nekt Express** as your workspace type.
2. Complete the short configuration steps in the app.
3. Your workspace will be provisioned and ready in a few minutes. Data is stored and processed in a secure environment managed by Nekt.
Use Nekt Express when:
* You don’t have an AWS or GCP account (or don’t want to use one for Nekt).
* You want to avoid dealing with CloudFormation/Terraform, IAM, or billing in your own cloud.
* You prefer a fully managed setup so your team can focus on data, not infrastructure.
Billing for Nekt usage is included in your Nekt subscription; there is no separate cloud provider bill for the workspace itself.
**Your own cloud:** If you prefer to run Nekt in your own AWS or GCP account, use one of the options below.
1. To run your workspace with AWS, we need to create a stack with all the necessary resources. Before we do that, make sure your AWS account meets the following requirements:
* You have an active **paid AWS account**
Free-tier accounts are not supported because some resources needed for the stack are only available on paid plans.
* No previous Nekt workspace exists in this AWS account.
If you've gone through this process before, delete any related stack to avoid conflicts.
2. Select the AWS region for your Nekt workspace. If unsure, keep the default value; it's our recommendation for the most cost-effective option. Click **Continue**.
3. Click **Open AWS CloudFormation** to be redirected to AWS. This will open a new tab on the CloudFormation console with the Nekt stack. For more information on the resources that will be created, check the section below.
The CloudFormation stack provisions the following resources in your AWS environment:
#### **Core Infrastructure**
1. **IAM Roles & Permissions**
* **ECS Service Roles**: Enable source and destination connectors to access data and store it in S3
* **EMR Service Roles**: Allow sources and transformations to access your data catalog and perform data operations
* **Frontend Access Role**: Enables data preview and exploration through the Nekt application
* **Server Orchestration Role**: Allows Nekt to manage and coordinate all sources, destinations, and transformations
2. **Lambda Functions**\
Custom functions that handle CloudFormation deployment tasks, including VPC IP retrieval, resource identifier notifications, and deployment completion notifications.
3. **Networking Infrastructure**\
An isolated VPC with secure network architecture:
* 1 public subnet and 2 private subnets
* NAT instance with Elastic IP for secure outbound connectivity
* Internet gateway for public subnet access
* Security groups configured for each resource type
**Important**: Ensure your AWS region has sufficient quota for "VPCs per
Region" and "Internet gateways per Region" before deployment.
4. **Data Storage & Catalog**
* **S3 Bucket**: Secure storage location for all your data
* **AWS Glue Databases**: Managed data catalog that integrates with Nekt's platform for metadata management
5. **Container & Processing Infrastructure**
* **ECR Repository**: Private Docker registry for custom connector and transformation images
* **EMR Studio**: Spark processing environment for data transformations and analysis
* **ECS Cluster**: Container orchestration platform for running connectors
#### **On-Demand Resources**
The following resources are provisioned automatically when you run data operations:
1. **AWS Fargate Tasks**\
Serverless container execution for source and destination connectors
2. **EMR Application Jobs**\
Spark-based processing jobs for data sources and transformations
3. **Amazon Athena**\
Interactive query service used when exploring data through Nekt's interface
> **Note:** The workspace setup process is automated, so you don't need to manually configure any resources beyond running the stack.
***
4. Keep the pre-filled default values for the stack resources.
5. Scroll down and check all checkboxes at the end of the page to allow the resources to be created.
6. Click **Create stack** to start the creation process. This entire process should take less than 5 minutes.
7. Click the refresh button to reload the page and check the status. When the stack creation process is completed, you'll see the stack status change from **CREATE\_IN\_PROGRESS** to **CREATE\_COMPLETE**.
8. Go back to the Nekt app and you'll be automatically redirected to the **Set up your profile** page.
1. To run your workspace with GCP, you need to have a Google Cloud Platform project. Make sure your you check the following requirements:
* **A Google account** (a regular Google account)
* **A credit or debit card**
* If something goes wrong with the card, verify that:
* It is enabled for online transactions
* It accepts charges in **USD**
* It allows **verification transactions**
2. Creating your GCP Account
* Go to **[https://cloud.google.com](https://cloud.google.com)** and complete the initial setup:
* Click **Get Started for free**
* Enter your Google account information
* Add your payment details
* Click **Start Free**
* Verify your payment information
* Complete the initial profile setup
* After completing the setup, the **Dashboard** will open with a project already created.
* Open the **Billing** module of the newly created project.\
When the page loads, a modal will appear indicating that the project is not linked to any billing account.\
**Create this billing account.**
* The **Project ID** will be used when configuring the Nekt workspace.
3. In the Nekt app, just follow the steps to inform the Project ID and authenticate with your Google Account. In a few minutes you'll have your workspace ready.
#### **Core Infrastructure**
1. Service Accounts & Permissions
* **Cloud Run Service Roles**: Enable source and destination connectors to access data and store it in GCS
* **Dataproc Service Roles**: Allow sources and transformations to access your data catalog and perform data operations
* **MCP Server Access Role**: Enables data preview and exploration through the Nekt application
* **Server Orchestration Roles**: Allow Nekt to manage and coordinate all sources, destinations, and transformations
2. Cloud Function
Custom function that handles job completion events, including status tracking, resource notifications, and orchestration callbacks.
3. Networking Infrastructure
An isolated VPC with secure network architecture:
* 1 public subnet and 1 private subnet
* Cloud NAT with static IP for secure outbound connectivity
* Cloud Router for network routing
* Firewall rules configured for each resource type
4. Data Storage & Catalog
GCS Bucket: Secure storage location for all your data
BigQuery: Managed data warehouse that integrates with Nekt's platform for metadata management and queries
5. Event Infrastructure
Pub/Sub Topic: Messaging service for coordinating job completion events
Log Sinks: Capture and route completion events from processing services
**On-Demand Resources**
1. Cloud Run Jobs
Serverless container execution for source and destination connectors
2. Dataproc Serverless Batches
Spark-based processing jobs for data transformations
3. BigQuery
Interactive query service used when exploring data through Nekt's interface
### **Step 4: Set up your profile**
1. Add your profile picture.
2. Review your name and email information. Click **Continue**.
3. Enter the current role you have at your company.
4. You'll be redirected to the Nekt **Overview** page.
***
**Now, your workspace is ready! 🚀🚀**
➡️ Move on to the next page of this guide to learn how to [Invite team members](./invite-team-members).
***
#### **Troubleshooting**
1. A few actions (like creating sources for example) might not work if your AWS Quotas are too low, specially if you have a new AWS account. Please check if you have at least 16 applied quotas on **EMR Serverless service** and request an increase if necessary.
* [Here's the AWS official guideline](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html) for doing so. We also demonstrate the process on the video below.
You don't need to increase the Fargate quotas as shown on the video unless explicitly requested.
> If you encounter any issues during onboarding or need assistance, feel free to reach out to our support team. We're here to help you get started.
2. If AWS takes too long to reply to your support request to increase your quotas, **you can try reaching out to them via live chat**, following the steps in the video below.
If they ask more details about the use case, you can share the following information:
> *We’re setting up a product called Nekt (nekt.com), which builds a Lakehouse on AWS for us, and to process data they use mostly EMR Serverless. The increase to at least 16vCPU is crucial to unblock my team from moving on with our data projects.*
# From static report to live app
Source: https://docs.nekt.com/guides/live-report
Turn a static HTML report built with Claude or Codex into a live web app powered by Nekt.
When you connect Claude or Codex to Nekt through the [MCP Server](/mcp-server/introduction), it can explore your data and generate a polished HTML report in minutes. The problem is that report is frozen: the numbers are baked into the file, and the moment your data changes it is out of date.
This guide turns that static report into a **live app**, accessible over the web, that always reflects your current data.
**You do not need to write code.** Each step below gives you a prompt to paste into your AI agent. The agent creates the tables in Nekt, builds the app, and deploys it. The text around each prompt explains what is happening and how to confirm it worked.
## How this works
The plan has three core steps, plus two optional enhancements:
Turn the SQL behind your report into scheduled Queries that precompute clean, fast tables.
The agent adds a small server function that reads those tables from Nekt through the Data API, and wires your report to load from it.
Publish the app to a public URL, optionally connected to GitHub.
Two optional enhancements, [caching](#optional-add-caching) and [authentication](#optional-add-authentication), are covered along the way. Add them if your report needs them, or skip them for a first version.
### Architecture
The key idea is that your Nekt API key stays on the server. Your report never talks to Nekt directly. It calls a small server function that queries Nekt, caches the result, and returns clean data.
```mermaid theme={null}
flowchart LR
A[Browser
your report] -->|loads data| B[Server function]
B -->|cached copy| A
B -->|Data API| C[Nekt]
C -->|materialized tables| D[(Catalog)]
```
## Before you start
You need an agent that can create files and run commands in a project folder, such as **Claude Code** or the **Codex CLI**, connected to Nekt. Follow [Connect Claude](/integrations/claude) or [Connect Codex](/integrations/codex) if you have not already.
Claude.ai on the web can create the Nekt Queries in Step 1, but it cannot build and deploy the app. For the full flow, use a coding agent.
Keep your static report file (the HTML the agent generated) and the SQL queries behind it. If the SQL is embedded in the HTML, that is fine, the agent can extract it.
In Nekt, go to **Workspace Settings > API Keys** and [create an API key](https://app.nekt.ai/settings/api-keys). You will paste it into the agent during deployment. Requires a **paid Nekt plan**.
Sign up at [vercel.com](https://vercel.com). This is where your live report will be hosted.
In a hurry? Skip to the [all-in-one prompt](#fast-path-one-prompt) that runs the whole flow at once, then come back to any step you want to adjust.
***
## Step 1: Materialize the report data in Nekt
Your report was built from one or more SQL queries that join and aggregate raw tables. Running those joins every time someone opens the report is slow. Instead, you precompute them once into clean, ready-to-read tables that the app loads instantly.
A report is usually made of several sections (a KPI row, a revenue chart, a top-customers table, and so on), and each section is backed by its own query. You create **one materialized table per section**. A simple report may need just one table; a richer one may need several.
Paste this into your agent:
```text Prompt theme={null}
Using the Nekt MCP tools, take the SQL queries behind my report and turn each one
into a materialized table in Nekt.
For every query:
- Create a scheduled Query in Nekt that writes its results to a clean table in the
trusted layer.
- Name each table report_, for example report_monthly_metrics or
report_top_customers.
- Schedule it to refresh daily.
- Shape each table so it matches exactly what that section of the report shows
(one row per data point), so the app does not have to transform anything.
My queries are here: [paste your SQL, or say: they are embedded in report.html,
please extract them first].
When done, list the tables you created and their columns so I can confirm.
```
**Confirm it worked:** open your [Nekt Catalog](https://app.nekt.ai) and check that each new `report_*` table exists and holds the expected rows.
For logic SQL cannot express (machine learning, external API calls, complex reshaping), ask the agent to use a [Notebook](/notebooks/overview) instead. The result lands in the Catalog the same way, so the rest of this guide is unchanged.
***
## Step 2: Turn the report into a live app
Now the agent builds a small app around your existing report. It keeps your layout and charts, and adds a server function that reads the materialized tables from Nekt's [Data API](/data-api/introduction) and hands the data to the page.
Paste this into your agent:
```text Prompt theme={null}
Turn my static report into a live web app that loads current data from Nekt.
- Keep my existing report layout, styling, and charts.
- Add a Vercel serverless function at api/report.js that reads my materialized
Nekt tables through the Data API:
- Endpoint: POST https://api.nekt.ai/api/v1/sql-query/
- Header: x-api-key, read from the NEKT_API_KEY environment variable
- Body: {"sql": "", "mode": "csv"}
- The response returns presigned_urls; download the CSV from the first URL
and parse it into rows.
Run one query per report table (report_monthly_metrics, report_top_customers,
and so on), in parallel, and return all datasets as JSON keyed by section.
- Update the report to fetch /api/report when it loads and render each dataset
into the matching section.
- Keep my Nekt API key server-side only. Never put it in the browser code.
Show me the project file tree and run it locally so I can preview it.
```
**Confirm it worked:** the agent runs the app locally (usually `http://localhost:3000`) and your report shows live data pulled from Nekt.
The Data API runs a SQL query and returns a presigned URL to download the results:
```bash Request theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sql-query/ \
--header 'x-api-key: YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"sql": "SELECT month, revenue, new_customers FROM \"trusted\".\"report_monthly_metrics\" ORDER BY month",
"mode": "csv"
}'
```
```json Response theme={null}
{
"state": "SUCCEEDED",
"presigned_urls": [
"https://storage.googleapis.com/bucket/query-....csv?..."
],
"data_scanned_in_bytes": 11250,
"execution_time_in_millis": 408
}
```
Presigned URLs expire after **1 hour**. The app caches the parsed data, never the URL.
The function lists every table in one place, runs the queries in parallel, and returns the datasets keyed by section:
```javascript api/report.js theme={null}
// One entry per section of your report.
const DATASETS = {
metrics: `SELECT month, revenue, new_customers
FROM "trusted"."report_monthly_metrics" ORDER BY month`,
topCustomers: `SELECT name, total_spent
FROM "trusted"."report_top_customers" ORDER BY total_spent DESC`,
kpis: `SELECT * FROM "trusted"."report_kpis"`,
};
export default async function handler(req, res) {
const entries = await Promise.all(
Object.entries(DATASETS).map(async ([key, sql]) => [key, await runQuery(sql)])
);
return res.status(200).json(Object.fromEntries(entries));
}
async function runQuery(sql) {
const query = await fetch("https://api.nekt.ai/api/v1/sql-query/", {
method: "POST",
headers: {
"Content-Type": "application/json",
"x-api-key": process.env.NEKT_API_KEY,
},
body: JSON.stringify({ sql, mode: "csv" }),
});
const result = await query.json();
if (result.state !== "SUCCEEDED" || !result.presigned_urls?.length) {
throw new Error(`Query failed: ${result.state_change_reason ?? result.state}`);
}
const csv = await fetch(result.presigned_urls[0]).then((r) => r.text());
return parseCsv(csv);
}
function parseCsv(text) {
const [header, ...lines] = text.trim().split("\n");
const cols = header.split(",");
return lines.map((line) =>
Object.fromEntries(cols.map((c, i) => [c, line.split(",")[i]]))
);
}
```
***
## Optional: Add caching
Without caching, every visitor and every refresh triggers a new query to Nekt. Since your tables only change when the scheduled Queries rerun, you can safely serve a cached copy for a while. This makes the report load instantly and reduces load on Nekt.
Paste this into your agent:
```text Prompt theme={null}
Add caching to api/report.js so the app does not query Nekt on every page load.
Set the response header:
Cache-Control: s-maxage=3600, stale-while-revalidate=86400
so Vercel's edge network serves a cached copy for one hour, then refreshes it in
the background. Cache the parsed data, not the presigned URLs (they expire after
an hour). If my tables refresh less often than hourly, use a longer cache time.
```
**Confirm it worked:** reload the report a few times. After the first load it should return instantly without querying Nekt again.
For finer control or a cache shared across regions, ask the agent to use [Vercel KV](https://vercel.com/docs/storage/vercel-kv) with an explicit expiration instead of the header.
***
## Optional: Add authentication
If your report is meant to be public, skip this. If it holds internal data, gate it.
Paste one of these into your agent:
```text Prompt: simplest, no code theme={null}
My report is deployed on Vercel and I want to keep it private. Walk me through
turning on Password Protection (or Vercel Authentication) in the Vercel project
settings, so only people with the password or my Vercel team can open it.
```
```text Prompt: app-level password theme={null}
Add a simple password gate to my report using a REPORT_TOKEN environment variable.
Require the token in api/report.js, and add a small login prompt on the page that
stores it in the browser after the first entry. Do not expose the token in the
code.
```
**Confirm it worked:** open the report in a private browser window. You should be asked to authenticate before seeing any data.
Whichever method you choose, your Nekt API key stays server-side only. Never let the agent place it in browser code.
***
## Step 3: Deploy with Vercel
Finally, the agent publishes the app to a public URL. Connecting a GitHub repository is recommended (every change you make later redeploys automatically) but optional.
Paste this into your agent:
```text Prompt theme={null}
Deploy this app to Vercel and give me the live URL.
- Set the environment variable NEKT_API_KEY to this value: [paste your Nekt API key].
(Also set REPORT_TOKEN if we added a password.)
- Deploy to production.
- Recommended: initialize a git repository and push it to GitHub, then connect the
repo to Vercel so future changes deploy automatically.
- If there is any step you cannot do for me (like authorizing GitHub or Vercel in
the browser), stop and give me clear instructions.
```
**Confirm it worked:** open the URL the agent returns. Your report is now live and shows current data from Nekt.
Set your Nekt API key as a Vercel **environment variable** only, as in the prompt above. Never commit it to your code or GitHub repository.
***
## Fast path: one prompt
If you would rather run the whole flow at once, paste this single prompt into your agent (connected to Nekt) after completing [Before you start](#before-you-start):
```text Prompt theme={null}
I have a static HTML report built on my Nekt data. Help me turn it into a live web
app deployed on Vercel. Do the following end to end, pausing only when you need
something from me (like my API key or a browser authorization):
1. Using the Nekt MCP tools, turn each SQL query behind my report into a scheduled
Nekt Query that materializes a clean table in the trusted layer, named
report_, refreshing daily. My queries: [paste SQL, or point to the
report file].
2. Build a Vercel app around my existing report. Add api/report.js that reads each
materialized table via the Nekt Data API
(POST https://api.nekt.ai/api/v1/sql-query/, header x-api-key from NEKT_API_KEY,
body {"sql": ..., "mode": "csv"}, download the returned presigned CSV and parse
it). Query all tables in parallel and return them as JSON keyed by section.
Update the report to fetch /api/report on load. Keep the API key server-side.
3. Add edge caching with Cache-Control: s-maxage=3600, stale-while-revalidate=86400.
4. Ask me whether the report should be private. If yes, help me enable Vercel
Password Protection or add a REPORT_TOKEN gate.
5. Deploy to Vercel with NEKT_API_KEY set as an environment variable, and give me
the live URL. Offer to connect a GitHub repo for automatic redeploys.
```
***
## Keeping it fresh
Your report now updates itself: the scheduled Queries refresh the tables in Nekt, and the app reads the latest data on each cache cycle.
* **Change what it shows:** ask the agent to update the Nekt Query or the SQL in `api/report.js`.
* **Change how it looks:** ask the agent to edit the report and redeploy (automatic if you connected GitHub).
Endpoint details, output formats, and limitations.
Schedule and orchestrate the queries that feed your report.
Build and edit the report with Claude over MCP.
Build and edit the report with Codex over MCP.
## Need help?
Contact our support team if you run into issues while building your live report.
# From data to app with Lovable
Source: https://docs.nekt.com/guides/lovable-app
Build a Lovable app powered by your Nekt data, using the MCP Server, the Data API, or a Supabase destination.
[Lovable](https://lovable.dev) turns prompts into working web applications. On its own, it builds an app with mock data. Connected to Nekt, it builds an app on top of your real, governed tables, and keeps showing current data after you ship it.
This guide covers the three ways to connect the two, when to use each, and the exact prompts to paste into Lovable.
**You do not need to write code.** Each step gives you a prompt to paste into Lovable or your agent. The text around each prompt explains what is happening and how to confirm it worked.
## Build time vs. runtime
The single most useful thing to understand before you start: the three methods do not compete, they solve different halves of the problem.
* The **MCP Server** is how the *Lovable agent* understands your data while it builds. It reads your schemas, previews rows, and generates an app grounded in tables that actually exist. It is not used by your app's end users.
* The **Data API** and the **Supabase destination** are how the *deployed app* gets data at runtime, when a real user opens a page.
A typical project uses MCP plus one runtime method.
```mermaid theme={null}
flowchart LR
subgraph BUILD["Build time"]
L[Lovable agent] -->|MCP Server| N[Nekt]
end
subgraph RUN["Runtime"]
A[Your app] -->|Edge Function
Data API| N2[Nekt]
A -->|SQL| S[(Supabase)]
N2b[Nekt] -->|Destination
on a schedule| S
end
```
## Choosing a runtime method
| | **Data API** | **Supabase destination** |
| :- | :- | :- |
| **How data arrives** | The app queries Nekt on demand | Nekt pushes tables into Supabase on a schedule |
| **Latency** | Seconds per query (analytical engine) | Milliseconds (Postgres) |
| **Best for** | Dashboards, reports, aggregates that change slowly | Search, filters, pagination, per-user views |
| **Data volume** | Any size, filtered in SQL | Whatever fits comfortably in your Postgres |
| **Freshness** | As fresh as the underlying table | As fresh as the last destination run |
| **Setup effort** | One Edge Function and an API key | A destination in Nekt, no app code |
| **Joins with app data** | Not directly | Yes, it is the same database |
If your app has interactive filtering or search over more than a few thousand rows, use the **Supabase destination**. If it renders precomputed charts and KPIs, use the **Data API**. When in doubt, Supabase is the more forgiving default for an app.
## Before you start
You need at least one table in the [Catalog](/catalog/overview). If you are starting from scratch, add a [Source](/sources/overview) first.
Sign up at [lovable.dev](https://lovable.dev). New Lovable projects come with a Supabase backend, which matters for two of the three methods below.
The MCP Server requires a **paid Nekt plan**, and custom MCP servers require a **paid Lovable plan**. The Data API requires a **paid Nekt plan**. The Supabase destination works on any plan that includes Destinations.
***
## Step 1: Shape app-ready tables in Nekt
Whichever method you choose, do this first. An app should read tables that are already shaped the way the screen displays them: one row per data point, columns already named and typed, no joins or heavy aggregation left for runtime.
Precomputing in Nekt makes the app faster, keeps the business logic in one governed place, and means you can change what the app shows without touching the app.
Paste this into Claude, Codex, or any agent connected to the [Nekt MCP Server](/mcp-server/introduction):
```text Prompt theme={null}
Using the Nekt MCP tools, create the tables my app will read.
I am building an app with these screens: [describe each screen, for example:
a KPI header, a monthly revenue chart, and a searchable customer list].
For each screen:
- Create a scheduled Query in Nekt that writes a clean table to the trusted layer.
- Name each table app_, for example app_kpis, app_monthly_revenue,
app_customers.
- Shape each table so it matches exactly what the screen shows, one row per data
point, so the app does not have to transform anything.
- Give every table a stable, unique identifier column I can use as a primary key.
- Schedule it to refresh daily.
When done, list the tables you created with their columns and types.
```
**Confirm it worked:** open your [Catalog](https://app.nekt.ai) and check that each `app_*` table exists and holds the expected rows.
For logic SQL cannot express (machine learning, external API calls, complex reshaping), ask the agent to use a [Notebook](/notebooks/overview) instead. The result lands in the Catalog the same way, so the rest of this guide is unchanged.
***
## Method 1: Connect the MCP Server
This gives the Lovable agent direct, read-only access to your Catalog while it builds. Instead of inventing a schema, it lists your real tables, previews rows, and writes the UI against columns that exist.
Requires a **paid Lovable plan** (for custom MCP servers) and a **paid Nekt plan** (for MCP Server access).
### Setup
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**, and click **Enable now** if the gateway is not enabled yet. Copy the **Endpoint** URL. See the [MCP setup guide](/mcp-server/setup) for cloud-specific steps.
In the **Access tokens** section, create a token with **Custom scope**. Scope it to only the `app_*` tables from Step 1 rather than granting full access, since this token defines everything the agent can read.
In Lovable, go to **Settings → Connectors → Personal connectors** and click **New MCP server**.
* **Server name**: `Nekt`
* **Server URL**: the Endpoint you copied
Authenticate with **OAuth** (recommended) by clicking **Add & authorize** and logging in with your Nekt credentials, or select the bearer token option and paste the token you created.
Nekt appears in your list of personal connectors. Ask Lovable *"List the tables available in Nekt"* and check that your `app_*` tables come back.
### Build the app
Paste this into Lovable:
```text Prompt theme={null}
You have access to my Nekt MCP server.
1. List the tables available in Nekt and preview a few rows of each app_* table.
2. Build a dashboard with one section per table:
- app_kpis as a KPI header row
- app_monthly_revenue as a line chart
- app_customers as a searchable, paginated table
3. Use the real column names and types from Nekt. Do not invent fields.
4. For now, seed the UI with the sample rows you previewed so I can review the
layout. I will wire up live data next.
```
**Confirm it worked:** the preview renders with your actual column names and realistic values, not placeholder data like "Product A" or "[user@example.com](mailto:user@example.com)".
The MCP connection belongs to your Lovable account, not to the deployed app. When a user opens your published app, it cannot reach Nekt through MCP. Keep going to Method 2 or Method 3 to give the app live data.
***
## Method 2: Query the Data API at runtime
The app calls a small server-side function, that function queries Nekt with SQL, and the results come back as JSON. Your Nekt API key never leaves the server.
In a Lovable project, that server-side function is a **Supabase Edge Function**, which every Lovable project can create.
### Architecture
```mermaid theme={null}
flowchart LR
A[Browser
your Lovable app] -->|invoke| B[Supabase
Edge Function]
B -->|cached JSON| A
B -->|Data API
x-api-key| C[Nekt]
C -->|app_* tables| D[(Catalog)]
```
### Setup
Go to **Workspace Settings → API Keys** and [create an API key](https://app.nekt.ai/settings/api-keys). Copy it, you will paste it into Lovable as a secret.
Paste the prompt below. When Lovable asks where to store the API key, add it as a Supabase secret named `NEKT_API_KEY`, never as a plain value in the code.
```text Prompt theme={null}
Add live data from Nekt to this app using the Nekt Data API.
Create a Supabase Edge Function called nekt-data that:
- Reads my API key from the NEKT_API_KEY secret. Never expose it to the browser.
- Calls POST https://api.nekt.ai/api/v1/sql-query/
- Header: x-api-key: , Content-Type: application/json
- Body: {"sql": "", "mode": "csv"}
- Reads presigned_urls from the response, downloads the CSV from the first URL,
and parses it into rows.
- Runs one query per table (app_kpis, app_monthly_revenue, app_customers) in
parallel and returns all datasets as JSON keyed by section.
- Only runs SQL from a fixed list defined inside the function. Never accept SQL
from the client.
- Caches each result for 1 hour, since the tables refresh daily.
Then update the app to call this function on load and render each dataset into
its matching section, with loading and error states.
```
**Confirm it worked:** open the preview. The sections fill with current Nekt data, and the Edge Function logs in Supabase show a successful call.
The Data API runs SQL and returns a presigned URL to download the results:
```bash Request theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sql-query/ \
--header 'x-api-key: YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"sql": "SELECT month, revenue FROM \"trusted\".\"app_monthly_revenue\" ORDER BY month",
"mode": "csv"
}'
```
```json Response theme={null}
{
"state": "SUCCEEDED",
"presigned_urls": [
"https://storage.googleapis.com/bucket/query-....csv?..."
],
"data_scanned_in_bytes": 11250,
"execution_time_in_millis": 408
}
```
```javascript supabase/functions/nekt-data/index.ts theme={null}
// One entry per section. The client picks a key, never sends SQL.
const DATASETS = {
kpis: `SELECT * FROM "trusted"."app_kpis"`,
revenue: `SELECT month, revenue FROM "trusted"."app_monthly_revenue" ORDER BY month`,
customers: `SELECT id, name, total_spent FROM "trusted"."app_customers"`,
};
async function runQuery(sql: string) {
const res = await fetch("https://api.nekt.ai/api/v1/sql-query/", {
method: "POST",
headers: {
"Content-Type": "application/json",
"x-api-key": Deno.env.get("NEKT_API_KEY")!,
},
body: JSON.stringify({ sql, mode: "csv" }),
});
const result = await res.json();
if (result.state !== "SUCCEEDED" || !result.presigned_urls?.length) {
throw new Error(`Query failed: ${result.state_change_reason ?? result.state}`);
}
const csv = await fetch(result.presigned_urls[0]).then((r) => r.text());
return parseCsv(csv);
}
```
Presigned URLs expire after **1 hour**, and the results themselves live in your own cloud storage, not on Nekt servers. Cache the parsed rows, never the URL.
### What to watch out for
* **Cache aggressively.** The Data API is powered by Athena or BigQuery, so a query takes seconds, not milliseconds. Without caching, every page load pays that cost and requeries data that only changes daily.
* **Keep the SQL server-side.** The Edge Function should expose a fixed set of named datasets. Accepting SQL from the browser hands any visitor full read access to your Catalog.
* **All CSV values are strings.** Ask Lovable to cast numbers and dates when parsing. Or use `parquet` mode, reviewing the [known limitations](/data-api/known-limitations) first.
* **This is not a transactional API.** Filtering and pagination against the Data API on every keystroke will feel slow. That is what Method 3 is for.
***
## Method 3: Send data to Supabase
Nekt writes your tables directly into the Postgres database your Lovable app already uses. The app queries them with its normal Supabase client, with no Nekt-specific code anywhere, and Nekt refreshes them on a schedule.
This is the best fit for real applications: filters, search, pagination, and joins against your app's own tables all work at Postgres speed.
### Architecture
```mermaid theme={null}
flowchart LR
D[(Nekt Catalog)] -->|Destination
scheduled| S[(Supabase Postgres)]
S -->|supabase-js| A[Your Lovable app]
```
### Setup
In your Lovable project, open the linked Supabase project. Click **Connect** in the top navigation and select the **Session pooler** tab. Copy the host, port, user, password, and database.
You must use the **Session pooler** connection. Transaction mode and direct connections are not supported.
If your Supabase project has **Network Restrictions** enabled (**Project Settings → Database → Network Restrictions**), add Nekt's public IP in CIDR notation, for example `203.0.113.42/32`. The IP is shown on the connector configuration page in Nekt.
Go to [Destinations](https://app.nekt.ai/destinations), click **Add destination**, and select **Supabase**. Enter the Session pooler credentials and set the target schema.
Load Nekt tables into a dedicated schema such as `nekt`, not `public`. It keeps a clear line between tables your app owns and tables Nekt manages, and it prevents a load method from ever touching your application data.
Choose the `app_*` tables from Step 1, define a primary key for each, pick a load method (**upsert** is the usual choice for app tables), and set how often the data should refresh.
See the full [Supabase destination guide](/destinations/supabase) for load methods, SSL, and record metadata.
Trigger the destination manually with the refresh icon, then confirm the tables appear in the Supabase Table Editor.
### Secure the tables
Tables loaded by Nekt arrive without row level security. If they sit in an API-exposed schema, anyone with your app's public key can read every row.
Before publishing your app, enable RLS on every Nekt-managed table and add explicit read policies. This is the most common mistake in this setup.
Paste this into Lovable:
```text Prompt theme={null}
Nekt loads tables into my Supabase database on a schedule. Secure them:
1. Enable row level security on every table in the nekt schema.
2. Add a policy that allows authenticated users to SELECT, and nothing else.
No INSERT, UPDATE, or DELETE from the client, since Nekt owns this data and
any writes will be overwritten on the next sync.
3. If a table should only be visible to certain users, ask me which column
identifies the owner and scope the policy to it.
```
### Build the app
```text Prompt theme={null}
Build the app against the Nekt-managed tables in my Supabase database:
- app_kpis renders as a KPI header row.
- app_monthly_revenue renders as a line chart.
- app_customers renders as a table with server-side search, sorting, and
pagination, querying Supabase directly rather than loading everything at once.
Treat these tables as read-only. Never write to them from the app.
```
**Confirm it worked:** search and paginate the customer table in the preview. It should respond instantly, and the results should match what you see in the Nekt Catalog.
### What to watch out for
* **The tables are read-only.** Anything the app writes into them is lost on the next sync. Keep app-generated data (user preferences, comments, statuses) in separate tables your app owns, and join to the Nekt tables.
* **Match the schedule to the need.** The app is exactly as fresh as the last destination run. Hourly for operational screens, daily for reporting is a reasonable starting point.
* **Set a primary key you trust.** Upsert matches on it. An unstable key produces duplicates.
* **Mind the size.** Supabase is a Postgres instance, not a warehouse. Aggregate and filter in Nekt in Step 1 rather than syncing raw tables of tens of millions of rows.
***
## Putting it together
For most projects, the full flow looks like this:
Scheduled Queries produce clean `app_*` tables (Step 1).
Method 1. The agent sees your actual columns and generates a UI that fits them.
Method 3 (Supabase) for interactive apps, Method 2 (Data API) for dashboards over precomputed data. Nothing stops you from using both: Supabase for the interactive tables, the Data API for an occasional heavy aggregate.
RLS policies on Supabase tables, secrets for the Nekt API key, then publish from Lovable.
## Keeping it fresh
Your app now updates itself. Scheduled Queries refresh the tables in Nekt, and the destination or the Edge Function carries the new data to the app.
* **Change what it shows:** update the Nekt Query. The app picks up the new values with no redeploy.
* **Change the shape of a table:** update the Query, then tell Lovable which columns changed.
* **Change how it looks:** prompt Lovable and republish.
Connection reference, including the Google Sheets option.
Full setup, load methods, and troubleshooting.
Endpoint details, output formats, and limitations.
Tools, tokens, and permissions.
## Need help?
Contact our support team if you run into issues while building your app.
# History
Source: https://docs.nekt.com/histories/overview
Track historical changes in your data over time with a no-code SCD Type 2 implementation.
The History module lets you create a complete history of any table using a no-code SCD Type 2 template.
Unlike Type 1 (which overwrites old data) or Type 3 (which stores limited history), the History module automatically creates new records for each change, making it perfect for tracking changes in customer information, product details, or employee records over time.
### Creating a History
Choose the source table containing the data you want to version. Make sure the table has a reliable way to detect changes.
Specify the business keys that uniquely identify each record (e.g., `customer_id`, `product_id`). Nekt automatically creates a surrogate key for versioning.
Select the output layer, name the history table, and set the run schedule. Choose from schedule-based, event-based, or manual triggers. See [Triggers documentation](/runs/scheduling-and-triggers) for details.
### Best Practices
Choose primary keys that are stable and will not change over the lifetime of a record. Changing primary keys mid-stream causes incorrect version boundaries and corrupts the historical chain.
Document the business rules that determine what counts as a meaningful change for each History. This context is easy to lose and hard to reconstruct later.
Monitor storage growth and the number of changes detected per run. A sudden spike usually indicates an upstream data issue rather than genuine business activity.
### Use Cases
ML models need versioned behavior history (plan downgrades, engagement drops), not just today's snapshot. History provides that automatically without custom pipelines.
CRM records change constantly. Versioned snapshots let scoring models understand when a lead's profile improved or degraded over time, improving model accuracy.
RAG pipelines and AI agents often need to know what was true at a specific moment. History makes it possible to reconstruct the state of any record at any point in time.
ML monitoring compares current feature values against historical baselines. History creates and maintains those baselines without custom code.
## API & MCP
Histories are transformations: list, update, trigger, and delete them.
No dedicated history tool yet.
# ChatGPT
Source: https://docs.nekt.com/integrations/chatgpt
Connect ChatGPT to your data in Nekt using the MCP Server with OAuth authentication.
ChatGPT can query and analyze your data when connected to Nekt through the MCP Server. The connection uses OAuth with **Dynamic Client Registration (DCR)**, so ChatGPT registers itself with your MCP Server and you authenticate with your Nekt credentials directly: no client IDs, secrets, or manual token management required.
Requires a **paid Nekt plan** (for MCP Server access) and a **ChatGPT Plus, Team, or Enterprise** subscription (for MCP support).
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. OAuth endpoints
Nekt's MCP Server is available at `https://mcp.nekt.com`, and the OAuth endpoints all share that host:
| Field in ChatGPT | Value |
| :- | :- |
| **Server URL** | `https://mcp.nekt.com/mcp` |
| **Authentication URL** | `https://mcp.nekt.com/authorize` |
| **Token URL** | `https://mcp.nekt.com/token` |
| **Registration URL** | `https://mcp.nekt.com/register` |
| **Authorization server base URL** | `https://mcp.nekt.com/` |
| **Resource** | `https://mcp.nekt.com/mcp` |
ChatGPT discovers these values from the MCP Server, so they are usually filled in for you. Confirm them against the table and fill in anything missing.
If your workspace runs on your own cloud with a dedicated MCP Server host, use the
**Endpoint** value shown in **MCP Gateway** instead, and replace the
`/mcp` suffix to get the other endpoints.
## 3. Connect ChatGPT to Nekt
Go to [chatgpt.com/plugins](https://chatgpt.com/plugins) and click the **+** button in the search bar to add a new MCP server.
Give the plugin a name (for example, `nekt`). The description and icon are optional.
For **Connection**, keep **Server URL** selected and enter `https://mcp.nekt.com/mcp`.
Set **Authentication** to **OAuth**.
Open **Advanced OAuth settings** and select **Dynamic Client Registration (DCR)** as the client configuration method.
Review the OAuth endpoints against the table above, then leave the default scopes empty.
Nekt no longer issues static OAuth client IDs and secrets for this
integration. Do not select the manual client configuration method; the MCP
Server registers a client for ChatGPT through the `/register` endpoint. If
the **Registration URL** is missing, dynamic registration fails.
Leave **OIDC enabled** off. The MCP Server does not publish an OIDC
configuration URL, so OpenID support is unavailable.
ChatGPT warns that custom MCP servers have not been reviewed by OpenAI. Select **I understand and want to continue**, then create the plugin.
ChatGPT will redirect you to the Nekt login page. Sign in with your Nekt credentials.
After authenticating, a list of your MCP tokens will be displayed.
Tokens authenticate your requests to Nekt. Each token defines what ChatGPT can access.
* If you haven't created any custom tokens yet, you'll see a default one created automatically with Full Access, meaning the same access your Nekt user has.
* If you already have custom tokens, select the one you want to use.
Click **Select Token** to confirm.
Tokens created during the OAuth flow use the creating user's permissions by
default. Tokens created manually can be scoped per resource type and per
tool. See [Permissions](/mcp-server/permissions).
The Nekt MCP Server provides a set of tools that AI agents use to interact with your data. You can learn more about them [here](/mcp-server/tools).
When ChatGPT prompts you to approve the Nekt tools, confirm to allow access.
***
## 4. Explore your data
Once connected, ask ChatGPT a question about your data. For example:
```
Give me a summary of the your_table_name table from Nekt.
```
ChatGPT will use the MCP tools to query your data and return an answer. You can see which tools were called and the queries executed in the response.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect ChatGPT to Nekt through the MCP Server, ChatGPT does not receive direct database credentials or unrestricted access to your infrastructure. Instead, ChatGPT interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data ChatGPT can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
When using ChatGPT, your interactions are subject to OpenAI's privacy settings and policies. OpenAI allows users to control whether their conversations may be used for model training through their Data Controls settings.
We recommend reviewing the [official OpenAI privacy documentation](https://openai.com/policies/privacy-policy) to understand their data handling policies and selecting the configuration that best aligns with your organization's security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# Claude
Source: https://docs.nekt.com/integrations/claude
Connect Claude to your data in Nekt and activate it with flexibility and ease.
Claude is a powerful and versatile AI tool, but it becomes even stronger when connected to your data. You can connect Claude to Nekt through **Claude.ai** (web) or **Claude Code** (CLI).
Requires a **paid Nekt plan** (for MCP Server access).
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. Connect Claude to Nekt
Choose the setup that matches how you use Claude.
Check the video below for a visual guide:
### Add the MCP as a custom connector
Access Claude, and click on the `+` icon on the prompt box to add a connector.
In the menu, select **Connectors**, hover over **Add connector**, and click **Add custom connector**.
Fill in the required information, which you'll find in the **Connection fields** section inside Nekt.
1. Name: we suggest "Nekt"
2. Remote MCP Server URL: paste the value of the `Endpoint` field
3. \[Optional] Open the advanced settings section and paste the values of the `OAuth Client ID` and `OAuth Client Secret` fields.
Click on **Add**.
### Connect to Nekt
Now you should have a new item called "Nekt" (or the name you previously defined) in the list of **Not Connected** connectors.
Click on it and then on the **Connect** button in the middle of the screen. A Nekt authentication screen will open with two options:
* **Connect with a token**: paste your Nekt MCP token directly and click **Connect**. No Nekt account or login required.
* **Log in with your credentials**: click the link at the bottom of the screen to sign in with your Nekt user and password, then select which token you want to use.
Tokens authenticate your requests to Nekt and define what Claude can access.
* If you use the credentials flow and haven't created any custom tokens yet, a default one is created automatically with Full Access, meaning the same access your Nekt user has.
* If you already have custom tokens, they appear in the list; pick the one you want and click **Select Token**.
Tokens created during the OAuth flow use the creating user's permissions by
default. Tokens created manually can be scoped per resource type and per
tool. See [Permissions](/mcp-server/permissions).
Once connected, you'll see a success message and be redirected back to Claude.
### Manage tool permissions
The Nekt MCP Server provides tools that AI agents can use to interact with your data. You can learn more about them [here](/mcp-server/tools).
Make sure to grant the **Always allow** permission to these tools.
### Get your endpoint and token
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**. You'll need two values:
* **Endpoint**: the MCP server URL displayed at the top of the page
* **Token**: a bearer token created in the **Access tokens** section
If you don't have a token yet, create one, choosing **Full access** or **Custom scope**.
### Add the MCP server to your configuration
You can add the MCP server using the CLI or by editing the configuration file manually.
**Option A: Using the CLI**
```bash theme={null}
claude mcp add --transport http Nekt MCP_ENDPOINT \
--header "Authorization: Bearer MCP_TOKEN"
```
**Option B: Editing the configuration file**
Open your `.claude.json` file (in your project root or home directory) and add the following block:
```json theme={null}
{
"mcpServers": {
"Nekt": {
"type": "http",
"url": "MCP_ENDPOINT",
"headers": {
"Authorization": "Bearer MCP_TOKEN"
}
}
}
}
```
Replace `MCP_ENDPOINT` with the endpoint from the MCP Gateway page and `MCP_TOKEN` with the token you created.
***
## 3. Explore your data
Once connected, ask Claude a question about your data. Here's an example:
```
Using /nekt give me insight about your_table_name table.
```
While it processes your request, you can see the steps taken and the queries executed to get to a final answer. It may take a few minutes.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect Claude to Nekt through the MCP Server, Claude does not receive direct database credentials or unrestricted access to your infrastructure. Instead, Claude interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data Claude can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
When using Claude (via Claude.ai or Claude Code), your interactions are subject to Anthropic’s privacy settings and policies. Anthropic allows users to control whether their conversations may be used for model training through their Privacy Settings.
We recommend reviewing the official privacy documentation from the respective providers to understand their data handling policies and selecting the configuration that best aligns with your organization’s security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# Codex
Source: https://docs.nekt.com/integrations/codex
Connect OpenAI Codex to your data in Nekt using the MCP Server with bearer token authentication.
Codex is OpenAI's AI coding agent. When connected to Nekt through the MCP Server, it can query and analyze your data directly from the CLI. The connection uses bearer token authentication via an environment variable.
Requires a **paid Nekt plan** (for MCP Server access) and the **Codex CLI** installed (`codex` command available).
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. Get your endpoint and token
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**. You'll need two values:
* **Endpoint**: the MCP server URL displayed at the top of the page
* **Token**: a bearer token created in the **Access tokens** section
If you don't have a token yet, create one, choosing **Full access** or **Custom scope**.
Tokens can be scoped per resource type (tables, volumes, secrets, context layer, and live connections) and per tool. Picking individual resources requires the Growth or Custom plan. See [Permissions](/mcp-server/permissions).
## 3. Connect Codex to Nekt
Export the token in your current shell session:
```bash theme={null}
export NEKT_MCP_BEARER_TOKEN="YOUR_TOKEN_HERE"
```
To persist it across sessions, add it to your shell profile:
```bash theme={null}
echo 'export NEKT_MCP_BEARER_TOKEN="YOUR_TOKEN_HERE"' >> ~/.zshrc
source ~/.zshrc
```
You can register the MCP server using the CLI or by editing the configuration file directly.
**Option A: Using the CLI**
```bash theme={null}
codex mcp add nekt --url https://mcp.nekt.com/mcp --bearer-token-env-var NEKT_MCP_BEARER_TOKEN
```
**Option B: Editing the configuration file**
Open `~/.codex/config.toml` and add the following block:
```toml theme={null}
[mcp_servers.nekt]
url = "https://mcp.nekt.com/mcp"
bearer_token_env_var = "NEKT_MCP_BEARER_TOKEN"
```
Confirm the server was registered correctly:
```bash theme={null}
codex mcp list
codex mcp get nekt --json
```
The server `nekt` should appear as enabled, with the URL `https://mcp.nekt.com/mcp` and auth mode showing bearer token via env var.
You can also ask Codex to list the available Nekt tools to confirm the connection is working. You should see the tools provided by the Nekt MCP Server. You can learn more about them [here](/mcp-server/tools).
The OAuth login flow (`codex mcp login nekt`) is not supported for Nekt. Use bearer token authentication as described above.
***
## 4. Explore your data
Once connected, ask Codex a question about your data. For example:
```
Using Nekt, give me a summary of the your_table_name table.
```
Codex will use the MCP tools to query your data and return an answer.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect Codex to Nekt through the MCP Server, Codex does not receive direct database credentials or unrestricted access to your infrastructure. Instead, Codex interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data Codex can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
When using Codex, your interactions are subject to OpenAI's privacy settings and policies. We recommend reviewing the [official OpenAI privacy documentation](https://openai.com/policies/privacy-policy) to understand their data handling policies and selecting the configuration that best aligns with your organization's security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# Cursor
Source: https://docs.nekt.com/integrations/cursor
Connect Cursor to your data in Nekt using the MCP Server with bearer token authentication.
Cursor is an AI-powered code editor. When connected to Nekt through the MCP Server, it can query and analyze your data directly while you code. The connection uses bearer token authentication via Cursor's `mcp.json` configuration.
Requires a **paid Nekt plan** (for MCP Server access) and **Cursor** installed.
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. Get your endpoint and token
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**. You'll need two values:
* **Endpoint**: the MCP server URL displayed at the top of the page
* **Token**: a bearer token created in the **Access tokens** section
If you don't have a token yet, create one, choosing **Full access** or **Custom scope**.
Tokens can be scoped per resource type (tables, volumes, secrets, context layer, and live connections) and per tool. Picking individual resources requires the Growth or Custom plan. See [Permissions](/mcp-server/permissions).
## 3. Connect Cursor to Nekt
In Cursor, open **Settings → Tools & Integrations → MCP**, then click **Add new MCP server**. This opens (or creates) the configuration file.
You can also edit the file manually:
* **Global** (available in every project): `~/.cursor/mcp.json`
* **Project-scoped** (committed with the repo): `.cursor/mcp.json` in the project root
Add the following block to `mcp.json`:
```json theme={null}
{
"mcpServers": {
"nekt": {
"url": "MCP_ENDPOINT",
"headers": {
"Authorization": "Bearer MCP_TOKEN"
}
}
}
}
```
Replace `MCP_ENDPOINT` with the endpoint from the MCP Gateway page and `MCP_TOKEN` with the token you created.
To avoid storing the token in plaintext, set it as an environment variable in your shell profile and reference it with `${env:NEKT_MCP_BEARER_TOKEN}` in the headers value. This is especially important for project-scoped `mcp.json` files committed to version control.
Go back to **Settings → Tools & Integrations → MCP**. The `nekt` server should appear with a green status indicator and the list of available tools.
You should see the tools provided by the Nekt MCP Server. You can learn more about them [here](/mcp-server/tools).
***
## 4. Explore your data
Once connected, ask Cursor's chat a question about your data. For example:
```
Using Nekt, give me a summary of the your_table_name table.
```
Cursor will use the MCP tools to query your data and return an answer.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect Cursor to Nekt through the MCP Server, Cursor does not receive direct database credentials or unrestricted access to your infrastructure. Instead, Cursor interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data Cursor can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
When using Cursor, your interactions are subject to Cursor's privacy settings and policies. We recommend reviewing the [official Cursor privacy documentation](https://cursor.com/privacy) to understand their data handling policies and selecting the configuration that best aligns with your organization's security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# Data API
Source: https://docs.nekt.com/integrations/data-api
Query your data using SQL through the Nekt Data API.
The Data API lets you run SQL queries against your Catalog tables over HTTP, enabling programmatic access to your data from any application or service.
Query your data using SQL over HTTP.
# Grafana
Source: https://docs.nekt.com/integrations/grafana
Connect Grafana to your data in Nekt for operational dashboards and monitoring.
Connect Grafana to your data in Nekt. How you connect depends on your cloud deployment.
| | Connector | Notes |
| :- | :- | :- |
| **Nekt Express** | Google BigQuery | Via the Grafana BigQuery plugin |
| **GCP** | Google BigQuery | Via the Grafana BigQuery plugin |
| **AWS** | Amazon Athena | Via the Grafana Athena plugin |
Nekt does not have a native connector for Grafana. Registering an integration in the Nekt platform has no effect on the Grafana connection: it only helps you stay organized about your existing dashboards. The connection is configured directly in Grafana.
***
## Nekt Express
Nekt Express uses GCP infrastructure. Connect Grafana via the **BigQuery data source plugin**.
Detailed BigQuery plugin setup for Grafana is available in the [Grafana BigQuery documentation](https://grafana.com/grafana/plugins/grafana-bigquery-datasource/). You will need a Google service account with BigQuery read access to your Nekt project (`nekt-hosted-infrastructure`).
1. In Grafana, go to **Connections** → **Add new connection** and search for **Google BigQuery**.
2. Install the plugin and add it as a data source.
3. Authenticate using a Google service account with access to the `nekt-hosted-infrastructure` project.
4. Select your dataset (each layer is a dataset in BigQuery) and start building dashboards.
***
## GCP
Your Nekt workspace is on your own GCP account. Connect Grafana using the **BigQuery plugin** with your own project credentials.
You will need a Google service account with BigQuery read access to your GCP project. Refer to the [Grafana BigQuery plugin documentation](https://grafana.com/grafana/plugins/grafana-bigquery-datasource/) for setup details.
1. In Grafana, go to **Connections** → **Add new connection** and search for **Google BigQuery**.
2. Install the plugin and add it as a data source.
3. Authenticate using your GCP service account credentials.
4. Select your project, dataset, and tables.
***
## AWS
Your Nekt workspace is on your own AWS account. Connect Grafana using the **Amazon Athena plugin**. You will need AWS credentials (Access Key ID and Secret Access Key).
> Generate AWS credentials using [this AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-key-self-managed.html#Using_CreateAccessKey).
### Step 1: Install the Athena plugin
1. In Grafana, go to **Connections** → **Add new connection** and search for **Amazon Athena**.
2. Filter by **Data sources** if needed.
3. Click **Install via grafana.com**.
![]()
On the plugin page, click **Install plugin**.
![]()
### Step 2: Configure the Athena data source
1. Go to **Connections** → **Data Sources** → **Add data source** and select **Amazon Athena**.
![]()
2. Keep the default authentication provider (**Access and secret key**) and enter your **Access Key ID** and **Secret Access Key**.
![]()
3. Enter the **AWS region** where your Nekt workspace is hosted.
4. Fill in the **Athena details** section:
* **Data Source**: `AwsDataCatalog`
* **Database**: the layer you want to connect (each layer is a database)
* **Workgroup**: `primary`
* **Output Location**: `s3://nekt-lakehouse-{your_company_slug}/_athena`
Contact Nekt support to get your company slug.
### Step 3: Save and test
Click **Save & Test**. If the connection is successful, you can start building dashboards.
> For more details, see [Using the Amazon Athena plugin for Grafana](https://grafana.com/blog/2021/12/13/query-and-analyze-amazon-s3-data-with-the-new-amazon-athena-plugin-for-grafana/).
# Hermes
Source: https://docs.nekt.com/integrations/hermes
Connect Hermes Agent to your data in Nekt using the MCP Server with bearer token authentication.
[Hermes Agent](https://github.com/NousResearch/hermes-agent) by Nous Research is an AI agent with a native MCP client. When you add the Nekt MCP Server to its configuration, Hermes connects on startup, discovers every available tool, and registers each one with the `mcp_nekt_*` prefix across all its platforms (CLI, Telegram, Discord, Slack, and others).
Hermes is the **agent**, while the underlying **LLM** is configured separately: you can run it on any model, including DeepSeek, OpenAI, or others. Nekt's MCP Server exposes the tools; Hermes passes their schemas to whichever model you've configured, and the model decides which tools to call.
Once connected, you can simply ask *"what was the revenue per customer last month?"* and Hermes uses the Nekt tools to discover the right tables, generate SQL, run the query, and answer.
Requires a **paid Nekt plan** (for MCP Server access), **Hermes Agent** installed, and the **`mcp` Python package** (`pip install mcp`). Without the `mcp` package, Hermes silently ignores the `mcp_servers` section.
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. Get your endpoint and token
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**. You'll need two values:
* **Endpoint**: the MCP server URL displayed at the top of the page
* **Token**: a bearer token created in the **Access tokens** section
If you don't have a token yet, create one, choosing **Full access** or **Custom scope**.
Tokens can be scoped per resource type (tables, volumes, secrets, context layer, and live connections) and per tool. Picking individual resources requires the Growth or Custom plan. See [Permissions](/mcp-server/permissions).
## 3. Connect Hermes to Nekt
Keep secrets out of the config file by adding them to `~/.hermes/.env`:
```bash theme={null}
NEKT_MCP_URL=https://your-endpoint.mcp.nekt.ai
NEKT_MCP_TOKEN=your-token-here
```
Replace the values with the endpoint and token from the Nekt platform.
Open `~/.hermes/config.yaml` and add the server under the `mcp_servers` key. Hermes interpolates `${VAR_NAME}` from your `.env`:
```yaml theme={null}
mcp_servers:
nekt:
url: ${NEKT_MCP_URL}
headers:
Authorization: "Bearer ${NEKT_MCP_TOKEN}"
timeout: 120 # seconds per tool call (default: 120)
connect_timeout: 60 # seconds for the initial connection (default: 60)
```
The Nekt MCP Server uses HTTP transport with a bearer token, which Hermes supports natively. Increase `timeout` (for example, to `300`) if you run heavy queries that return large result sets.
Reload Hermes so it picks up the new server:
* **CLI**: exit and start again (`hermes`)
* **Gateway** (Telegram, Discord, etc.): send `/restart` in the chat
* **Inline**: run `/reload-mcp` to reload without leaving the session (a full restart is more reliable)
Hermes does not support the interactive OAuth 2.1 flow (browser redirect) for Nekt. Use bearer token authentication as described above. The token does not refresh automatically: when it expires, update `NEKT_MCP_TOKEN` in `.env` and restart Hermes.
Hermes profiles (`default`, `analista`, etc.) do **not** inherit the `mcp_servers` section. If you use profiles, replicate the configuration in each profile's `config.yaml` (e.g. `~/.hermes/profiles/analista/config.yaml`).
## 4. Verify the connection
Each Nekt tool is registered with the `mcp_nekt_` prefix. Confirm the connection from the Hermes chat:
```
Use mcp_nekt_list_layers()
```
Or from the CLI:
```bash theme={null}
hermes mcp test nekt
```
You should see the tools provided by the Nekt MCP Server. You can learn more about them [here](/mcp-server/tools).
***
## 5. Explore your data
Once connected, ask Hermes a question about your data. For example:
```
What were the top 5 customers by revenue this month?
```
Hermes orchestrates the Nekt tools in sequence: discovering relevant tables (`mcp_nekt_get_relevant_tables_ddl`), generating SQL (`mcp_nekt_generate_sql`), executing it (`mcp_nekt_execute_sql`), and interpreting the results, then answers in natural language, without you writing a line of SQL.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Troubleshooting
| Symptom | Likely cause | Action |
| - | - | - |
| No `mcp_nekt_*` tools appear | `mcp` package not installed | `pip install mcp`, then restart |
| `"No MCP servers configured"` | Empty or mis-indented `mcp_servers` | Check `config.yaml` |
| `401 Unauthorized` | Expired or invalid token | Renew `NEKT_MCP_TOKEN` and restart |
| Connection timeout | Endpoint unreachable | Test with `curl -H "Authorization: Bearer $NEKT_MCP_TOKEN" $NEKT_MCP_URL` |
| Works in `default` but not another profile | Profiles don't inherit MCP servers | Replicate `mcp_servers` in that profile |
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect Hermes to Nekt through the MCP Server, Hermes does not receive direct database credentials or unrestricted access to your infrastructure. Instead, Hermes interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data Hermes can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
When using Hermes with a third-party model such as DeepSeek or OpenAI, your interactions are also subject to that provider's privacy policies. We recommend reviewing the relevant provider's documentation to understand their data handling practices and selecting the configuration that best aligns with your organization's security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# Looker Studio
Source: https://docs.nekt.com/integrations/looker-studio
Connect Looker Studio to your data in Nekt to build reports and dashboards.
Connect Looker Studio to your data in Nekt. How you connect depends on your cloud deployment.
| | Connector | Token required |
| :- | :- | :- |
| **Nekt Express** | Google BigQuery (native) | No |
| **GCP** | Google BigQuery (native) | No |
| **AWS** | Nekt custom connector | Yes |
***
## Nekt Express
Nekt Express uses GCP infrastructure. Connect via the native BigQuery connector using the shared Nekt project.
### Step 1: Register the integration in Nekt
1. In the Nekt platform, go to **Integrations** and click **Add integration**.
2. Select **Looker Studio** from the list.
3. Enter the email address associated with your Looker Studio account.
4. Add a description to identify this integration.
### Step 2: Choose the BigQuery connector
1. [Sign in to Looker Studio](https://lookerstudio.google.com/). Use the same email registered in step 1.
2. Click **Create** → **Data Source**.
3. Select the **BigQuery** connector.
4. If prompted, click **AUTHORIZE** and allow the necessary permissions.
### Step 3: Set the Project ID
When prompted, enter the following Project ID:
```
nekt-hosted-infrastructure
```
Press **Enter** to confirm. This is only required the first time; subsequent connections will recognize it automatically.
### Step 4: Select the data and explore
1. Browse to the dataset and table you want to connect.
2. Click **Connect** in the top-right corner.
3. You will see a column preview. Click **Explore** or **Create Report** to proceed.
***
## GCP
Your Nekt workspace is hosted on your own GCP account. Connect directly to BigQuery using your project.
### Step 1: Choose the BigQuery connector
1. [Sign in to Looker Studio](https://lookerstudio.google.com/).
2. Click **Create** → **Data Source**.
3. Select the **BigQuery** connector.
4. If prompted, authorize with your GCP account email and click **Allow**.
### Step 2: Select the data and explore
1. Under **My Projects**, find and select the table you want to connect.
2. Click **Connect** in the top-right corner.
3. Click **Explore** or **Create Report** to proceed.
***
## AWS
Your Nekt workspace is hosted on your own AWS account. Connection is made through **Nekt's custom Looker Studio connector**, which requires a scoped access token.
### Step 1: Generate a token
1. In the Nekt platform, go to **Integrations**, find **Looker Studio**, and click **Configure**.
2. Create an access token and select the tables you want to make available. You can select tables from any Lakehouse layer.
The token grants access only to the tables selected here. You can create multiple tokens to give access to different sets of tables, which is useful for data governance.
3. Click **Next**.
4. Give the integration a description (this helps you identify it later). Optionally, add a link to the report.
5. Click **Next**, then copy the generated token from the Integrations page.
### Step 2: Open Nekt's connector
Access Nekt's Looker Studio connector directly via this link:
[Open Nekt connector in Looker Studio](https://lookerstudio.google.com/datasources/create?connectorId=AKfycbwFfHPxHQzcxVuHeIbCoMywA1UjGCv0YY2gD9N1pnQ)
On your first use, click **AUTHORIZE** and complete the Google sign-in. Click **Allow** to confirm permissions.
### Step 3: Configure the connection
Fill in the access parameters:
1. **Nekt data visualization token**: paste the token generated in Step 1.
2. **Table**: select the table to connect (Looker Studio connects one table at a time).
![]()
Click **NEXT** and configure optional parameters:
* **Date range column name**: the column to use when applying date range filters in reports.
* **Row limit**: maximum rows per query. Default is `1000`. Set to `-1` to fetch all rows.
![]()
You can name the data source at the top of the page.
### Step 4: Connect and explore
Click **Connect** in the top-right corner. You will see a column preview. Click **Explore** or **Create Report** to proceed.
# Lovable
Source: https://docs.nekt.com/integrations/lovable
Connect Lovable to your data in Nekt.
Lovable is an AI-powered app builder that turns prompts into working web applications. These are the most common ways to connect it to your data in Nekt.
For an end-to-end walkthrough with prompts, from shaping tables in Nekt to publishing the app, see [From data to app with Lovable](/guides/lovable-app).
| Method | Best for | How it works |
| :- | :- | :- |
| **MCP Server** | AI-driven exploration, prototyping from live schemas | Lovable queries Nekt in real-time through the MCP protocol |
| **Supabase** | Production apps that need a live database | Nekt sends data to the Supabase instance your Lovable app already uses |
| **Google Sheets** | Quick setups, read-only data, non-technical teams | Nekt sends data to a spreadsheet that Lovable can read from |
| **Data API** | Custom integrations, on-demand SQL queries from app code | Your Lovable app queries Nekt directly via HTTP using SQL |
## MCP Server
Connect Lovable directly to Nekt's MCP Server so the Lovable Agent can explore your schemas, preview data, and generate apps grounded in your actual tables.
Requires a **paid Lovable plan** (for custom MCP servers) and a **paid Nekt plan** (for MCP Server access).
### Setup
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**, and enable the gateway. See the [MCP setup guide](/mcp-server/setup) for detailed instructions.
In Lovable, go to **Settings > Connectors > Personal connectors** and click **New MCP server**.
* **Server name**: enter `Nekt`
* **Server URL**: paste the **Endpoint** from the MCP Gateway page in Nekt
Choose one of the following methods:
**OAuth (recommended)**: select OAuth and click **Add & authorize**. You will be redirected to log in with your Nekt credentials and select a token.
**Bearer token**: select the bearer token option and paste a token from the **Access tokens** section in Nekt.
Tokens created during the OAuth flow use the creating user's permissions by
default. Tokens created manually can be scoped per resource type and per
tool. See [Permissions](/mcp-server/permissions).
Nekt should now appear in your list of personal connectors. You can verify it is active by clicking on the connector in your settings.
### Example prompts
| Scenario | Prompt |
| :- | :- |
| Explore available data | *List the tables available in Nekt and describe their schemas.* |
| Build a dashboard | *Build a dashboard that shows monthly revenue trends using the `orders` table from Nekt.* |
| Prototype from real data | *Preview the `products` table and build a product catalog page based on its structure.* |
***
## Supabase Destination
If your Lovable app already uses Supabase as its database, you can send processed data from Nekt directly into the same Supabase instance. Your app reads from Supabase as usual; Nekt keeps the data fresh on a schedule.
### Setup
In your Supabase project, click **Connect** in the top navigation bar and select the **Session pooler** tab. Copy the host, port, user, password, and database values.
You must use the **Session pooler** connection method. Transaction mode and direct connections are not supported.
Go to **Destinations** in Nekt, click **Add destination**, and select **Supabase**. Enter the Session pooler credentials from the previous step and choose a load method (append, upsert, or overwrite).
See the full [Supabase destination guide](/destinations/supabase) for details on IP whitelisting, SSL, and load methods.
Choose which tables from your Catalog should be sent to Supabase. Define a primary key for each table and set a schedule for how often data should be refreshed.
Your Lovable app can now query the synced tables through its standard Supabase connection. No additional configuration is needed in Lovable; the data appears in your existing database.
### When to use this method
* Your Lovable app needs to **query data at runtime** (filters, search, pagination).
* You want Lovable to work with **production-ready tables** managed by Nekt.
* You need **scheduled refreshes** so the app always has current data.
***
## Google Sheets Destination
For simpler setups, you can send data from Nekt to a Google Sheet and have Lovable read from it. This works well for read-only data, small datasets, or teams that want to get started quickly.
### Setup
Go to **Destinations** in Nekt, click **Add destination**, and select **Google Sheets**. Paste the link to your spreadsheet and grant Editor access to the email address provided by Nekt.
See the full [Google Sheets destination guide](/destinations/google-sheets) for details on tab mapping and cell range configuration.
Choose which tables from your Catalog should be sent. Each table is written to a separate tab in the spreadsheet. Set a schedule for how often data should be refreshed.
In your Lovable prompt, reference the Google Sheet URL and ask the agent to read from it. Lovable can parse the sheet data to build tables, charts, or app views.
Example: *Read the data from this Google Sheet \[URL] and build a reporting dashboard.*
### When to use this method
* You need a **quick, low-effort setup** without database configuration.
* The data is **read-only** and doesn't require filtering or pagination at runtime.
* Non-technical team members want to **review or edit the data** in the spreadsheet before it reaches the app.
***
## Data API
If your Lovable app needs to query Nekt data on demand from its own code, you can use the Data API. Your app sends a SQL query over HTTP and receives the results as a downloadable file (Parquet or CSV).
Requires a **paid Nekt plan** and an **API key** created in your workspace settings.
### Setup
Go to **Workspace Settings > API Keys** in Nekt and [create an API key](https://app.nekt.ai/settings/api-keys).
In your Lovable prompt, ask the agent to add a server-side function (e.g. a Supabase Edge Function) that calls the Nekt Data API endpoint with a SQL query. The API returns a download link to the results.
Example: *Add an Edge Function that queries the `orders` table from Nekt using the Data API and returns the results as JSON.*
See the full [Data API documentation](/data-api/introduction) for endpoint details and output formats.
Parse the downloaded file in your app logic and display the data in your UI. The download link is valid for 1 hour after each query.
### When to use this method
* Your app needs to **run arbitrary SQL** against Nekt data at runtime.
* You want **on-demand queries** rather than scheduled syncs.
* You are building a **custom integration** that doesn't fit the MCP, Supabase, or Sheets patterns.
***
## Privacy and security
Regardless of the method you choose, Nekt enforces authentication and permission rules at every layer. MCP access is scoped using tokens, Supabase connections use encrypted sessions, and Google Sheets access is limited to the spreadsheets you explicitly share.
For details on how Lovable handles data, refer to [Lovable's security documentation](https://docs.lovable.dev/integrations/mcp-servers#security-and-permissions).
***
## Need help?
Contact our support team if you encounter issues during setup.
# Metabase
Source: https://docs.nekt.com/integrations/metabase
Connect Metabase to your data in Nekt for self-serve analytics.
Connect Metabase to your data in Nekt. How you connect depends on your cloud deployment.
| | Connector | Notes |
| :- | :- | :- |
| **Nekt Express** | Google BigQuery | Native Metabase connector |
| **GCP** | Google BigQuery | Native Metabase connector |
| **AWS** | Amazon Athena | Native Metabase connector |
Nekt does not have a native connector for Metabase. Registering an integration in the Nekt platform has no effect on the Metabase connection: it only helps you stay organized about your existing dashboards. The connection is configured directly in Metabase.
***
## Nekt Express
Nekt Express uses GCP infrastructure. Connect Metabase using the **Google BigQuery** database connector.
You will need a Google service account with BigQuery read access to the `nekt-hosted-infrastructure` project. Please get in touch with Nekt support to get the credentials.
1. In Metabase, go to **Admin settings** → **Databases** → **Add a database**.
2. Select **Google BigQuery** as the database type.
3. Authenticate using a service account with access to the `nekt-hosted-infrastructure` project.
4. Select the dataset (each layer is a dataset in BigQuery) and save.
If the initial connection to BigQuery exceeds Metabase's default 10-second timeout, increase it to 30 seconds. For a self-hosted Metabase, set the environment variable `MB_DB_CONNECTION_TIMEOUT_MS=30000`. On Metabase Cloud, this depends on the options available on the platform, or may require opening a support ticket with Metabase.
***
## GCP
Your Nekt workspace is on your own GCP account. Connect Metabase using **Google BigQuery** with your own project.
Refer to the [Metabase BigQuery documentation](https://www.metabase.com/docs/latest/databases/connections/bigquery) for full setup details.
1. In Metabase, go to **Admin settings** → **Databases** → **Add a database**.
2. Select **Google BigQuery**.
3. Authenticate with your GCP service account.
4. Select your project and datasets.
If the initial connection to BigQuery exceeds Metabase's default 10-second timeout, increase it to 30 seconds. For a self-hosted Metabase, set the environment variable `MB_DB_CONNECTION_TIMEOUT_MS=30000`. On Metabase Cloud, this depends on the options available on the platform, or may require opening a support ticket with Metabase.
***
## AWS
Your Nekt workspace is on your own AWS account. Connect Metabase using the **Amazon Athena** database connector.
> You will need AWS credentials: Access Key ID and Secret Access Key. Generate them using [this AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-key-self-managed.html#Using_CreateAccessKey).
Replace `{your_company_slug}` throughout with your company slug. Contact Nekt support if you don't know it.
### Step 1: Select Amazon Athena as the database
If setting up Metabase for the first time, select your database during onboarding.
For an existing Metabase account:
1. Go to **Admin settings** (gear icon menu) → **Databases** → **Add a database**.
![]()
In both cases, select **Amazon Athena** as the database type.
![]()
### Step 2: Add connection details
| Field | Value |
| :- | :- |
| **Display name** | Nekt |
| **Region** | `us-east-1` (confirm your actual region) |
| **Workgroup** | `primary` |
| **S3 staging directory** | `s3://nekt-lakehouse-{your_company_slug}` |
| **Catalog** | `AwsDataCatalog` |
| **Access key** | your IAM Access Key ID |
| **Secret key** | your IAM Secret Access Key |
Click **Save**.
![]()
### Step 3: Add the IAM policy
If using an IAM user to generate the credentials, attach the following policy to grant Metabase the necessary access:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"athena:BatchGetQueryExecution",
"athena:CancelQueryExecution",
"athena:GetCatalogs",
"athena:GetExecutionEngine",
"athena:GetExecutionEngines",
"athena:GetNamespace",
"athena:GetNamespaces",
"athena:GetQueryExecution",
"athena:GetQueryExecutions",
"athena:GetQueryResults",
"athena:GetQueryResultsStream",
"athena:GetTable",
"athena:GetTables",
"athena:ListQueryExecutions",
"athena:RunQuery",
"athena:StartQueryExecution",
"athena:StopQueryExecution",
"athena:ListWorkGroups",
"athena:ListEngineVersions",
"athena:GetWorkGroup",
"athena:GetDataCatalog",
"athena:GetDatabase",
"athena:GetTableMetadata",
"athena:ListDataCatalogs",
"athena:ListDatabases",
"athena:ListTableMetadata",
"athena:GetQueryRuntimeStatistics",
"athena:CreatePreparedStatement",
"athena:UpdatePreparedStatement",
"athena:GetPreparedStatement",
"athena:DeletePreparedStatement",
"athena:ListPreparedStatements"
],
"Resource": ["*"],
"Effect": "Allow"
},
{
"Action": [
"glue:GetDatabase",
"glue:GetDatabases",
"glue:GetTable",
"glue:GetTables",
"glue:GetPartition",
"glue:GetPartitions",
"glue:BatchGetPartition"
],
"Resource": ["*"],
"Effect": "Allow"
},
{
"Action": ["s3:GetBucketLocation"],
"Resource": ["arn:aws:s3:::nekt-lakehouse-{your_company_slug}"],
"Effect": "Allow"
},
{
"Action": ["lakeformation:GetDataAccess"],
"Resource": ["*"],
"Effect": "Allow"
},
{
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::nekt-lakehouse-{your_company_slug}/*",
"arn:aws:s3:::nekt-lakehouse-{your_company_slug}"
],
"Effect": "Allow"
}
]
}
```
### Step 4: Explore your data
Once the database is configured and the IAM policy is attached, browse your tables in Metabase.
> For more details, see the [Metabase Amazon Athena connector documentation](https://www.metabase.com/docs/latest/databases/connections/athena).
# Microsoft 365 Copilot
Source: https://docs.nekt.com/integrations/microsoft-365-copilot
Connect Microsoft 365 Copilot to your data in Nekt using the MCP Server and Copilot Studio.
Microsoft 365 Copilot can query and analyze your data when connected to Nekt through the MCP Server. The connection is built in **Microsoft Copilot Studio**, which is where MCP servers are registered and attached to agents.
Unlike Claude or ChatGPT, Microsoft doesn't let you plug an MCP server straight
into the chat experience. You must create (or reuse) a **Copilot Studio agent**
and attach the Nekt MCP Server to it as a tool. The agent then runs side by side
with Microsoft 365 Copilot. This means you'll be asked for agent details such as
a name, description, and instructions. That's expected, and the defaults are
fine if you only want tool access.
Requires a **paid Nekt plan** (for MCP Server access) and a **Microsoft Copilot Studio** license in your Power Platform environment.
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. Get your endpoint and token
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**. You'll need two values:
* **Endpoint**: the MCP server URL displayed at the top of the page
* **Token**: a bearer token created in the **Access tokens** section
If you don't have a token yet, create one, choosing **Full access** or **Custom scope**.
Tokens can be scoped per resource type (tables, volumes, secrets, context layer, and live connections) and per tool. Picking individual resources requires the Growth or Custom plan. See [Permissions](/mcp-server/permissions).
## 3. Connect Copilot Studio to Nekt
Sign in to [Copilot Studio](https://copilotstudio.microsoft.com) and select the environment that will host the agent.
Open an existing agent or select **Create** to build a new one. Give it a name and a description, for example, "Nekt Data Agent: answers questions about company data stored in Nekt."
Open the agent's **Tools** page, then select **Add a tool > New tool > Model Context Protocol**.
Complete the wizard fields:
| Field | Value |
| :- | :- |
| **Server name** | Any name you prefer; we suggest "Nekt" |
| **Server description** | A short description of what the server does. The agent's orchestrator uses this to decide when to call Nekt, for example, "Queries company data stored in the Nekt data platform." |
| **Server URL** | The `Endpoint` value from the MCP Gateway page in Nekt |
Copilot Studio only supports the Streamable HTTP transport. The Nekt MCP
Server uses HTTP transport, so it's compatible.
Select **API key** as the authentication type, then fill in the fields that appear:
| Field | Value |
| :- | :- |
| **Parameter label** | `Bearer Token` (this is only the display label shown when someone creates the connection) |
| **Parameter name** | `Authorization` |
| **Parameter location** | `Header` |
Don't put your token, or the word `Bearer` followed by the token, in any of
these fields. They only describe *where* the credential goes. The token value
itself is requested in the next step. Entering it here results in `401
Unauthorized` errors when the agent calls Nekt.
Select **Create**.
The **Add tool** dialog appears. Select **Create a new connection**.
Copilot Studio now prompts you for the credential, using the **Parameter label** you defined. Enter your Nekt token prefixed with `Bearer` and a space:
```
Bearer MCP_TOKEN
```
Replace `MCP_TOKEN` with the token from the **Access tokens** section in Nekt. Because the parameter name is `Authorization`, the whole value is sent as the header, so the `Bearer` prefix must be included here.
Select **Create**, then **Add to agent**.
Open the **Connections** section of the agent (or the tool's detail panel) and confirm the Nekt connection shows as connected. If it isn't, select **Connect** and complete the prompt.
Once connected, Copilot Studio queries the server and lists the available Nekt tools under the connector. You can learn more about them [here](/mcp-server/tools).
Go to the agent's **Overview** page and confirm the Nekt MCP server is listed and enabled under the agent's tools. An agent won't call a tool that's connected but not enabled on it.
To reuse the same server on another agent, go to **Tools > Add a tool** and
search for the name you gave it. Filter the connector types by **Model Context
Protocol**; otherwise the server won't appear in the results.
***
## 4. Test the agent
Use the **Test your agent** panel in Copilot Studio to check the connection before publishing. Ask a question about your data, for example:
```
Give me a summary of the your_table_name table from Nekt.
```
The test panel shows which tools the agent calls. You should see the Nekt tools being invoked and their results feeding the answer.
## 5. Publish the agent
Changes made in Copilot Studio are not available to end users until the agent is published.
Select **Publish** in the agent. This applies to the initial setup and to every later change, including adding or reconfiguring the Nekt MCP server.
Go to the agent's **Channels** section and add **Microsoft 365 Copilot** (and **Microsoft Teams**, if you want it there too). Depending on your tenant's governance settings, an admin may need to approve the agent in the Microsoft 365 admin center before users can access it.
It can take up to an hour after publishing for the latest version of an agent to
take effect, and users receive it the next time they message the agent. In
channels with persistent conversations, send `start over` in the existing chat
to force a new session with the published version.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Troubleshooting
This is almost always a misconfigured API key. Check that:
* **Parameter name** is exactly `Authorization` and **Parameter location** is `Header`.
* The token value was entered when creating the **connection**, not in the **Parameter label** field.
* The connection value includes the `Bearer ` prefix: `Bearer MCP_TOKEN`.
* The token is still active in **MCP Gateway → Access tokens** in Nekt.
To correct the credential, delete the connection and create a new one; editing the tool configuration alone does not update a stored connection.
Publish the agent again. Copilot Studio serves the last published version, so edits made in the authoring canvas, including newly added tools, don't reach end users until you select **Publish**.
If it's already published, allow up to an hour for propagation and send `start over` in the chat to reset the session.
In the **Add a tool** dialog, filter the connector types by **Model Context Protocol**. Servers added through the MCP wizard are not returned by the default search filter.
Confirm the server is enabled on the agent's **Overview** page, and that the **Server description** clearly states what the server does. The agent's orchestrator uses that description to decide when to route a request to Nekt, so a vague description makes it less likely to be called.
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect Microsoft 365 Copilot to Nekt through the MCP Server, Copilot does not receive direct database credentials or unrestricted access to your infrastructure. Instead, it interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data the agent can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
Because the connection uses a shared API key rather than per-user OAuth, every user of the agent queries Nekt with the same token. Scope the token to the tables that everyone with access to the agent should be able to read.
Access to MCP servers in Copilot Studio relies on Power Platform connectors, so any data loss prevention policy applied to Power Platform connectors also governs access to the Nekt MCP Server and its tools.
When using Microsoft 365 Copilot, your interactions are subject to Microsoft's privacy settings and policies. We recommend reviewing the [official Microsoft privacy documentation](https://learn.microsoft.com/en-us/copilot/microsoft-365/microsoft-365-copilot-privacy) to understand their data handling policies and selecting the configuration that best aligns with your organization's security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# OpenClaw
Source: https://docs.nekt.com/integrations/openclaw
Connect OpenClaw to your data in Nekt using the MCP Server.
OpenClaw is an open-source personal AI assistant that can query and analyze your data when connected to Nekt through the MCP Server. The connection is made via OpenClaw's **mcporter** skill, which handles MCP server discovery and configuration.
Requires a **paid Nekt plan** (for MCP Server access).
## 1. Activate your MCP Server
The connection is made by activating the MCP Server in your workspace. Check [this guide](/mcp-server/setup) for instructions.
## 2. Get your endpoint and token
In Nekt, click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**. You'll need two values:
* **Endpoint**: the MCP server URL displayed at the top of the page
* **Token**: a bearer token created in the **Access tokens** section
If you don't have a token yet, create one, choosing **Full access** or **Custom scope**.
Tokens can be scoped per resource type (tables, volumes, secrets, context layer, and live connections) and per tool. Picking individual resources requires the Growth or Custom plan. See [Permissions](/mcp-server/permissions).
## 3. Connect OpenClaw to Nekt
If you haven't already, install the mcporter skill in your OpenClaw instance. Refer to the [OpenClaw documentation](https://openclaw.ai/) for skill installation instructions.
Use mcporter to register the Nekt MCP server. Run the following in your OpenClaw environment:
```bash theme={null}
mcporter add --name Nekt \
--url MCP_ENDPOINT \
--header "Authorization: Bearer MCP_TOKEN"
```
Replace `MCP_ENDPOINT` with the endpoint from the MCP Gateway page and `MCP_TOKEN` with the token you created.
Ask OpenClaw to list the available Nekt tools to confirm the connection is working:
```
List the tools available from the Nekt MCP server.
```
You should see the tools provided by the Nekt MCP Server. You can learn more about them [here](/mcp-server/tools).
***
## 4. Explore your data
Once connected, ask OpenClaw a question about your data. For example:
```
Using Nekt, give me a summary of the your_table_name table.
```
OpenClaw will use the MCP tools to query your data and return an answer.
We'd love to know what you're building with Nekt! Share your projects with us through any of our communication channels.
***
## Privacy & Security
Your data privacy is a core principle of the Nekt platform.
When you connect OpenClaw to Nekt through the MCP Server, OpenClaw does not receive direct database credentials or unrestricted access to your infrastructure. Instead, OpenClaw interacts with your data exclusively through the secure tools exposed by the Nekt MCP Server, which act as a controlled gateway to the Nekt API. These tools enforce the same authentication, authorization, and permission rules used throughout the Nekt platform.
A full access token can do everything its user can, including writes. Use a scoped token to limit the tools and data OpenClaw can reach. Every request is recorded in the [token access log](/platform-api/token-access-logs/list) (7 days on Starter, 30 on Growth, 90 on Custom; not available on Free).
When using OpenClaw, your interactions are subject to OpenClaw's privacy settings and the policies of the AI model you have configured (Claude, GPT-4, or a local model). We recommend reviewing the relevant privacy documentation to understand their data handling policies and selecting the configuration that best aligns with your organization's security requirements.
***
## Need help?
Contact our support team if you encounter issues during setup.
# Integrations
Source: https://docs.nekt.com/integrations/overview
Connect your data in Nekt to BI tools, AI agents, and external APIs.
Once your data is organized in the Catalog, Integrations let you put it to work: in dashboards, AI agents, or any system that can query an API.
## AI
Connect Claude to your data through Claude.ai or Claude Code.
Build apps with your Nekt data via MCP, Supabase, Google Sheets, or the Data API.
Connect ChatGPT to your data via OAuth.
Connect Microsoft 365 Copilot to your data through a Copilot Studio agent.
Connect OpenClaw to your data via MCP.
Connect OpenAI Codex to your data via MCP from the CLI.
## APIs
Query your data directly over HTTP using SQL.
## Visualizations
Connect directly to your data via BigQuery (Nekt Express / GCP) or Nekt's custom connector (AWS).
Build reports against your data using BigQuery (Nekt Express / GCP) or the ODBC connector (AWS).
Query your data via BigQuery or Athena and build operational dashboards.
Self-serve analytics on your data with the BigQuery or Athena connector.
Connect Tableau to your data via BigQuery or the Amazon Athena connector.
Connect Qlik Sense to your data via BigQuery or the Amazon Athena connector.
Build data apps with direct access to your tables using a scoped token.
# Power BI
Source: https://docs.nekt.com/integrations/power-bi
Connect Power BI to your data in Nekt to build reports and dashboards.
Connect Power BI to your data in Nekt. How you connect depends on your cloud deployment.
| | Connector | Token required |
| :- | :- | :- |
| **Nekt Express** | Google BigQuery (native) | No |
| **GCP** | Google BigQuery (native) | No |
| **AWS** | ODBC + Nekt custom connector (`nekt.mez`) | Yes |
***
## Nekt Express
Nekt Express uses GCP infrastructure. Connect via the native BigQuery connector using the shared Nekt project.
The new BigQuery connector implementation must be **disabled** for Nekt Express workspaces. This is a required step: see below.
### Step 1: Register the integration in Nekt
1. In the Nekt platform, go to **Integrations** and click **Add integration**.
2. Select **Power BI** from the list.
3. Enter an email address from a Google account, as that will be used to authenticate against BigQuery connector on Power BI.
4. Add a description to identify this integration.
### Step 2: Disable the new BigQuery connector implementation
1. Open **Power BI Desktop**.
2. Go to **File** → **Options and settings** → **Options**.
3. Under **Preview features**, uncheck **"Use new Google BigQuery connector implementation"**.
4. Click **OK**.
![]()
This falls back to the ODBC connector instead of ADBC, which is required for Nekt Express.
### Step 3: Choose the BigQuery connector
1. Click **Home** → **Get Data** → **More...**.
2. Search for **Google BigQuery** and select it. Click **Connect**.
3. On your first use, sign in with the Google account registered in Step 1. Click **Allow** to grant permissions.
### Step 4: Select the data and load
1. In the **Navigator** panel, expand the project to browse datasets and tables.
2. Select the table(s) you want.
3. Click **Load** to import directly, or **Transform Data** to open Power Query Editor first.
***
## GCP
Your Nekt workspace is on your own GCP account. Connect using the native BigQuery connector in Power BI Desktop or Power BI Online.
### Power BI Desktop
1. Open **Power BI Desktop** and go to **Get Data**.
2. Search for and select **Google BigQuery**.
3. Optionally, expand **Advanced options** to specify a **Billing Project ID**.
4. Choose your authentication method:
* **Organizational login**: standard Google account OAuth.
* **Service Account Login**: paste the service account email and the JSON key contents (ensure the JSON has all newlines removed).
5. Click **Sign In**, complete the Google authentication prompt, then click **Connect**.
6. In the **Navigator**, browse your BigQuery datasets and tables.
7. Select a table and click **Load** or **Transform Data**.
### Power BI Online (Service)
1. In the Power BI web app, choose **Get Data**.
2. Select **Google BigQuery**.
3. Create a new connection or reuse an existing one.
4. Select your authentication method (organizational account or service account).
5. Sign in and browse to your BigQuery tables. Click **Next**.
### Authentication notes
* **Organizational Account**: standard Google OAuth with your user account.
* **Service Account**: for Power BI Service or Power Query Online, use **Basic authentication**: username = service account email, password = flattened JSON key (no newlines).
* If you encounter permission errors, verify your IAM roles include sufficient BigQuery permissions.
* Nested fields may appear as JSON strings; use Power Query to expand or parse them.
***
## AWS
Your Nekt workspace is on your own AWS account. Connection uses an ODBC driver and Nekt's custom Power BI connector (`nekt.mez`). A scoped access token is required.
### Requirements
* Power BI Desktop installed from the **official website** (not the Microsoft Store; Store installations block custom connectors).
### Step 1: Generate a token
1. In the Nekt platform, go to **Integrations** and click **Add integration**.
2. Select **Power BI** as the integration type.
3. Select the tables you want to make available.
The token grants access only to the tables selected here. You can create multiple tokens for different sets of tables.
4. Click **Next**.
5. Give the integration a description and optionally add a report link.
6. Click **Next**, then copy the **Identifier** (e.g. `power-bi-abcd`) and the **Token** from the Integrations page.
### Step 2: Install the ODBC driver
Download and install the **Amazon Athena ODBC 2.x** driver for Windows from [AWS documentation](https://docs.aws.amazon.com/athena/latest/ug/odbc-v2-driver.html).
![]()
### Step 3: Download and place the Nekt connector
[Download `nekt.mez`](https://public.nekt.ai/bi-connectors/pbi/nekt_v1_1_2.mez) and place it in:
```
[Documents]\Microsoft Power BI Desktop\Custom Connectors\
```
Create the folder if it doesn't exist.
The exact folder path may vary. Make sure the file is inside the Custom Connectors folder within the Power BI Desktop directory.
![]()
### Step 4: Allow external connectors
1. In Power BI Desktop, go to **File** → **Options and settings** → **Options**.
2. Go to **Security** → **Data Extensions**.
3. Select **"Allow any extension to load without validation or warning"**.
![]()
### Step 5: Connect
1. Open a new file in Power BI Desktop and click **Get Data**.
2. Search for and select **Nekt** as the data source.
3. Enter the **Identifier** from your integration (e.g. `power-bi-abcd`). Click **OK**.
4. Enter the **Token**. Click **Connect**.
5. Select the tables you want and proceed normally.
### Troubleshooting
* Use the most recent version of Power BI Desktop, the ODBC driver, and the `nekt.mez` file.
* Confirm Power BI was installed from the official website, not the Microsoft Store.
***
### Automatic refresh with a gateway (AWS)
To enable scheduled data refresh in Power BI Service, you need an **On-Premises Data Gateway** running on a machine that has the ODBC driver and `nekt.mez` installed.
#### On the gateway machine
Ensure the following are installed and up to date on the gateway machine:
* Power BI Desktop
* On-Premises Data Gateway
* Amazon Athena ODBC 2.x driver
* `nekt.mez` in the `[Documents]\Power BI Desktop\Custom Connectors` folder
* External connectors allowed in Power BI's Data Extensions settings
#### Configure the On-Premises Data Gateway
1. Open the **On-Premises Data Gateway** app and complete installation.
2. In the **'Email address to use with this gateway'** field, enter the email used in Power BI Service.
3. Log in with your Power BI Service account.
4. Select **Register a new gateway on this computer**.
5. Give the gateway a name (e.g. `aws-gateway`) and set a recovery key. Click **Configure**.
The gateway is now online.
**Grant the gateway user access to the connector folder:**
1. Check which user the gateway runs as: open the gateway app → **Service Settings**. The default is `NT SERVICE\PBIEgwService`.
2. Right-click the Custom Connectors folder → **Properties** → **Security** tab → **Edit**.
3. Click **Add**, type `NT SERVICE\PBIEgwService`, click **Check Names**, then **OK**.
4. Grant **Read & execute**, **List folder contents**, and **Read** permissions. Click **Apply** then **OK**.
5. In the gateway app, go to **Connectors** and point it to your Custom Connectors folder. Click **Apply**. The gateway will restart. If the Nekt connector doesn't appear, restart the gateway from **Service Settings**.
#### Publish and configure in Power BI Service
1. **Publish** your semantic model from Power BI Desktop while logged in to your Power BI Service account.
2. In Power BI Service, go to **Settings** → **Manage connections and gateways**. The gateway should appear under **On-premises data gateways**.
3. Open the gateway settings:
* **General**: enable "Distribute requests across all active gateways in this cluster".
* **Power BI**: enable "Allow user's cloud data sources to refresh through this gateway cluster" and "Allow user's custom data connectors to refresh through this gateway cluster". Click **Save**.
#### Add the data source to the gateway
1. Open the semantic model's settings → **Gateway and cloud connections**.
2. The data source will show "Not properly configured". Click the play icon → **Add manually to gateway**.
3. Configure:
* **Connection name**: use the Integration Identifier (e.g. `power-bi-abcd`)
* **Visualization identifier**: the Integration Identifier
* **Authentication method**: Visualization Token
* **Visualization token**: the token from your integration in Nekt
4. Click **Create**.
5. In the dropdown, select the gateway for this semantic model. Click **Apply**.
Repeat this process for each semantic model you want to configure with the gateway.
#### Set up automatic refresh
In the semantic model settings → **Refresh**, configure the schedule and click **Apply**. All reports using this model will refresh on that schedule.
If you don't have a local Windows machine to run the gateway, you can use a Windows EC2 instance on AWS.
#### 1. Launch a Windows EC2 instance
1. Log in to the AWS Console and go to **EC2** → **Launch instance**.
2. Configure:
* **Name**: `power-bi-gateway` (or any descriptive name)
* **OS**: Microsoft Windows Server 2025 Base (Quick Start tab)
* **Instance type**: `t3.medium` minimum (2 vCPU, 4 GiB RAM, \~\$0.06/hr)
* **Key pair**: select or create one (needed for RDP access)
* **Network settings**: create a security group and allow RDP traffic from your IP
* **Storage**: 60 GiB gp3 root volume
3. Click **Launch instance**.
#### 2. Connect via Remote Desktop (RDP)
1. In EC2, select the instance → **Connect** → **RDP Client** tab.
2. Click **Get Password**, upload the key pair file, and click **Decrypt password**.
3. Note the **Public DNS**, **Username**, and **Password**.
4. Install **Microsoft Remote Desktop** on your local machine.
5. Add a new PC: paste the Public DNS, add a user account with the Username and Password, then connect.
#### 3. Configure Power BI on the instance
Inside the Windows instance, install:
* [Power BI Desktop (latest)](https://www.microsoft.com/download/details.aspx?id=58494)
* [On-Premises Data Gateway (latest)](https://www.microsoft.com/download/details.aspx?id=53127)
Then follow the [gateway setup](#automatic-refresh-with-a-gateway-aws) steps above to complete the configuration.
# Qlik Sense
Source: https://docs.nekt.com/integrations/qlik-sense
Connect Qlik Sense to your data in Nekt for analysis and dashboards.
Connect Qlik Sense to your data in Nekt. How you connect depends on your cloud deployment.
| | Connector | Notes |
| :- | :- | :- |
| **Nekt Express** | Google BigQuery | Via the Qlik BigQuery connector |
| **GCP** | Google BigQuery | Via the Qlik BigQuery connector |
| **AWS** | Amazon Athena | Via the Qlik Athena ODBC connector |
Nekt does not have a native connector for Qlik Sense. Registering an integration in the Nekt platform has no effect on the Qlik Sense connection: it only helps you stay organized about your existing dashboards. The connection is configured directly in Qlik Sense.
***
## Nekt Express
Nekt Express uses GCP infrastructure. Connect Qlik Sense using the native **Google BigQuery** connector.
You will need a Google account with access to the `nekt-hosted-infrastructure` BigQuery project. Refer to [Qlik's BigQuery connector documentation](https://help.qlik.com/en-US/connectors/Subsystems/ODBC_connector_help/Content/Connectors_ODBC/BigQuery/BigQuery-connector.htm) for full setup details.
1. In Qlik Sense, go to **Add new data** → **Data Connection**.
2. Search for and select **Google BigQuery**.
3. Authenticate with your Google account that has access to `nekt-hosted-infrastructure`.
4. Browse to your dataset (each layer is a dataset in BigQuery) and select tables.
***
## GCP
Your Nekt workspace is on your own GCP account. Connect Qlik Sense using the native **Google BigQuery** connector with your own project.
Refer to [Qlik's BigQuery connector documentation](https://help.qlik.com/en-US/connectors/Subsystems/ODBC_connector_help/Content/Connectors_ODBC/BigQuery/BigQuery-connector.htm) for full setup details.
1. In Qlik Sense, go to **Add new data** → **Data Connection**.
2. Select **Google BigQuery**.
3. Authenticate with your GCP credentials.
4. Select your project, datasets, and tables.
***
## AWS
Your Nekt workspace is on your own AWS account. Connect Qlik Sense using the **Amazon Athena** ODBC connector.
> You will need AWS credentials: Access Key ID and Secret Access Key. Generate them using [this AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-key-self-managed.html#Using_CreateAccessKey).
### Step 1: Find the Athena connector
1. In the Qlik Sense main page, go to **Add new data** → **Data Connection**.
![]()
2. Select **Amazon Athena**.
There is a second Athena option labeled "via Direct Access Gateway"; do not use it. This guide requires the standard **Amazon Athena** connector.
![]()
### Step 2: Enter connection credentials
Update the following fields (you can leave others as default):
* **AWS Region**: your workspace region
* **Data source**: `AwsDataCatalog`
* **Database**: the layer to connect
* **Workgroup**: `primary`
* **Access key ID**: your IAM Access Key ID
* **Secret Key**: your IAM Secret Access Key
* **S3 output location**: `s3://nekt-lakehouse-{your_company_slug}/_athena`
Contact Nekt support for your company slug and region.
Click **Test Connection** to verify.
![]()
### Step 3: Connect and select data
Click **Connect and Analyze**. Use the checkboxes to select the tables and columns you need, then click **Next** to start building dashboards.
![]()
> For more details, see the [Qlik Sense Amazon Athena connector documentation](https://help.qlik.com/en-US/connectors/Subsystems/ODBC_connector_help/Content/Connectors_ODBC/Athena/Athena-connector.htm).
# Streamlit
Source: https://docs.nekt.com/integrations/streamlit
Build data apps in Streamlit with direct access to your data in Nekt.
Connect Streamlit to Nekt using the Nekt SDK and a scoped access token. The token grants read access to the specific tables you choose, and can be used across all cloud deployments.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
***
## Generate a token
In the Nekt platform, go to the [Access Tokens](https://app.nekt.ai/notebooks/access-tokens) page.
Click **Create a token**.
Select the tables you want to give access to. You can select tables from any layer of your Lakehouse.
The generated token grants access only to the tables selected here. You can create multiple tokens to give access to different sets of tables, useful for scoping access per application or user.
Click **Create token**.
A success message will confirm the token was created. Go to the [Access Tokens](https://app.nekt.ai/notebooks/access-tokens) page to view it and copy your token.
***
## Connect with the Nekt SDK
The Nekt SDK lets your Streamlit app load tables directly from your Lakehouse as Spark DataFrames.
### Project setup
Your Streamlit project needs three files alongside your app code:
**`requirements.txt`** (Python dependencies):
```txt theme={null}
streamlit
git+https://github.com/nektcom/nekt-sdk-py.git#egg=nekt-sdk
```
**`packages.txt`** (system packages required by the SDK's Spark runtime):
```txt theme={null}
openjdk-17-jdk
```
The `packages.txt` file is used by Streamlit Community Cloud to install system-level dependencies. If you deploy elsewhere, install Java 17 through your platform's package manager or Docker image.
### Store the token as a secret
Never hardcode your token. Use [Streamlit secrets management](https://docs.streamlit.io/deploy/streamlit-community-cloud/deploy-your-app/secrets-management) to store it securely.
Create a `.streamlit/secrets.toml` file in your project root:
```toml theme={null}
DATA_ACCESS_TOKEN = "your-nekt-token-here"
```
Add `.streamlit/secrets.toml` to your `.gitignore` to keep the token out of version control.
When deploying to Streamlit Community Cloud, add the same key-value pair in your app's **Secrets** settings.
### Load data in your app
Initialize the SDK with the token from secrets, then call `nekt.load_table()` to fetch tables from your Lakehouse:
```python theme={null}
import streamlit as st
import nekt
nekt.data_access_token = st.secrets["DATA_ACCESS_TOKEN"]
df = nekt.load_table(layer_name="Raw", table_name="orders")
st.dataframe(df.toPandas())
```
`load_table` returns a Spark DataFrame. Call `.toPandas()` to convert it for use with Streamlit's display components.
You can also select specific columns before converting:
```python theme={null}
df = nekt.load_table(
layer_name="Raw",
table_name="orders"
).select("id", "customer_name", "total")
st.dataframe(df.toPandas())
```
### Full example
```python theme={null}
import streamlit as st
import nekt
nekt.data_access_token = st.secrets["DATA_ACCESS_TOKEN"]
st.title("Sales Dashboard")
orders_df = nekt.load_table(
layer_name="Raw",
table_name="orders"
).select("id", "customer_name", "total")
st.metric("Total orders", orders_df.count())
st.dataframe(orders_df.toPandas())
```
For a complete working project you can fork and deploy, see the [streamlit-demo repository](https://github.com/nektcom/streamlit-demo).
***
## Deploy to Streamlit Community Cloud
Your repository should contain at least `streamlit_app.py`, `requirements.txt`, and `packages.txt`. Do not commit `.streamlit/secrets.toml`.
Go to [share.streamlit.io](https://share.streamlit.io), click **New app**, and select your repository, branch, and main file path.
In the app's **Advanced settings**, paste your secret:
```toml theme={null}
DATA_ACCESS_TOKEN = "your-nekt-token-here"
```
Click **Deploy**. Streamlit will install the system packages from `packages.txt`, the Python dependencies from `requirements.txt`, and start your app.
Once deployed, you're ready to go!
***
## SDK reference
The following SDK methods are available for use in your Streamlit app. See the full [SDK documentation](/notebooks/sdk) for details on all methods.
| Method | Description |
| :- | :- |
| `nekt.load_table(layer_name, table_name)` | Load a table from your Lakehouse as a Spark DataFrame |
| `nekt.get_spark_session()` | Access the Spark session for advanced operations |
***
## Need help?
Contact our support team if you encounter issues during setup.
# Tableau
Source: https://docs.nekt.com/integrations/tableau
Connect Tableau to your data in Nekt for visualization and analysis.
Connect Tableau to your data in Nekt. How you connect depends on your cloud deployment.
| | Connector | Notes |
| :- | :- | :- |
| **Nekt Express** | Google BigQuery | Native Tableau connector |
| **GCP** | Google BigQuery | Native Tableau connector |
| **AWS** | Amazon Athena | Requires ODBC driver |
Nekt does not have a native connector for Tableau. Registering an integration in the Nekt platform has no effect on the Tableau connection: it only helps you stay organized about your existing dashboards. The connection is configured directly in Tableau.
***
## Nekt Express
Nekt Express uses GCP infrastructure. Connect Tableau using the native **Google BigQuery** connector.
You will need a Google account with access to the `nekt-hosted-infrastructure` BigQuery project. Refer to [Tableau's BigQuery connector documentation](https://help.tableau.com/current/pro/desktop/en-us/examples_googlebigquery.htm) for full setup details.
1. In Tableau, go to **Connect** → **To a Server** → **Google BigQuery**.
2. Sign in with your Google account.
3. Select the `nekt-hosted-infrastructure` project, then browse to the dataset and table you want.
***
## GCP
Your Nekt workspace is on your own GCP account. Connect Tableau using the native **Google BigQuery** connector with your own project.
Refer to [Tableau's BigQuery connector documentation](https://help.tableau.com/current/pro/desktop/en-us/examples_googlebigquery.htm) for full setup details.
1. In Tableau, go to **Connect** → **To a Server** → **Google BigQuery**.
2. Sign in with your Google account.
3. Select your GCP project, dataset, and tables.
***
## AWS
Your Nekt workspace is on your own AWS account. Connect Tableau using the **Amazon Athena** connector, which requires an ODBC driver.
> You will need AWS credentials: Access Key ID and Secret Access Key. Generate them using [this AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-key-self-managed.html#Using_CreateAccessKey).
### Step 1: Find the Athena connector
In Tableau's **Start Page**, go to **Connect** → **To a Server** → **More**. Find and select **Amazon Athena**.
![]()
### Step 2: Install the required driver
If the ODBC driver is not installed, Tableau will display an error with a download link. Visit [Tableau's driver download page](https://www.tableau.com/support/drivers) for the correct driver version.
### Step 3: Enter your credentials
After selecting Amazon Athena, enter the required connection details including your AWS region, workgroup, S3 staging directory, and credentials. Contact Nekt support for your company-specific values.
![]()
Click **Sign In** to connect.
> For more details, see [Tableau's Amazon Athena connector documentation](https://help.tableau.com/current/pro/desktop/en-us/examples_amazonathena.htm).
# Authentication
Source: https://docs.nekt.com/mcp-server/authentication
Connect AI agents to the MCP Server using OAuth 2.1 or bearer tokens.
The MCP Server supports two authentication methods depending on how your AI agent connects.
## OAuth 2.1
For AI desktop applications like **Claude Desktop** or **Cursor**, the MCP Server supports OAuth 2.1 with PKCE. The application opens a browser window where you log in with your Nekt credentials (or SSO), select which token to use, and the app receives access automatically. No manual token copying is needed.
To connect, point your AI application to your MCP server URL using the **HTTP Streamable** transport protocol.
## Bearer Token
For automation platforms like **N8N**, **Make**, or custom integrations, you can use a bearer token directly. Copy the token from the Nekt platform and paste it into your tool's configuration.
Tokens can also be minted and managed from the [Platform API](/platform-api/mcp-tokens/create) with an API key (useful when provisioning an agent is part of a script rather than a one-off). The secret itself is never returned to the script: the response carries a one-time link that opens in the creating user's browser, so a person still collects and pastes it.
Tokens created during the OAuth flow use the creating user's permissions by
default. Tokens created manually can be scoped per resource type (tables,
volumes, secrets, context layer, and live connections) and per tool. Picking
individual resources requires the Growth or Custom plan. See
[Permissions](/mcp-server/permissions).
# Introduction
Source: https://docs.nekt.com/mcp-server/introduction
Give your AI agents natural language access to your data in Nekt.
## What is an MCP Server?
[MCP (Model Context Protocol)](https://modelcontextprotocol.io/) is an open protocol that standardizes how applications provide context to LLMs. It gives AI agents a structured way to discover data, call tools, and take actions on your behalf, all through a single integration point.
## What can Nekt's MCP Server do?
Nekt's MCP Server gives your AI agent native access to your data through natural language. Instead of writing SQL or navigating dashboards, you can ask questions and get answers directly from the tools you already use: Claude, Cursor, or any MCP-compatible client.
* **Explore your data**: discover which tables are available, preview their contents, and understand schemas without leaving your AI agent.
* **Query with natural language**: ask questions in plain English and let the server generate and execute SQL against your data warehouse (BigQuery or Athena, depending on your cloud).
* **Read files**: browse layers, volumes, and files stored in Nekt, and read their contents directly.
* **Build and run pipelines**: create SQL Queries and Python or PySpark Notebooks, schedule them, and trigger runs from your AI agent.
* **Learn about Nekt**: ask the server about platform features, configuration, and capabilities without switching context.
Access is scoped using tokens. A token can be restricted per resource type (tables, volumes, secrets, context layer, and live connections) and per tool, so you control what your AI agent can see and do.
The MCP Server is available on all **paid plans**.
## Get started
Enable the MCP Server on your cloud and create access tokens.
Connect AI agents using OAuth 2.1 or bearer tokens.
Explore the tools available to AI agents.
# Permissions
Source: https://docs.nekt.com/mcp-server/permissions
How platform permissions control what AI agents can see and do through the MCP Server.
What your AI agent can access depends on the kind of token it uses:
* **Full access token:** the agent reaches exactly what the token's creator can reach on the platform, as defined by their [permissions](/workspace/permissions). If the creator cannot see a layer, that layer does not appear in `list_layers`.
* **Scoped token:** the token has its own scope, per resource type and per tool, and it is always capped by what its creator can still reach. See [Scoped tokens](#scoped-tokens).
Table-level restriction of queries is enforced by your cloud's access controls, not by reading the SQL. If you use your own GCP cloud, make sure it is on cloud version **1.0.10** or later; below that, `execute_sql` is not restricted per user or per token, although the other tools still list only the tables the token can access. You can check your cloud version in **Settings → Cloud**.
Data-level permissions are only enforced on **Growth or Custom plans**. On Starter, permissions are not enforced and every member can access all data, so a full access token sees everything in the workspace. A scoped token is still limited to what it was scoped to: see [Scoping on Starter and Free](#scoping-on-starter-and-free).
## Token types and permission behavior
When you create a token (in the **Access tokens** section of the [MCP Gateway](https://app.nekt.ai/mcp) page or through the [Platform API](/platform-api/mcp-tokens/create)), you choose between two scoping options, shown in the app as **Full access** and **Custom scope**. The choice determines how the token relates to platform permissions.
| | **Full access token** | **Scoped token** |
| - | - | - |
| **Visibility** | Everything the creating user can access on the platform | Only what you select, chosen separately for each kind of resource (tables, volumes, secrets, context-layer documents, live connections), plus which tools it may call |
| **Reflects platform permission changes** | Yes, in both directions: access gained and access lost | **Narrows only.** Access the creator loses is dropped from the token immediately; access they gain is not added |
| **Cannot reach** | Anything its creator cannot reach | Sources, destinations, Queries and Notebooks, triggers, runs, GitHub, permission and token management, and the token access log. See [What a scoped token can reach](#what-a-scoped-token-can-reach) |
| **Best for** | Individual users connecting their own AI agent | Automation, integrations, or agents that need a stable, narrow scope |
### Full access tokens
A full access token inherits the permissions of the user who created it and stays in sync dynamically. If that user gains access to a new layer, the token immediately reflects it. If their access to a table is revoked, the token loses that access automatically, with no token rotation needed.
A full access token can do everything its creator can, including writes: editing pipelines, granting permissions, and creating other tokens. It cannot be limited to certain tools. To limit what an agent can do, use a scoped token.
Tokens created during the [OAuth flow](/mcp-server/authentication) are full access tokens tied to the authenticating user.
### Scoped tokens
A scoped token carries its own selection, chosen per resource type when you create it: which tables, which volumes, which secrets, which context-layer documents, which live connections, and which tools it may call. Each is independent: a token can name three tables and no secrets at all.
That selection is not the last word. On every request it is narrowed to what the creating user can **still** access, so a scoped token can only ever be narrower than its creator:
* **Access the creator loses is dropped from the token immediately.** If their permission to `trusted.customer_orders` is revoked, the token stops seeing that table on its next call; no rotation, no editing.
* **Access the creator gains is not added.** The selection stays the selection. To widen a scoped token, edit it.
A token can only be scoped to resources the creating user has access to at the time of creation, and it keeps only the ones they still have.
### What a scoped token can reach
A scoped token reaches only surfaces where you can select what it sees:
* **Tables:** listing, searching, previewing, and querying the tables in its scope, and generating SQL over them.
* **Context layer:** `get_semantic_context` returns only the documents in its scope.
* **Live Data:** the live connections selected on it, and within each one the tools you chose.
* **Layers, volumes, files, and context-layer folders and documents:** browsing layers and volumes, reading and uploading files, and working with the context-layer folders and documents in its scope are available to a scoped token only when scoped-token browsing has been enabled for your workspace by Nekt support. Without it, these tools return an error for a scoped token.
Everything else is refused to a scoped token whatever its scope says: sources, destinations, Queries and Notebooks, triggers, runs, GitHub, creating layers, failure alerts, permission management, MCP token management, and the token access log. Those tools require a full access token.
A scoped token is not read-only. If you do not restrict its tools, it can use the write tools that apply to its scope: for example, `update_resource_description` can change the descriptions of the tables in its scope. To keep a token read-only, select only read tools when you create it.
### Scoping on Starter and Free
Choosing individual resources is part of the permissions model, and permissions are enforced on Growth and Custom only. On **Starter** and **Free** a scoped token therefore works one whole resource type at a time: for each kind of resource you choose **all of it or none of it**.
| Switch | What the token reaches |
| - | - |
| All tables | Every table in the workspace, including tables created later |
| All volumes | Every volume |
| All secrets | Every secret |
| All context layer | Every document and folder |
| All live connections | Every live connection |
The five are independent. A token with **All tables** and **All secrets** on and the rest off reaches every table and every secret, and no volumes whatsoever: off means none, not "inherit". Turning all five on is still a scoped token, not a full access one.
Workspaces that were already scoping tokens to individual tables when this changed keep that ability. If your Starter workspace still offers a table picker, that is why.
## Who can create tokens, and how long they last
* **Who can create tokens:** in **Access tokens**, the **Who can create tokens** setting decides which roles can create and manage tokens in the workspace: everyone (the default), only Admins and Owners, or only Owners.
* **Expiration:** a token lasts one year from creation by default and can never be valid for more than a year. Rotating a token in the app renews it for the same period.
* **A token depends on its creator:** if the person who created a token is deactivated or removed from the workspace, the token stops working, whatever its scope. Before someone leaves, create replacement tokens for the automations that use theirs.
* **Access log:** every request a token makes is recorded in the [token access log](/platform-api/token-access-logs/list), including the SQL of each query, so you can see which tables an agent queried. It is available on Starter (7 days), Growth (30 days), and Custom (90 days), not on Free. Owners and Admins see every token; other members see the tokens they created.
## Practical examples
### Example 1: User receives Viewer access to an entire layer
**Scenario:** A member is granted Viewer access to the `raw` layer, which contains dozens of tables and several volumes.
**Action on the platform:** An admin grants Viewer access to the `raw` layer under **Workspace → Permissions**.
**What to expect in the MCP:**
* `list_layers` now includes the `raw` layer.
* `list_volumes` returns volumes inside `raw`.
* `list_tables` and `get_relevant_tables_ddl` include tables from `raw`.
* `generate_sql` and `execute_sql` can query those tables.
***
### Example 2: Table permission is revoked for a specific user
**Scenario:** A member previously had Viewer access to `trusted.customer_orders`. That permission is now revoked directly on the table.
**Action on the platform:** The permission is removed from the table: no group change is involved.
**What to expect in the MCP:**
* `execute_sql` returns an error for any query that references `trusted.customer_orders` (if you use your own GCP cloud, it must be on cloud version 1.0.10 or later; see the note at the top of this page).
* `get_table_preview` returns an error for that table.
* `get_relevant_tables_ddl` no longer returns its schema.
* `list_tables` no longer includes it.
***
### Example 3 (Troubleshooting): Group permission revoked but agent still has access
**Scenario:** An admin revokes the `raw` layer permission from a group. An AI agent using a full access token from a user in that group can still query tables in `raw`.
**Most likely causes:**
1. **Conflicting group permissions.** If the user belongs to another group (including the built-in **All** group) that still has access to `raw`, Nekt grants the highest permission the user holds from any assignment. Revoking access from one group alone does not remove access if another group retains it.
2. **Misconfigured change.** Verify the permission was saved correctly in **Workspace → Groups** or on the layer's Permissions tab.
Nekt always applies the **highest** permission a user has from any group or direct assignment. Revoking access in one group has no effect if another group (or a direct assignment) still grants it.
## How permissions apply to each tool
Most tools filter their output automatically to what the token can access. Tools that operate on resources outside the token's scope return an error rather than partial results.
| Tool | Permission behavior |
| - | - |
| `ask_nekt` | No warehouse data access. Reads the workspace's connected sources to match playbooks |
| `list_layers` | Returns only layers the token can access. A scoped token needs scoped-token browsing enabled for the workspace |
| `create_layer` | Full access token created by an Owner or Admin. Every other token is refused |
| `list_volumes` | Returns only volumes in accessible layers. A scoped token needs scoped-token browsing enabled for the workspace |
| `list_files` | Returns only files in accessible volumes |
| `read_file` | Returns an error for files outside the token's scope |
| `list_tables` | Returns only accessible tables |
| `get_relevant_tables_ddl` | Semantic search is scoped to accessible tables only |
| `get_semantic_context` | Returns only the context-layer documents the token's user may read, and for a scoped token only the documents in its scope (see [Context layer permissions](#context-layer-permissions)) |
| `list_semantic_layer_folders` | Returns only visible folders, each with your permission level on it; `writable_only=True` narrows it to the folders that can receive a document |
| `generate_sql` | Generates SQL using only accessible table schemas |
| `execute_sql` | Returns an error if the query references an inaccessible table, enforced by your cloud's access controls |
| `get_table_preview` | Returns an error for inaccessible tables |
| `create_query` / `create_notebook` | Full access token only. Requires at least Editor access to the output layer |
| `run_pipeline` / `get_pipeline_status` | Full access token only. Requires access to the pipeline's input and output tables |
| Source, destination, trigger, run, and GitHub tools | Full access token only |
| Permission tools (`list_permissions`, `grant_permission`, and the rest) | Full access token only. Limited to what the creator could grant in the app |
| MCP token tools and token access log tools | Full access token only |
| `update_alerts` | Full access token only: a subscription needs a real user account to email, so a scoped token is refused |
| `create_semantic_layer_folder` / `create_context_document` | Editor on the destination folder, or on the root for a top-level item. Any member on Free and Starter |
| `update_context_document` | Editor on the document. Available only to people who hold Editor on at least one document. Any member on Free and Starter |
| `delete_context_document` | Manager on the document. Any member on Free and Starter |
A scoped token reads only the context-layer documents and folders it was scoped to, always capped
by its creator's access. Beyond `get_semantic_context`, the context-layer tools are available to a
scoped token only when scoped-token browsing is enabled for the workspace.
## Context layer permissions
The context layer (the context documents `get_semantic_context` reads) is organized into
**folders, each with its own access rules**, and a token sees exactly what its user sees.
Like data-level permissions, context-layer permissions are enforced on **Growth and Custom plans**.
On Free and Starter they are not enforced: every member of the workspace can read, create, edit,
and delete context-layer documents and folders.
Two consequences worth knowing when an agent comes back empty-handed:
* **An empty `get_semantic_context` result is ambiguous.** It can mean the question matched no
document, or that the relevant documents are in folders this token's user cannot read. Rephrasing
fixes the first and never fixes the second, so the agent will say there may be documentation you
don't have access to instead of retrying.
* **On Growth and Custom, creating a document needs a folder you can write to.** A document created
without one lands at the root of the context layer, which most users cannot write to. Agents call
`list_semantic_layer_folders(writable_only=True)` first to choose a destination; an empty result
there means the user cannot create documents anywhere and the agent should stop rather than retry.
## Troubleshooting
The most common causes:
* **Group membership:** The token's user may belong to a group (including the built-in **All** group) that still has access to the resource. Check effective permissions in **Workspace → Members**.
* **The token's own selection:** A scoped token reaches whatever was selected on it, which may be broader than you remember. Revoking a permission from the **creating user** removes the resource from the token as well, but a resource the creator still has access to stays in scope until you edit the token. Check its scope with **View token access** in the **Access tokens** section of the [MCP Gateway](https://app.nekt.ai/mcp) page.
The most common causes:
* **Permission not yet assigned:** Verify the permission is set correctly in **Workspace → Permissions** or on the resource's own Permissions tab.
* **Scoped token missing the table:** the table may not have been selected on the token. Edit the token to add it.
* **The creating user lost access:** a scoped token never reaches more than its creator. If their permission to the table was revoked, the token loses it too. Restore the creator's access, or use a token created by someone who has it.
* **A scoped token on a surface it cannot reach:** sources, pipelines, runs, and permission management always require a full access token. See [What a scoped token can reach](#what-a-scoped-token-can-reach).
* **The token's creator left the workspace:** a token stops working when its creator is deactivated or removed.
To confirm which type of token you're using, go to **Access tokens** on the [MCP Gateway](https://app.nekt.ai/mcp) page.
# Setup
Source: https://docs.nekt.com/mcp-server/setup
Enable the MCP Server on your cloud and create access tokens.
Setup steps depend on your cloud deployment. Choose your cloud below.
### Step 1: Enable the MCP Server
Click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**, and click **Enable now**. Nekt will provision the server automatically.
### Step 2: Create tokens
Once the server is active, go to the **Access tokens** section and create one or more access tokens. Each token either has **Full access**, based on the creating user's permissions, or a **Custom scope** chosen per resource type and per tool.
If you connect via OAuth and don't have any tokens yet, one will be automatically created with the same permissions as your user account.
### Step 1: Enable the MCP Server
Click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**, and click **Enable now**. The server is provisioned as a Google Cloud Run Service in your GCP project.
### Step 2: Create tokens
Once the server is active, go to the **Access tokens** section and create one or more access tokens. Each token either has **Full access**, based on the creating user's permissions, or a **Custom scope** chosen per resource type and per tool.
If you connect via OAuth and don't have any tokens yet, one will be automatically created with the same permissions as your user account.
### Step 1: Enable the MCP Server
Click **MCP** in the top-right corner of the header, then **Manage MCP Gateway**, and click **Enable now**. Enter the full domain you want to use as your MCP server endpoint: for example, `mcp.company.co`, not just `company.co`.
### Step 2: Verify the HTTPS certificate
Add the CNAME record provided by Nekt to your DNS provider to validate the HTTPS certificate. Allow 5-10 minutes for verification.
### Step 3: Add the MCP URL to your DNS
Still in your DNS provider, add the second CNAME record to point your custom domain to the MCP server. Allow an additional 5-10 minutes for this to propagate.
Your MCP server is now live.
### Step 4: Create tokens
Go to the **Access tokens** section and create one or more access tokens. Each token either has **Full access**, based on the creating user's permissions, or a **Custom scope** chosen per resource type and per tool.
If you connect via OAuth and don't have any tokens yet, one will be automatically created with the same permissions as your user account.
## Infrastructure
Managed by Nekt and billed through your Nekt credits.
The MCP Server runs on [Google Cloud Run](https://cloud.google.com/run) in your GCP project. Costs are based on Cloud Run's per-second billing model for CPU and memory while instances are active. With typical usage, expect approximately **\$20-25/month**. The server scales to zero when idle, so actual costs depend on how often it handles requests.
Services used:
* [Cloud Run](https://cloud.google.com/run): 1 vCPU, 2 GiB memory, scales to zero when idle
* [Secret Manager](https://cloud.google.com/secret-manager): stores server credentials
See [Cloud Run pricing](https://cloud.google.com/run/pricing) for current rates in your region.
The MCP Server runs on [AWS Fargate](https://aws.amazon.com/fargate/) in your AWS account as an always-on service. In `us-east-1`, the fixed cost is approximately **\$34/month**.
Services used:
* [AWS Fargate](https://aws.amazon.com/fargate/): 0.5 vCPU, 2 GiB memory
* [Network Load Balancer](https://aws.amazon.com/elasticloadbalancing/network-load-balancer/): routes HTTPS traffic to the server
* [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/): stores server credentials
Costs vary by AWS region.
## Prerequisites
If you use your own cloud account, make sure it is on at least the cloud version below before enabling the MCP Server.
| Cloud | Minimum cloud version |
| :- | :- |
| **Nekt Express** | No requirement, always up to date |
| **AWS** | **2.0.26** |
| **GCP** | **1.0.4** |
You can check your current cloud version in **Settings → Cloud**.
***
## Need help?
Contact our support team if you encounter issues during setup.
## API & MCP
List, create, rotate, and scope MCP tokens.
`create_mcp_token`, `list_mcp_tokens`, and `rotate_mcp_token`.
# Tools
Source: https://docs.nekt.com/mcp-server/tools
Explore the tools available to AI agents through the Nekt MCP Server.
The MCP Server provides tools that AI agents can use to interact with your data, files, and pipelines in Nekt.
Most tools appear in your agent's tool list the moment it connects. A handful of rarely-used ones are
kept off that list and reached through a dispatcher (`get_tool_details` → `run_tool`), and Live Data
tools are always reached through a dispatcher (`discover_live_tools` → `run_live_tool`). Both are
explained below; the tools themselves are fully available either way.
**Have skills, automations, or saved prompts that call Nekt tools by name?** Several tools were
merged into generic versions that take a `kind` argument, and the old names no longer exist. See
[Changed tool names](#changed-tool-names) at the bottom of this page.
## Asking about Nekt
### `ask_nekt`
Your guide to Nekt: how the platform works and what to build with it. With a question, it answers
from Nekt's documentation and returns matching playbooks from Nekt Academy (real use cases proven by
Nekt customers) together. `focus` narrows it: `product` for how Nekt works only, `playbooks` for
what to build only. If one of the two lookups fails, the other still comes back.
Without a question, it returns the playbook catalog, filterable by goal, stage, and content type.
With an `item_id` from a previous result, it returns that full playbook or guide.
Each playbook is matched against the sources your workspace has already connected: `ready` means
you can build it today, while `one_source_away` and `not_yet` list what's missing. To build one,
your agent creates it one item at a time (source, Query, Notebook, destination, alert, context
document) and asks you to confirm each.
This tool does not query your data. For questions about your own numbers, your agent uses
`get_semantic_context` and `generate_sql`.
SQL inside a playbook uses example table names. Your agent maps it to your real tables before
generating and running anything.
**Example prompts:** "what use cases can I build with my HubSpot and Stripe sources?", "how do
incremental syncs work?"
## Querying data
The path is: read the business context, find the right tables, generate SQL, run it.
### `get_semantic_context`
Retrieves the most relevant context-layer documents for a question using vector search over your
business knowledge. Returns curated documentation that explains the business meaning of tables,
fields, and metrics, with each document's referenced warehouse resources already resolved. This is
the **first step** of the query workflow: the context it returns grounds `generate_sql` in your
organization's own definitions and rules. It is also what decides whether a question should be
answered from the warehouse or from a [Live Data](#live-data) connection. For creating semantic
context, see [Adding context](#adding-context).
**Example prompt:** "explain how we define monthly recurring revenue"
**Results are filtered by permission.** The context layer is organized into folders, each with
its own access rules, and on Growth and Custom plans this tool returns only the documents the
token's user is allowed to read. So an empty result has two different causes: the question matched
nothing, or there is documentation the caller cannot see. Your agent handles them differently: it
rephrases into narrower sub-questions for the first, and for the second it tells you there may be
documentation you don't have access to rather than retrying. See
[Permissions](/mcp-server/permissions).
### `get_relevant_tables_ddl`
Discovers which tables are relevant to a given question using semantic search. Returns the full
schema (CREATE TABLE DDL) for each matched table, useful for understanding available columns and
types before generating SQL.
**Example prompt:** "which tables should I use to analyze customer churn?"
### `generate_sql`
Converts a natural language question into a SQL query based on your table schemas and the business
context retrieved above. The generated query is scoped to the tables accessible by the token. You can
then execute it using the `execute_sql` tool.
**Example prompt:** "write me a query that returns active users in the last 30 days"
### `execute_sql`
Executes a SQL query against your data warehouse and returns the results as structured data. Supports
pagination for large result sets (up to 1,000 rows per page). The SQL dialect adapts automatically to
your cloud: BigQuery for GCP and Nekt Express, Athena for AWS.
When a query fails, the error comes back with a likely explanation rather than the raw engine text
alone: in particular it distinguishes a table that exists in your Catalog but has not been
materialized yet (the pipeline that writes it hasn't produced data) from a table name that is simply
wrong, so your agent stops guessing at names that were already correct.
**Example prompt:** "run this query and show me the results: SELECT COUNT(\*) FROM orders WHERE status = 'paid'"
### `get_table_preview`
Returns a small sample (10 rows) from a table to help understand its contents. Useful for inspecting
column values, data formats, and categorical fields before writing queries.
**Example prompt:** "show me a sample of the orders table"
### `list_tables`
Lists tables in the Catalog one page at a time (100 tables per page). Use this when you need a
deterministic, complete enumeration of available tables: for example, when browsing the Catalog
without a specific question, or when you need a table's `id`. For question-driven discovery, prefer
`get_relevant_tables_ddl`.
**Example prompt:** "list all tables in the trusted layer"
## Browsing what's connected
Two tools cover everything in your workspace: sources, destinations, Queries, and Notebooks. Both
take a `kind` argument.
### `list_resources`
The single browse tool over all resource types. `kind` accepts:
| `kind` | Returns |
| :- | :- |
| `"source"` | Connected sources (pipelines that pull data in) |
| `"destination"` | Connected destinations (pipelines that push data out) |
| `"transformation"` | Queries and Notebooks together |
| `"query"` | SQL transformations only |
| `"notebook"` | Python / PySpark transformations only |
Supports free-text `search`, plus `status`, `tag`, `connector_slug`, and `output_table_name` filters
and pagination (100 per page). Pass `include_trigger=True` to also project each row's current
trigger: that's how you review existing schedules, and how your agent discovers the pipeline slugs
available to an event trigger. For transformations, each row also carries the GitHub repo the code is
mirrored to, if any.
**Example prompt:** "show me all my connected sources"
### `get_resource`
Returns the full detail of a single source or destination by slug, including its streams and
triggers for a source. Configuration values are redacted for security. For a transformation's code,
use `get_code` instead.
**Example prompt:** "what streams is the Shopify source pulling?"
## Data layers
A layer (lakehouse) is the container every table and pipeline writes into. Nothing can be
published without one: a source needs an output layer, and so does a Query or a Notebook.
### `list_layers`
Lists all data layers that your token has access to. Returns each layer's ID, name, slug, and
description. It is the starting point for choosing where a pipeline writes, and for browsing
[files in volumes](#browsing-files-in-volumes). When your workspace has no layers yet, the result
says so explicitly and points at `create_layer`, so your agent offers to create the first one
instead of reporting "nothing found".
With a scoped token, layer and volume browsing is available only when scoped-token browsing has
been enabled for your workspace by Nekt support. See
[What a scoped token can reach](/mcp-server/permissions#what-a-scoped-token-can-reach).
**Example prompt:** "what layers do I have access to?"
### `create_layer`
Creates a new data layer. Your agent will raise this on its own in two situations: when
`list_layers` comes back empty (nothing can be published yet), and when you are setting up a
source or a transformation and none of the existing layers fit what you're building.
It is **two-phase**: the first call returns a preview (the layers you already have, any name
collision, and a reminder that the name is permanent) and creates nothing. Only a second call
with `confirm=True` creates the layer.
**The name is a one-way decision.** Nekt derives both the layer's slug and its physical database
name from the name you give it, and that database name is what every SQL query, every source, and
every context-layer document then references. **A layer cannot be renamed, and tables and
pipelines cannot be moved between layers**: the closest thing to a fix is building a second
layer and rebuilding everything that pointed at the first. Approve the exact wording before
confirming.
Ask for a name that will still be right in a year, and favour the role the data plays over a
project, team, or tool name. If you have no convention yet, your agent will suggest the common
stage-based one: `Raw` (landed as the source produced it), `Trusted` or `Silver` (cleaned, typed,
deduplicated), `Service` or `Gold` (modelled for consumption), though organizing by domain
(`Finance`, `Marketing`) is equally valid.
Creating a layer needs a **full access token created by an Owner or an Admin**. Other tokens get
a clear message saying so, and layers can always be created in the Nekt web app instead.
**Example prompt:** "create a silver layer for cleaned data"
## Connecting sources
Sources pull data *into* Nekt from external systems: Shopify, HubSpot, Postgres, and more. New
sources are created through a **draft → publish** flow: create a draft, hand the user a secure
browser link to complete OAuth or enter secrets (credentials never pass through the chat), validate
the live connection, then finalize.
Every step is a cross-kind tool called with `kind="source"`:
1. **`list_connectors(kind="source")`** / **`get_connector(kind="source")`**: find the connector and read its config schema to see required and secret fields.
2. **`create_draft(kind="source")`**: create a draft (connector slug and a description are required; no tables or pipeline yet).
3. **`get_setup_link(kind="source")`**: for OAuth/secret connectors, hand the user a deeplink to finish setup in the browser.
4. **`validate_source_connector_config`**: test the real connection.
5. **`check_connector_validation`**: poll until validation succeeds and the available streams come back.
6. **`complete_pipeline(kind="source")`**: finalize the draft: choose streams, output layer, and trigger. Tables are provisioned and the pipeline is deployed.
Connectors with no secret fields can skip step 3: pass the full config directly at `create_draft`.
### `list_connectors`
Lists the connector types you can connect, returning each one's slug and metadata. Use it to discover
the slug needed to start a draft. Pass `kind="source"` or `kind="destination"`.
**Example prompt:** "what sources can I connect to Nekt?"
### `get_connector`
Returns a connector's configuration schema (`config_template`): every field, which are required, and
which are secret, plus a convenience split so your agent can decide the flow. Connectors with secret
fields (OAuth, API keys, passwords) go through the browser deeplink; connectors without secrets can
be configured entirely in chat. Pass `kind="source"` or `kind="destination"`.
**Example prompt:** "what do I need to connect Postgres as a source?"
### `create_draft`
Creates a draft source or destination: the first step of connecting one. A draft has no provisioned
tables and no deployed pipeline; the connector slug and a description are required, so you can start
the connection before any secrets exist. For no-secret connectors, you can pass the full config here
directly.
**Example prompt:** "start connecting our Shopify store as a source"
### `get_setup_link`
Mints a time-limited (24h) deeplink for a draft so the user can complete OAuth or enter secret values
in the browser: **secrets never pass through the chat**. Present the link and wait for the user to
finish before continuing; the token auto-invalidates once the resource leaves draft. The page fills
only the connector config, not streams, tables, or triggers.
**Example prompt:** "give me the link to enter the Shopify credentials"
### `validate_source_connector_config`
Tests the connector configuration by actually connecting to the external system, catching bad
credentials or wrong hosts before the source is finalized. For drafts set up via the deeplink, it
re-validates against the config stored on the backend, so secrets never round-trip through the chat.
Returns a validation ID to poll. This step is **source-only**: destinations publish directly.
**Example prompt:** "check that the Shopify connection works"
### `check_connector_validation`
Polls a validation job until it finishes. On success, it returns the available **streams**: the
tables/objects the source can pull, each with its supported sync types, fields, primary keys, and
replication keys. Present these so the user can choose which streams to sync before completing the
source.
**Example prompt:** "which tables can I pull from this source?"
### `complete_pipeline`
Finalizes a draft. For a **source** (`kind="source"`), it requires a description, an output layer, at
least one stream, and a trigger (`manual`, `cron`, or `event`); each stream specifies a destination
table name and a sync type: `INCREMENTAL` (append new/changed rows) or `FULL_SYNC` (replace all).
For a **destination** (`kind="destination"`), it takes the tables to deliver and a trigger.
Completion is one-way: it provisions warehouse tables and deploys the pipeline.
For a source, the destination table names are **prefixed with the connector's name** automatically
(`hubspot_deals`, `hubspot_contacts`), matching what the web app does. That is what keeps a second
source from colliding with the first over generic stream names like `users` or `contacts`, and a
table name cannot be changed after it is created. The final names come back in the result, so your
agent reports the tables you actually got.
**Example prompt:** "pull the orders and customers streams into the raw layer and sync them daily at 6am"
## Managing a live source
Once a source is published, these change it in place.
### `update_source_streams`
Adds new streams or edits which streams a live source pulls, applied in place without a redeploy. It
is non-destructive: it never removes streams or drops tables; to stop a stream, disable it. Removing
streams and dropping tables is deliberately left to the web app.
**Example prompt:** "also start pulling the products stream from Shopify"
### `update_trigger`
Changes the trigger of an already-live source: `manual`, `cron` (a schedule), or `event` (run when
upstream pipelines finish). Use `kind="source"`. The same tool handles transformation triggers; see
[Creating and editing pipelines](#creating-and-editing-pipelines).
**Example prompt:** "run the Shopify source every weekday at 7am instead"
### `run_pipeline`
Triggers an extraction run of a live source on demand, optionally as a full sync or for selected
streams. Use `kind="source"`. This **consumes credits**, so the assistant confirms before running.
**Example prompt:** "run the Shopify source now"
## Connecting destinations
Destinations deliver data *out* of Nekt to external systems: Slack, Google Sheets, Postgres,
webhooks, ad platforms, and more. The flow is the same shape as a source, with `kind="destination"`,
and one difference: **destinations have no validation step; they publish directly**.
1. **`list_connectors(kind="destination")`** / **`get_connector(kind="destination")`**: find the connector and read its config schema.
2. **`create_draft(kind="destination")`**: create a draft (connector slug and a description; input tables optional here).
3. **`get_setup_link(kind="destination")`**: for OAuth/secret connectors, hand the user a deeplink to finish setup in the browser.
4. **`complete_pipeline(kind="destination")`**: finalize: choose the tables to deliver (each with its fields, and optional primary keys or field renames) and a trigger. The delivery pipeline is deployed.
Browsing destinations is `list_resources(kind="destination")` and `get_resource(kind="destination")`;
running one on demand is `run_pipeline(kind="destination")`: fire-and-forget, with the assistant
confirming first. Run history comes from `list_pipeline_runs` and `get_pipeline_run_logs`.
**Example prompt:** "start sending revenue alerts to our #revenue Slack channel"
## Creating and editing pipelines
### `create_query`
Deploys a SQL Query as a pipeline in Nekt. Accepts the SQL code, a description, the output layer and
table name, and an optional trigger: either a cron schedule or an event trigger that runs the Query
when one or more upstream pipelines finish.
It is **two-phase**: the first call returns an authoring guide and a preview of exactly what will be
created, without creating anything; your agent shows you that and only creates the pipeline on a
second call with `confirm=True`. The pipeline then starts in `preparing` status; poll
`get_pipeline_status` until it leaves `preparing`, then trigger execution with `run_pipeline` once
you confirm.
Optionally, the Query can be created already connected to a GitHub repo: pass the repository full
name (and optionally a branch), and Nekt creates, connects, and pushes the first commit in one step.
The folder in the repo is always the generated pipeline slug. See
[Syncing transformation code to GitHub](#syncing-transformation-code-to-github).
**Example prompt:** "create a daily Query that aggregates revenue by customer into the trusted layer"
### `create_notebook`
Deploys a Python or PySpark Notebook as a pipeline in Nekt. Accepts the notebook code, a description,
the runtime type (`python-base`, `pyspark`, or `python-browser`), optional pip dependencies, and an
optional trigger. For PySpark notebooks, Spark driver and executor sizing can also be configured.
Inputs and outputs are extracted automatically from `nekt.load_table` and `nekt.save_table` calls in
the code, so the notebook must use the Nekt SDK for all table I/O.
Like `create_query`, it is **two-phase** (preview first, then `confirm=True`), follows the same
deploy-then-run lifecycle, and can be created already connected to a GitHub repo.
**Example prompt:** "create a PySpark Notebook that joins orders and customers and runs every hour"
### `get_code`
Returns the current code of a Query or Notebook by slug, along with its configuration and whether it
is connected to a GitHub repo. Use it before editing so changes are made against what is actually
deployed.
**Example prompt:** "show me the code of the daily\_revenue Query"
### `update_transformation`
Edits an existing transformation, a Query or a Notebook, with no `kind` argument; it infers the
type. Updates the code, description, dependencies (Notebooks only), active state, and retry settings.
If the transformation is connected to GitHub, changing its code auto-commits the change; no extra
parameters are needed.
**Example prompt:** "update the daily\_revenue Query to also exclude refunded orders"
### `update_trigger`
Replaces the trigger of an existing pipeline. Supports three trigger types: `manual` (runs only on
demand), `cron` (runs on a schedule, with a cron expression and timezone), and `event` (runs whenever
upstream pipelines finish). For event triggers, provide the triggering pipeline slugs and an optional
rule: `any` runs it when any of them completes, `all` waits for all of them. Use
`list_resources(kind=..., include_trigger=True)` to discover the available slugs. Defaults to
`kind="transformation"`; pass `kind="source"` for a source.
**Example prompt:** "run the daily\_revenue Query whenever the orders and customers sources finish"
## Running and monitoring pipelines
### `run_pipeline`
Triggers execution of a deployed pipeline by slug. Defaults to `kind="transformation"` (Queries and
Notebooks); pass `kind="source"` to trigger an extraction or `kind="destination"` to trigger a
delivery. Returns immediately without waiting for the run to finish: use `get_pipeline_status` to
track progress.
**Example prompt:** "run the daily\_revenue pipeline"
### `get_pipeline_status`
Returns the current status of any pipeline by slug: Sources, Queries, Notebooks, or Destinations.
Used to track deployment progress after `create_query` or `create_notebook` (until status leaves
`preparing`), and to monitor execution after `run_pipeline` until it reaches a terminal state
(`success` or `failed`).
**Example prompt:** "what's the status of the daily\_revenue pipeline?"
### `list_pipeline_runs`
Lists recent runs of any pipeline by slug. Returns a summary for each run including its `id`,
`status`, `start` and `end` times, and `trigger_type`. Paginated (25 per page by default, up to 100).
Use this to inspect a pipeline's execution history, such as checking for recent failures or
confirming that a trigger fired.
**Example prompt:** "show me the last runs of the daily\_revenue pipeline"
### `get_pipeline_run_logs`
Fetches the logs of a specific run, or of the most recent run when only a pipeline slug is given.
Returns the `ERROR`-level entries and, when available, Nekt's AI analysis of the failure: a summary
of what happened, the technical details, and a suggested fix. Use this to diagnose a failed run
without reading the full log.
**Example prompt:** "why did the last run of the daily\_revenue pipeline fail?"
### `update_alerts`
Turns failure alerts on or off for one source, destination, or transformation, by slug. While
subscribed, every **failed** run of that pipeline emails you. Pass `kind` to say which resource
type the slug belongs to: `"source"`, `"destination"`, or `"transformation"` (`"query"` and
`"notebook"` are accepted too, since that's how `list_resources` names them).
**Alerts are per user and failure-only.** Subscribing emails the Nekt account this connection
authenticated as: it is not a workspace-wide switch, and it doesn't touch the workspace's global
alert emails configured in the app. Unsubscribing stops *your* emails and nobody else's.
Subscribing twice is a no-op, and so is unsubscribing something you were never subscribed to. The
tool reports the state after the call, not the state you asked for. To check the current state
without changing it, `get_resource` (sources and destinations) and `get_code` (transformations)
already carry it; no extra call needed.
Alert subscriptions are tied to a real user account, so this needs a **full access token**; a
table-scoped token has no user to email and is refused with a message saying so.
**Example prompt:** "email me whenever the Shopify source fails"
## Adding context
The context layer is the curated business knowledge your agent reads before writing SQL: what
tables and fields *mean*, how metrics are defined, and which source to prefer for which question.
`get_semantic_context` (above) reads it; these tools curate it.
### `update_resource_description`
Updates the Description of a resource in your Catalog. Supported `resource_type` values are `layer`,
`table`, `field`, `folder`, `volume_file`, and `derived_field`. Returns a preview by default: pass
`confirm=True` to apply the change.
When `resource_type` is `derived_field`, the tool first identifies the Query or Notebook that
produced the target table (via `list_resources(kind="transformation")`), then enriches the field
Description with structured tags inferred from the transformation code: `@pk`, `@fk .`,
`@measure additive|non-additive`, `@attribute`, and `@grain=`. Use this when annotating fact,
aggregated, or dimension tables so downstream consumers and AI agents can interpret each column's
role.
**Example prompt:** "I want to add descriptions to this table: sessions\_reviews\_consolidated"
### `list_semantic_layer_folders`
Lists the folders of the context layer, so a document can be filed in the right one. On Growth and
Custom plans **each folder has its own access rules**, so where a document goes is a real decision:
a document filed in the wrong place is invisible to the people who needed it, and creating one
without a folder targets the root, which most users cannot write to.
Returns each folder's ID, name, readable path (`Marketing / Metrics`) and your permission level on
it. Pass `writable_only=True` to see only the folders you can actually save into: your agent calls
it that way *before* creating a document, not after being denied. When several folders plausibly
fit, it will ask you which one rather than guessing.
**Example prompt:** "where can I save context documents?"
### `create_semantic_layer_folder`
Creates a folder in the context layer, at the root or inside an existing folder. Your agent offers
it when `list_semantic_layer_folders` comes back empty, or when no existing folder fits the document
you are filing. It is **two-phase**: the first call checks the name against the folders that
already exist at that level and creates nothing; a second call with `confirm=True` creates the
folder.
A subfolder needs Editor on its parent folder, and a top-level folder needs Editor on the root of
the context layer, which Owners and Admins hold by default. On Free and Starter, where
context-layer permissions are not enforced, every member can create folders.
**Example prompt:** "create a Finance folder in the context layer for our revenue definitions"
### `create_context_document`
Creates a new context-layer context document: reusable business knowledge such as metric
definitions, segment logic, or table interpretations that should inform future analyses. Documents
support annotations (`@table`, `@field`, `@layer`, `@measure`, `@transformation`, `@source`,
`@destination`, `@context`) that link the prose to specific warehouse resources. Returns a preview by
default: pass `confirm=True` to save the document after reviewing it.
It also takes the **folder** to file the document in. Pick that first with
[`list_semantic_layer_folders`](#list_semantic_layer_folders): a document created without a folder
lands at the root of the context layer, which most users are not allowed to write to, and needs
editor or manager access on the folder that receives it.
Context documents are also how you steer [Live Data](#live-data) routing: a document that says which
questions should be answered from a connected app rather than the warehouse makes that routing
automatic next time.
**Example prompt:** "save a definition of net revenue that excludes refunds and chargebacks"
### `update_context_document`
Updates an existing context-layer context document when a definition is refined, a segment is added,
or business rules change. The full name and content are replaced, so the saved version always
reflects the complete document. Returns a preview showing the current versus new content by default:
pass `confirm=True` to apply the change.
On Growth and Custom plans, editing a document needs Editor on it (a grant on its folder counts),
and the tool is available only to people who hold Editor on at least one document.
**Example prompt:** "update the revenue definition to also exclude partial refunds"
### `delete_context_document`
Reached through the dispatcher: see [More tools](#more-tools).
Soft-deletes a context-layer context document so it no longer influences analyses or appears in
semantic search. Use it for obsolete, duplicated, or incorrect knowledge; prefer
`update_context_document` when a document is merely out of date. Returns a preview of the document to
be removed by default: pass `confirm=True` to delete it (there is no undo).
On Growth and Custom plans, deleting a document needs Manager on it, one level above what creating
and editing need.
**Example prompt:** "delete the old churn definition document, it's outdated"
### `resolve_references`
Reached through the dispatcher: see [More tools](#more-tools).
Follows the annotations inside a context-layer document to pull in related tables, fields, measures,
or other context documents that weren't part of the initial match. Use it to traverse the semantic
graph when a returned document points to definitions it doesn't fully contain. Returns the resolved
entities along with their names, layers, and descriptions.
**Example prompt:** "how does the revenue metric connect back to the orders table?"
## Live Data
Live Data tools read from and write to your connected apps in real time, such as HubSpot, Slack, Grafana, or
an ad platform, rather than reading the copy of that data in your warehouse.
Because the set of live tools grows with every connector you attach, they are **not** listed
individually in your agent's tool list. Two tools cover all of them.
Find the list of MCP connectors, with the tools each one exposes, in
[MCP connectors](/mcps/overview#mcp-connectors).
### `discover_live_tools`
Lists your organization's connected Live Data tools, grouped by connection: the connector, its
namespace, and for each tool its exact name, description, input schema, and whether it writes. It
refreshes on demand, so a connector you linked *after* your client connected shows up without
reconnecting.
**Example prompt:** "what live tools do I have connected?"
### `run_live_tool`
Calls a live tool by name, passing arguments built from its input schema. Since live tools aren't
listed individually, this is how every one of them is invoked.
**Example prompt:** "create a HubSpot deal for Acme Corp at \$12,000"
The call order is:
```
get_semantic_context(question) → warehouse or live? and which connection?
discover_live_tools() → the exact tool name and its input schema
run_live_tool(tool_name, arguments) → run it
```
**Why discovery comes first.** Live tool names are specific to your organization and can change, so
an agent should resolve them at run time rather than hardcoding them into a skill or automation.
Discovery also disambiguates two connections of the same app: a Marketing HubSpot and a CS
HubSpot show up as separate entries, so your agent can ask which one you mean.
A full access token sees every live connection its user can use. A scoped token sees only the live
connections (and, within each, the tools) selected on it; a token scoped only to tables sees none.
If your organization has no Live Data connections, these tools simply return nothing and the
routing guidance is omitted from your agent's tool list entirely.
Write tools are enabled one by one per connection, and your agent confirms before calling one. After
routing a question to a live tool, your agent will typically offer to save a context document
recording that routing, so the next similar question goes to the right place automatically.
## Browsing files in volumes
Volumes are file stores that live inside a layer, so browsing starts from
[`list_layers`](#list_layers). The file tools below, including the two that upload a file, are
reached through the dispatcher: see [More tools](#more-tools).
### `list_volumes`
Reached through the dispatcher: see [More tools](#more-tools).
Lists volumes (file stores) within a layer, or across all layers if no layer is specified. Returns
each volume's name, slug, and description. Use this after `list_layers` to discover available file
containers.
**Example prompt:** "what volumes are available in the raw layer?"
### `list_files`
Reached through the dispatcher: see [More tools](#more-tools).
Lists files stored inside a specific volume, one page at a time (100 per page). Requires the layer
slug and volume slug to scope the results. Returns file metadata including name, type, and size: use
this to find the file ID needed to read a file's content.
**Example prompt:** "list the files in the contracts volume"
### `read_file`
Reached through the dispatcher: see [More tools](#more-tools).
Downloads and returns the content of a file stored in a Nekt volume. Requires the layer slug, volume
slug, and file ID. Text files (md, txt, csv, json, yaml, xml) come back as UTF-8 strings and PDFs as
extracted text, so the agent can answer from the content directly; other binaries come back as
base64. Maximum inline size is 2 MB.
When you want the original file rather than its content (to forward, attach, or save it), the
`download` parameter returns a presigned URL, a base64 string, or an MCP resource blob instead (up to
10 MB).
**Example prompt:** "open the latest invoice in the documents volume and summarize it"
### `create_volume_upload_url`
Reached through the dispatcher: see [More tools](#more-tools).
Starts uploading a file from your machine into a volume. It does not move the file: it returns
presigned URLs (one per 100 MB part) that your agent then uploads the bytes to directly, so the file
never passes through the MCP Server or the conversation. This requires an agent that can run shell
commands (`curl`) on the machine holding the file; in a chat client without a filesystem, use the
upload button on the volume page instead. Uploading a name that already exists in the volume
replaces that file. Needs Editor access on the volume.
**Example prompt:** "upload ./reports/q3-board-deck.pdf to the documents volume in the raw layer"
### `complete_volume_upload`
Reached through the dispatcher: see [More tools](#more-tools).
Finishes an upload started by `create_volume_upload_url`, using the ETag returned for each uploaded
part. Until it runs, the file appears in `list_files` but cannot be read. Your agent calls it right
after the parts are uploaded.
## Syncing transformation code to GitHub
Query and Notebook code can be mirrored into your own GitHub repository (one GitHub App installation
per organization). The repo is a code mirror: a transformation's trigger, output layer, and settings
still live in Nekt. There are two paths: create a transformation already connected to a repo (via
`create_query` / `create_notebook`), or connect transformations that already exist. Once a
transformation is connected, editing its code with `update_transformation` auto-commits the change:
no extra parameters needed.
To see which transformations are already connected and where, use
`list_resources(kind="transformation")`: each row carries its GitHub repo.
### `list_github_repositories`
Reached through the dispatcher: see [More tools](#more-tools).
Lists the repositories the Nekt GitHub App can write to for your organization. Call it once to
discover the repo full name (e.g. `my-org/analytics`) needed when creating or connecting
transformations.
**Example prompt:** "which GitHub repos can Nekt push transformations to?"
### `connect_transformations_to_github`
Reached through the dispatcher: see [More tools](#more-tools).
Connects one or more existing Queries or Notebooks to a GitHub repo, uploading each one's current
code as a commit. This is the only way to connect a transformation that already exists: there is no
GitHub parameter on the update tool. Connecting overwrites any existing repo binding, so your agent
warns before re-pointing a transformation that's already connected elsewhere.
**Example prompt:** "sync my daily\_revenue and churn notebooks to the analytics repo"
## Managing permissions
These tools read and change who can access what, on the three permission axes: `catalog` (layers,
folders, tables, and volumes), `others` (secrets and live connections), and `semantic_layer`
(context-layer folders and documents). Pass the axis as `axis`; it defaults to `catalog`.
The ceiling is always the person behind the token: an agent can grant only what that person could
grant in the app. A Member holding Manager on a resource can grant, change, or revoke Viewer and
Editor there; only Owners and Admins can create or remove Manager grants. Grants are enforced on
Growth and Custom plans. All of these tools need a **full access token**; a scoped token is refused.
See [Permissions](/workspace/permissions) for the model itself.
### `list_permissions`
Lists the grants on one axis, optionally narrowed to one resource. Owners and Admins see every grant
in the workspace; a Member sees their own, the ones their groups carry, and every grant on a
resource they manage.
**Example prompt:** "who has access to the finance layer?"
### `list_permission_recipients`
Lists the people and groups that can still be added to a resource, so your agent can turn a name
into an id before granting. It leaves out Owners and Admins, who already have access to everything,
and anyone who already holds a grant on that resource.
**Example prompt:** "who can I still share the churn dashboard table with?"
### `list_permission_groups`
Lists the workspace's permission groups and their members, with optional filters by name or by
person. Use it to explain access a person has through a group rather than a grant of their own. The
built-in **All** group contains every member automatically and cannot be edited or deleted.
**Example prompt:** "which groups is Ana in?"
### `grant_permission`
Grants a user or a group `viewer`, `editor`, or `manager` access to a resource. It is
**two-phase**: the first call returns exactly what would be written; a second call with
`confirm=True` applies it.
**Example prompt:** "give the Finance group viewer access to the revenue table"
### `update_permission_level`
Changes the level of an existing grant in place. Prefer it over revoking and granting again:
revoking removes the resource from any scoped MCP token that relied on that access, and granting it
again does not put it back. Two-phase, like `grant_permission`.
**Example prompt:** "downgrade Pedro from editor to viewer on the raw layer"
### `revoke_permission`
Removes a grant. The resource is also removed from any scoped MCP token that relied on it, and
re-granting later does not restore those tokens; their owners have to select the resource again.
Two-phase.
**Example prompt:** "remove the Contractors group's access to the customer secrets"
### `manage_permission_group`
Creates, renames, or deletes a permission group, or adds and removes its members. Adding a person to
a group gives them everything the group can reach, on all three axes. Owners and Admins only, and
two-phase. The **All** group cannot be changed.
**Example prompt:** "create a Marketing group with Ana and Bruno in it"
## Managing MCP tokens
These tools let an agent create and maintain MCP tokens for other agents and automations. They need
a **full access token**, so a scoped token can never widen itself or mint a wider one, and they
only see tokens created by the person behind the current token. A new token can never reach
more than that person.
### `get_mcp_token_available_scopes`
Lists the tools and live connections a new token can be scoped to, for your organization's MCP
Server version. Tables, volumes, secrets, and documents come from their own listing tools.
**Example prompt:** "what can I scope a new MCP token to?"
### `list_mcp_tokens`
Lists the tokens created by the person behind the current token, with each one's description,
expiration, and scopes. The token secret is never included.
**Example prompt:** "which MCP tokens have I created, and when do they expire?"
### `create_mcp_token`
Creates a token, scoped per resource type and per tool. It is **two-phase**: the first call
previews the scope and creates nothing. The result never contains the secret: it carries a one-time
link, valid for 15 minutes, that opens only in the creator's own browser session. Open it yourself
and paste the token into the tool that will use it. Tokens last one year.
**Example prompt:** "create a token for our n8n workflow that can only query the sales tables"
### `update_mcp_token_scopes`
Changes what an existing scoped token can reach, one resource type at a time. Each value you pass
replaces that axis entirely; axes you omit stay as they are. A full access token takes no scope.
Two-phase.
**Example prompt:** "add the marketing folder to the n8n token's tables"
### `rotate_mcp_token`
Replaces a token's secret and keeps its description, scopes, and expiration date. Everything still
using the old secret stops working immediately, including OAuth connections built on it. Like
creation, it returns a one-time link instead of the secret. A token cannot rotate itself. Two-phase.
**Example prompt:** "rotate the n8n token, I think it leaked"
## Token access log
Every request made with an API key or an MCP token is recorded in the token access log: which
token, who created it, which action, which endpoint, the status, and the request content. Because a
query's SQL is part of the request, this is where you find out which tables an AI agent queried.
You can browse it in the app under [Settings → Audit logs](/workspace/audit-logs), and the same log
is available through the [Platform API](/platform-api/token-access-logs/list). To report on a whole
period at once, with each log's full detail and what its Athena queries scanned and cost,
[export it as a Parquet file](/platform-api/token-access-logs/export) through the Platform API.
Every tool an MCP token calls is recorded as its own entry, named after the tool (for example,
`execute_sql`, `generate_sql`, `get_table_preview`), so you see which tool an agent called, not
just which endpoint it hit. This includes tools that never reach the API, such as `generate_sql`,
`get_semantic_context`, and the [Live Data](#live-data) tools, as well as calls the token's scope
refused. Opening an entry shows the tool's arguments and the requests that tool triggered.
Owners and Admins see every token in the workspace; other members see only the tokens they created.
The log is not available on the Free plan, and it covers the last 7 days on Starter, 30 on Growth,
and 90 on Custom. Both tools need a **full access token**.
### `list_token_access_logs`
Lists the log newest first, filtered by date range, token, token kind (API token or MCP token),
token creator, action, transport (HTTP or WebSocket), or status code. Without a start date it
covers the last 30 days. Results are paginated; to sync incrementally, pass the newest
`created_at` you already have as the lower bound.
**Example prompt:** "what did the n8n token do last night, and did any request fail?"
### `get_token_access_log`
Returns one log line together with the recorded request content: the request body (including the
SQL of a query), the query string, and a short list of headers. Credentials are masked. The content
can occasionally be unavailable while the line itself is intact.
**Example prompt:** "show me the exact SQL the agent ran in that request"
## More tools
A long tool list measurably degrades how accurately an agent picks the right tool, so nine
rarely-used tools are kept off the main list. They remain fully available through two dispatcher
tools, and they are documented in their own sections above.
### `get_tool_details`
With no argument, returns a catalog of every tool kept off the main list. With a tool name, returns
that tool's full documentation and input schema, plus whether it writes or is destructive, so your
agent can confirm with you before firing a write.
### `run_tool`
Runs one of those tools by name, passing the arguments built from its schema. Two-phase tools behave
exactly as documented: call again with `confirm: true` in the arguments to commit.
The tools reached this way:
| Tool | What it does |
| :- | :- |
| `list_volumes` | Lists file stores within a layer, or across all layers. |
| `list_files` | Lists files inside a volume, with metadata. |
| `read_file` | Downloads and returns the content of a file from a volume. |
| `create_volume_upload_url` | Returns presigned URLs to upload a local file into a volume. |
| `complete_volume_upload` | Finishes an upload started by `create_volume_upload_url`. |
| `resolve_references` | Follows annotations inside context-layer documents to pull in related context. |
| `delete_context_document` | Soft-deletes a context document. Two-phase, and there is no undelete. |
| `list_github_repositories` | Lists the repositories the Nekt GitHub App can write to. |
| `connect_transformations_to_github` | Connects existing Queries or Notebooks to a GitHub repo and pushes their current code. |
These tools stay off the main list for every session, so each use goes through
`get_tool_details` → `run_tool`. Nothing changes about what they do or what they accept, only
how they're reached.
## Changed tool names
Several tools were merged into generic versions that take a `kind` argument. **The old names no
longer exist.** If you have saved skills, automations, or prompts that call them, update them using
this table: the arguments are otherwise unchanged.
| Old name | Now |
| :- | :- |
| `list_sources` | `list_resources(kind="source")` |
| `list_destinations` | `list_resources(kind="destination")` |
| `list_queries` | `list_resources(kind="query")` |
| `list_notebooks` | `list_resources(kind="notebook")` |
| `get_source_details` | `get_resource(kind="source")` |
| `get_destination` | `get_resource(kind="destination")` |
| `list_source_connectors` | `list_connectors(kind="source")` |
| `list_destination_connectors` | `list_connectors(kind="destination")` |
| `get_source_connector` | `get_connector(kind="source")` |
| `get_destination_connector` | `get_connector(kind="destination")` |
| `create_draft_source` | `create_draft(kind="source")` |
| `create_draft_destination` | `create_draft(kind="destination")` |
| `get_source_setup_link` | `get_setup_link(kind="source")` |
| `get_destination_setup_link` | `get_setup_link(kind="destination")` |
| `complete_source` | `complete_pipeline(kind="source")` |
| `complete_destination` | `complete_pipeline(kind="destination")` |
| `run_source` | `run_pipeline(kind="source")` |
| `run_destination` | `run_pipeline(kind="destination")` |
| `update_query` | `update_transformation` |
| `update_notebook` | `update_transformation` |
| `update_transformation_trigger` | `update_trigger` |
| `update_source_trigger` | `update_trigger(kind="source")` |
| `list_pipeline_options` | `list_resources(kind=..., include_trigger=True)` |
| `ask_about_nekt` | `ask_nekt` (now also returns Academy playbooks; use `focus="product"` for the old behavior) |
`list_pipeline_options` is the one tool that was **removed rather than renamed**. It returned every
pipeline in a single unpaginated response; `list_resources` is paginated and filtered by `kind`, so
an automation that relied on getting everything at once needs its logic adjusted, not just its name
swapped.
Ask your agent to do the migration for you: point it at your skill or automation files and give it
this table. It can find and update every reference in one pass.
# AWS as an MCP
Source: https://docs.nekt.com/mcps/aws
Let your AI agents use AWS in real time through the Nekt MCP Gateway.
AWS is a comprehensive cloud computing platform. By attaching AWS as an MCP, your agents can inspect and manage your Step Functions executions, CloudWatch metrics, CloudWatch Logs, and Lambda functions directly from a conversation.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding AWS as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **AWS**.
**Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
**Setup access**. Generate an access key pair in AWS for the IAM user or role the agent should act as.
* **Default AWS region**: The AWS region used when a tool call does not specify one, e.g. us-east-1 or ca-central-1. (Required)
* **Access key ID**: Access key ID of the IAM user. Grant this user read-only permissions unless the write tools will be used. (Required)
* **Secret access key**: Secret access key matching the access key ID. (Required)
* **Session token**: Optional. Only needed when the access key pair is a temporary credential.
**Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "List the recent executions for our data pipeline state machine."
* "Check the logs for the cache invalidation Lambda function over the last hour."
* "Get the current CPU utilization metrics for the production cluster."
* "Invoke the reporting Lambda function with the daily payload."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `execute_cloudwatch_logs_read_only` | Call a read-only AWS CloudWatch Logs (boto3 `logs`) operation and return its response. For reading logs: `filter_log_events` and `get_log_events` to fetch log lines, `describe_log_groups` / `describe_log_streams` to find them, and `start_query` + `get_query_results` for Logs Ins… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
| `execute_cloudwatch_read_only` | Call a read-only AWS CloudWatch metrics/alarms (boto3 `cloudwatch`) operation. For metrics and alarm state: `get_metric_data` and `get_metric_statistics` to read datapoints, `list_metrics` to discover them, `describe_alarms` / `describe_alarm_history` for alarm state. Pass \`oper… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
| `execute_lambda_read_only` | Call a read-only AWS Lambda (boto3 `lambda`) operation and return its response. For inspecting functions: `get_function` and `get_function_configuration` for a function's settings, `list_functions` to enumerate them, `get_function_concurrency`, `list_event_source_mappings`, \`get… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
| `execute_stepfunctions_read_only` | Call a read-only AWS Step Functions (boto3 `stepfunctions`) operation and return its response. For inspecting state machines and executions: `describe_execution`, `get_execution_history`, `list_executions`, `describe_state_machine`, `list_state_machines`, `describe_map_run`, \`li… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `execute_cloudwatch_logs_write` | Call any AWS CloudWatch Logs (boto3 `logs`) operation, including mutating ones. Covers `create_log_group`, `delete_log_group`, `put_log_events`, `put_retention_policy`, `put_subscription_filter`, `create_export_task` and tagging. `parameters` is the AWS request object. For readi… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
| `execute_cloudwatch_write` | Call any AWS CloudWatch metrics/alarms (boto3 `cloudwatch`) operation, including mutating ones. Covers `put_metric_data`, `put_metric_alarm`, `delete_alarms`, `set_alarm_state`, `put_dashboard` and tagging. `parameters` is the AWS request object. For reading metrics and alarm st… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
| `execute_lambda_write` | Call any AWS Lambda (boto3 `lambda`) operation, including invoking and mutating functions. Covers `invoke` (which runs the function), `create_function`, `update_function_code`, `update_function_configuration`, `delete_function`, `add_permission` and concurrency settings. \`parame… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
| `execute_stepfunctions_write` | Call any AWS Step Functions (boto3 `stepfunctions`) operation, including mutating ones. Covers `start_execution`, `start_sync_execution`, `stop_execution`, `create_state_machine`, `update_state_machine`, `delete_state_machine`, `send_task_success`/`send_task_failure` and tagging… | `operation` (string, required), `region` (string), `max_items` (integer), `parameters` (object) |
## Troubleshooting
* **Access expired or revoked**: If your temporary session token expires or the IAM access key is revoked, the MCP shows that it needs to reconnect. Go to the MCP's page in Nekt and update the credentials to sign in again.
* **Missing permissions**: The agent can only perform operations allowed by the IAM policy attached to your access key. If a tool call fails, verify the IAM identity has the required permissions for that AWS service.
## Need help?
Contact our support team if you have trouble connecting AWS.
# Conta Azul as an MCP
Source: https://docs.nekt.com/mcps/contaazul
Let your AI agents use Conta Azul in real time through the Nekt MCP Gateway.
Conta Azul is a Brazilian ERP and cloud accounting platform. With this MCP, your agents can read and write financial records, manage sales, and organize customers or suppliers directly in your Conta Azul account.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Conta Azul as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → MCPs**, click **New MCP** and choose **Conta Azul**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. Click the sign-in button and authorize with an account that can see the data you want the agent to use. Actions run with that account's permissions.
* **Sign in with Conta Azul**: Connect your Conta Azul account via OAuth.
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "List all our approved sales from last month."
* "Create a new customer profile for Acme Corp with the CNPJ 12.345.678/0001-90."
* "What are our total accounts payable for next week, and how many of them are already paid?"
* "Register a payment settlement for the pending installment of 500 BRL using our primary financial account."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
**What changed in v1.12.** Updated: `atualizar_venda`, `consultar_protocolo`, `criar_conta_a_pagar`, `criar_conta_a_receber`, `criar_venda`, `listar_parcelas_do_evento`.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `consultar_parcela` | Retrieve full details of a specific installment by ID in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/docs/financial-apis-openapi/v1/getinstallmentbyid](https://developers.contaazul.com/docs/financial-apis-openapi/v1/getinstallmentbyid) Args: creds: Injected authentication credentials. parcela\_id: UUID of the installment (from \`listar\_parc… | `parcela_id` (string, required) |
| `consultar_protocolo` | Retrieve the status and result of an asynchronous creation protocol in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/docs/protocol-apis-openapi/v1/buscarprotocoloporid](https://developers.contaazul.com/docs/protocol-apis-openapi/v1/buscarprotocoloporid) Use this tool after calling `criar_conta_a_pagar` or `criar_conta_a_receber` to verify wh… | `protocolo_id` (string, required) |
| `consultar_proximo_numero_venda` | Get the next sequential sale number available in Conta Azul ERP. Useful before creating a new sale to ensure a valid next sequence number is used. Args: creds: Injected authentication credentials. Returns: Next available sale number (integer or dict response from API). | - |
| `consultar_venda` | Retrieve full details of a specific sale in Conta Azul by ID. Args: creds: Injected authentication credentials. venda\_id: UUID or legacy ID of the sale to retrieve. Returns: Dict containing full sale details. Top-level keys, as measured against the live API by the production tap… | `venda_id` (string, required) |
| `listar_baixas_da_parcela` | List all settlements (payments/receipts) registered for a specific installment in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/listarbaixas](https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/listarbaixas) Args: creds: Injected authentication credentials. parcela\_id: UUID of the installme… | `parcela_id` (string, required) |
| `listar_categorias` | List financial categories from Conta Azul. Args: creds: Injected authentication credentials. pagina: Page number for pagination (default: 1). tamanho\_pagina: Number of items per page (default: 50). Returns: Dict with keys: `itens_totais` (int): Total count of categories; \`it… | `pagina` (integer), `tamanho_pagina` (integer) |
| `listar_contas_a_pagar` | List accounts payable (expenses) from Conta Azul. Args: creds: Injected authentication credentials. data\_vencimento\_de: Required start due date filter in YYYY-MM-DD format. data\_vencimento\_ate: Required end due date filter in YYYY-MM-DD format. pagina: Page number for pagination… | `data_vencimento_de` (string, required), `data_vencimento_ate` (string, required), `pagina` (integer), `status` (array), `descricao` (string), `ids_categorias` (array), `tamanho_pagina` (integer), `data_pagamento_de` (string), `data_pagamento_ate` (string), `ids_contas_financeiras` (array) |
| `listar_contas_a_receber` | List accounts receivable (incomes) from Conta Azul. Args: creds: Injected authentication credentials. data\_vencimento\_de: Required start due date filter in YYYY-MM-DD format. data\_vencimento\_ate: Required end due date filter in YYYY-MM-DD format. pagina: Page number for paginati… | `data_vencimento_de` (string, required), `data_vencimento_ate` (string, required), `pagina` (integer), `status` (array), `descricao` (string), `ids_clientes` (array), `ids_categorias` (array), `tamanho_pagina` (integer), `data_pagamento_de` (string), `data_pagamento_ate` (string), `ids_contas_financeiras` (array) |
| `listar_contas_financeiras` | List financial accounts (bank accounts, cash drawers) from Conta Azul. Args: creds: Injected authentication credentials. pagina: Page number for pagination (default: 1). tamanho\_pagina: Number of items per page (default: 50). Returns: Dict with keys: `itens_totais` (int): Tota… | `pagina` (integer), `tamanho_pagina` (integer) |
| `listar_parcelas_do_evento` | List installments linked to a specific financial event in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/docs/financial-apis-openapi/v1/getinstallmentsbyeventid](https://developers.contaazul.com/docs/financial-apis-openapi/v1/getinstallmentsbyeventid) Args: creds: Injected authentication credentials. evento\_id: UUID or legacy ID of the financial e… | `evento_id` (string, required) |
| `listar_pessoas` | List persons/contacts (customers, suppliers) from Conta Azul. Args: creds: Injected authentication credentials. pagina: Page number for pagination (default: 1). tamanho\_pagina: Number of items per page (must be one of: 10, 20, 50, 100, 200, 500, 1000; default: 50). Returns: Dict… | `pagina` (integer), `tamanho_pagina` (integer) |
| `listar_vendas` | List sales from Conta Azul with filtering options. Args: creds: Injected authentication credentials. pagina: Page number for pagination (default: 1). tamanho\_pagina: Number of items per page (default: 20). data\_inicio: Start date of sale issuance in YYYY-MM-DD format. data\_fim:… | `tipos` (array), `pagina` (integer), `data_fim` (string), `situacoes` (array), `data_inicio` (string), `termo_busca` (string), `ids_clientes` (array), `ids_vendedores` (array), `tamanho_pagina` (integer), `data_criacao_de` (string), `data_criacao_ate` (string), `data_alteracao_de` (string), `data_alteracao_ate` (string) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `atualizar_baixa` | Update details of an existing settlement (baixa) by ID in Conta Azul (partial PATCH update). OpenAPI documentation: [https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/atualizarbaixa](https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/atualizarbaixa) Useful for partial settlement adjustments or correcting payment details (date, amo… | `baixa_id` (string, required), `nsu` (string), `taxa` (number), `juros` (number), `multa` (number), `versao` (integer), `desconto` (number), `observacao` (string), `valor_bruto` (number), `data_pagamento` (string), `conta_financeira` (string), `metodo_pagamento` (string) |
| `atualizar_parcela` | Update specific fields of an installment in Conta Azul (partial PATCH update). OpenAPI documentation: [https://developers.contaazul.com/docs/financial-apis-openapi/v1/updateinstallment](https://developers.contaazul.com/docs/financial-apis-openapi/v1/updateinstallment) Note on field names: The PATCH endpoint uses `vencimento`, `composicao_valor` and \`id\_conta\_f… | `parcela_id` (string, required), `nsu` (string), `nota` (string), `taxa` (number), `juros` (number), `multa` (number), `versao` (integer), `desconto` (number), `descricao` (string), `perda_data` (string), `vencimento` (string), `perda_valor` (number), `valor_bruto` (number), `valor_liquido` (number), `metodo_pagamento` (string), `pagamento_agendado` (boolean), `id_conta_financeira` (string), `data_pagamento_esperado` (string) |
| `atualizar_venda` | Update an existing sale in Conta Azul by ID. Note: Sale update is synchronous (HTTP 200 OK) and requires `versao` for concurrency control. Args: creds: Injected authentication credentials. venda\_id: Required UUID of the sale to update. id\_cliente: Required UUID of the customer.… | `itens` (array, required), `numero` (integer, required), `versao` (integer, required), `situacao` (string, required), `venda_id` (string, required), `data_venda` (string, required), `id_cliente` (string, required), `condicao_pagamento` (any, required), `id_vendedor` (string), `observacoes` (string), `composicao_de_valor` (any), `id_natureza_operacao` (string), `observacoes_pagamento` (string) |
| `baixar_parcela` | Register settlement (payment or receipt) for an installment in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/criarbaixa](https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/criarbaixa) Note: Registering settlement is synchronous (HTTP 200 OK) and immediately returns the created settlement… | `valor` (number, required), `parcela_id` (string, required), `data_pagamento` (string, required), `conta_financeira` (string, required), `observacao` (string), `metodo_pagamento` (string) |
| `criar_conta_a_pagar` | Create an account payable (expense) in Conta Azul, paid in one or more installments. OpenAPI documentation: [https://developers.contaazul.com/docs/financial-apis-openapi/v1/createpayablefinancialevent](https://developers.contaazul.com/docs/financial-apis-openapi/v1/createpayablefinancialevent) Installments: use exactly ONE mode: 1. Single payment (default): pass \`data\_ve… | `valor` (number, required), `contato` (string, required), `descricao` (string, required), `id_categoria` (string, required), `conta_financeira` (string, required), `data_competencia` (string, required), `parcelas` (array), `observacao` (string), `data_vencimento` (string), `intervalo_meses` (integer), `numero_parcelas` (integer), `metodo_pagamento` (string) |
| `criar_conta_a_receber` | Create an account receivable (income) in Conta Azul, received in one or more installments. OpenAPI documentation: [https://developers.contaazul.com/docs/financial-apis-openapi/v1/createreceivablefinancialevent](https://developers.contaazul.com/docs/financial-apis-openapi/v1/createreceivablefinancialevent) Installments: use exactly ONE mode: 1. Single payment (default): pass… | `valor` (number, required), `contato` (string, required), `descricao` (string, required), `id_categoria` (string, required), `conta_financeira` (string, required), `data_competencia` (string, required), `parcelas` (array), `observacao` (string), `data_vencimento` (string), `intervalo_meses` (integer), `numero_parcelas` (integer), `metodo_pagamento` (string) |
| `criar_pessoa` | Create a new person/contact (customer, supplier, carrier) in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/open-api-docs/open-api-person/v1/criarpessoa](https://developers.contaazul.com/open-api-docs/open-api-person/v1/criarpessoa) Note: Creation is synchronous (HTTP 201 Created) and returns the created person record. Args: creds: Inje… | `nome` (string, required), `tipo_pessoa` (string, required), `rg` (string), `cpf` (string), `cnpj` (string), `ativo` (boolean), `email` (string), `codigo` (string), `perfis` (array), `enderecos` (array), `inscricoes` (array), `observacao` (string), `nome_fantasia` (string), `agencia_publica` (boolean), `data_nascimento` (string), `optante_simples` (boolean), `outros_contatos` (array), `telefone_celular` (string), `telefone_comercial` (string), `contato_cobranca_faturamento` (object) |
| `criar_venda` | Create a new sale in Conta Azul. Note: Sale creation is synchronous (HTTP 200 OK) and immediately returns the created sale object. Args: creds: Injected authentication credentials. id\_cliente: Required UUID of the customer (from listar\_pessoas). numero: Required sequential sale… | `itens` (array, required), `numero` (integer, required), `situacao` (string, required), `data_venda` (string, required), `id_cliente` (string, required), `condicao_pagamento` (any, required), `id_vendedor` (string), `observacoes` (string), `id_categoria` (string), `id_centro_custo` (string), `composicao_de_valor` (any), `observacoes_pagamento` (string) |
| `excluir_baixa` | Delete a settlement (baixa), reversing the payment/receipt and reopening the installment in Conta Azul. OpenAPI documentation: [https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/deletarbaixa](https://developers.contaazul.com/docs/acquittance-apis-openapi/v1/deletarbaixa) Important: Deleting a settlement directly impacts the financial balance a… | `baixa_id` (string, required) |
| `excluir_venda` | Delete one or more sales in Conta Azul (batch or single deletion). Note: Batch deletion is synchronous (HTTP 200/204) and deletes the specified sales. Args: creds: Injected authentication credentials. venda\_id: Optional UUID or identifier of a single sale to delete. ids\_vendas:… | `venda_id` (string), `ids_vendas` (array) |
## Troubleshooting
If the agent loses access to Conta Azul (usually because the authorization expired or was revoked), the MCP will show that it needs to reconnect. You can resolve this by going to the MCP's page and signing in again.
## Need help?
Contact our support team if you have trouble connecting Conta Azul.
# Facebook Ads as an MCP
Source: https://docs.nekt.com/mcps/facebook-ads
Let your AI agents use Facebook Ads in real time through the Nekt MCP Gateway.
Meta (Facebook and Instagram) Ads allows you to run and manage paid marketing campaigns. By connecting Facebook Ads as an MCP, your agent can analyze ad performance, retrieve collected leads, and create or update campaigns, ad sets, and ads directly.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Facebook Ads as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **Facebook Ads**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. Click the sign-in button and authorize with an account that can see the data you want the agent to use. Actions run with that account's permissions.
* **Sign in with Meta**: Sign in with a Meta account that has access to the ad accounts (and pages, for Lead Ads) you want to read from and manage. (Required)
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "What was the total spend and number of impressions for our active campaigns over the last 30 days?"
* "Fetch the latest leads submitted to our spring promotion lead form."
* "Create a new paused traffic campaign with a daily budget of \$100."
* "Pause any active ads in the 'Retargeting' ad set."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
**What changed in v1.10.** Added: `create_ad_copy`, `create_ad_set_copy`, `create_campaign_copy`, `create_lead_form`, `create_lead_form_copy`, `update_ad_creative`. Updated: `create_ad_set`, `create_image_ad`, `create_link_ad`, `create_video_ad`, `delete_ads_object`, `update_ad_set`, `upload_image`, `upload_video`.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `get_ad_performance` | Get performance metrics from a Meta ad account broken down at ad or adset level for a date range (YYYY-MM-DD; defaults to the last 30 days). level: ad or adset. | `account_id` (string, required), `level` (string), `end_date` (string), `start_date` (string) |
| `get_campaign_performance` | Get per-campaign performance metrics (impressions, clicks, spend, reach, actions) from a Meta ad account for a date range (YYYY-MM-DD; defaults to the last 30 days). | `account_id` (string, required), `end_date` (string), `start_date` (string) |
| `get_leads` | Get leads submitted to a Meta Lead Ads form (field data, created time, ad/campaign ids); since filters by creation date (YYYY-MM-DD). | `form_id` (string, required), `limit` (integer), `since` (string) |
| `list_ad_accounts` | List the Meta ad accounts accessible to the authenticated user, with name, id, currency and status. | - |
| `list_ad_sets` | List ad sets in a Meta ad account (optionally filtered by campaign id), with budget, optimization goal, targeting summary and schedule. | `account_id` (string, required), `campaign_id` (string) |
| `list_ads` | List ads in a Meta ad account (optionally filtered by ad set or campaign), with status and creative summary. | `account_id` (string, required), `ad_set_id` (string), `campaign_id` (string) |
| `list_campaigns` | List campaigns in a Meta ad account with status, objective, budget and schedule. | `account_id` (string, required), `include_archived` (boolean) |
| `list_lead_forms` | List Lead Ads forms of a Facebook page, with id, name, status and leads count. | `page_id` (string, required) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `create_ad_copy` | Duplicate an ad via Meta /copies endpoint. Parameters: ad\_id: Source ad id; ad\_set\_id: Target ad set id (optional, defaults to current ad set); rename\_options: Dict e.g. \{'rename\_strategy': 'ONLY\_TOP\_LEVEL\_RENAME', 'rename\_suffix': ' - Copy'}; status\_option: 'ACTIVE', 'P… | `ad_id` (string, required), `ad_set_id` (string), `status_option` (string), `rename_options` (object), `creative_parameters` (object) |
| `create_ad_set` | Create an ad set in a Meta campaign (created PAUSED by default). Parameters: daily\_budget: Budget in account currency (omit if campaign uses CBO); bid\_amount: Bid in account currency (required for LOWEST\_COST\_WITH\_BID\_CAP or TARGET\_COST); bid\_strategy: Strategy e.g. LOWEST… | `name` (string, required), `account_id` (string, required), `campaign_id` (string, required), `status` (string), `age_max` (integer), `age_min` (integer), `page_id` (string), `countries` (array), `bid_amount` (number), `bid_strategy` (string), `daily_budget` (number), `billing_event` (string), `destination_type` (string), `device_platforms` (array), `optimization_goal` (string), `threads_positions` (array), `advantage_audience` (boolean), `facebook_positions` (array), `instagram_positions` (array), `messenger_positions` (array), `publisher_platforms` (array), `audience_network_positions` (array) |
| `create_ad_set_copy` | Duplicate an ad set via Meta /copies endpoint. Parameters: ad\_set\_id: Source ad set id; campaign\_id: Target campaign id (optional, defaults to same campaign); deep\_copy: If True, duplicates child ads (max 3 sync, up to 51 async); rename\_options: Dict e.g. \{'rename\_strate… | `ad_set_id` (string, required), `end_time` (string\|integer), `deep_copy` (boolean), `start_time` (string\|integer), `campaign_id` (string), `status_option` (string), `rename_options` (object) |
| `create_campaign` | Create a campaign in a Meta ad account (created PAUSED by default). objective: OUTCOME\_TRAFFIC, OUTCOME\_LEADS, OUTCOME\_SALES, OUTCOME\_ENGAGEMENT, OUTCOME\_AWARENESS or OUTCOME\_APP\_PROMOTION; daily\_budget in the account currency (only for Campaign Budget Optimization; omit to set… | `name` (string, required), `account_id` (string, required), `status` (string), `objective` (string), `daily_budget` (number), `special_ad_categories` (array) |
| `create_campaign_copy` | Duplicate a campaign via Meta /copies endpoint. Parameters: campaign\_id: Source campaign id; deep\_copy: If True, duplicates all child ad sets and ads (max 51 total ads); rename\_options: Dict e.g. \{'rename\_strategy': 'ONLY\_TOP\_LEVEL\_RENAME', 'rename\_suffix': ' - Copy'}; s… | `campaign_id` (string, required), `end_time` (string\|integer), `deep_copy` (boolean), `start_time` (string\|integer), `status_option` (string), `rename_options` (object), `parameter_overrides` (object) |
| `create_image_ad` | Create a single-image ad in a Meta ad set, PAUSED by default. `image_hash` comes from upload\_image. | `name` (string, required), `message` (string, required), `page_id` (string, required), `ad_set_id` (string, required), `account_id` (string, required), `image_hash` (string, required), `link` (string), `status` (string), `headline` (string), `description` (string), `call_to_action` (string), `lead_gen_form_id` (string), `enable_multi_advertiser_ads` (boolean), `enable_advantage_plus_creative` (boolean), `advantage_plus_creative_features` (object) |
| `create_lead_form` | Create a Lead Ads form for a Facebook Page. Parameters: page\_id: Facebook Page id (must have admin/manage rights); name: Form name; questions: List of questions (e.g. \[\{'type': 'FULL\_NAME', 'key': 'full\_name'}, \{'type': 'EMAIL', 'key': 'email'}]); privacy\_policy: Dict wi… | `name` (string, required), `page_id` (string, required), `questions` (array, required), `locale` (string), `context_card` (object), `cover_photo_id` (string), `privacy_policy` (object), `thank_you_page` (object), `tracking_parameters` (object\|array), `follow_up_action_url` (string), `is_optimized_for_quality` (boolean), `question_page_custom_headline` (string), `block_display_for_non_targeted_viewer` (boolean) |
| `create_lead_form_copy` | Duplicate an existing Lead Ads form with optional field overrides. Lead forms in Meta cannot be updated after creation; this tool fetches the original form, applies the requested changes, and creates a new form. | `form_id` (string, required), `page_id` (string, required), `name` (string), `locale` (string), `questions` (array), `context_card` (object), `privacy_policy` (object), `thank_you_page` (object), `follow_up_action_url` (string) |
| `create_link_ad` | Create a link ad (creative + ad) in a Meta ad set, created PAUSED by default. Parameters: link: Destination URL; message: Primary ad copy; headline: Optional ad title; description: Optional ad subtitle/description; call\_to\_action: Action button (e.g. LEARN\_MORE, SHOP\_N… | `link` (string, required), `name` (string, required), `message` (string, required), `page_id` (string, required), `ad_set_id` (string, required), `account_id` (string, required), `status` (string), `headline` (string), `description` (string), `call_to_action` (string), `lead_gen_form_id` (string), `enable_multi_advertiser_ads` (boolean), `enable_advantage_plus_creative` (boolean), `advantage_plus_creative_features` (object) |
| `create_video_ad` | Create a video ad in a Meta ad set, PAUSED by default. `video_id` comes from upload\_video. `thumbnail_url` is required by Meta. | `name` (string, required), `message` (string, required), `page_id` (string, required), `video_id` (string, required), `ad_set_id` (string, required), `account_id` (string, required), `thumbnail_url` (string, required), `link` (string), `status` (string), `headline` (string), `call_to_action` (string), `lead_gen_form_id` (string), `enable_multi_advertiser_ads` (boolean), `enable_advantage_plus_creative` (boolean), `advantage_plus_creative_features` (object) |
| `delete_ads_object` | Permanently delete a Meta ads object by id: campaign, ad set or ad. | `object_id` (string, required) |
| `update_ad_creative` | Update or replace the creative content (image, text, headline, link, CTA) of an ad. AdCreative objects in Meta are immutable. This tool reads the existing creative, applies the changes to create a new creative, and either replaces it on the existing ad (mode='replace') or create… | `ad_id` (string, required), `account_id` (string, required), `link` (string), `mode` (string), `message` (string), `headline` (string), `image_hash` (string), `description` (string), `call_to_action` (string) |
| `update_ad_set` | Update a Meta ad set: status (ACTIVE/PAUSED), name, daily budget, bid amount and/or manual placements. | `ad_set_id` (string, required), `name` (string), `status` (string), `bid_amount` (number), `daily_budget` (number), `device_platforms` (array), `threads_positions` (array), `facebook_positions` (array), `instagram_positions` (array), `messenger_positions` (array), `publisher_platforms` (array), `audience_network_positions` (array) |
| `update_ad_status` | Change a Meta ad's status: ACTIVE to activate or PAUSED to pause it. | `ad_id` (string, required), `status` (string, required) |
| `update_campaign` | Update a Meta campaign: status (ACTIVE to activate, PAUSED to pause), name and/or daily budget in the account currency (CBO campaigns only). | `campaign_id` (string, required), `name` (string), `status` (string), `daily_budget` (number) |
| `upload_image` | Upload an image to a Meta ad account's media library from a public URL, returning the image\_hash that create\_image\_ad needs. | `image_url` (string, required), `account_id` (string, required), `filename` (string) |
| `upload_video` | Upload a video to a Meta ad account's media library from a public URL, returning the video\_id that create\_video\_ad needs. Meta fetches the URL itself, so the file has to be reachable without authentication. | `name` (string, required), `video_url` (string, required), `account_id` (string, required) |
## Troubleshooting
**Expired or revoked access**
If your Meta session expires or you revoke Nekt's access from your Meta account settings, the MCP will show that it needs to reconnect. Go to the MCP's page in Nekt and sign in again to restore access.
**Missing leads or pages**
To read lead information, the Meta account you used to authorize the connection must be an admin of the corresponding Facebook page. If the agent cannot access leads, verify your page roles in Meta.
## Need help?
Contact our support team if you have trouble connecting Facebook Ads.
# Github as an MCP
Source: https://docs.nekt.com/mcps/github
Let your AI agents use Github in real time through the Nekt MCP Gateway.
GitHub is a developer platform for code hosting, collaboration, and CI/CD. By connecting Github as an MCP, your agent can read repositories, manage issues and pull requests, trigger GitHub Actions, and interact with GitHub Copilot directly.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Github as an MCP
Before you start, make sure the [MCP Gateway](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate** and click **MCPs**. Click **New MCP** and choose **Github**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. To connect using an API key, you need a personal access token from GitHub:
1. Go to your GitHub account settings and generate a personal access token.
2. For company-wide access, create a fine-grained personal access token with the organization as the resource owner and grant it access to the necessary repositories.
Provide the following field:
* **GitHub token** (required): A GitHub personal access token authenticating every request to GitHub's hosted MCP server. For company-wide access, create a fine-grained token with the organization as the resource owner and grant it access to all repositories. If the organization requires approval, the token stays pending, with read access to public resources only, until an organization owner approves it. The token's permissions control read vs. write: a read-only token makes this a read-only connection, enforced by GitHub.
4. **Select tools**. Pick the tools your agents can use. See [MCP tools](/mcp-server/tools) to learn how they work.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Check the latest pull requests in our main repository and summarize the pending reviews."
* "List the failed GitHub Actions workflows from yesterday and fetch the job logs for the failures."
* "Create a new issue describing the login bug, then assign GitHub Copilot to investigate it."
* "Search our codebase for usages of the deprecated logging function and list the files that need updating."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `actions_get` | Get details of GitHub Actions resources (workflows, workflow runs, jobs, and artifacts) | `repo` (string, required), `owner` (string, required), `method` (string, required), `resource_id` (string, required) |
| `actions_list` | List GitHub Actions workflows in a repository | `repo` (string, required), `owner` (string, required), `method` (string, required), `page` (number), `perPage` (number), `resource_id` (string), `workflow_jobs_filter` (object), `workflow_runs_filter` (object) |
| `custom_properties_read` | Read custom properties | `level` (string, required), `org` (string), `repo` (string), `owner` (string), `enterprise` (string) |
| `get_code_quality_finding` | Get code quality finding | `repo` (string, required), `owner` (string, required), `findingNumber` (number, required) |
| `get_code_scanning_alert` | Get code scanning alert | `repo` (string, required), `owner` (string, required), `alertNumber` (number, required) |
| `get_commit` | Get commit details | `sha` (string, required), `repo` (string, required), `owner` (string, required), `page` (number), `detail` (string), `perPage` (number) |
| `get_copilot_space` | Get Copilot Space | `name` (string, required), `owner` (string, required) |
| `get_dependabot_alert` | Get dependabot alert | `repo` (string, required), `owner` (string, required), `alertNumber` (number, required) |
| `get_discussion` | Get discussion | `repo` (string, required), `owner` (string, required), `discussionNumber` (number, required) |
| `get_discussion_comments` | Get discussion comments | `repo` (string, required), `owner` (string, required), `discussionNumber` (number, required), `after` (string), `perPage` (number), `includeReplies` (boolean) |
| `get_file_contents` | Get file or directory contents | `repo` (string, required), `owner` (string, required), `ref` (string), `sha` (string), `path` (string), `fields` (array) |
| `get_gist` | Get Gist Content | `gist_id` (string, required) |
| `get_global_security_advisory` | Get a global security advisory | `ghsaId` (string, required) |
| `get_job_logs` | Get GitHub Actions workflow job logs | `repo` (string, required), `owner` (string, required), `job_id` (number), `run_id` (number), `tail_lines` (number), `failed_only` (boolean), `return_content` (boolean) |
| `get_label` | Get a specific label from a repository | `name` (string, required), `repo` (string, required), `owner` (string, required) |
| `get_latest_release` | Get latest release | `repo` (string, required), `owner` (string, required) |
| `get_me` | Get my user profile | - |
| `get_notification_details` | Get notification details | `notificationID` (string, required) |
| `get_release_by_tag` | Get a release by tag name | `tag` (string, required), `repo` (string, required), `owner` (string, required) |
| `get_repository_tree` | Get repository tree | `repo` (string, required), `owner` (string, required), `tree_sha` (string), `recursive` (boolean), `path_filter` (string) |
| `get_secret_scanning_alert` | Get secret scanning alert | `repo` (string, required), `owner` (string, required), `alertNumber` (number, required) |
| `get_tag` | Get tag details | `tag` (string, required), `repo` (string, required), `owner` (string, required) |
| `get_team_members` | Get team members | `org` (string, required), `team_slug` (string, required) |
| `get_teams` | Get teams | `user` (string) |
| `github_support_docs_search` | Retrieve documentation relevant to answer GitHub product and support questions | `query` (string, required) |
| `issue_read` | Get issue details | `repo` (string, required), `owner` (string, required), `method` (string, required), `issue_number` (number, required), `page` (number), `perPage` (number) |
| `list_branches` | List branches | `repo` (string, required), `owner` (string, required), `page` (number), `perPage` (number) |
| `list_code_scanning_alerts` | List code scanning alerts | `repo` (string, required), `owner` (string, required), `ref` (string), `page` (number), `state` (string), `perPage` (number), `severity` (string), `tool_name` (string) |
| `list_commits` | List commits | `repo` (string, required), `owner` (string, required), `sha` (string), `page` (number), `path` (string), `since` (string), `until` (string), `author` (string), `fields` (array), `perPage` (number) |
| `list_copilot_spaces` | List Copilot Spaces | - |
| `list_dependabot_alerts` | List dependabot alerts | `repo` (string, required), `owner` (string, required), `after` (string), `state` (string), `perPage` (number), `severity` (string) |
| `list_discussion_categories` | List discussion categories | `owner` (string, required), `repo` (string) |
| `list_discussions` | List discussions | `owner` (string, required), `repo` (string), `after` (string), `orderBy` (string), `perPage` (number), `category` (string), `direction` (string) |
| `list_gists` | List Gists | `page` (number), `since` (string), `perPage` (number), `username` (string) |
| `list_global_security_advisories` | List global security advisories | `cwes` (array), `type` (string), `cveId` (string), `ghsaId` (string), `affects` (string), `updated` (string), `modified` (string), `severity` (string), `ecosystem` (string), `published` (string), `isWithdrawn` (boolean) |
| `list_issue_fields` | List issue fields | `owner` (string, required), `repo` (string) |
| `list_issue_types` | List available issue types | `owner` (string, required), `repo` (string) |
| `list_issues` | List issues | `repo` (string, required), `owner` (string, required), `after` (string), `since` (string), `state` (string), `fields` (array), `labels` (array), `orderBy` (string), `perPage` (number), `direction` (string), `field_filters` (array) |
| `list_label` | List labels from a repository | `repo` (string, required), `owner` (string, required) |
| `list_notifications` | List notifications | `page` (number), `repo` (string), `owner` (string), `since` (string), `before` (string), `filter` (string), `perPage` (number) |
| `list_org_repository_security_advisories` | List org repository security advisories | `org` (string, required), `sort` (string), `state` (string), `direction` (string) |
| `list_pull_requests` | List pull requests | `repo` (string, required), `owner` (string, required), `base` (string), `head` (string), `page` (number), `sort` (string), `state` (string), `fields` (array), `perPage` (number), `direction` (string) |
| `list_releases` | List releases | `repo` (string, required), `owner` (string, required), `page` (number), `fields` (array), `perPage` (number) |
| `list_repository_collaborators` | List repository collaborators | `repo` (string, required), `owner` (string, required), `page` (number), `perPage` (number), `affiliation` (string) |
| `list_repository_security_advisories` | List repository security advisories | `repo` (string, required), `owner` (string, required), `sort` (string), `state` (string), `direction` (string) |
| `list_secret_scanning_alerts` | List secret scanning alerts | `repo` (string, required), `owner` (string, required), `page` (number), `state` (string), `perPage` (number), `resolution` (string), `secret_type` (string) |
| `list_starred_repositories` | List starred repositories | `page` (number), `sort` (string), `perPage` (number), `username` (string), `direction` (string) |
| `list_tags` | List tags | `repo` (string, required), `owner` (string, required), `page` (number), `perPage` (number) |
| `projects_get` | Get details of GitHub Projects resources | `method` (string, required), `owner` (string), `fields` (array), `item_id` (number), `view_id` (string), `field_id` (number), `owner_type` (string), `field_names` (array), `project_number` (number), `status_update_id` (string) |
| `projects_list` | List GitHub Projects resources | `owner` (string, required), `method` (string, required), `after` (string), `query` (string), `before` (string), `fields` (array), `perPage` (number), `owner_type` (string), `field_names` (array), `project_number` (number) |
| `pull_request_read` | Get details for a single pull request | `repo` (string, required), `owner` (string, required), `method` (string, required), `pullNumber` (number, required), `page` (number), `after` (string), `perPage` (number) |
| `repository_ruleset_read` | Read repository rulesets | `level` (string, required), `method` (string, required), `org` (string), `ref` (string), `page` (number), `repo` (string), `owner` (string), `branch` (string), `perPage` (number), `actor_name` (string), `enterprise` (string), `ruleset_id` (number), `time_period` (string), `rule_suite_id` (number), `evaluate_status` (string), `repository_name` (string), `includes_parents` (boolean), `rule_suite_result` (string) |
| `search_code` | Search code | `query` (string, required), `page` (number), `sort` (string), `order` (string), `fields` (array), `perPage` (number) |
| `search_commits` | Search commits | `query` (string, required), `page` (number), `sort` (string), `order` (string), `perPage` (number) |
| `search_issues` | Search issues | `query` (string, required), `page` (number), `repo` (string), `sort` (string), `order` (string), `owner` (string), `fields` (array), `perPage` (number) |
| `search_orgs` | Search organizations | `query` (string, required), `page` (number), `sort` (string), `order` (string), `perPage` (number) |
| `search_pull_requests` | Search pull requests | `query` (string, required), `page` (number), `repo` (string), `sort` (string), `order` (string), `owner` (string), `fields` (array), `perPage` (number) |
| `search_repositories` | Search repositories | `query` (string, required), `page` (number), `sort` (string), `order` (string), `perPage` (number), `minimal_output` (boolean) |
| `search_users` | Search users | `query` (string, required), `page` (number), `sort` (string), `order` (string), `perPage` (number) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `actions_run_trigger` | Trigger GitHub Actions workflow actions | `repo` (string, required), `owner` (string, required), `method` (string, required), `ref` (string), `inputs` (object), `run_id` (number), `workflow_id` (string) |
| `add_comment_to_pending_review` | Add review comment to the requester's latest pending pull request review | `body` (string, required), `path` (string, required), `repo` (string, required), `owner` (string, required), `pullNumber` (number, required), `subjectType` (string, required), `line` (number), `side` (string), `startLine` (number), `startSide` (string) |
| `add_issue_comment` | Add comment to issue or pull request | `repo` (string, required), `owner` (string, required), `issue_number` (number, required), `body` (string), `reaction` (string), `comment_id` (integer) |
| `add_reply_to_pull_request_comment` | Add reply to pull request comment | `repo` (string, required), `owner` (string, required), `commentId` (number, required), `body` (string), `reaction` (string), `pullNumber` (number) |
| `assign_copilot_to_issue` | Assign Copilot to issue | `repo` (string, required), `owner` (string, required), `issue_number` (number, required), `base_ref` (string), `custom_instructions` (string) |
| `assign_copilot_to_issue_with_intent` | Assign Copilot to issue with intent | `repo` (string, required), `owner` (string, required), `rationale` (string, required), `confidence` (string, required), `issue_number` (number, required), `is_suggestion` (boolean, required), `base_ref` (string), `custom_instructions` (string) |
| `create_branch` | Create branch | `repo` (string, required), `owner` (string, required), `branch` (string, required), `from_branch` (string) |
| `create_gist` | Create Gist | `content` (string, required), `filename` (string, required), `public` (boolean), `description` (string) |
| `create_or_update_file` | Create or update file | `path` (string, required), `repo` (string, required), `owner` (string, required), `branch` (string, required), `content` (string, required), `message` (string, required), `sha` (string), `allow_symlink_write` (boolean) |
| `create_pull_request` | Open new pull request | `base` (string, required), `head` (string, required), `repo` (string, required), `owner` (string, required), `title` (string, required), `body` (string), `draft` (boolean), `reviewers` (array), `maintainer_can_modify` (boolean) |
| `create_pull_request_with_copilot` | Perform task with GitHub Copilot coding agent | `repo` (string, required), `owner` (string, required), `title` (string, required), `problem_statement` (string, required), `base_ref` (string) |
| `create_repository` | Create repository | `name` (string, required), `private` (boolean), `autoInit` (boolean), `description` (string), `organization` (string) |
| `create_repository_ruleset` | Create repository ruleset | `name` (string, required), `level` (string, required), `rules` (array, required), `enforcement` (string, required), `org` (string), `repo` (string), `owner` (string), `target` (string), `conditions` (object), `enterprise` (string), `bypass_actors` (array) |
| `custom_properties_write` | Set custom properties | `level` (string, required), `properties` (array, required), `org` (string), `repo` (string), `owner` (string), `enterprise` (string) |
| `delete_file` | Delete file | `path` (string, required), `repo` (string, required), `owner` (string, required), `branch` (string, required), `message` (string, required) |
| `discussion_comment_write` | Manage discussion comments | `method` (string, required), `body` (string), `repo` (string), `owner` (string), `commentNodeID` (string), `discussionNumber` (number) |
| `dismiss_notification` | Dismiss notification | `state` (string, required), `threadID` (string, required) |
| `fork_repository` | Fork repository | `repo` (string, required), `owner` (string, required), `organization` (string) |
| `issue_write` | Create or update issue/pull request | `repo` (string, required), `owner` (string, required), `method` (string, required), `body` (string), `type` (string), `state` (string), `title` (string), `labels` (array), `assignees` (array), `milestone` (number), `parent_repo` (string), `duplicate_of` (number), `issue_fields` (array), `issue_number` (number), `parent_owner` (string), `state_reason` (string), `parent_issue_number` (number) |
| `label_write` | Write operations on repository labels | `name` (string, required), `repo` (string, required), `owner` (string, required), `method` (string, required), `color` (string), `new_name` (string), `description` (string) |
| `manage_notification_subscription` | Manage notification subscription | `action` (string, required), `notificationID` (string, required) |
| `manage_repository_notification_subscription` | Manage repository notification subscription | `repo` (string, required), `owner` (string, required), `action` (string, required) |
| `mark_all_notifications_read` | Mark all notifications as read | `repo` (string), `owner` (string), `lastReadAt` (string) |
| `merge_pull_request` | Merge pull request | `repo` (string, required), `owner` (string, required), `pullNumber` (number, required), `commit_title` (string), `merge_method` (string), `commit_message` (string), `expectedHeadSha` (string) |
| `projects_write` | Manage GitHub Projects | `owner` (string, required), `method` (string, required), `body` (string), `name` (string), `items` (array), `title` (string), `filter` (string), `layout` (string), `status` (string), `item_id` (number), `view_id` (string), `item_repo` (string), `item_type` (string), `field_name` (string), `item_owner` (string), `iterations` (array), `owner_type` (string), `start_date` (string), `target_date` (string), `issue_number` (number), `updated_field` (object), `project_number` (number), `visible_fields` (array), `iteration_duration` (number), `pull_request_number` (number), `visible_field_names` (array) |
| `pull_request_review_write` | Write operations (create, submit, delete) on pull request reviews | `repo` (string, required), `owner` (string, required), `method` (string, required), `pullNumber` (number, required), `body` (string), `event` (string), `commitID` (string), `threadId` (string) |
| `push_files` | Push files to repository | `repo` (string, required), `files` (array, required), `owner` (string, required), `branch` (string, required), `message` (string, required) |
| `request_copilot_review` | Request Copilot review | `repo` (string, required), `owner` (string, required), `pullNumber` (number, required) |
| `star_repository` | Star repository | `repo` (string, required), `owner` (string, required) |
| `sub_issue_write` | Change sub-issue | `repo` (string, required), `owner` (string, required), `method` (string, required), `issue_number` (number, required), `sub_issue_id` (number, required), `after_id` (number), `before_id` (number), `replace_parent` (boolean) |
| `unstar_repository` | Unstar repository | `repo` (string, required), `owner` (string, required) |
| `update_gist` | Update Gist | `content` (string, required), `gist_id` (string, required), `filename` (string, required), `description` (string) |
| `update_issue_comment` | Update issue comment | `body` (string, required), `repo` (string, required), `owner` (string, required), `comment_id` (integer, required) |
| `update_pull_request` | Edit pull request | `repo` (string, required), `owner` (string, required), `pullNumber` (number, required), `base` (string), `body` (string), `draft` (boolean), `state` (string), `title` (string), `reviewers` (array), `maintainer_can_modify` (boolean) |
| `update_pull_request_branch` | Update pull request branch | `repo` (string, required), `owner` (string, required), `pullNumber` (number, required), `expectedHeadSha` (string) |
## Troubleshooting
* **Expired or revoked access**: If your personal access token expires or is revoked, the MCP will show that it needs to reconnect. Go to the MCP's page in Nekt and update the token to restore access.
* **Missing permissions**: If the agent fails to perform certain actions (especially write operations like creating branches or modifying issues), ensure your token has the correct repository permissions granted. A read-only token will restrict the agent to read-only tools.
* **Pending organization approval**: If you created a fine-grained token for an organization that requires approval, the token will only have read access to public resources until an administrator approves it in GitHub.
# Google Ads as an MCP
Source: https://docs.nekt.com/mcps/google-ads
Let your AI agents use Google Ads in real time through the Nekt MCP Gateway.
Google Ads is Google's online advertising platform. By connecting it as an MCP, your AI agents can read campaign performance, manage budgets, create or pause ads, and adjust negative keywords in real time.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Google Ads as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **Google Ads**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. Click the sign-in button and authorize with a Google account that can see the data you want the agent to use. Actions run with that account's permissions.
* **Sign in with Google** (Required): Sign in with a Google account that has access to the Google Ads accounts you want to read from and manage.
* **Manager Account ID**: Fill this field if you authenticated using a manager (MCC) account. This value is sent in the login-customer-id header on every API call.
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "What was the total spend and conversion count for the 'Spring Sale' campaign over the last 30 days?"
* "Increase the daily budget of the 'Retargeting' campaign by 20 percent."
* "Add 'free' and 'cheap' as negative keywords to the 'Enterprise Software' campaign."
* "Pause all ad groups in the 'Summer Promo' campaign."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `find_geo_targets` | Look up Google Ads location ids by name, for set\_campaign\_geo. Searches place names ("Brasil", "São Paulo", "Rio Grande do Sul") and returns each match with its id, canonical name and type (Country, State, City...). Without this there is no way to discover an id short of hardcod… | `query` (string, required), `locale` (string), `country_code` (string) |
| `get_campaign_performance` | Get per-campaign performance metrics (impressions, clicks, cost, conversions) from Google Ads for a date range (YYYY-MM-DD; defaults to the last 30 days). | `customer_id` (string, required), `end_date` (string), `start_date` (string) |
| `list_accessible_customers` | List the Google Ads customer account ids directly accessible to the authenticated user. | - |
| `list_account_hierarchy` | List all Google Ads client accounts under a customer (useful for manager/MCC accounts). | `customer_id` (string, required) |
| `list_ad_groups` | List ad groups in a Google Ads account, optionally filtered by campaign id. | `customer_id` (string, required), `campaign_id` (string) |
| `list_ads` | List ads in a Google Ads account (responsive search ad texts, final URLs and status), optionally filtered by ad group or campaign id. | `customer_id` (string, required), `ad_group_id` (string), `campaign_id` (string) |
| `list_campaigns` | List campaigns in a Google Ads account with status, type, budget and bidding strategy. | `customer_id` (string, required), `include_removed` (boolean) |
| `list_negative_keywords` | List the negative keywords of one Google Ads negative keyword list (shared\_set\_id) or of one campaign (campaign\_id); pass exactly one. Returns each term with its criterion\_id, which is what remove\_negative\_keywords takes to undo a negativation. | `customer_id` (string, required), `campaign_id` (string), `shared_set_id` (string) |
| `list_shared_sets` | List the shared sets (negative keyword lists) of a Google Ads account and the campaigns each one is attached to; use it to find the shared\_set\_id to pass to add\_negative\_keywords. | `customer_id` (string, required) |
| `run_gaql` | Run an arbitrary GAQL (Google Ads Query Language) query against a Google Ads account; use for any report or lookup not covered by the other tools; add LIMIT to keep results small. | `query` (string, required), `customer_id` (string, required) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `add_keywords` | Add keywords to a Google Ads ad group; match\_type: BROAD, PHRASE or EXACT. | `keywords` (array, required), `ad_group_id` (string, required), `customer_id` (string, required), `match_type` (string) |
| `add_negative_keywords` | Add negative keywords to a Google Ads account so the given terms stop triggering ads. Pass exactly one target: shared\_set\_id applies the terms to every campaign attached to that negative keyword list (use list\_shared\_sets to find it), campaign\_id blocks them in that one campaign… | `keywords` (array, required), `customer_id` (string, required), `match_type` (string), `campaign_id` (string), `shared_set_id` (string) |
| `attach_shared_set_to_campaign` | Attach a negative keyword list (shared set) to a Google Ads campaign, so every term in the list stops triggering that campaign's ads. | `campaign_id` (string, required), `customer_id` (string, required), `shared_set_id` (string, required) |
| `create_ad_group` | Create an ad group inside a Google Ads campaign; cpc\_bid is the max CPC in the account currency (optional). | `name` (string, required), `campaign_id` (string, required), `customer_id` (string, required), `status` (string), `cpc_bid` (number) |
| `create_campaign` | Create a campaign in Google Ads (created PAUSED by default for safety). channel\_type: SEARCH, DISPLAY, PERFORMANCE\_MAX...; bidding\_strategy: MAXIMIZE\_CLICKS, MAXIMIZE\_CONVERSIONS, MAXIMIZE\_CONVERSION\_VALUE, MANUAL\_CPC or TARGET\_IMPRESSION\_SHARE; dates are YYYY-MM-DD. Bidding lim… | `name` (string, required), `budget_id` (string, required), `customer_id` (string, required), `status` (string), `end_date` (string), `start_date` (string), `target_cpa` (number), `target_roas` (number), `channel_type` (string), `cpc_bid_ceiling` (number), `display_network` (boolean), `search_partners` (boolean), `bidding_strategy` (string), `impression_share_percent` (number), `impression_share_location` (string), `contains_eu_political_advertising` (boolean) |
| `create_campaign_budget` | Create a daily campaign budget in Google Ads; daily\_amount is in the account currency (e.g. 50.0 = R\$50/day). Returns the budget resource name to use in create\_campaign. | `name` (string, required), `customer_id` (string, required), `daily_amount` (number, required), `explicitly_shared` (boolean) |
| `create_responsive_search_ad` | Create a responsive search ad in a Google Ads ad group (created PAUSED by default); 3-15 headlines (max 30 chars each), 2-4 descriptions (max 90 chars each). | `headlines` (array, required), `final_urls` (array, required), `ad_group_id` (string, required), `customer_id` (string, required), `descriptions` (array, required), `path1` (string), `path2` (string), `status` (string) |
| `create_shared_set` | Create a negative keyword list (a shared set) in a Google Ads account. The list blocks nothing until it is attached to campaigns with attach\_shared\_set\_to\_campaign. | `name` (string, required), `customer_id` (string, required) |
| `remove_ads_object` | Permanently remove a Google Ads object; object\_type: campaign, ad\_group, ad, keyword or budget (campaign\_budget is accepted as an alias for budget; ad and keyword also require ad\_group\_id). Removal cannot be undone: the object is kept in historical reports but can never be reac… | `object_id` (string, required), `customer_id` (string, required), `object_type` (string, required), `ad_group_id` (string) |
| `remove_negative_keywords` | Remove negative keywords, so the terms can trigger ads again. Takes criterion ids (get them from list\_negative\_keywords, not the term text) and exactly one target: shared\_set\_id to take them off a negative keyword list, campaign\_id to take them off a single campaign. | `customer_id` (string, required), `criterion_ids` (array, required), `campaign_id` (string), `shared_set_id` (string) |
| `set_campaign_ad_schedule` | Set the days and hours a Google Ads campaign may serve, REPLACING the previous schedule. Each entry is \{"day": "MONDAY", "start\_hour": 9, "end\_hour": 18} and may carry "start\_minute"/"end\_minute", which must be 0, 15, 30 or 45. Hours are 0-24 in the account's timezone; end\_hour… | `schedule` (array, required), `campaign_id` (string, required), `customer_id` (string, required) |
| `set_campaign_device_bid_modifier` | Raise or lower a Google Ads campaign's bids on one device type. device: MOBILE, DESKTOP, TABLET, CONNECTED\_TV or OTHER. modifier is a multiplier: 1.0 is no change, 1.2 bids 20% more, 0.8 bids 20% less, and 0 stops serving on that device entirely. Only the given device is touche… | `device` (string, required), `modifier` (number, required), `campaign_id` (string, required), `customer_id` (string, required) |
| `set_campaign_geo` | Set which locations a Google Ads campaign targets, REPLACING whatever it targeted before. location\_ids come from find\_geo\_targets (Brazil is 2076). Set negative=True to exclude the given locations instead of targeting them; note that this replaces the campaign's exclusions, not… | `campaign_id` (string, required), `customer_id` (string, required), `location_ids` (array, required), `negative` (boolean) |
| `set_campaign_languages` | Set which languages a Google Ads campaign targets, REPLACING whatever it targeted before. Language ids are Google constants: Portuguese is 1014, English 1000, Spanish 1003. This matches the user's Google interface language, not the language the ad is written in. | `campaign_id` (string, required), `customer_id` (string, required), `language_ids` (array, required) |
| `update_ad_group` | Update a Google Ads ad group: status (ENABLED/PAUSED), name and/or max CPC bid in the account currency. | `ad_group_id` (string, required), `customer_id` (string, required), `name` (string), `status` (string), `cpc_bid` (number) |
| `update_ad_status` | Change a Google Ads ad's status: ENABLED to activate or PAUSED to pause it. | `ad_id` (string, required), `status` (string, required), `ad_group_id` (string, required), `customer_id` (string, required) |
| `update_campaign` | Update a Google Ads campaign: status (ENABLED to activate, PAUSED to pause), name, start\_date or end\_date (YYYY-MM-DD). | `campaign_id` (string, required), `customer_id` (string, required), `name` (string), `status` (string), `end_date` (string), `start_date` (string) |
| `update_campaign_bidding` | Change a Google Ads campaign's bidding strategy and its limits. Same arguments as create\_campaign: cpc\_bid\_ceiling for MAXIMIZE\_CLICKS or TARGET\_IMPRESSION\_SHARE, target\_cpa for MAXIMIZE\_CONVERSIONS, target\_roas for MAXIMIZE\_CONVERSION\_VALUE, and the impression\_share pair for TA… | `campaign_id` (string, required), `customer_id` (string, required), `bidding_strategy` (string, required), `target_cpa` (number), `target_roas` (number), `cpc_bid_ceiling` (number), `impression_share_percent` (number), `impression_share_location` (string) |
| `update_campaign_budget` | Update a Google Ads campaign budget's daily amount (account currency) and/or name. | `budget_id` (string, required), `customer_id` (string, required), `name` (string), `daily_amount` (number) |
| `update_campaign_networks` | Choose where a Google Ads search campaign may serve: Google Search itself, Google's search partner sites, and the Display Network. Turning display\_network on makes the ads run on third-party sites, a different surface, not just more of the same one. Turning google\_search off wh… | `campaign_id` (string, required), `customer_id` (string, required), `google_search` (boolean), `display_network` (boolean), `search_partners` (boolean) |
## Troubleshooting
* **Expired or revoked access**: If the MCP shows that it needs to reconnect (for example, if your Google session expired or access was revoked), go to the MCP's page and sign in again.
* **Rate limits**: If your agent encounters a quota error (RESOURCE\_EXHAUSTED), the Google Ads API rate limit has been reached. Wait until the reset time indicated in the error message before trying again.
* **Missing permissions**: Ensure the Google account you used to sign in has the appropriate access levels to read or edit the campaigns you are querying.
## Need help?
Contact our support team if you have trouble connecting Google Ads.
# Grafana as an MCP
Source: https://docs.nekt.com/mcps/grafana
Let your AI agents use Grafana in real time through the Nekt MCP Gateway.
Grafana is an open-source platform for monitoring and observability. By connecting Grafana to Nekt, your AI agent can query metrics, search dashboards, analyze logs and traces, and create annotations across your infrastructure in real time.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Grafana as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **Grafana**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. To authenticate, you need a service account token from your Grafana instance.
1. In Grafana, create a new service account.
2. Grant it the Viewer role, or the Editor role if you want the agent to use write tools.
3. Generate a token for the service account.
Provide the following fields:
* **Grafana URL** (Required): Base URL of your Grafana instance, a Grafana Cloud stack (`https://.grafana.net`) or a self-hosted deployment.
* **Service account token** (Required): Token of a Grafana service account with access to the dashboards, datasources and alerts you want to use.
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Are there any active alerts firing right now?"
* "Search for the infrastructure overview dashboard and summarize its panels."
* "Run a PromQL query to show the error rate over the last 30 minutes."
* "Check the Loki logs for any authentication failures in the past hour."
* "Create a new annotation marking a production deployment."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `get_active_alerts` | Get the alerts currently firing in Grafana's alertmanager, with labels, state and start time. | - |
| `get_dashboard` | Get a Grafana dashboard by uid: by default a summary (panel titles, types and queries); set full\_json=true for the complete dashboard model (large). | `uid` (string, required), `full_json` (boolean) |
| `get_trace` | Get a full trace by id from a Tempo datasource in Grafana (spans, timings, attributes). | `trace_id` (string, required), `datasource_uid` (string) |
| `list_alert_rules` | List Grafana-managed alert rule definitions, with uid, title, folder and paused state. | - |
| `list_datasources` | List datasources configured in Grafana, with uid, name, type and default flag. | - |
| `query_loki` | Run a LogQL query on a Loki datasource in Grafana over a time range (RFC3339; defaults to the last hour), returning up to `limit` log lines or metric samples. | `query` (string, required), `end` (string), `limit` (integer), `start` (string), `datasource_uid` (string) |
| `query_prometheus` | Run a PromQL query on a Prometheus datasource in Grafana: instant query when no start is given, range query otherwise (RFC3339 timestamps; step e.g. '1m', '15s'). | `query` (string, required), `end` (string), `step` (string), `start` (string), `datasource_uid` (string) |
| `search_dashboards` | Search Grafana dashboards by title (empty query lists all), returning uid, title, folder and URL. | `limit` (integer), `query` (string) |
| `search_traces` | Search traces on a Tempo datasource in Grafana using a TraceQL query (e.g. '\{resource.service.name="api" && duration > 1s}') over a time range (RFC3339; defaults to the last hour). | `query` (string, required), `end` (string), `limit` (integer), `start` (string), `datasource_uid` (string) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `create_annotation` | Create a Grafana annotation (e.g. a deploy or incident marker), global by default, or attached to a dashboard/panel; time\_iso RFC3339 (defaults to now). | `text` (string, required), `tags` (array), `panel_id` (integer), `time_iso` (string), `dashboard_uid` (string) |
| `create_or_update_dashboard` | Create or update a Grafana dashboard from its JSON model (omit `id` and set `uid` to update an existing one); folder\_uid places it in a folder, message describes the change. | `dashboard` (object, required), `message` (string), `overwrite` (boolean), `folder_uid` (string) |
## Troubleshooting
* **Token expired or revoked**: If your service account token is deleted or expires, the MCP shows that it needs to reconnect. Generate a new token in Grafana and update the credentials from the MCP's page in Nekt.
* **Permission denied on writes**: If the agent fails to create annotations or update dashboards, verify that the Grafana service account has the Editor role. The Viewer role is only sufficient for reading data.
## Need help?
Contact our support team if you have trouble connecting Grafana.
# Instagram as an MCP
Source: https://docs.nekt.com/mcps/instagram
Let your AI agents use Instagram in real time through the Nekt MCP Gateway.
Instagram is a social media platform for sharing photos and videos. As an MCP, agents can read engagement insights, list published media and comments, moderate comments, and publish new posts, carousels, or reels to your professional (Business or Creator) accounts.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Instagram as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **Instagram**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. Click the sign-in button and authorize with a Meta account that can see the data you want the agent to use. Actions run with that account's permissions.
* **Sign in with Meta**: Sign in with a Meta account that has access to the Facebook Pages linked to the Instagram professional (Business/Creator) accounts you want to read from and publish to. (Required)
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Publish a new photo to our main Instagram account using this image URL and the caption 'Exciting news coming soon!'."
* "What were our top-performing reels by total interactions over the last 30 days?"
* "Check our current API publishing quota to see if we have room for another post today."
* "List the recent comments on our latest carousel post and hide any that contain spam keywords."
* "Get the follower count and profile views from the last week for our professional account."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `get_account_insights` | Get account-level Instagram insights (reach, views, profile views, accounts engaged, total interactions) for a date range (YYYY-MM-DD, defaults to the last 30 days on Meta's side); metrics accepts a custom comma-separated metric list. | `ig_user_id` (string, required), `metrics` (string), `end_date` (string), `start_date` (string) |
| `get_account_profile` | Get an Instagram professional account's profile: username, name, biography, website, followers/following and media counts. | `ig_user_id` (string, required) |
| `get_media_insights` | Get engagement metrics for one Instagram media: reach, likes, comments, saves, shares, views and total interactions. media\_product\_type: FEED (posts/carousels), REELS or STORY; story insights are only available for 24h. | `media_id` (string, required), `media_product_type` (string) |
| `get_publishing_limit` | Get how many API-published posts the Instagram account has used of its rolling 24h publishing quota. | `ig_user_id` (string, required) |
| `list_comments` | List comments on an Instagram media with author username, text, timestamp, like count, hidden flag and replies. | `media_id` (string, required), `limit` (integer) |
| `list_instagram_accounts` | List the Instagram professional accounts accessible to the authenticated user, with the IG user id (used by every other tool), username, followers and the linked Facebook page. | - |
| `list_media` | List an Instagram account's published media (posts, carousels and reels) with caption, type, permalink, timestamp and like/comment counts; since filters by creation date (YYYY-MM-DD). | `ig_user_id` (string, required), `limit` (integer), `since` (string) |
| `list_stories` | List an Instagram account's currently active stories (last 24h) with media type, URL and timestamp. | `ig_user_id` (string, required) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `delete_media` | Permanently delete a published Instagram media: post, reel, story or an entire carousel album (individual carousel items cannot be deleted; ad posts are not supported). | `media_id` (string, required) |
| `moderate_comment` | Moderate a comment on an Instagram media. action: hide, unhide or delete (delete is permanent). | `action` (string, required), `comment_id` (string, required) |
| `publish_carousel` | Publish a carousel post (2-10 images) to an Instagram professional account's feed. Each URL must be a publicly accessible JPEG; the caption cannot be edited after publishing. | `ig_user_id` (string, required), `image_urls` (array, required), `caption` (string) |
| `publish_photo` | Publish a single photo to an Instagram professional account's feed. image\_url must be a publicly accessible JPEG; the caption cannot be edited after publishing. | `image_url` (string, required), `ig_user_id` (string, required), `caption` (string), `alt_text` (string) |
| `publish_reel` | Publish a reel to an Instagram professional account. video\_url must be a publicly accessible MP4/MOV (Meta processes it asynchronously; this tool waits up to \~2min); the caption cannot be edited after publishing. | `video_url` (string, required), `ig_user_id` (string, required), `caption` (string), `cover_url` (string), `share_to_feed` (boolean) |
| `reply_to_comment` | Reply to a comment on an Instagram media as the account owner. | `message` (string, required), `comment_id` (string, required) |
| `update_media` | Enable or disable comments on a published Instagram media (the only supported update on published media; captions cannot be edited via the API). | `media_id` (string, required), `comment_enabled` (boolean, required) |
## Troubleshooting
* **Expired or revoked access:** If your Meta session expires or the app's access is revoked, the MCP will show that it needs to reconnect. Go to the MCP's page and sign in again.
* **Missing accounts:** Standard Instagram accounts are not supported. The Meta account used for authorization must have access to the Facebook Page linked to an Instagram professional (Business or Creator) account.
* **Publishing limitations:** The Instagram API does not allow editing captions after a post is published. Any media URLs provided to the agent for publishing must be publicly accessible.
* **Reel processing delays:** Reels are processed asynchronously by Meta. If the agent publishes a reel, it may take up to a few minutes for the API to confirm the post.
* **Rate limits:** API-published posts count against a rolling 24-hour quota per account. The agent can check your remaining quota if needed.
## Need help?
Contact our support team if you have trouble connecting Instagram.
# iRancho as an MCP
Source: https://docs.nekt.com/mcps/irancho
Let your AI agents use iRancho in real time through the Nekt MCP Gateway.
iRancho is a livestock and farm management platform. Through the Nekt MCP Gateway, your AI agents can look up animal records, manage products, check inventory, and record stock purchases directly in your iRancho account.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding iRancho as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **iRancho**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. To connect, you need an integration token from iRancho.
1. Log into your iRancho account.
2. Navigate to your account settings to find or generate your integration token.
3. Fill in the following field in Nekt:
* **Api Token**: Token de integração gerado nas configurações da conta no iRancho (enviado no header x-access-token-ws). (Required)
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Find the animal with the farm ID 998877."
* "List all the stock locations we have configured in our account."
* "Check if the supplier with CNPJ 12345678000199 exists. If they don't, register them as 'Agro Supplies'."
* "Register a new product called 'Premium Cattle Feed' in the system and give me its new ID."
* "Record a stock entry for the purchase of items on invoice number 5544 from our supplier, placing them into the main warehouse."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
**What changed in v1.2.** Updated: `alterar_produto`, `incluir_produto`, `registrar_entrada_estoque`.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `buscar_animal` | Busca um animal específico por ID da fazenda, registro de nascimento ou definitivo. Endpoint: GET /animal/buscar-animal | `id_animal_fazenda` (string), `registro_definitivo` (string), `registro_nascimento` (string), `filtra_animais_inativos` (boolean) |
| `buscar_entrada_estoque` | Busca movimentações de estoque no iRancho (por padrão, entradas e compras). Endpoint: GET /produto/\{idProduto}/movimentacao (id\_produto "0" = todos os produtos) Datas em dd/mm/aaaa ou AAAA-MM-DD (data\_final padrão = data\_inicial). Lê todas as páginas. Filtros no conector: numero… | `data_inicial` (string, required), `tipos` (array), `numero_nf` (string), `data_final` (string), `id_produto` (string), `fornecedor_nome` (string) |
| `buscar_pessoa_por_documento` | Busca pessoas/contatos (fornecedores, funcionários...) no iRancho por id, CPF/CNPJ ou nome. Endpoint: GET /pessoa/\{id} ou GET /pessoa (sem filtro na API: o filtro é feito no conector) cpf\_cnpj: com ou sem formatação; compara o número inteiro (só dígitos). nome: trecho do nome, s… | `nome` (string), `cpf_cnpj` (string), `id_pessoa` (string) |
| `buscar_produto` | Busca produtos no iRancho por id, "ID único" ou nome (inclui inativos). Endpoint: GET /produto?allProducts=true (sem filtro na API: o filtro é feito no conector) id\_produto: id do iRancho (o que vai em idProduto numa entrada). id\_unico: o "ID único" da tela (campo id\_produto\_ass… | `nome` (string), `id_unico` (string), `id_produto` (string) |
| `listar_animais` | Lista animais cadastrados no iRancho com filtros opcionais. Endpoint: GET /animal | `lote` (integer), `raca` (integer), `sexo` (integer), `limite` (integer), `pagina` (integer), `fazenda` (integer), `categoria` (integer), `subdivisao` (integer) |
| `listar_categorias_produto` | Lista as categorias de produto do iRancho com as subcategorias (idSubcategoria é obrigatório em incluir\_produto). Endpoints: GET /lista/categoria-produto e GET /lista/subcategoria-produto?id\_produto\_categoria= Sem id\_categoria: categorias (\{value, label}) com "subcategorias" ani… | `id_categoria` (integer) |
| `listar_embalagens` | Lista as embalagens utilizáveis para uma unidade de medida. Endpoint: GET /produto/embalagem/\{idUnidadeMedida} | `id_unidade_medida` (integer, required) |
| `listar_estoques` | Lista os locais de estoque do iRancho (id\_produto\_estoque usado em registrar\_entrada\_estoque). Endpoint: GET /produto/estoques | — |
| `listar_fazendas` | Lista todas as fazendas cadastradas na conta do iRancho. Endpoint: GET /fazenda | — |
| `listar_unidades_medida` | Lista as unidades de medida do iRancho (\{value, label}). Endpoint: GET /lista/unidade-medida | — |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `alterar_produto` | Altera um produto existente no iRancho. Endpoint: PUT /produto/\{id\_produto} (corpo Produto) O Swagger não diz se o PUT substitui o produto inteiro ou só os campos enviados: por isso os quatro obrigatórios da inclusão são exigidos aqui também. Envie o produto completo. id\_unico:… | `nome` (string, required), `id_produto` (string, required), `id_categoria` (integer, required), `id_subcategoria` (integer, required), `id_unidade_base` (integer, required), `carencia` (integer), `id_unico` (string), `monitorar` (boolean), `embalagens` (array), `usado_pasto` (boolean), `idade_maxima` (integer), `idade_minima` (integer), `id_fabricante` (string), `pct_materia_seca` (number), `produzido_fazenda` (boolean), `quantidade_maxima` (number), `quantidade_minima` (number), `pct_proteina_bruta` (number), `especifico_para_sexo` (boolean), `pct_nutrientes_digestiveis_totais` (number) |
| `incluir_fornecedor` | Cadastra um fornecedor no iRancho só quando o CPF/CNPJ ainda não existe. Endpoint: POST /pessoa (corpo Pessoa) num\_cpf\_cnpj (obrigatório, é a chave): procura antes pelo CPF/CNPJ; se achar, devolve o existente com replay\_idempotente=true e NÃO altera o cadastro. tipo\_pessoa: pess… | `nome` (string, required), `num_cpf_cnpj` (string, required), `cep` (string), `email` (string), `endereco` (string), `categoria` (array), `id_estado` (string), `sobrenome` (string), `tipo_pessoa` (string), `id_municipio` (string), `tel_comercial` (string), `tel_residencial` (string), `num_insc_estadual` (string), `endereco_complemento` (string) |
| `incluir_produto` | Cadastra um produto no iRancho. Endpoint: POST /produto (corpo Produto) Obrigatórios: nome, id\_categoria, id\_subcategoria (listar\_categorias\_produto) e id\_unidade\_base (listar\_unidades\_medida). id\_unico: "ID único" do produto (código Omie; id\_produto\_assinatura na leitura). Com… | `nome` (string, required), `id_categoria` (integer, required), `id_subcategoria` (integer, required), `id_unidade_base` (integer, required), `carencia` (integer), `id_unico` (string), `monitorar` (boolean), `embalagens` (array), `usado_pasto` (boolean), `idade_maxima` (integer), `idade_minima` (integer), `id_fabricante` (string), `pct_materia_seca` (number), `produzido_fazenda` (boolean), `quantidade_maxima` (number), `quantidade_minima` (number), `pct_proteina_bruta` (number), `especifico_para_sexo` (boolean), `pct_nutrientes_digestiveis_totais` (number) |
| `registrar_entrada_estoque` | Registra no iRancho a entrada de produtos de uma nota fiscal de compra. Endpoint: POST /produto/movimentacao (corpo PedidoDeMovimentacao) fornecedor: id\_pessoa do fornecedor (buscar\_pessoa\_por\_documento / incluir\_fornecedor). numero\_nf: obrigatório e só dígitos (chave de idempot… | `estoque` (integer, required), `produtos` (array, required), `numero_nf` (string, required), `dt_entrada` (string, required), `fornecedor` (string, required), `frete` (number), `tp_movimentacao` (integer) |
## Troubleshooting
* **Revoked or expired token**: If your integration token is regenerated or deleted in iRancho, the MCP will show that it needs to reconnect. Go to the MCP's page in Nekt and update the token to restore access.
* **Write operation failures**: Adding products or recording stock entries requires exact internal IDs for categories, units of measure, and storage locations. If the agent cannot find a matching configuration in your account, it will fail to save the record. Ensure your iRancho account has the necessary categories and locations set up.
## Need help?
Contact our support team if you have trouble connecting iRancho.
# Omie as an MCP
Source: https://docs.nekt.com/mcps/omie
Let your AI agents use Omie in real time through the Nekt MCP Gateway.
Omie is a cloud-based ERP platform. By connecting it to Nekt, your AI agent can securely read your company's financial records, sales orders, clients, and inventory in real time to answer questions about your business.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Omie as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → MCPs**, click **New MCP** and choose **Omie**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. To authenticate with Omie, you need your application keys:
1. Log in to the Omie application.
2. Go to your application settings and navigate to **Configurações > Integrações** or **Resumo do App**.
3. Copy the keys for the company you want to connect. Each connection corresponds to one company.
* **App Key**: App Key obtida no aplicativo Omie em Configurações > Integrações ou Resumo do App. Um par por empresa. (Required)
* **App Secret**: App Secret obtido no aplicativo Omie em Configurações > Integrações ou Resumo do App. Mantido em segredo. (Required)
4. **Select tools**. Pick the tools your agents can use. See Available tools below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Can you pull our financial summary for today?"
* "Show me all accounts payable with a due date this week."
* "Create a new accounts payable entry for the cleaning service due next Friday."
* "Look up the contact details and address for the client with CNPJ 12.345.678/0001-90."
* "Update the purchase order number 12345 with the new delivery date."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
**What changed in v1.3.** Added: `alterar_pedido_compra`, `alterar_produto`, `consultar_lancamento_cc`, `incluir_ajuste_estoque`, `incluir_conta_pagar`, `incluir_fornecedor`, `incluir_lancamento_cc`, `incluir_pedido_compra`, `incluir_produto`, `posicao_estoque`. Updated: `consultar_cliente`, `consultar_cliente_por_cnpj`, `consultar_conta_pagar`, `consultar_pedido_compra`, `consultar_produto`, `excluir_ajuste_estoque`, `excluir_pedido_compra`, `listar_ajustes_estoque`, `listar_empresas`, `listar_movimentos_estoque`, `listar_ordens_servico`, `pesquisar_titulos`. Removed: `atualizar_pedido_compra`, `atualizar_produto`, `criar_ajuste_estoque`, `criar_anexo`, `criar_conta_pagar`, `criar_fornecedor`, `criar_pedido_compra`, `criar_produto`, `obter_posicao_estoque`.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `consultar_api` | Consulta genérica de leitura para qualquer endpoint disponível na API Omie. Endpoint: POST /\/ (Call: \) Doc: [https://developer.omie.com.br/](https://developer.omie.com.br/) Por segurança, apenas chamadas de leitura são permitidas nesta versão. O nome da `call` deve iniciar obrigatoriamente… | `call` (string, required), `modulo_servico` (string, required), `param` (object\|array) |
| `consultar_cliente` | Consulta um cliente/fornecedor na Omie pelo código Omie ou pelo código de integração. Endpoint: POST /geral/clientes/ (Call: ConsultarCliente) Doc: [https://app.omie.com.br/api/v1/geral/clientes/](https://app.omie.com.br/api/v1/geral/clientes/) A chave de ConsultarCliente só aceita esses dois códigos; para buscar por CNPJ/CPF u… | `codigo_cliente_omie` (integer), `codigo_cliente_integracao` (string) |
| `consultar_cliente_por_cnpj` | Busca clientes/fornecedores na Omie pelo CNPJ ou CPF (com ou sem formatação). Endpoint: POST /geral/clientes/ (Call: ListarClientes com clientesFiltro.cnpj\_cpf) Doc: [https://app.omie.com.br/api/v1/geral/clientes/](https://app.omie.com.br/api/v1/geral/clientes/) Retorna \{"encontrado", "cliente" (o primeiro), "clientes"}; só reg… | `cnpj_cpf` (string, required) |
| `consultar_conta_pagar` | Consulta uma conta a pagar na Omie, com a situação e a data de pagamento do título. Endpoints: POST /financas/contapagar/ (Call: ConsultarContaPagar) e, se incluir\_pagamento, POST /financas/pesquisartitulos/ (Call: PesquisarLancamentos) para a data do pagamento. Doc: `https://app…` | `incluir_pagamento` (boolean), `codigo_lancamento_omie` (integer), `codigo_lancamento_integracao` (string) |
| `consultar_lancamento_cc` | Consulta um lançamento de conta corrente na Omie. Endpoint: POST /financas/contacorrentelancamentos/ (Call: ConsultaLancCC) Doc: [https://app.omie.com.br/api/v1/financas/contacorrentelancamentos/](https://app.omie.com.br/api/v1/financas/contacorrentelancamentos/) Requer nCodLanc ou cCodIntLanc. Devolve \{"encontrado": false} se não existir. | `nCodLanc` (integer), `cCodIntLanc` (string) |
| `consultar_pedido_compra` | Consulta um pedido de compra na Omie (cabeçalho com a etapa, itens, parcelas, frete). Endpoint: POST /produtos/pedidocompra/ (Call: ConsultarPedCompra) Doc: [https://app.omie.com.br/api/v1/produtos/pedidocompra/](https://app.omie.com.br/api/v1/produtos/pedidocompra/) Requer nCodPed, cCodIntPed ou cNumero. Devolve \{"encontrado": false}… | `cNumero` (string), `nCodPed` (integer), `cCodIntPed` (string) |
| `consultar_produto` | Consulta os dados completos de um produto na Omie. Endpoint: POST /geral/produtos/ (Call: ConsultarProduto) Doc: [https://app.omie.com.br/api/v1/geral/produtos/](https://app.omie.com.br/api/v1/geral/produtos/) Requer codigo\_produto, codigo\_produto\_integracao ou codigo. Devolve \{"encontrado": false} se não existir. | `codigo` (string), `codigo_produto` (integer), `codigo_produto_integracao` (string) |
| `listar_ajustes_estoque` | Lista os ajustes de estoque na Omie. Endpoint: POST /estoque/ajuste/ (Call: ListarAjusteEstoque) Doc: [https://app.omie.com.br/api/v1/estoque/ajuste/](https://app.omie.com.br/api/v1/estoque/ajuste/) Datas no formato dd/mm/aaaa. Retorna `ajuste_estoque_lista`. | `tipo` (string), `motivo` (string), `pagina` (integer), `id_prod` (integer), `cod_int_ajuste` (string), `data_movimento_de` (string), `data_movimento_ate` (string), `codigo_local_estoque` (integer), `registros_por_pagina` (integer) |
| `listar_categorias` | Lista o plano de contas / categorias financeiras cadastradas na Omie. Endpoint: POST /geral/categorias/ (Call: ListarCategorias) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `categoria_cadastro` com código, descrição e classificação contábil. | `pagina` (integer), `registros_por_pagina` (integer) |
| `listar_clientes` | Lista os clientes e fornecedores cadastrados na Omie. Endpoint: POST /geral/clientes/ (Call: ListarClientes) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `clientes_cadastro` com dados de contato, endereço, tags e paginação. Datas no formato dd/mm/aaaa. | `pagina` (integer), `filtrar_por_data_de` (string), `apenas_importado_api` (boolean), `filtrar_por_data_ate` (string), `registros_por_pagina` (integer) |
| `listar_contas_correntes` | Lista as contas correntes bancárias e caixas cadastradas na Omie. Endpoint: POST /geral/contacorrente/ (Call: ListarContasCorrentes) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `ListarContasCorrentes` com dados bancários, tipo de conta e saldo. | `pagina` (integer), `apenas_ativas` (boolean), `registros_por_pagina` (integer) |
| `listar_contas_pagar` | Lista os lançamentos de contas a pagar na Omie. Endpoint: POST /financas/contapagar/ (Call: ListarContasPagar) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `conta_pagar_cadastro` com título, fornecedor, vencimento, valor e status. | `pagina` (integer), `filtrar_por_status` (string), `filtrar_por_data_de` (string), `apenas_importado_api` (boolean), `filtrar_por_data_ate` (string), `registros_por_pagina` (integer) |
| `listar_contas_receber` | Lista os lançamentos de contas a receber na Omie. Endpoint: POST /financas/contareceber/ (Call: ListarContasReceber) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `conta_receber_cadastro` com título, cliente, vencimento, valor e status. | `pagina` (integer), `filtrar_por_status` (string), `filtrar_por_data_de` (string), `apenas_importado_api` (boolean), `filtrar_por_data_ate` (string), `registros_por_pagina` (integer) |
| `listar_empresas` | Lista as empresas cadastradas no Omie. Endpoint: POST /geral/empresas/ (Call: ListarEmpresas) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna a lista de empresas (`empresas_cadastro`) com dados cadastrais e paginação. | `pagina` (integer), `registros_por_pagina` (integer) |
| `listar_extrato` | Lista o extrato de movimentações de uma conta corrente na Omie em um período. Endpoint: POST /financas/extrato/ (Call: ListarExtrato) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) nCodCC: Código da conta corrente na Omie. dPeriodoInicial / dPeriodoFinal: datas no formato dd/m… | `nCodCC` (integer, required), `dPeriodoFinal` (string, required), `dPeriodoInicial` (string, required) |
| `listar_movimentos_estoque` | Lista as movimentações de estoque registradas na Omie. Endpoint: POST /estoque/consulta/ (Call: ListarMovimentoEstoque) Doc: [https://app.omie.com.br/api/v1/estoque/consulta/](https://app.omie.com.br/api/v1/estoque/consulta/) Datas no formato dd/mm/aaaa. Filtra por produto (idProd ou cCodInt) e local de estoque. | `idProd` (integer), `pagina` (integer), `cCodInt` (string), `dDtFinal` (string), `dDtInicial` (string), `codigo_local_estoque` (integer), `registros_por_pagina` (integer) |
| `listar_nfse` | Lista as Notas Fiscais de Serviço Eletrônicas (NFS-e) emitidas na Omie. Endpoint: POST /servicos/nfse/ (Call: ListarNFSEs) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `nfseEncontradas` com número, tomador, valores e retenções. | `pagina` (integer), `dEmiFinal` (string), `dEmiInicial` (string), `registros_por_pagina` (integer) |
| `listar_notas_fiscais` | Lista as Notas Fiscais Eletrônicas (NF-e de produto) emitidas na Omie. Endpoint: POST /produtos/nfconsultar/ (Call: ListarNF) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `nfCadastro` com número, chave de acesso, destinatário, valor e status. | `pagina` (integer), `dEmiFinal` (string), `dEmiInicial` (string), `registros_por_pagina` (integer) |
| `listar_ordens_servico` | Lista as Ordens de Serviço cadastradas na Omie. Endpoint: POST /servicos/os/ (Call: ListarOS) Doc: [https://app.omie.com.br/api/v1/servicos/os/](https://app.omie.com.br/api/v1/servicos/os/) Datas em dd/mm/aaaa (inclusão/alteração). Retorna `osCadastro`. | `pagina` (integer), `filtrar_por_data_de` (string), `filtrar_por_data_ate` (string), `registros_por_pagina` (integer) |
| `listar_pedidos_venda` | Lista os pedidos de venda cadastrados na Omie. Endpoint: POST /produtos/pedido/ (Call: ListarPedidos) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `pedido_venda_produto` com cabeçalho, itens, parcelas e totais do pedido. | `pagina` (integer), `filtrar_por_etapa` (string), `filtrar_por_data_de` (string), `apenas_importado_api` (boolean), `filtrar_por_data_ate` (string), `registros_por_pagina` (integer) |
| `listar_produtos` | Lista os produtos cadastrados de forma resumida na Omie. Endpoint: POST /geral/produtos/ (Call: ListarProdutosResumido) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Retorna `produto_servico_resumido` com código, descrição, preços e estoque. | `pagina` (integer), `filtrar_por_data_de` (string), `apenas_importado_api` (boolean), `filtrar_por_data_ate` (string), `registros_por_pagina` (integer), `filtrar_apenas_omiepdv` (boolean) |
| `obter_resumo_financas` | Obtém o resumo financeiro da empresa em uma data específica ou na data atual. Endpoint: POST /financas/resumo/ (Call: ObterResumoFinancas) Doc: [https://developer.omie.com.br/service-list/](https://developer.omie.com.br/service-list/) Data no formato dd/mm/aaaa (dDia). | `dDia` (string) |
| `pesquisar_titulos` | Pesquisa títulos financeiros (a receber e a pagar) na Omie, com baixas e resumo do pagamento. Endpoint: POST /financas/pesquisartitulos/ (Call: PesquisarLancamentos) Doc: [https://app.omie.com.br/api/v1/financas/pesquisartitulos/](https://app.omie.com.br/api/v1/financas/pesquisartitulos/) Datas em dd/mm/aaaa. cNatureza: P (a pagar) ou R (… | `pagina` (integer), `cStatus` (string), `dDtRegDe` (string), `cNatureza` (string), `dDtRegAte` (string), `dDtVencDe` (string), `dDtPagtoDe` (string), `dDtVencAte` (string), `dDtPagtoAte` (string), `registros_por_pagina` (integer) |
| `posicao_estoque` | Consulta o saldo de estoque de um produto na Omie, em um local e data. Endpoint: POST /estoque/consulta/ (Call: PosicaoEstoque) Doc: [https://app.omie.com.br/api/v1/estoque/consulta/](https://app.omie.com.br/api/v1/estoque/consulta/) Requer id\_prod ou cod\_int. Data em dd/mm/aaaa. Retorna saldo, fisico, reservado, cmc. | `data` (string), `cod_int` (string), `id_prod` (integer), `apenas_saldo` (boolean), `codigo_local_estoque` (integer) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `alterar_pedido_compra` | Altera (corrige) um pedido de compra existente na Omie. Endpoint: POST /produtos/pedidocompra/ (Call: AlteraPedCompra) Doc: [https://app.omie.com.br/api/v1/produtos/pedidocompra/](https://app.omie.com.br/api/v1/produtos/pedidocompra/) cabecalho (cabecalho\_alterar): nCodPed ou cCodIntPed (obrigatório um) e os mesmos campos da inclusão,… | `cabecalho` (object, required), `frete` (object), `parcelas` (array), `produtos` (array), `departamentos` (array) |
| `alterar_produto` | Altera um produto existente na Omie. Endpoint: POST /geral/produtos/ (Call: AlterarProduto) Doc: [https://app.omie.com.br/api/v1/geral/produtos/](https://app.omie.com.br/api/v1/geral/produtos/) Requer codigo\_produto ou codigo\_produto\_integracao. Envia só os campos informados; inativo não pode ser enviado. Retorna codigo\_omie =… | `ncm` (string), `unidade` (string), `descricao` (string), `codigo_familia` (integer), `codigo_produto` (integer), `valor_unitario` (number), `campos_adicionais` (object), `codigo_produto_integracao` (string) |
| `excluir_ajuste_estoque` | Exclui um ajuste de estoque na Omie (ex.: desfazer uma baixa lançada errada). Endpoint: POST /estoque/ajuste/ (Call: ExcluirAjusteEstoque) Doc: [https://app.omie.com.br/api/v1/estoque/ajuste/](https://app.omie.com.br/api/v1/estoque/ajuste/) A API só aceita id\_ajuste; se vier cod\_int\_ajuste, o id é resolvido antes por ListarAjus… | `id_ajuste` (integer), `cod_int_ajuste` (string) |
| `excluir_pedido_compra` | APAGA um pedido de compra na Omie. O histórico do pedido se perde no Omie. Endpoint: POST /produtos/pedidocompra/ (Call: ExcluirPedCompra) Doc: [https://app.omie.com.br/api/v1/produtos/pedidocompra/](https://app.omie.com.br/api/v1/produtos/pedidocompra/) A API da Omie não tem operação de cancelar pedido de compra: excluir é a única fo… | `nCodPed` (integer), `cCodIntPed` (string) |
| `incluir_ajuste_estoque` | Inclui um ajuste de estoque na Omie (ex.: saída de diesel por abastecimento). Endpoint: POST /estoque/ajuste/ (Call: IncluirAjusteEstoque) Doc: [https://app.omie.com.br/api/v1/estoque/ajuste/](https://app.omie.com.br/api/v1/estoque/ajuste/) cod\_int\_ajuste: código do app (obrigatório, até 60). Produto: id\_prod ou cod\_int. tipo:… | `data` (string, required), `quan` (number, required), `tipo` (string, required), `motivo` (string, required), `cod_int_ajuste` (string, required), `obs` (string), `valor` (number), `origem` (string), `cod_int` (string), `id_prod` (integer), `codigo_local_estoque` (integer) |
| `incluir_conta_pagar` | Inclui uma conta a pagar na Omie (ex.: reembolso), idempotente pelo código de integração. Endpoint: POST /financas/contapagar/ (Call: IncluirContaPagar) Doc: [https://app.omie.com.br/api/v1/financas/contapagar/](https://app.omie.com.br/api/v1/financas/contapagar/) codigo\_lancamento\_integracao: código do app (obrigatório, até 60). Fa… | `data_vencimento` (string, required), `valor_documento` (number, required), `codigo_lancamento_integracao` (string, required), `categorias` (array), `observacao` (string), `distribuicao` (array), `data_previsao` (string), `numero_pedido` (string), `codigo_categoria` (string), `numero_documento` (string), `id_conta_corrente` (integer), `codigo_tipo_documento` (string), `codigo_cliente_fornecedor` (integer), `codigo_cliente_fornecedor_integracao` (string) |
| `incluir_fornecedor` | Inclui um fornecedor na Omie só quando ele ainda não existe (também vale para funcionário). Endpoint: POST /geral/clientes/ (Call: IncluirCliente) Doc: [https://app.omie.com.br/api/v1/geral/clientes/](https://app.omie.com.br/api/v1/geral/clientes/) Antes de criar: ConsultarCliente pelo codigo\_cliente\_integracao e, se não achar,… | `cnpj_cpf` (string, required), `razao_social` (string, required), `codigo_cliente_integracao` (string, required), `email` (string), `endereco` (object), `nome_fantasia` (string), `telefone1_ddd` (string), `telefone1_numero` (string) |
| `incluir_lancamento_cc` | Inclui um lançamento em conta corrente na Omie (ex.: compra do cartão corporativo). Endpoint: POST /financas/contacorrentelancamentos/ (Call: IncluirLancCC) Doc: [https://app.omie.com.br/api/v1/financas/contacorrentelancamentos/](https://app.omie.com.br/api/v1/financas/contacorrentelancamentos/) codigo\_integracao: código do app (cCodIntLanc, obri… | `valor` (number, required), `data_lancamento` (string, required), `codigo_favorecido` (integer, required), `codigo_integracao` (string, required), `codigo_conta_corrente` (integer, required), `categorias` (array), `observacao` (string), `departamentos` (array), `tipo_documento` (string), `codigo_categoria` (string), `numero_documento` (string) |
| `incluir_pedido_compra` | Inclui um pedido de compra na Omie (idempotente pelo cCodIntPed). Endpoint: POST /produtos/pedidocompra/ (Call: IncluirPedCompra) Doc: [https://app.omie.com.br/api/v1/produtos/pedidocompra/](https://app.omie.com.br/api/v1/produtos/pedidocompra/) cabecalho (cabecalho\_incluir): cCodIntPed (obrigatório, até 20), dDtPrevisao (dd/mm/aaaa),… | `produtos` (array, required), `cabecalho` (object, required), `frete` (object), `parcelas` (array), `departamentos` (array) |
| `incluir_produto` | Inclui um produto na Omie, idempotente pelo codigo\_produto\_integracao. Endpoint: POST /geral/produtos/ (Call: IncluirProduto) Doc: [https://app.omie.com.br/api/v1/geral/produtos/](https://app.omie.com.br/api/v1/geral/produtos/) campos\_adicionais: outros campos de produto\_servico\_cadastro (exceto inativo). Se o código já existir… | `codigo` (string, required), `unidade` (string, required), `descricao` (string, required), `codigo_produto_integracao` (string, required), `ncm` (string), `codigo_familia` (integer), `valor_unitario` (number), `campos_adicionais` (object) |
## Troubleshooting
If the MCP shows that it needs to reconnect, your API keys may have been rotated or revoked by an administrator. Generate a new App Key and App Secret in the Omie application settings, then sign in again from the MCP's page in Nekt.
## Need help?
Contact our support team if you have trouble connecting Omie.
# MCPs
Source: https://docs.nekt.com/mcps/overview
Attach external MCP servers to Nekt so your agent can use their tools alongside your catalog data through a single endpoint.
MCPs are available on all **paid plans**.
## What are MCPs?
MCPs let you attach third-party MCP servers (Slack, HubSpot, Stripe, Grafana, PostgreSQL, Supabase, custom internal servers, and more) to your Nekt workspace. Once attached, their tools are exposed through the **Nekt MCP Gateway** together with your catalog data, Context Layer, and context documents.
Your agent connects to a single endpoint (your Nekt MCP Gateway) and gets:
* **Nekt's own tools** for querying tables, running Queries and Notebooks, and reading context documents.
* **Tools from every MCP you've attached**, discovered and invoked through the same dispatcher.
This lets the agent reason across your warehoused data and your live systems in one conversation, using the context documents in your [Context Layer](/semantic-layer/overview) to route each question to the right place.
## Attach an MCP
Go to **Activate → MCPs** in the sidebar and click **+ New MCP**.
Choose from the list of supported MCP integrations. Custom MCP servers can also be added by URL. See [MCP connectors](#mcp-connectors) for each connector's setup steps and tools.
Give the MCP a short name and a description explaining what it's used for. Descriptions are read by the agent to help it choose between MCPs of the same type (for example, "Slack for marketing" vs "Slack for engineering").
Connect to the MCP server using its native auth flow. OAuth for most SaaS connectors, API key or connection string for databases. If someone else holds the account, [share a setup link](/mcps/share-setup-link) with them instead.
Pick which tools from the MCP your agent will be allowed to use. Each tool is annotated as **Read** or **Write** so you can enable them intentionally.
On Growth and Custom plans, choose which users and groups in your workspace can use this MCP through the Gateway. See [Access levels](#access-levels).
## How the agent uses attached MCPs
Tools from your MCPs are not listed individually to the agent. Instead, the Nekt MCP Gateway exposes two dispatcher tools that cover all of them:
* [`discover_live_tools`](/mcp-server/tools#discover_live_tools) returns the attached MCPs, their tools, and the input schemas.
* [`run_live_tool`](/mcp-server/tools#run_live_tool) invokes a specific tool with arguments.
The agent typically calls `get_semantic_context` first to decide whether a question should be answered from warehoused data or routed to a live tool. After a successful routing, it can save a context document so the next similar question goes to the right place automatically.
Write-capable tools require explicit confirmation from the agent before each call.
## MCP connectors
Every MCP connector Nekt supports has its own page under **MCPs → Connectors** in the sidebar, with
how to set up its access and the full list of the tools it exposes, each marked **Read** or
**Write**. A connector's page is updated every time a new version of it is released, including what
changed.
## Attach an MCP over the API
Everything above is also available through the [Platform API](/platform-api/introduction), which is
what you want when each of your own clients brings their own credentials: create one connection per
client, send each of them a setup link, and let a callback webhook tell you when they are done.
See [Connect an MCP with a setup link](/platform-api/mcp-connections/setup-link) for the full flow.
## Managing MCPs
From the MCPs list you can:
* **Enable or disable** an MCP without deleting it.
* **Tag** MCPs to group them by team, purpose, or environment.
* **Edit permissions** to change who can use it.
* **Update the tool selection** as the underlying MCP server adds or removes tools.
### Access levels
On Growth and Custom plans, access is per connection:
| Level | What it allows |
| - | - |
| **Viewer** | Use the MCP through the Gateway. |
| **Editor** | Everything Viewer allows, plus editing the MCP's description and tags. |
| **Manager** | Full control: configuration, tool selection, enabling or disabling, setup links, sharing, and deletion. |
The person who creates an MCP becomes its Manager. A Viewer or Editor grant can be limited to a subset of the MCP's tools. On other paid plans, access is not restricted per connection.
## API & MCP
List, create, and manage MCP connections and their tools.
`discover_live_tools` and `run_live_tool` reach your connected apps.
## Related
The endpoint your agent connects to.
How attached MCPs surface to agents technically.
Context documents that steer routing.
Enable the Gateway on Express, GCP, or AWS.
Setup links and callback webhooks, one connection per client.
Every supported connector, with its setup steps and tools.
# PostgreSQL as an MCP
Source: https://docs.nekt.com/mcps/postgres
Let your AI agents use PostgreSQL in real time through the Nekt MCP Gateway.
PostgreSQL is a powerful, open-source object-relational database system. By connecting it to Nekt, your AI agent can query your live database, analyze results, and safely execute schema or data changes directly from chat.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding PostgreSQL as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **PostgreSQL**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. Provide your connection credentials to allow the agent to connect to your database.
* **Host** (Required): The host address of your Postgres database.
* **Port** (Required): The port used for connecting to your Postgres database. The default port for Postgres is 5432, but your port might be different.
* **Database** (Required): The name of a database in your Postgres account.
* **User** (Required): The user name for accessing your Postgres database. Read-only tools are always run inside a READ ONLY transaction, so a user with write access is only able to write through the write tool.
* **Password** (Required): The password used for accessing your Postgres database.
* **Enable SSL**: Whether or not to use SSL to verify the server's identity.
* **SSL Mode**: SSL protection method.
* **SSH Tunnel**: SSH Tunnel to be used in your database connection
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Count the number of new user signups we had in our database over the last 30 days."
* "Update the status of order #84920 to 'shipped' in the orders table."
* "Create a new table called 'marketing\_campaigns' with columns for id, name, budget, and start date."
* "Show me the top 5 most expensive products currently in our inventory."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `execute_read_only_sql` | Run one read-only SQL statement against the PostgreSQL database and return the rows. Accepts a single SELECT, WITH, EXPLAIN, SHOW, TABLE or VALUES statement: it runs inside a READ ONLY transaction, so it can never change data or schema even if the connection user has write acce… | `sql` (string, required), `row_limit` (integer), `parameters` (array), `statement_timeout_seconds` (integer) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `execute_write_sql` | Run a SQL statement that changes the PostgreSQL database (INSERT, UPDATE, DELETE, DDL, ...). The statement runs in a transaction that is committed on success and rolled back on error. Rows are returned when the statement produces a result set (for example INSERT ... RETURNING),… | `sql` (string, required), `row_limit` (integer), `parameters` (array), `statement_timeout_seconds` (integer) |
## Troubleshooting
If your database password changes or your user access is revoked, the MCP will show that it needs to reconnect. You can update your credentials and reconnect directly from the MCP's page.
If the agent fails to read or write data, verify that the configured PostgreSQL **User** has the necessary permissions (such as `SELECT`, `INSERT`, `UPDATE`, or `DELETE`) for the specific tables the agent is trying to access.
## Need help?
Contact our support team if you have trouble connecting PostgreSQL.
# RD Station CRM as an MCP
Source: https://docs.nekt.com/mcps/rd-station-crm
Let your AI agents use RD Station CRM in real time through the Nekt MCP Gateway.
RD Station CRM is a sales pipeline and customer relationship management platform. With this connector, your agent can query and update contacts, deals, products, and sales pipelines directly in your RD Station account.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding RD Station CRM as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **RD Station CRM**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. To generate your connection key:
1. Go to [RD Station Mentor](https://mcp.rdstationmentor.com/crm).
2. Click **Conectar neste servidor** and authorize your RD Station account.
3. Copy the value immediately following `key=` in the resulting connection URL.
Provide the following configuration:
* **Connection key**: The connection key generated at RD Station Mentor. (Required)
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Create a new deal for João Silva and add a follow-up task to call him tomorrow."
* "List all deals in our main sales pipeline and sort them by the most recently updated."
* "Update the price of the Enterprise SaaS product in our catalog to 150."
* "Find the contact information for Maria and add a note to her latest deal about our meeting today."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `campaigns_get` | Retrieves a specific campaign by its unique ID. | `id` (string, required) |
| `campaigns_list` | Lists all campaigns in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `contacts_get` | Retrieves a specific contact by its unique ID. | `id` (string, required) |
| `contacts_list` | Lists all contacts in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `custom_fields_get` | Retrieves a specific custom field by its unique ID. | `id` (string, required) |
| `custom_fields_list` | Lists all custom fields in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `deals_get` | Retrieves a specific deal by its unique ID. | `id` (string, required) |
| `deals_list` | Lists all deals in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `deals_notes_list` | Lists all notes associated with a specific deal. | `deal_id` (string, required), `page` (object), `sort` (object), `filter` (string) |
| `deals_products_list` | Lists all products associated with a specific deal. | `deal_id` (string, required) |
| `funnel_list` | Lists all sales pipelines in the account. | `page` (object), `sort` (object) |
| `funnel_stages_get` | Retrieves a specific stage from a specific sales pipeline. | `id` (string, required), `pipeline_id` (string, required) |
| `funnel_stages_list` | Lists all stages for a specific sales pipeline. | `pipeline_id` (string, required), `page` (object), `sort` (object) |
| `funnels_get` | Retrieves a specific sales pipeline by its unique ID. | `id` (string, required) |
| `lost_reasons_get` | Retrieves a specific lost reason by its unique ID. | `id` (string, required) |
| `lost_reasons_list` | Lists all lost reasons in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `organizations_get` | Retrieves a specific organization by its unique ID. | `id` (string, required) |
| `organizations_list` | Lists all organizations in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `products_get` | Retrieves complete information for a specific product by its unique ID. | `id` (string, required) |
| `products_list` | Lists all products in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `segments_get` | Retrieves a specific organization segment by its unique ID. | `id` (string, required) |
| `segments_list` | Lists all segments in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `sources_get` | Retrieves a specific deal source by its unique ID. | `id` (string, required) |
| `sources_list` | Lists all deal sources in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `tasks_get` | Retrieves a specific task by its unique ID. | `id` (string, required) |
| `tasks_list` | Lists all tasks in the account, allowing for filtering and pagination. | `page` (object), `sort` (object), `filter` (string) |
| `teams_get` | Retrieves a specific team by its ID, including the list of its member user IDs. | `id` (string, required) |
| `teams_list` | Lists all teams in the account, including their names and IDs. | `page` (object) |
| `users_get` | Retrieves a specific user by their unique ID. | `id` (string, required) |
| `users_list` | Lists all users in the account, allowing for filtering and pagination. | `page` (object), `filter` (string) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `campaigns_create` | Creates a new campaign to register new marketing initiatives. | `data` (object) |
| `campaigns_update` | Updates the data of an existing campaign, such as its name or description. | `id` (string, required), `data` (object) |
| `contacts_create` | Creates a person (contact), not a deal/lead. Fields: name, emails, phones. | `data` (object) |
| `contacts_update` | Updates the fields of an existing contact. | `id` (string, required), `data` (object) |
| `deals_create` | Creates a deal/opportunity in the pipeline (this is the CRM "lead"), not a person. | `data` (object) |
| `deals_notes_create` | Adds a new note to a specific deal, useful for logging calls or meetings. | `deal_id` (string, required), `data` (object) |
| `deals_products_create` | Adds a catalog product to an existing deal, specifying quantity, price, and discount. | `deal_id` (string, required), `data` (object) |
| `deals_products_update` | Updates a product that is already associated with a deal, such as its quantity or price. | `id` (string, required), `deal_id` (string, required), `data` (object) |
| `deals_update` | Updates a deal's fields. Use this to move a deal between stages or change its owner. | `id` (string, required), `data` (object) |
| `funnel_stages_create` | Creates a new stage within an existing sales pipeline. | `pipeline_id` (string, required), `data` (object) |
| `funnel_stages_update` | Updates the name or display order of an existing stage within a pipeline. | `id` (string, required), `pipeline_id` (string, required), `data` (object) |
| `funnels_create` | Creates a new sales pipeline, which is automatically populated with default stages. | `data` (object) |
| `funnels_update` | Updates the name or display order of an existing sales pipeline. | `id` (string, required), `data` (object) |
| `lost_reasons_create` | Creates a new lost reason, expanding options for classifying lost deals. | `data` (object) |
| `lost_reasons_update` | Updates the name of an existing lost reason. | `id` (string, required), `data` (object) |
| `organizations_create` | Creates a new organization. Organization names must be unique within the account. | `data` (object) |
| `organizations_update` | Updates the fields of an existing organization. | `id` (string, required), `data` (object) |
| `products_create` | Creates a new product in the catalog with details like name, price, and description. | `data` (object) |
| `products_update` | Updates the data of an existing product in the catalog, such as its price or name. | `id` (string, required), `data` (object) |
| `segments_create` | Creates a new segment used for classifying organizations. | `data` (object) |
| `segments_update` | Updates the name of an existing organization segment. | `id` (string, required), `data` (object) |
| `sources_create` | Creates a new deal source, allowing tracking of new acquisition channels. | `data` (object) |
| `sources_update` | Updates an existing deal source. | `id` (string, required), `data` (object) |
| `tasks_create` | Creates a follow-up task on a deal (deal\_id). Use this for next step / outreach (e.g. a draft email). | `data` (object) |
| `tasks_update` | Updates a task's fields, such as its subject, due date, or completion status. | `id` (string, required), `data` (object) |
## Troubleshooting
If the agent loses access or returns an invalid connection key error, your connection may have expired or the authorization was revoked. The MCP Gateway will show that it needs to reconnect. Go to **Activate → MCPs**, open your RD Station CRM connection, generate a new connection key from RD Station Mentor, and update the **Connection key** field.
## Need help?
Contact our support team if you have trouble connecting RD Station CRM.
# Salesforce as an MCP
Source: https://docs.nekt.com/mcps/salesforce
Let your AI agents use Salesforce in real time through the Nekt MCP Gateway.
Salesforce is a leading customer relationship management (CRM) platform. Connecting it as an MCP lets your AI agent query records, perform searches, and read object schemas across your Salesforce organization in real time.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Salesforce as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **Salesforce**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**. Click the sign-in button and authorize with a Salesforce account that can see the data you want the agent to use. Actions run with that account's permissions. Configure the following fields:
* **Sign in with Salesforce**: Click the button to authenticate.
* **Salesforce URL** (Required): Your org's My Domain URL, e.g. [https://yourcompany.my.salesforce.com](https://yourcompany.my.salesforce.com) (sandbox: [https://yourcompany--sandboxname.sandbox.my.salesforce.com](https://yourcompany--sandboxname.sandbox.my.salesforce.com)). Find it in Setup > My Domain.
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Search for the Acme account and summarize its current details."
* "List all open opportunities with a value over \$50,000."
* "What custom fields are configured on our Lead object?"
* "Get the latest 10 closed-won deals from this month."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `get_object_schema` | Retrieve detailed metadata and field definitions for a given sObject (schema describe). Args: creds: The connection credentials. sobject\_name: The API name of the sObject (e.g., 'Lead', 'Case', 'Check\_up\_Call\_\_c'). Returns: Dict containing object metadata, field list, types, pic… | `sobject_name` (string, required) |
| `get_record` | Retrieve a single record of any standard or custom sObject by its Salesforce Id. Args: creds: The connection credentials. sobject\_name: The API name of the sObject (e.g., 'Account', 'Opportunity', 'PartnerTier\_\_c'). record\_id: The 15 or 18 character Salesforce record Id. fields:… | `record_id` (string, required), `sobject_name` (string, required), `fields` (array) |
| `list_objects` | List all available standard and custom sObjects in the Salesforce organization. Args: creds: The connection credentials. Returns: Dict with 'sobjects' containing list of objects, their labels, API names, and capabilities. | - |
| `query_records` | Execute a SOQL query against Salesforce or retrieve the next batch of records. Args: creds: The connection credentials. q: The SOQL query string (e.g., "SELECT Id, Name, StageName FROM Opportunity WHERE IsClosed = false"). next\_records\_url: The nextRecordsUrl returned from a pre… | `q` (string), `next_records_url` (string) |
| `search_records` | Execute a SOSL search across multiple sObjects. Args: creds: The connection credentials. q: The SOSL search query (e.g., "FIND \{Acme} IN ALL FIELDS RETURNING Account(Id, Name), Contact(Id, Name)"). Returns: Dict or list containing the matching search records grouped by entity. | `q` (string, required) |
### Write tools
This connector has no write tools.
## Troubleshooting
If the MCP shows that it needs to reconnect because access expired or was revoked, go to the MCP's page in Nekt and sign in again. If the agent cannot find a specific object or record, ensure the connected Salesforce account has the necessary permissions to read that object.
## Need help?
Contact our support team if you have trouble connecting Salesforce.
# Share a setup link
Source: https://docs.nekt.com/mcps/share-setup-link
Invite a client or third party to connect an MCP connector without giving them access to your workspace.
When adding a new MCP, you can generate a shareable link so that someone outside your workspace can complete the access setup on your behalf. This is useful when the account, credentials, or OAuth sign-in belong to a client, a partner, or a colleague in another team, for example, an agency connecting HubSpot on a client's CRM, or an IT team that prefers to enter a database password themselves.
The recipient opens the link, signs in through OAuth (for connectors like HubSpot or Slack) or fills in the credentials the connector asks for, and the values are saved to the draft MCP connection in your workspace. You can then resume the setup, select the tools your agents can use, and save the MCP.
***
### **How it works**
While adding a new MCP, the **Setup access** step shows a **Share link** button next to "Need someone else to connect …?". Generating the link returns a URL you can copy and send to the recipient.
Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP**, and choose the MCP connector you want to set up.
Give the MCP connection a name and a description. Continuing creates it as a draft in your workspace, which is what the setup link fills in.
On the **Setup access** step, click **Share link**. Nekt returns a setup link you can copy and share.
Share the URL with the client or third party through your preferred channel (email, Slack, WhatsApp, etc.). They do not need a Nekt account.
The recipient opens the link and sees a page identifying who requested the connection and which MCP connector is being set up. They sign in with the provider, or enter the credentials and click **Submit**, depending on what the connector requires.
Once the recipient finishes, the credentials are saved to the draft MCP connection. Return to it in your workspace, continue to **Select tools**, choose which tools your agents can use, and click **Save**. The MCP becomes active and its tools are available through the Nekt MCP Gateway.
***
### **Partial information is allowed**
The recipient does not have to fill in every field. They can submit whatever information they have access to and leave the rest blank. You can complete the missing fields yourself afterwards, or send another setup link to a different person to fill in the remaining values.
***
### **Security and access**
The setup link is scoped strictly to the MCP connection it was generated for. The recipient cannot reach any other part of your workspace, see your other MCPs or their tools, or query catalog data. Credentials entered through the link are encrypted and stored by Nekt; your agents never see them.
The setup link contains a token that **expires 24 hours after it is generated**. It also stops working once the MCP connection is active. If the recipient does not complete the setup in time, generate a new link from the same draft.
The recipient sees only the setup form for the specific MCP connector they were invited to connect. Access ends as soon as the token expires or the MCP connection is activated.
***
### **When to use a setup link**
* A client owns the CRM, chat workspace, or ad account you want your agents to work with, and you need them to sign in or paste credentials.
* An IT or security team controls the credentials for an internal system, such as a database, and they prefer to enter the values themselves.
* A teammate without a Nekt seat needs to provide access details for an MCP you are setting up.
Connecting one MCP per client? The same flow is available through the Platform API, with a callback webhook that tells you when each client is done. See [Connect an MCP with a setup link](/platform-api/mcp-connections/setup-link).
## Skills for agents
Share setup link documentation as plain markdown, for use in AI agent contexts.
## API & MCP
Create a draft MCP connection and share a setup link, with setup tokens.
No dedicated tool yet.
# Stripe as an MCP
Source: https://docs.nekt.com/mcps/stripe
Let your AI agents use Stripe in real time through the Nekt MCP Gateway.
Stripe is a financial infrastructure platform that lets businesses accept payments and manage revenue online. Through Nekt, your AI agents can read and write to the full Stripe API, search the official documentation, and help you plan your Stripe implementation.
| | Available |
| :- | :- |
| **Nekt Express** | Yes |
| **GCP** | Yes |
| **AWS** | Yes |
## Adding Stripe as an MCP
Before you start, make sure the [MCP Server](/mcp-server/setup) is set up. See [MCPs](/mcps/overview) for how MCPs work.
1. Go to **Activate → [MCPs](https://app.nekt.ai/mcps)**, click **New MCP** and choose **Stripe**.
2. **Name your MCP** and describe what it is used for. The agent reads the description to choose between MCPs of the same provider.
3. **Setup access**.
1. Log in to your Stripe Dashboard.
2. Navigate to the API keys section and create a restricted API key.
3. Grant the key only the permissions you want the AI to have. Note that test-mode keys also work and will scope the connection to your test environment.
* **API key** (Required): A Stripe secret or restricted API key. We strongly recommend a restricted key with only the permissions you want the AI to have: a read-only restricted key makes this a read-only connection, enforced by Stripe.
4. **Select tools**. Pick the tools your agents can use. See [Available tools](#available-tools) below.
5. Click **Save**. On Growth and Custom plans, choose who can use it; see [Access levels](/mcps/overview#access-levels).
## Example prompts
* "Search the Stripe documentation for how to implement webhook signatures in Node.js."
* "Look up the most recent invoices for the customer with the email address [billing@example.com](mailto:billing@example.com)."
* "Create a new payment intent for \$50 USD for the customer we just found."
* "Help me plan a Stripe integration for a subscription-based SaaS product."
## Available tools
Choose which of these tools your agents can use in **Select tools**.
### Read tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `search_stripe_documentation` | Search the Stripe documentation for the given question and language | `question` (string, required), `language` (string), `search_only_api_ref` (boolean) |
| `stripe_api_details` | Get detailed parameter information for a specific Stripe API method. | `stripe_api_operation_id` (string, required) |
| `stripe_api_read` | Read data with any Stripe API GET method | `parameters` (object, required), `stripe_api_operation_id` (string, required) |
| `stripe_api_search` | Search for Stripe API methods by keyword | `intent` (string, required), `resource` (string, required), `limit` (integer) |
| `stripe_implementation_planner` | Guides the user through Stripe products to help users accept payments, sell products online, set up billing, or build any Stripe integration | `message` (string, required), `accept` (boolean), `guide_id` (string) |
### Write tools
| Tool | Description | Parameters |
| :- | :- | :- |
| `send_stripe_mcp_feedback` | Submit feedback from user or agent about Stripe’s MCP server tools | `quote` (string, required), `source` (string, required), `context` (string, required), `sentiment` (string, required), `tool_name` (string) |
| `stripe_api_write` | Write data with any Stripe API POST, PATCH, PUT and DELETE method | `parameters` (object, required), `stripe_api_operation_id` (string, required), `human_confirmation` (object) |
## Troubleshooting
If the agent receives permission errors during a conversation, verify that your Stripe restricted API key has the necessary read or write permissions for the specific API resources the agent is trying to use. If your API key expires, is rolled, or is revoked, the MCP will show that it needs to reconnect. You can supply a new key by updating the credentials directly from the MCP's page.
## Need help?
Contact our support team if you have trouble connecting Stripe.
# Notebooks
Source: https://docs.nekt.com/notebooks/overview
Build and deploy Python and PySpark notebooks to process and transform your data.
Our Notebooks module allows you to manipulate data using **PySpark** or **Python**. When creating a Python notebook, you also choose an **image**: Base (standard Python) or Browser (includes a headless Chrome instance for web automation).
Whether you're processing large-scale datasets, running external API calls, or scraping web content, our templates make it easy to develop and deploy your code.
***
## Choosing the Right Notebook Type
Best for very large datasets. Runs on a distributed Spark cluster for maximum performance.
Best for simpler datasets and external API calls. Uses Pandas without a Spark cluster.
Best for web scraping. Includes a headless Chrome instance for browser automation.
| | PySpark | Python: Base | Python: Browser |
| :- | :- | :- | :- |
| **Best for** | Very large datasets | Simpler datasets & APIs | Web scraping |
| **Processing** | Distributed (Spark) | Single-node (Pandas) | Single-node + browser |
| **External requests** | Limited | ✅ | ✅ |
| **Browser automation** | ❌ | ❌ | ✅ |
The notebook type and Python image are set when you create the notebook and **cannot be changed afterwards**. If you need a different configuration, create a new notebook.
### PySpark
Use PySpark when you need to process **very large datasets** (millions of rows or more). It runs on a distributed Spark cluster, so transformations are parallelized across multiple nodes for maximum performance.
**Ideal for:**
* Aggregating or joining large tables from your Lakehouse
* Complex business logic applied to high-volume data
* ETL pipelines that would be slow or memory-intensive in plain Python
```python theme={null}
import nekt
from pyspark.sql import functions as F
# Load a large orders table
df = nekt.load_table(layer_name="raw", table_name="orders")
# Aggregate revenue by customer
result = df.groupBy("customer_id").agg(
F.sum("amount").alias("total_revenue"),
F.count("order_id").alias("order_count")
)
nekt.save_table(
df=result,
layer_name="trusted",
table_name="customer_revenue"
)
```
***
### Python: Base Image
Use the Python Base image when you're working with **simpler or moderately sized datasets**, or when you need to **make external API/HTTP requests** as part of your transformation. It uses Pandas and doesn't require a Spark cluster.
**Ideal for:**
* Calling external REST APIs and enriching your data with the responses
* Transformations on smaller tables that fit comfortably in memory
* Prototyping logic before scaling it to PySpark
```python theme={null}
import nekt
import requests
# Load a contacts table
df = nekt.load_table(layer_name="raw", table_name="contacts")
# Enrich each record with data from an external API
def enrich_contact(row):
response = requests.get(f"https://api.example.com/contacts/{row['id']}")
data = response.json()
row["score"] = data.get("score", 0)
return row
df = df.apply(enrich_contact, axis=1)
nekt.save_table(
df=df,
layer_name="trusted",
table_name="contacts_enriched"
)
```
***
### Python: Browser Image
Use the Python Browser image when your transformation requires **automating a real browser**, such as scraping websites that rely on JavaScript rendering or logging into a web portal to extract data. It includes **Selenium** with a headless Chrome instance pre-configured.
**Ideal for:**
* Scraping dynamic web pages (React, Vue, etc.) that don't expose a public API
* Automating form submissions or login flows to collect data
* Extracting structured data from web UIs that require interaction
```python theme={null}
import nekt
import os
import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
def create_driver():
options = Options()
options.add_argument("--headless=new")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--window-size=1920,1080")
possible_paths = [
"/usr/local/bin/chromedriver",
"/usr/bin/chromedriver",
"/usr/bin/chromium-driver",
]
for path in possible_paths:
if os.path.exists(path):
return webdriver.Chrome(service=Service(executable_path=path), options=options)
return webdriver.Chrome(options=options)
def scrape_holidays():
driver = create_driver()
results = []
try:
driver.get("https://www.timeanddate.com/holidays/brazil/2025")
wait = WebDriverWait(driver, 15)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "table.table")))
rows = driver.find_elements(By.CSS_SELECTOR, "table.table tbody tr")
for row in rows:
cols = row.find_elements(By.TAG_NAME, "td")
if len(cols) >= 3:
results.append({
"date": cols[0].text.strip(),
"name": cols[1].text.strip(),
"type": cols[2].text.strip(),
})
finally:
driver.quit()
return pd.DataFrame(results)
df = scrape_holidays()
nekt.save_table(
df=df,
layer_name="raw",
table_name="brazil_holidays_2025"
)
```
***
## Creating a Notebook
Regardless of the notebook type you choose, the workflow is the same.
### Step 1: Generate a Token
To allow Jupyter Notebooks to access your data:
1. Navigate to the **Notebooks** module.
2. [Generate a token](https://app.nekt.ai/notebooks/access-tokens) with appropriate access permissions.
* You can create multiple tokens, each with specific access levels for better security.
3. Select a notebook template to get started.
Notebook templates are pre-configured with the necessary imports and setup to access data from your Lakehouse. It's used for local testing and development.
***
### Step 2: Develop and Test Your Transformation
Test your transformation locally to validate its logic before running it on Nekt. Nekt provides several environment options for running Jupyter Notebooks. You can find it on the Notebooks module, under the **Templates** button:
* **Google Colab Notebook**: Run a Jupyter notebook in the cloud with minimal setup. Requires a Google account.
* **GitHub Codespaces**: Use a cloud-hosted Jupyter notebook powered by GitHub for easy setup. Requires a GitHub account.
* **Local Dev Container**: Set up a Jupyter notebook on your local machine using an isolated environment with all dependencies pre-installed.
* **Local Jupyter Notebook**: Manually configure a Jupyter notebook on your local machine. Best for users with advanced knowledge of Python environments.
**How to Start**:
1. Copy your token and paste it into the template as instructed.
2. Use the Nekt SDK to load the data you need for your transformation.
```python theme={null}
import nekt
df = nekt.load_table(
layer_name="layer_name",
table_name="table_name"
)
```
3. Develop and test your transformation code within the notebook environment.
***
### Step 3: Deploy Your Notebook to Nekt
Once your code is validated locally, deploy it to Nekt to run it on a schedule, trigger it based on pipeline events, or execute it on demand.
1. Navigate back to the **Add Notebook** page in Nekt.
2. Select the **Notebook type**: **PySpark** or **Python**.
* If you selected Python, also choose the **Python image**: **Base** for standard notebooks, or **Browser** if your code requires a headless browser.
3. Copy your notebook code into Nekt.
4. Use the Nekt SDK `.save_table` method to save transformed dataframes as new tables in your Lakehouse. You can call it multiple times to save several dataframes in the same notebook.
```python theme={null}
import nekt
nekt.save_table(
df=df,
layer_name="layer_name",
table_name="table_name",
folder_name="folder_name" # optional
)
```
5. If you need external dependencies, add them in the **Define dependencies** section.
6. Save the notebook to make it available for execution.
***
## Best Practices
1. **Choose the right runtime**: Use PySpark for large-scale data, Python (Base) for API-heavy or moderate workloads, and Python (Browser) only when a real browser is strictly necessary; it carries more overhead than the Base image.
2. **Development**: Use Notebook Templates to reduce setup time and start with simple notebooks before scaling complexity.
3. **Testing**: Validate your notebook logic locally before deployment. Test with sample data of various sizes and verify edge cases.
4. **Performance**:
* For PySpark: leverage built-in Spark functions and avoid `.collect()` on large datasets.
* For Python/Browser: keep external requests minimal and handle timeouts gracefully.
5. **Maintenance**: Document complex logic, keep dependencies up to date, and use version control for your notebooks. Direct integration with Github is available on the Growth plan.
***
## Need Help?
If you encounter challenges with your notebooks, our team is ready to assist. You can:
* Use our example notebooks as templates
* Contact our support team for additional guidance
## API & MCP
List, update, trigger, and delete transformations (Queries and Notebooks).
`create_notebook`, `get_code`, `update_transformation`, and `run_pipeline`.
# SDK
Source: https://docs.nekt.com/notebooks/sdk
Use Nekt SDK to interact with your data through code.
Nekt SDK provides a simple interface to read and write data from your Lakehouse. It supports two processing engines:
* **Python**: Uses Pandas DataFrames. Best for smaller datasets and general-purpose data processing.
* **Spark**: Uses PySpark DataFrames. Best for large-scale distributed data processing.
Explore the SDK using [Google Colab](https://colab.research.google.com/drive/1y_CEwfmNEahxuBrD9hRQMeCsnGBBBl5N) or check all templates navigating to [**Notebooks**](https://app.nekt.ai/notebooks) > **Templates**.
## Getting started
[Generate a token](https://app.nekt.ai/notebooks/access-tokens) with access to the resources you need (tables, volumes, secrets).
The SDK is already included in Nekt Notebook templates. To install it in your own environment:
```bash theme={null}
pip install git+https://github.com/nektcom/nekt-sdk-py.git#egg=nekt-sdk
```
Set your access token and choose an engine before calling any SDK method:
```python theme={null}
import nekt
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "python" # or "spark"
```
```python theme={null}
df = nekt.load_table(layer_name="Bronze", table_name="pipedrive_deals")
```
## Configuration
You must set `data_access_token` and `engine` before using any SDK method.
| Attribute | Required | Description |
| - | - | - |
| `nekt.data_access_token` | Yes | Access token for authenticating with the Nekt API. [Generate one here](https://app.nekt.ai/notebooks/access-tokens). |
| `nekt.engine` | Yes | Processing engine: `"python"` (Pandas) or `"spark"` (PySpark). |
```python theme={null}
import nekt
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "python"
```
Configuration is locked after the first SDK operation. If you need to change the engine or token, restart your Python session.
## Reading data
Load a table from your Lakehouse as a DataFrame using the `.load_table()` method. Returns a Pandas DataFrame when `engine="python"` or a Spark DataFrame when `engine="spark"`.
**Parameters:**
* `layer_name` (str): The name of the layer where the table is located
* `table_name` (str): The name of the table to load
Layer and table names must match the exact capitalization used in your Catalog.
**Returns:** Pandas DataFrame (`engine="python"`) or Spark DataFrame (`engine="spark"`)
**Example with Python engine:**
```python theme={null}
import nekt
import pandas as pd
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "python"
df: pd.DataFrame = nekt.load_table(
layer_name="Bronze",
table_name="pipedrive_deals"
)
```
**Example with Spark engine:**
```python theme={null}
import nekt
from pyspark.sql import DataFrame
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "spark"
df: DataFrame = nekt.load_table(
layer_name="Bronze",
table_name="pipedrive_deals"
)
```
Load a secret value from your organization's secrets vault using the `.load_secret()` method.
**Parameters:**
* `key` (str): The secret key to retrieve
The key name must match the exact capitalization used in your Catalog.
**Returns:** str
**Example:**
```python theme={null}
import nekt
api_key = nekt.load_secret(key="MY_SECRET_API_KEY")
```
Your access token must have permission to access the secret.
List files in a volume from your Lakehouse using the `.load_volume()` method.
**Parameters:**
* `layer_name` (str): The name of the layer where the volume is located
* `volume_name` (str): The name of the volume
Layer and volume names must match the exact capitalization used in your Catalog.
**Returns:** List of dictionaries containing file paths
**Example:**
```python theme={null}
import nekt
files = nekt.load_volume(
layer_name="Raw",
volume_name="csv_uploads"
)
for file in files:
print(file["path"])
```
Load a Delta table object from your Lakehouse using the `.load_delta_table()` method. This gives you access to Delta Lake features like ACID transactions and time travel.
**Parameters:**
* `layer_name` (str): The name of the layer where the table is located
* `table_name` (str): The name of the table to load
This method requires `engine="spark"`. It raises an error if the engine is set to `"python"`.
Layer and table names must match the exact capitalization used in your Catalog.
**Returns:** DeltaTable object
**Example:**
```python theme={null}
import nekt
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "spark"
delta_table = nekt.load_delta_table(
layer_name="Transformation",
table_name="customer_data"
)
# View table history
delta_table.history().show()
```
## Writing data
`save_table` is only available when running inside Nekt Notebooks. When running locally, this method prints a message indicating it is not available.
Save a DataFrame as a table in your Lakehouse using the `.save_table()` method. Supports overwrite, append, and merge (upsert) modes with automatic schema evolution.
**Parameters:**
* `df` (DataFrame): The DataFrame to save (Pandas or Spark, depending on your engine)
* `layer_name` (str): The name of the target layer
* `table_name` (str): The name of the table to create or update
* `folder_name` (str, optional): Folder within the layer. Defaults to the layer root
* `mode` (str, optional): Write mode: `"overwrite"`, `"append"`, or `"merge"`. Defaults to `"overwrite"`
* `merge_keys` (list\[str], optional): Column names used as keys for merge mode. Required when `mode="merge"`
* `schema_evolution` (str, optional): Schema evolution strategy: `"merge"`, `"strict"`, or `"overwrite"`. Defaults to `"merge"`
Layer, table, and folder names must match the exact capitalization used in your Catalog.
**Example (overwrite, default):**
```python theme={null}
import nekt
nekt.save_table(
df=transformed_df,
layer_name="Transformation",
table_name="customer_metrics"
)
```
**Example (append):**
```python theme={null}
nekt.save_table(
df=new_records,
layer_name="Raw",
table_name="events",
mode="append"
)
```
**Example (merge, upsert):**
```python theme={null}
nekt.save_table(
df=updated_records,
layer_name="Transformation",
table_name="users",
mode="merge",
merge_keys=["user_id"]
)
```
**Schema evolution strategies:**
| Strategy | Behavior |
| - | - |
| `"merge"` | New columns are added automatically. Existing columns are preserved. |
| `"strict"` | Schema must match exactly. Raises an error if columns differ. |
| `"overwrite"` | The table schema is replaced with the DataFrame schema. |
Upload a local file to a volume in your Lakehouse using the `.save_file()` method. The file is uploaded with a multipart upload in 100 MB chunks, so it works with large files.
**Parameters:**
* `layer_name` (str): The name of the layer where the volume is located
* `volume_name` (str): The name of the target volume
* `file_path` (str): Local path to the file to upload
* `file_name` (str, optional): Name for the file in the volume. Defaults to the local file's name
* `description` (str, optional): Description for the file
Layer and volume names must match the exact capitalization used in your Catalog.
**Returns:** Dictionary with file metadata (`id`, `name`, `file_size`, `file_type`, `description`)
**Example:**
```python theme={null}
import nekt
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "python"
file = nekt.save_file(
layer_name="Raw",
volume_name="csv_uploads",
file_path="/path/to/local/report.csv",
description="Monthly sales report",
)
print(file["id"])
```
## Logging
The SDK provides a built-in logger for tracking execution progress. Inside Nekt Notebooks, logs are visible in the Nekt UI. Outside Nekt, logs go to standard error output.
### Default logger
Use `nekt.logger` to log messages directly:
```python theme={null}
import nekt
nekt.logger.info("Starting transformation")
nekt.logger.warning("Skipping 5 invalid rows")
```
### Named loggers
Use `nekt.get_logger(name)` to create a named sub-logger. Messages are automatically prefixed with the logger name, making it easier to identify which part of your code produced each log entry.
```python theme={null}
import nekt
validation_logger = nekt.get_logger("validation")
validation_logger.info("Checking schema") # → "[validation] Checking schema"
validation_logger.error("Missing columns") # → "[validation] Missing columns"
etl_logger = nekt.get_logger("etl")
etl_logger.info("Processing batch 1") # → "[etl] Processing batch 1"
```
Use named loggers to organize logs by pipeline stage or domain. This makes it much easier to filter and debug issues in the Nekt UI.
## Utilities
Access the shared Spark session using the `.get_spark_session()` method. Useful for creating DataFrames manually or running Spark operations directly.
This method requires `engine="spark"`. It raises an error if the engine is set to `"python"`.
**Returns:** SparkSession object
**Example:**
```python theme={null}
import nekt
from pyspark.sql.types import StructType, StructField, StringType, IntegerType
nekt.data_access_token = "YOUR_TOKEN"
nekt.engine = "spark"
spark = nekt.get_spark_session()
# Create a DataFrame with custom schema
schema = StructType([
StructField("id", IntegerType(), True),
StructField("name", StringType(), True)
])
data = [(1, "Alice"), (2, "Bob")]
custom_df = spark.createDataFrame(data, schema)
# Read files directly
csv_df = spark.read.csv("path/to/file.csv", header=True)
```
***
### Need Help?
If you encounter any issues with the SDK or have feedback, reach out to our support team.
# Spark Configuration
Source: https://docs.nekt.com/notebooks/spark_configuration
Optimize and configure Spark configuration for your transformations.
Spark is the engine that powers Nekt's transformations. It is responsible for executing the transformations and managing the data processing.
You can configure the Spark configuration for your transformations in the **Settings** tab of each transformation.
### **Optimizing Transformation Performance**
In data workflows, performance can vary based on the size of your data and the complexity of the operations. Configuring cloud resources for your transformations allows you to strike the perfect balance between speed and cost efficiency.
Nekt lets you adjust key parameters for each transformation in the **Settings** tab, giving you control over resource allocation to meet your organization’s specific needs.
***
### **Spark Configuration**
Here are the adjustable parameters available in transformation settings and how they affect performance:
1. **Spark Driver Cores**\
The number of CPU cores allocated to the **Spark Driver**, which coordinates the overall execution of the Spark application.
* **Impact:** Increasing driver cores can improve the application's ability to manage multiple tasks concurrently.
* **Recommendation:** Adjust based on the complexity of your transformations and the number of tasks managed by the driver.
2. **Spark Driver Memory**\
The amount of memory allocated to the **Spark Driver**, which keeps track of task states and stores temporary results.
* **Impact:** Allocating sufficient memory prevents crashes due to out-of-memory errors during coordination.
* **Recommendation:** Allocate more memory if the application manages a large number of tasks or processes metadata-heavy transformations.
3. **Spark Executor Cores**\
The number of CPU cores allocated to each **Spark Executor**, which executes individual tasks within the transformation.
* **Impact:** Higher cores per executor can speed up task execution but may lead to diminishing returns if tasks are smaller.
* **Recommendation:** Balance the cores based on the size of your data partitions and the nature of your tasks.
4. **Spark Executor Instances**\
The number of **Spark Executors** launched to run the tasks of the transformation.
* **Impact:** Increasing executor instances allows parallel processing of more tasks, improving throughput for large datasets.
* **Recommendation:** Adjust based on the size and complexity of your data, ensuring enough instances are available to prevent bottlenecks.
5. **Spark Executor Memory**\
The amount of memory allocated to each **Spark Executor**, which processes individual tasks and stores intermediate results.
* **Impact:** Insufficient memory allocation can lead to task failures or retries, slowing down execution.
* **Recommendation:** Allocate memory based on task memory requirements, ensuring enough room for large data processing.
***
### **Finding the Right Balance**
Configuring these parameters is a tradeoff between performance and cost. Over-allocating resources might improve execution speed but can increase cloud resource expenses. Conversely, under-allocating might lead to slower execution or even task failures.
#### **Tips for Optimization**:
1. **Start Small:** Begin with lower resource allocations and scale up as needed based on performance.
2. **Monitor Execution Metrics:** Use the execution logs and performance metrics to identify bottlenecks or inefficiencies.
3. **Iterative Tuning:** Gradually adjust parameters, testing the impact on performance after each change.
4. **Parallelism vs. Resource Usage:** For larger datasets, prioritize increasing the number of executor instances over cores per executor.
***
### **Real-World Scenarios**
* **Small Datasets:** Use fewer executor instances and moderate memory to keep resource usage minimal.
* **Large Transformations:** Increase executor instances and allocate more memory to executors for better parallelism and efficiency.
* **Complex Workflows:** Allocate additional driver memory and cores to ensure smooth coordination of tasks across the application.
***
### **Best Practices**
* Ensure resources match your workload's complexity and size to avoid underutilization or overspending.
* Periodically review resource configurations as data volumes grow or transformation complexity increases.
* Coordinate with your cloud provider's pricing model to ensure cost-effective resource allocation.
***
By leveraging these configurable parameters, you can tailor your transformation settings to your organization's unique needs, ensuring optimal performance without unnecessary overhead.
# Versioning
Source: https://docs.nekt.com/notebooks/versioning
Version control your notebooks with GitHub and deploy from commits.
GitHub Versioning lets you manage code for Notebooks and Queries through Git: link a Nekt resource to a GitHub repository, and every push to the configured branch (e.g. **main**) automatically triggers a redeploy in Nekt. You get version control, collaboration via pull requests, and a clear audit trail.
This requires a [workspace-wide GitHub connection](/workspace/github): an owner or admin connects your organization in **Settings → GitHub** before you can link repositories to individual Notebooks or Queries.
Once connected, you link a repo per resource (when creating or editing, or via the **Versioning** column in the list). After that, no manual "deploy" step is needed: Nekt stays in sync with your branch.
## Prerequisite: Connect GitHub (workspace)
Before linking repositories, an owner or admin must [connect your GitHub organization](/workspace/github) in **Settings → GitHub**. This step is mandatory before you can assign repositories to Notebooks.
## Prepare your repository
Use a GitHub repository to store your Notebook code. It can be empty. A dedicated repo per Notebook is recommended for clarity and isolation.
## Link the repository to a Notebook
You can link a repo when **creating** a new Notebook or **editing** an existing one.
### When creating a new Notebook
During creation, use the GitHub section to select the repository (and branch, if offered).
### When editing an existing Notebook
1. Open the Notebook.
2. Go to **Settings** and find the **GitHub** / versioning section.
3. Select the repository (and branch) to link.
You can also use the **Versioning** column in the Notebooks list: click **Connect repository** and choose the repo.
## Automatic deployments
Once a Notebook is linked, any push to the configured branch (typically **main**) automatically redeploys that Notebook in Nekt. No manual deploy step is required.
## API & MCP
Not available in the Platform API yet.
`connect_transformations_to_github` and `list_github_repositories`.
# Pipelines
Source: https://docs.nekt.com/pipelines/overview
Pipelines are the conceptual tunnel where your data moves through Nekt.
A **pipeline** in Nekt is a data workflow that moves data through the platform:
* **Source → Catalog** when a source extraction runs
* **Catalog → Catalog** when a query, notebook, or history runs
* **Catalog → Destination** when data is loaded into an external tool
Pipelines are the structure that handles data movement and keeps it orchestrated and monitored.
## Configuring pipelines
Whenever you create a source, query, notebook, history, or destination, the platform guides you to configure how its pipeline runs: when it runs, how data is synced, and so on.
As your workspace grows, you can orchestrate pipelines. For example: when source X finishes extracting, trigger query Y, and when that execution completes, send the result to destination Z. That connects the executions of a source, a query, and a destination pipeline.
## Controlling pipelines
Pipelines are controlled by **triggers** (scheduled, event-based, or manual) that determine when they run.
Each time a pipeline runs, it creates a [run](/runs/overview). You can monitor success via the run status and [logs](/runs/logs).
## API & MCP
Trigger pipelines and track their runs over REST.
`run_pipeline`, `complete_pipeline`, and `get_pipeline_status`.
# Activity object
Source: https://docs.nekt.com/platform-api/activities/activity
# List Activities
Source: https://docs.nekt.com/platform-api/activities/list
GET /api/v1/activities/
# API token object
Source: https://docs.nekt.com/platform-api/api-token/token
# Folder object
Source: https://docs.nekt.com/platform-api/catalog/folder
# Layer object
Source: https://docs.nekt.com/platform-api/catalog/layer
# List Layers
Source: https://docs.nekt.com/platform-api/catalog/list-layers
GET /api/v1/layers/
# Table object
Source: https://docs.nekt.com/platform-api/catalog/table
# Update Field
Source: https://docs.nekt.com/platform-api/catalog/update-field
PATCH /api/v1/tables/{table_pk}/fields/{name}/
# Update Layer
Source: https://docs.nekt.com/platform-api/catalog/update-layer
PATCH /api/v1/layers/{slug}/
# Update Table
Source: https://docs.nekt.com/platform-api/catalog/update-table
PATCH /api/v1/tables/{id}/
# Volume object
Source: https://docs.nekt.com/platform-api/catalog/volume
# Get Connector Validation
Source: https://docs.nekt.com/platform-api/connector-validations/details
GET /api/v1/connector-validations/{id}/
Retrieve the status of a connector validation started via the source `revalidate` endpoint. Poll this until `status` is `success` or `failed`.
# Get Connector Validation Logs
Source: https://docs.nekt.com/platform-api/connector-validations/logs
GET /api/v1/connector-validations/{id}/logs/
Fetch logs produced during a connector validation. Supports incremental fetching via the `start`/`end` query parameters.
# Connector object
Source: https://docs.nekt.com/platform-api/connectors/connector
# List Destination Connectors
Source: https://docs.nekt.com/platform-api/connectors/destination
GET /api/v1/destinations/connectors/
# List MCP Connectors
Source: https://docs.nekt.com/platform-api/connectors/mcp
GET /api/v1/live-connectors/
The catalog of Live Data connectors available to this workspace.
Each connector carries the configuration template its connections are created against —
expand ``current_version.config_template`` to get the JSON Schema of the credentials and
settings a connection for it needs.
# List Source Connectors
Source: https://docs.nekt.com/platform-api/connectors/source
GET /api/v1/sources/connectors/
# Connector version object
Source: https://docs.nekt.com/platform-api/connectors/version
# Bulk Delete Destinations
Source: https://docs.nekt.com/platform-api/destinations/bulk-delete
POST /api/v1/destinations/bulk-delete/
Delete several destinations at once with a single request.
Deletes several destinations in one call. The request body takes a list of pipeline **slugs**, not destination IDs.
This endpoint uses `POST` rather than `DELETE` because OpenAPI does not document a request body on `DELETE`. It deletes; it does not create anything.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `destinations` | array of strings | Yes | Slugs of the destinations to delete. |
## Response
Returns `200 OK` with a summary of what the action did, rather than the deleted objects.
```json theme={null}
{
"message": "Successfully deleted 2 destinations"
}
```
The operation is all-or-nothing: nothing is deleted unless **every** listed destination passes the same checks a single [Delete Destination](/platform-api/destinations/delete) call would apply. There is no undelete endpoint, so deletion cannot be reversed through the Platform API.
## Example
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/destinations/bulk-delete/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"destinations": ["google-sheets-weekly-report", "hubspot-contacts-sync"]
}'
```
## Python example: delete every destination carrying a given tag
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": API_KEY,
"Content-Type": "application/json",
}
destinations = requests.get(
f"{BASE_URL}/api/v1/destinations/",
headers=headers,
params={"expand[]": "tags"},
).json()
legacy_slugs = [
d["slug"]
for d in destinations["results"]
if any(tag["name"] == "legacy" for tag in d.get("tags", []))
]
response = requests.post(
f"{BASE_URL}/api/v1/destinations/bulk-delete/",
headers=headers,
json={"destinations": legacy_slugs},
)
print(response.json()["message"])
```
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header. MCP tokens cannot delete resources and are rejected with `403 Forbidden`.
## Related
* [Delete Destination](/platform-api/destinations/delete): remove a single destination.
* [List Destinations](/platform-api/destinations/list): collect the slugs to send.
# Delete Destination
Source: https://docs.nekt.com/platform-api/destinations/delete
DELETE /api/v1/destinations/{slug}/
Delete a single destination by its slug.
Deletes the destination identified by `slug`. The identifier is the pipeline **slug**, not the destination ID, the same value used by [Get Destination](/platform-api/destinations/details) and [Update Destination](/platform-api/destinations/update).
A successful call returns `204 No Content` with an empty body.
## Example
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/destinations/google-sheets-weekly-report/ \
--header "x-api-key: YOUR_API_KEY"
```
There is no undelete endpoint. Deleting a destination cannot be reversed through the Platform API. Double-check the slug before sending the request.
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header. MCP tokens cannot delete resources and are rejected with `403 Forbidden`.
## Related
* [Bulk Delete Destinations](/platform-api/destinations/bulk-delete): remove several destinations in a single request.
* [List Destinations](/platform-api/destinations/list): find the slug you want to delete.
# Destination object
Source: https://docs.nekt.com/platform-api/destinations/destination
# Get Destination
Source: https://docs.nekt.com/platform-api/destinations/details
GET /api/v1/destinations/{slug}/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# List Destinations
Source: https://docs.nekt.com/platform-api/destinations/list
GET /api/v1/destinations/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Trigger Destination
Source: https://docs.nekt.com/platform-api/destinations/trigger
POST /api/v1/destinations/{slug}/trigger/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Get Destination Trigger
Source: https://docs.nekt.com/platform-api/destinations/triggers/details
GET /api/v1/destinations/{destination_slug}/triggers/{id}/
# List Destination Triggers
Source: https://docs.nekt.com/platform-api/destinations/triggers/list
GET /api/v1/destinations/{destination_slug}/triggers/
# Update Destination
Source: https://docs.nekt.com/platform-api/destinations/update
PATCH /api/v1/destinations/{slug}/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Expanding Responses
Source: https://docs.nekt.com/platform-api/expanding-responses
Expand responses to get related objects inline.
Many objects allow you to request additional information as an expanded response by using the `expand[]` request parameter. This parameter is available on all API requests, and applies to the response of that request only.
The expandable label in this documentation indicates ID fields that you can expand into objects. When an object contains the ID of a related object in its response properties, you can expand these objects in line with the `expand[]` request parameter.
For example, when querying Sources, without expanding any fields:
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: "
```
The API will return:
```JSON theme={null}
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"id": "78f91ad6-fb65-470e-b193-4f34fe2ba024",
"slug": "json-content-9pVL",
"active": true,
...
"connector_version": "dc371ad6-388f-48df-89d6-d64898ba9835",
"last_run": "0dce33ce-59a8-4adb-a92c-1366798ab96d",
"created_by": 14
}
]
}
```
Note that `connector_version` field returned the ID of the Connector Version model related to this Source.
We can then expand this field, using the `expand[]` query parameter:
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: " \
--data "expand[]"=connector_version \
--get
```
The API will return:
```JSON theme={null}
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"id": "78f91ad6-fb65-470e-b193-4f34fe2ba024",
"slug": "json-content-9pVL",
"active": true,
...
"connector_version": {
"id": "dc371ad6-388f-48df-89d6-d64898ba9835",
"version": "0.1",
"editable": true,
"connector": "97323c4a-7bd0-420a-a243-0feaee555a55"
},
"last_run": "0dce33ce-59a8-4adb-a92c-1366798ab96d",
"created_by": 14
}
]
}
```
Note that `connector` field, inside the `connector_version` field returned the ID of the Connector model related to the Connector Version.
## Expanding recursively
You can expand recursively by specifying nested fields after a dot (`.`). For example, requesting `connector.customer` on a transaction expands the payment property into a full Payment object, then expands the customer property on that payment into a full Customer object.
Performing deep expansions might result in slower processing times.
This is an example expanding the `connector` field of the `connector_version` of a Source, using the `expand[]` query parameter:
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: " \
--data "expand[]"=connector_version.connector \
--get
```
The API will return:
```JSON theme={null}
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"id": "78f91ad6-fb65-470e-b193-4f34fe2ba024",
"slug": "json-content-9pVL",
"active": true,
...
"connector_version": {
"id": "dc371ad6-388f-48df-89d6-d64898ba9835",
"version": "0.1",
"editable": true,
"connector": {
"id": "97323c4a-7bd0-420a-a243-0feaee555a55",
"created_at": "2025-04-14T23:40:01.572568-03:00",
"updated_at": "2025-04-14T23:50:08.062632-03:00",
"type": "source",
"name": "JSON Content",
"slug": "json-content",
"description": "The Json Content connector enables the seamless capture and unification of JSON data available as a payload in a public single-page endpoint.",
"logo": "https://statics.nekt.ai/connector/logos/97323c4a-7bd0-420a-a243-0feaee555a55-2025-04-15T024001.5726250000.jpg",
"docs_available": true,
"external_docs_url": null,
"current_version": "dc371ad6-388f-48df-89d6-d64898ba9835"
}
},
"last_run": "0dce33ce-59a8-4adb-a92c-1366798ab96d",
"created_by": 14
}
]
}
```
## Maximum expansion depth
Expansions have a maximum depth of four levels.
## Methods that support expanding
You can use the `expand[]` parameter on any endpoint that returns expandable fields, including list, details, create, and update endpoints.
Performing deep expansions on numerous list requests might result in slower processing times.
You can expand multiple objects at the same time by repeating `expand[]`. A comma-separated list is read as one field name, not several, so it expands nothing:
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/sources/?expand[]=connector_version&expand[]=created_by" \
--header "x-api-key: "
```
# Introduction
Source: https://docs.nekt.com/platform-api/introduction
Manage your resources and automate workflows using the Platform API.
Nekt's Platform API helps organizations streamline data operations and gain valuable insights. It enables teams to build integrations that maximize the value of their Nekt implementation.
### Automate Workflows
You can integrate with your own systems to automate workflows:
* Schedule regular data extraction
* Trigger pipelines based on external events
* Build custom dashboards or monitoring solutions
* Integrate Nekt with your existing tools and services
## Base URL
The API is built on REST conventions with predictable URL structure, uses standard HTTP response codes, authentication and verbs, and returns JSON responses.
All requests must be made against:
```
https://api.nekt.ai
```
We enforce HTTPS in every request to improve data security, integrity, and
privacy.
# Authentication
The Nekt API uses API keys to authenticate requests. To authenticate, you need to add an `x-api-key` header with your API key as the value:
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: "
```
Your API key acts as **the person who created it**: same role, same permissions, re-checked on every
request. A key made by an owner or admin can do everything they can. Keep it secure, and do not share
it in publicly accessible areas such as GitHub, client-side code, and so forth.
## Create an API key
Go to your Workspace settings and [create an API Key](https://app.nekt.ai/settings/api-keys), it will be necessary to use all endpoints.
## Expanding Responses
Many objects allow you to request additional information as an expanded response by using the `expand[]` request parameter. This parameter is available on all API requests, and applies to the response of that request only.
The expandable label in this documentation indicates ID fields that you can expand into objects. When an object contains the ID of a related object in its response properties, you can expand these objects in line with the `expand[]` request parameter.
Check [Expanding Responses](./expanding-responses) for details.
## API Modules
Access and trigger your data sources
Access and trigger your data transformations
Access and trigger your data destinations
Track and cancel execution runs
Grant and revoke access to data, secrets, and live connections
Invite people, and set up their access before they accept
# Bulk Invite Users
Source: https://docs.nekt.com/platform-api/invitations/bulk-invite
POST /api/v1/invitations/bulk-invite/
Invite several people to your organization in one request.
Invites a list of people in one call. Every entry behaves exactly like a single [Invite User](/platform-api/invitations/invite) request: an address with no outstanding invitation is invited, and one that already has an invitation gets it resent, with a role change if the role differs. This is the common integration case for onboarding a team from a list.
**All-or-nothing.** One bad entry undoes the whole batch: nothing is written and nobody is emailed. Applying some entries and reporting the rest would make a retry ambiguous, and re-sending the batch would mail everyone who already succeeded a second time.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `invitations` | array | Yes | The people to invite. Each entry has the same shape as a single invite. |
Each entry in `invitations` accepts:
| Field | Type | Required | Description |
| - | - | - | - |
| `email` | string | Yes | The address to invite. Normalized before storing, so casing never creates a duplicate. |
| `role` | string | No | One of `owner`, `admin`, or `member`. Defaults to `member`. |
| `first_name` | string | No | Shown in the member list before the person accepts. |
| `last_name` | string | No | Shown in the member list before the person accepts. |
| `functional_area` | string | No | The invitee's functional area. |
| `other_functional_area` | string | No | Free-text functional area, when none of the predefined options fit. |
Each entry behaves exactly like a single [Invite User](/platform-api/invitations/invite) request, so the same fields and validation rules apply.
## Response
Returns `200 OK` with a summary of what the batch did, split by what happened to each address.
```json theme={null}
{
"message": "Successfully invited 2 user(s).",
"invited": ["analyst@example.com", "lead@example.com"],
"resent": []
}
```
| Field | Description |
| - | - |
| `message` | Human-readable summary of what the bulk invite did. |
| `invited` | Addresses that had no outstanding invitation and now do. |
| `resent` | Addresses whose outstanding invitation was resent. |
`invited` and `resent` together always account for every entry you sent.
## Example
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/invitations/bulk-invite/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"invitations": [
{"email": "analyst@example.com"},
{"email": "scientist@example.com"},
{"email": "lead@example.com", "role": "admin"}
]
}'
```
## Python example: onboard a team from a list
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": os.environ["NEKT_API_KEY"],
"Content-Type": "application/json",
}
emails = ["analyst@example.com", "scientist@example.com", "lead@example.com"]
response = requests.post(
f"{BASE_URL}/api/v1/invitations/bulk-invite/",
headers=headers,
# `role` omitted, so everyone comes in as a member.
json={"invitations": [{"email": email} for email in emails]},
)
response.raise_for_status()
body = response.json()
print(f"invited: {body['invited']}")
print(f"resent: {body['resent']}")
```
## Resending
Sending the same batch again is safe and is how you chase people who have not accepted. Nobody is duplicated: addresses that already have an invitation come back under `resent` with a fresh expiry, and their existing groups and grants are untouched.
```json theme={null}
{
"message": "Successfully invited 1 user(s) and resent 2 invitation(s).",
"invited": ["newcomer@example.com"],
"resent": ["analyst@example.com", "scientist@example.com"]
}
```
If an entry names a different role than the outstanding invitation has, the resend moves the person to it on both the invitation and their pending membership. Moving an existing `admin` or `owner` requires an owner.
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header.
Only an **owner or an admin** can invite, and nobody can invite somebody to a role above their own. For an API key, that ceiling is read from the role of the user who created the key.
## Errors
Every one of these rejects the **whole** batch.
| Status | When |
| - | - |
| `400` | An address already belongs to an active member of the organization. |
| `400` | The same address appears twice in one payload. |
| `400` | `invitations` is empty or missing, or an entry has no `email`. |
| `403` | The caller is a member, or an entry names a role above the caller's own. |
| `403` | The organization is on the Free plan, where only owners may be invited. |
Because one bad entry rejects everything, a long list built from an external source is worth filtering first: drop addresses that are already members, and de-duplicate. A rejected batch tells you what was wrong, but it writes nothing.
## Related
* [Invite User](/platform-api/invitations/invite): invite or resend for a single person.
* [Onboarding flow](/platform-api/invitations/overview): invite, group, and grant before somebody accepts.
* [List Invitations](/platform-api/invitations/list): what is still outstanding.
* [Invitation object](/platform-api/invitations/invitation): the shape of each entry.
# Invitation object
Source: https://docs.nekt.com/platform-api/invitations/invitation
# Invite User
Source: https://docs.nekt.com/platform-api/invitations/invite
POST /api/v1/invitations/
Invite one person to your organization, or resend the invitation they already have.
Invites a single person to your organization. The same call also **resends** an existing invitation: posting an address that already has a live invitation resends it rather than creating a second one.
Inviting someone provisions them immediately as a **pending member**, so you can add them to permission groups and grant them access before they accept. See [Pending users and pre-granted invitations](#pending-users-and-pre-granted-invitations) below.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `email` | string | Yes | The address to invite. Normalized before storing, so casing never creates a duplicate. |
| `role` | string | No | One of `owner`, `admin`, or `member`. Defaults to `member`. |
| `first_name` | string | No | Shown in the member list before the person accepts. |
| `last_name` | string | No | Shown in the member list before the person accepts. |
| `functional_area` | string | No | The invitee's functional area. |
| `other_functional_area` | string | No | Free-text functional area, when none of the predefined options fit. |
`first_name` and `last_name` are optional. An admin often has only an address, and the person names themselves afterward from their profile or the onboarding step. The invitation email greets an unnamed invitee by their address.
## Response
Returns the [Invitation](/platform-api/invitations/invitation) object. The status code tells you which of the two things happened:
* `201 Created`: a new invitation was created.
* `200 OK`: an existing invitation was resent.
## Example
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/invitations/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"email": "newcomer@example.com",
"role": "member"
}'
```
## Resending and role changes
Sending the same address again is safe and is how you chase someone who has not accepted. Nobody is duplicated: the existing invitation is resent with a fresh expiry, and the invitee's existing groups and grants are untouched.
If the `role` differs from the outstanding invitation, the resend is a role **change**: it moves the person on both the invitation and their pending membership.
Moving an existing `admin` or `owner` invitee to another role requires an owner. This is the same ceiling that applies to changing the role of an accepted member.
## Pending users and pre-granted invitations
Because an invitation provisions the person right away, they exist as a pending member the moment the call succeeds:
* **Grant access before acceptance.** A pending member is a normal target for every permission-assign endpoint, and can be [added to a permission group](/platform-api/permission-groups/add-members) with no separate flow.
* **Pending is not inactive.** In the member list a pending member carries its own status; it is not the same as a deactivated user.
* **They cannot log in until they accept.** The invitation token is the only proof that the person controls the address.
* **Revoking is destructive.** [Revoking](/platform-api/invitations/revoke) removes the invitee's groups and grants, not just the email.
The invitation email expires after 7 days; the pre-granted access does not. An admin resends and nothing has to be re-granted.
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header.
Only an **owner or an admin** can invite, and nobody can invite somebody to a role above their own. For an API key, that ceiling is read from the role of the user who created the key.
## Errors
| Status | When |
| - | - |
| `400` | The address already belongs to an active member of the organization. |
| `400` | `email` is missing or malformed. |
| `403` | The caller is a member, or the request names a role above the caller's own. |
| `403` | The request changes the role of an existing `admin` or `owner` invitee and the caller is not an owner. |
| `403` | The organization is on the Free plan, where only owners may be invited. |
## Related
* [Onboarding flow](/platform-api/invitations/overview): invite, group, and grant before somebody accepts.
* [Bulk Invite Users](/platform-api/invitations/bulk-invite): invite several people in one call.
* [List Invitations](/platform-api/invitations/list): what is still outstanding.
* [Revoke an Invitation](/platform-api/invitations/revoke): and what it takes back.
* [Invitation object](/platform-api/invitations/invitation): the full response schema.
# List Invitations
Source: https://docs.nekt.com/platform-api/invitations/list
GET /api/v1/invitations/
The invitations still waiting on a reply.
Returns the invitations that are still outstanding: not yet accepted, not revoked, and (for an expired one) still the most recent invitation sent to that address.
Once somebody accepts, they stop being an invitation and become a member, so they drop off this list. To see them after that, read the member list in the app under [Members](/workspace/members).
One row per address. Resending does not create a second invitation, so an address you have chased three times still appears once, with the latest `expires_at`.
## Response
An array of [invitation objects](/platform-api/invitations/invitation). This endpoint is not paginated.
```json theme={null}
[
{
"id": "c1f0a9d4-5b62-4e18-9a73-2d8e4f6b0c51",
"email": "analyst@example.com",
"first_name": "",
"last_name": "",
"role": "member",
"sent_at": "2026-09-24T10:14:02Z",
"expires_at": "2026-10-01T10:14:02Z",
"accepted_at": null,
"revoked_at": null
}
]
```
## Find what has gone stale
An expired invitation still holds every group and grant attached to it: the link stopped working, the access did not. Resending re-opens it without re-granting anything.
```python theme={null}
from datetime import datetime, timezone
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": API_KEY, "Content-Type": "application/json"}
now = datetime.now(timezone.utc)
invitations = requests.get(f"{BASE_URL}/api/v1/invitations/", headers=headers).json()
for invitation in invitations:
expires_at = datetime.fromisoformat(invitation["expires_at"].replace("Z", "+00:00"))
if expires_at < now:
print(f"{invitation['email']} expired on {expires_at:%Y-%m-%d} - resending")
requests.post(
f"{BASE_URL}/api/v1/invitations/",
headers=headers,
json={"email": invitation["email"], "role": invitation["role"]},
)
```
## Related
* [Invite User](/platform-api/invitations/invite): inviting and resending.
* [Revoke an Invitation](/platform-api/invitations/revoke): for the ones that are never coming.
# Onboarding flow
Source: https://docs.nekt.com/platform-api/invitations/overview
Invite someone, put them in groups, and grant them access before they accept.
Inviting someone through the API creates the user straight away. That is the part worth knowing before you build anything: an invitation is not a pending email with nothing behind it. It provisions a user and associates them with your organization at the moment you send it.
So you can finish somebody's onboarding in one run of your script and they walk into a workspace that is already set up for them.
Inviting requires an [API key](https://app.nekt.ai/settings/api-keys) created by an **Owner** or an **Admin**. A key created by a Member cannot invite. See [the ceiling rule](/platform-api/permissions/overview#the-ceiling-rule).
***
## What "pending" means
An invited user exists in three states, and the difference matters when you read them back:
| State | `is_active` | `is_pending` | What they can do |
| - | - | - | - |
| **Invited** | `false` | `true` | Hold groups and grants. Cannot sign in. |
| **Active** | `true` | `false` | Everything their role and grants allow. |
| **Deactivated** | `false` | `false` | Nothing. Grants are gone. |
A pending user is deliberately **not active**. Member counts and the billing contact leave them out until they accept. They do appear in the member list, marked as pending. What they *can* do is receive access, so that the moment they accept, everything is already there.
Because pending is not active, an invitee never appears in a count of your members. Accepting is what moves them.
***
## The flow
Only `email` is required. `role` defaults to `member`. If you do not know the person's name yet, leave it out; they will set it themselves when they accept.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/invitations/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"email": "analyst@example.com", "role": "member"}'
```
See [Invite User](/platform-api/invitations/invite).
The invitee now appears in the recipients endpoints alongside your existing members, which is what makes the next two steps possible.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/permission-recipients/?expand[]=users&search=analyst@example.com" \
--header "x-api-key: YOUR_API_KEY"
```
Keep the `id` from the response: groups and grants both target it.
Grants work exactly as they do for an accepted member, on all three axes.
You can grant to the invitee directly, or grant to a **group**: if they are already in one, whatever that group can reach, they can reach.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{"permission_level": "viewer", "users": ["USER_ID"], "layers": ["LAYER_ID"]}
]
}'
```
See [Permissions flow](/platform-api/permissions/overview).
**A group is usually the better target.** Grant the group once, then put the invitee in it:
they inherit everything it holds, and so does the next user you add. Pending invitees can
be grouped before they accept, exactly like accepted members. See
[Add Members](/platform-api/permission-groups/add-members).
The invitation email carries the link. Accepting sets their credential and clears the pending state.
**No permission is created, moved, or replayed at this point.** Everything you granted is already attached to them, which is why acceptance cannot half-fail and leave someone with the wrong access.
***
## Onboard someone end to end
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
EMAIL = "analyst@example.com"
GOLD_LAYER = "e4d5c6b7-a8b9-40c1-d2e3-f4a5b6c7d8e9"
headers = {"x-api-key": API_KEY, "Content-Type": "application/json"}
requests.post(
f"{BASE_URL}/api/v1/invitations/",
headers=headers,
json={"email": EMAIL, "role": "member"},
).raise_for_status()
# The invitee is a recipient from this moment on, so the id is available immediately.
recipients = requests.get(
f"{BASE_URL}/api/v1/permission-recipients/",
headers=headers,
params={"expand[]": "users", "search": EMAIL},
).json()
user_id = recipients["users"][0]["id"]
requests.post(
f"{BASE_URL}/api/v1/permissions/",
headers=headers,
json={
"assignments": [
{"permission_level": "viewer", "users": [user_id], "layers": [GOLD_LAYER]}
]
},
).raise_for_status()
```
***
## Things worth knowing before you build
**Only Members appear as permission recipients.** Invite someone as `admin` or `owner` and they will not show up in the recipients endpoints, not because the invitation failed, but because Owners and Admins bypass data-level permissions entirely, so there is nothing to grant them. Invite as `member` when you intend to grant.
**Expiry does not revoke.** The email link expires after seven days; the access does not. Resending re-opens the link and keeps every group and grant already attached, so you never re-grant on a resend.
**Revoking is destructive.** It removes every grant on all three axes and every group membership, and deletes the user if this was their only pending invitation. It is not the same as letting an invitation lapse. See [Revoke an Invitation](/platform-api/invitations/revoke).
**Someone who already has a Nekt account keeps it.** Inviting an existing user to a second organization adds a membership; it does not create a second user or touch their name or password.
***
## Related
* [Invite User](/platform-api/invitations/invite): including resending and changing the role.
* [List Invitations](/platform-api/invitations/list): the outstanding ones.
* [Revoke an Invitation](/platform-api/invitations/revoke): and what it takes back.
* [Permissions flow](/platform-api/permissions/overview): the three grant axes.
* [Members](/workspace/members): what each role can do, in the app.
# Revoke an Invitation
Source: https://docs.nekt.com/platform-api/invitations/revoke
DELETE /api/v1/invitations/{token}/
Cancel an invitation and take back everything it granted.
Revoking is not just cancelling an email. Because an invitation provisions the person the moment it is sent, they may already be carrying groups and grants, so revoking takes all of it back.
Revoking removes **every grant on all three permission axes**, and every group membership. If the invitation was the only reason that person existed in Nekt, the account is deleted too. This cannot be undone: re-inviting them starts from nothing.
## The path parameter is the id
The URL segment is named `token`, but the value is the invitation's **`id`**, the field you get from [List Invitations](/platform-api/invitations/list). Passing the invitation token instead returns `404`.
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/invitations/c1f0a9d4-5b62-4e18-9a73-2d8e4f6b0c51/ \
--header "x-api-key: YOUR_API_KEY"
```
## Response
`200`, with a confirmation body.
```json theme={null}
{ "detail": "The invitation has been revoked." }
```
## Revoke versus letting it lapse
These are different, and the difference is the access:
| | Link works | Groups and grants | Appears in the member list |
| - | - | - | - |
| **Expired** | No | Kept | Yes, as pending |
| **Revoked** | No | Removed | No |
An expired invitation is a pause: resend it and the person picks up exactly where you left them. A revoked one is a decision. Revoke when somebody is not joining; let one lapse when they are simply slow.
## Clean up invitations nobody answered
```python theme={null}
from datetime import datetime, timedelta, timezone
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
GIVE_UP_AFTER_DAYS = 90
headers = {"x-api-key": API_KEY, "Content-Type": "application/json"}
cutoff = datetime.now(timezone.utc) - timedelta(days=GIVE_UP_AFTER_DAYS)
for invitation in requests.get(f"{BASE_URL}/api/v1/invitations/", headers=headers).json():
sent_at = datetime.fromisoformat(invitation["sent_at"].replace("Z", "+00:00"))
if sent_at < cutoff:
# Takes back their groups and grants as well as the invitation.
requests.delete(f"{BASE_URL}/api/v1/invitations/{invitation['id']}/", headers=headers)
```
## Errors
| Status | When |
| - | - |
| `400` | The invitation has already been accepted. Accepted people are members now; remove them from [Members](/workspace/members) instead. |
| `403` | Your API key was created by a Member. Revoking requires an Owner's or Admin's key. |
| `404` | No invitation with that **id** in your organization; check you are not passing the token. |
## Related
* [Onboarding flow](/platform-api/invitations/overview): what an invitation carries.
* [List Invitations](/platform-api/invitations/list): where the `id` comes from.
# Archive MCP Connection
Source: https://docs.nekt.com/platform-api/mcp-connections/archive
POST /api/v1/live-connections/{slug}/archive/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# Create MCP Connection
Source: https://docs.nekt.com/platform-api/mcp-connections/create
POST /api/v1/live-connections/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# Delete MCP Connection
Source: https://docs.nekt.com/platform-api/mcp-connections/delete
DELETE /api/v1/live-connections/{slug}/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# MCP Connection Details
Source: https://docs.nekt.com/platform-api/mcp-connections/details
GET /api/v1/live-connections/{slug}/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# List MCP Connections
Source: https://docs.nekt.com/platform-api/mcp-connections/list
GET /api/v1/live-connections/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# MCP Connection object
Source: https://docs.nekt.com/platform-api/mcp-connections/mcp-connection
# Connect an MCP with a setup link
Source: https://docs.nekt.com/platform-api/mcp-connections/setup-link
Programmatically create a draft MCP connection, invite a third party to authenticate through a setup link, and get notified via a callback webhook.
This flow lets you build an MCP onboarding experience **on top of the Platform API**: create a connection in draft, hand a secure link to a client or third party so they can authenticate with their own credentials (without a Nekt account), receive a webhook when they finish, then activate the connection.
It mirrors the in-app [MCPs](/mcps/overview) flow end to end over the API, and it is the same shape as [Configure a source with a setup link](/platform-api/sources/setup-link): a different resource, and a different link prefix (`/lcl/` instead of `/scl/`).
The case this exists for: one connection per end client, each with their own credentials. Create as many connections of the same connector as you have clients; each has its own slug, its own credentials, and its own permissions.
## Overview
```mermaid theme={null}
sequenceDiagram
participant Client as Your app
participant API as Nekt API
participant ThirdParty as Third party
participant Hook as Your callback URL
Client->>API: GET /live-connectors/ (pick a connector)
Client->>API: POST /live-connections/ (callback_webhook)
API-->>Client: Draft connection
Client->>API: POST /live-connections/{slug}/setup-tokens/
API-->>Client: { token, expires_at }
Client->>ThirdParty: Share https://app.nekt.ai/lcl/{token}
ThirdParty->>API: Authenticate / fill in the config (via the link)
API->>Hook: POST callback_webhook (live_connection.setup_filled)
Client->>API: PATCH /live-connections/{slug}/ (active=true)
API-->>Client: Connection in service
```
## Prerequisites
* An [API key](/platform-api/introduction#create-an-api-key) for authentication.
* MCPs are available on all **paid plans**.
## Step 1: Pick a connector
List the connectors available to your workspace. Expand the current version to get the JSON Schema that a connection for it is configured against: that is what tells you which fields the third party will be asked for.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/live-connectors/?expand[]=current_version&include[]=current_version.config_template" \
--header "x-api-key: YOUR_API_KEY"
```
## Step 2: Create a draft connection with a callback webhook
The connection is created as a draft, pinned to the connector's current version.
Optionally attach a **`callback_webhook`**: Nekt POSTs to it once the third party submits the setup form. Use the optional custom header to authenticate the call on your side.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/live-connections/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"connector": "CONNECTOR_ID_OR_SLUG",
"slug": "hubspot-acme",
"description": "ACME'\''s HubSpot",
"callback_webhook": {
"url": "https://your-app.example.com/hooks/nekt",
"header_name": "X-Webhook-Secret",
"header_value": "your-shared-secret"
}
}'
```
| Field | Type | Required | Description |
| - | - | - | - |
| `connector` | string | Yes | The connector this connection is for. |
| `slug` | string | No | Stable identifier, generated when omitted. It is also the namespace the connection's tools are exposed under, so keep it recognisable. |
| `description` | string | No | Read by the agent to choose between connections of the same type ("HubSpot for marketing" vs "HubSpot for sales"). |
| `available_tools` | array | No | Which of the connector's tools this connection exposes. Defaults to all of them. |
| `callback_webhook` | object | No | See below. |
### `callback_webhook` object
| Field | Type | Required | Description |
| - | - | - | - |
| `url` | string | Yes | HTTPS URL Nekt POSTs to on setup completion. Must be a public `https://` endpoint. |
| `header_name` | string | No | Optional custom header name to send with the callback. |
| `header_value` | string | No | Value for the custom header. Required if `header_name` is set. Stored encrypted and never returned. |
`callback_webhook` is **write-only**. Responses never echo it back; they expose a read-only boolean `has_callback_webhook` so you can confirm one is configured. To remove a configured webhook, send `"callback_webhook": null` on an update.
For security, the URL must use `https` and resolve to a public host. URLs pointing at `localhost`, loopback, private (RFC 1918), or link-local addresses are rejected.
## Step 3: Create a setup token
Generate a time-limited token scoped to this connection, then build the link as `https://app.nekt.ai/lcl/{token}` and share it with the third party.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/live-connections/{slug}/setup-tokens/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{ "expires_in_seconds": 86400 }'
```
```json theme={null}
{
"id": "0d2c1b47-2902-4c47-8b30-0e7c3fd72609",
"token": "Qm9vR2k...redactedOnList",
"expires_at": "2026-06-18T12:00:00Z",
"connector_config_fields": null,
"created_at": "2026-06-17T12:00:00Z"
}
```
| Field | Type | Required | Description |
| - | - | - | - |
| `expires_in_seconds` | integer | No | Token lifetime. Defaults to 24 hours (`86400`). |
| `connector_config_fields` | array, object or boolean | No | Restricts which configuration fields the link exposes. Omit for the whole config template. See [Restricting the link](#restricting-what-the-link-exposes). |
The full `token` is returned only on creation. When listing tokens it is redacted. To revoke a link, delete the token: `DELETE /api/v1/live-connections/{slug}/setup-tokens/{id}/`.
## Step 4: The third party authenticates
The recipient opens `https://app.nekt.ai/lcl/{token}` and enters the credentials (or authenticates via OAuth for connectors that support it). No Nekt account is required, and the link is scoped strictly to this connection.
When they submit, Nekt POSTs to your `callback_webhook` (if configured):
```http theme={null}
POST https://your-app.example.com/hooks/nekt
Content-Type: application/json
X-Webhook-Secret: your-shared-secret
{
"event": "live_connection.setup_filled",
"live_connection": {
"id": "d2c1b472-902c-477a-b300-e7c3fd72609b",
"slug": "hubspot-acme",
"connector_slug": "hubspot",
"description": "ACME's HubSpot",
"draft": true,
"active": false,
"config_completed": true,
"missing_required_fields": []
}
}
```
| Field | Type | Description |
| - | - | - |
| `config_completed` | boolean | `true` when all required configuration fields are filled. |
| `missing_required_fields` | array | Dot-paths of required fields still missing. Empty when `config_completed` is `true`. |
Read `config_completed`, not `draft`. The connection stays `draft` until **you** activate it in the next step, so `draft` never answers "did my client finish?". A partially filled form produces `config_completed: false` with the remaining fields named; send another link to collect them.
## Step 5: Activate the connection
Activating clears `draft` and puts the connection in service: its tools become available to your agents through the Gateway.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/live-connections/{slug}/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{ "active": true }'
```
Setup links stop working once the connection is active. Any unused link for it is invalidated at that moment, and a link can be used again only if the connection later goes into an error state and needs re-authentication.
## Restricting what the link exposes
By default a setup link exposes the connector's whole configuration template. `connector_config_fields` narrows it: the setup page renders only the named fields, and anything else the recipient sends back is **ignored rather than written**; a hidden field keeps its stored value.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/live-connections/{slug}/setup-tokens/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{ "connector_config_fields": ["api_key"] }'
```
| Value | Meaning |
| - | - |
| omitted / `null` | No restriction: the whole template. |
| `["api_key", "domain"]` | Those fields, nothing else. |
| `{"auth": {"token": true}}` | Nested: only `auth.token`, and only `token` within `auth`. |
| `true` / `false` | Everything / nothing, at any level. |
A typical use: your systems already know the account id and region, so you send them as `config` when you create the connection and restrict the link to the one secret only your client has.
Naming a field the connector's template does not declare is rejected with `400` when the link is created, listing every unknown path at once, rather than producing a link whose page is silently empty.
## Choosing which tools the connection exposes
A connection is created exposing every tool of the connector. Send `available_tools` at creation, or replace the set later: the payload is the full desired set, so anything missing from the list stops being exposed.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/live-connections/{slug}/tools/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{ "tools": ["search_objects", "get_object"] }'
```
`GET` the same endpoint to list the connector's tools with an `is_write` flag on each, so you can enable write-capable tools deliberately.
The exact endpoints and request/response shapes are available in the OpenAPI schema at [api.nekt.ai/api/schema/](https://api.nekt.ai/api/schema/).
# Create Setup Token
Source: https://docs.nekt.com/platform-api/mcp-connections/setup-tokens/create
POST /api/v1/live-connections/{live_connection_slug}/setup-tokens/
Setup links for a connection, so a third party fills in credentials in their own browser.
Create one to get a token, then send its recipient to ``{WEB_APP_URL}/lcl/{token}``: they
configure the connection without a Nekt account, and the credentials never pass through
you. Optionally restrict what the link exposes with ``connector_config_fields``, and have
the connection announce completion with its ``callback_webhook``.
A link minted while the connection awaits setup (a draft) stops working once it is set up.
A link minted for a connection already in use updates it: it keeps working until it expires,
is deleted or the connection is archived, and a write through it must leave every required
field filled.
The token is returned in clear text only when it is created, and is redacted on every
later read. Managing setup links requires manager access on the connection — the token is
a config-write capability, so listing links is as sensitive as creating one.
# Delete Setup Token
Source: https://docs.nekt.com/platform-api/mcp-connections/setup-tokens/delete
DELETE /api/v1/live-connections/{live_connection_slug}/setup-tokens/{id}/
Setup links for a connection, so a third party fills in credentials in their own browser.
Create one to get a token, then send its recipient to ``{WEB_APP_URL}/lcl/{token}``: they
configure the connection without a Nekt account, and the credentials never pass through
you. Optionally restrict what the link exposes with ``connector_config_fields``, and have
the connection announce completion with its ``callback_webhook``.
A link minted while the connection awaits setup (a draft) stops working once it is set up.
A link minted for a connection already in use updates it: it keeps working until it expires,
is deleted or the connection is archived, and a write through it must leave every required
field filled.
The token is returned in clear text only when it is created, and is redacted on every
later read. Managing setup links requires manager access on the connection — the token is
a config-write capability, so listing links is as sensitive as creating one.
# List Setup Tokens
Source: https://docs.nekt.com/platform-api/mcp-connections/setup-tokens/list
GET /api/v1/live-connections/{live_connection_slug}/setup-tokens/
Setup links for a connection, so a third party fills in credentials in their own browser.
Create one to get a token, then send its recipient to ``{WEB_APP_URL}/lcl/{token}``: they
configure the connection without a Nekt account, and the credentials never pass through
you. Optionally restrict what the link exposes with ``connector_config_fields``, and have
the connection announce completion with its ``callback_webhook``.
A link minted while the connection awaits setup (a draft) stops working once it is set up.
A link minted for a connection already in use updates it: it keeps working until it expires,
is deleted or the connection is archived, and a write through it must leave every required
field filled.
The token is returned in clear text only when it is created, and is redacted on every
later read. Managing setup links requires manager access on the connection — the token is
a config-write capability, so listing links is as sensitive as creating one.
# List Connection Tools
Source: https://docs.nekt.com/platform-api/mcp-connections/tools
GET /api/v1/live-connections/{slug}/tools/
List the connection's tools (GET), or edit its available set (PATCH). The available
set is streams-style — a tool is either in it or gone, there is no per-tool status.
GET returns the caller's view: managers/bypassed principals (and flag-off orgs) get
the full snapshot with the transitional ``enabled`` flag (the management view);
everyone else gets only the tools their accesses give them (spec §5.7).
PATCH accepts the declarative replace ``{tools: [names]}`` (the full desired set,
mirror of stream selection) or, transitionally until the FE ships, the legacy
``{name, enabled}`` toggle mapped onto the set. Writes serialize on the connection
row lock, and removals cascade into existing access tool scopes (§5.3) — re-adding
a tool later does not restore explicitly-scoped accesses.
# Unarchive MCP Connection
Source: https://docs.nekt.com/platform-api/mcp-connections/unarchive
POST /api/v1/live-connections/{slug}/unarchive/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# Update MCP Connection
Source: https://docs.nekt.com/platform-api/mcp-connections/update
PATCH /api/v1/live-connections/{slug}/
Connections to external systems that AI agents can call tools against.
A connection is created as a draft pinned to the connector's current version. Fill its
credentials directly with ``config``, or mint a setup link and let a third party fill
them in their own browser; then ``PATCH {"active": true}`` to put it in service. Each
connector may hold several connections, one per account you connect, each with its own
slug — which is also the namespace its tools are exposed under.
On plans with access control, visibility and writes follow the access granted on each
connection: viewers see it, editors edit its description and tags, managers change
everything else — configuration, state, slug, the exposed tools, setup links and
disconnection. Admins and owners always have full access.
# Update Connection Tools
Source: https://docs.nekt.com/platform-api/mcp-connections/update-tools
PATCH /api/v1/live-connections/{slug}/tools/
List the connection's tools (GET), or edit its available set (PATCH). The available
set is streams-style — a tool is either in it or gone, there is no per-tool status.
GET returns the caller's view: managers/bypassed principals (and flag-off orgs) get
the full snapshot with the transitional ``enabled`` flag (the management view);
everyone else gets only the tools their accesses give them (spec §5.7).
PATCH accepts the declarative replace ``{tools: [names]}`` (the full desired set,
mirror of stream selection) or, transitionally until the FE ships, the legacy
``{name, enabled}`` toggle mapped onto the set. Writes serialize on the connection
row lock, and removals cascade into existing access tool scopes (§5.3) — re-adding
a tool later does not restore explicitly-scoped accesses.
# Create MCP Token
Source: https://docs.nekt.com/platform-api/mcp-tokens/create
POST /api/v1/mcp/tokens/
Mint a token for an assistant or automation with an API key.
The token is created **as the person who created the API key**. A full access token follows that
person's permissions as they change; a scoped one may only name resources they can reach.
```bash theme={null}
curl --request POST \
--url "https://api.nekt.ai/api/v1/mcp/tokens/" \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"description": "n8n reporting agent",
"all_tables": true,
"all_semantic_layer": true,
"tool_scope": ["get_semantic_context", "list_tables", "get_relevant_tables_ddl", "generate_sql", "execute_sql"]
}'
```
This token is scoped because it declares scope fields: it reaches every table and the whole semantic
layer the key's creator can see, and only the five read tools listed. To create a full access token,
send only a `description`. The token expires one year after creation.
## Getting the secret: the reveal link
The response carries **no `bearer_token`**. It carries a `reveal_url` instead:
```json theme={null}
{
"id": "0f9c7e4a-...",
"description": "n8n reporting agent",
"reveal_url": "/api/v1/mcp/tokens/reveal/UGxhdGZvcm0.../",
"...": "..."
}
```
**The reveal link needs a person.** It resolves only in the browser session of the user who
created the API key; calling it with `x-api-key` returns `401`/`403`, and it works once. A
script can therefore mint the token unattended, but a human has to open the link to collect the
secret and paste it into the tool that will use it.
This is deliberate: an MCP token lives up to a year, and returning it in an API response would
put it in whatever log, transcript or webhook payload the caller happens to keep.
## What an API key cannot do
| | |
| - | - |
| **Delete a token** | `403`. Creating and rotating are reversible, deleting is not. Revoke from the [MCP page](https://app.nekt.ai/mcp). |
| **See other people's tokens** | The listing shows only tokens created by the key's own creator, stricter than that person's own session, where an Owner or Admin sees every token in the workspace. |
| **Bypass the creation role** | If the workspace requires Admin to create MCP tokens, a key created by a member gets `403`, exactly as that member would. |
| **Read the scope pickers** | `GET /api/v1/mcp/tokens/available-scopes/` is not available to an API key. Take tool names from the [tools reference](/mcp-server/tools) and resource ids from `/api/v1/tables/`, `/api/v1/secrets/`, `/api/v1/semantic-layer/documents/` and `/api/v1/live-connections/`. |
## Related
* [MCP token object](/platform-api/mcp-tokens/mcp-token): every scope field, and what each one covers.
* [Rotate an MCP token](/platform-api/mcp-tokens/rotate): new secret, same token.
* [Permissions](/mcp-server/permissions): how full access and scoped tokens react to permission changes.
# List MCP Tokens
Source: https://docs.nekt.com/platform-api/mcp-tokens/list
GET /api/v1/mcp/tokens/
The tokens your API key's creator has minted.
Only tokens created by **the person who created this API key** are returned, deliberately
stricter than that same person's session, where an Owner or Admin sees every token in the
workspace. A session lasts a day and passed MFA or SSO; an API key lives until someone revokes it
and usually sits in a config file.
`bearer_token` is never included. Use the listing to audit what exists, when each token was last
used (`last_used_at`, taken from the [token access log](/platform-api/token-access-logs/list)) and
when it expires, then [rotate](/platform-api/mcp-tokens/rotate) or
[edit](/platform-api/mcp-tokens/update) it. To see what a token did, not only when, filter the token
access log by its id.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/mcp/tokens/?expand[]=created_by" \
--header "x-api-key: YOUR_API_KEY"
```
# MCP token object
Source: https://docs.nekt.com/platform-api/mcp-tokens/mcp-token
The credential an AI agent or automation uses to reach your workspace through the
[MCP Server](/mcp-server/introduction). An API key can mint, read, edit and rotate these; see
[Create an MCP token](/platform-api/mcp-tokens/create).
## The fields that decide what it reaches
| Field | |
| - | - |
| `use_created_by_permissions` | **Derived, not chosen.** A token that declares any scope field below is **scoped**: it carries its own selection. A token that declares none is **full access**: it follows the permissions of the person who created the API key, as those change. Sending `true` together with a scope field returns `400`. |
| `tool_scope` | Which MCP tools the token may call. Omit for all of them. Names come from the [tools reference](/mcp-server/tools); a name that is unknown or not available to the key's creator returns `400`. |
| `table_scopes`, `volume_scopes` | Catalog links, as `{"layers": [], "folders": [], "tables": []}`. Linking a layer or folder means everything inside it, now and in the future. |
| `secret_scopes` | Ids of secrets the token may reference. Stored and validated, but no tool a scoped token can reach uses secrets yet, so today this field has no effect. |
| `semantic_layer_scopes` | `{"folders": [], "documents": []}`. A folder covers its whole subtree. |
| `live_connection_scopes` | One entry per live connection, each with an optional `tools` subset. |
| `all_tables`, `all_volumes`, `all_secrets`, `all_semantic_layer`, `all_live_connections` | Take a whole axis instead of selecting within it. On **Starter** and **Free** this is the only form of scoping available; see [Permissions](/mcp-server/permissions#scoping-on-starter-and-free). |
Every axis is independent, and every one is narrowed on each request to what the API key's creator
can still reach. A scoped token can only ever be narrower than that person.
## Other fields
| Field | |
| - | - |
| `expires_at` | When the token stops working. One year after creation by default, and never more than a year. It cannot be changed after creation. |
| `last_used_at` | When the token last made a request, taken from the [token access log](/platform-api/token-access-logs/list). For a token idle for longer than the log keeps, or on the Free plan, where the log does not record, it falls back to the last value recorded before the log existed, or `null` if there is none. |
A token stops working if the person who created it is deactivated or removed from the workspace,
whatever its scope.
`bearer_token` is **never** returned to an API key, on any of these endpoints. The secret is
handed over through a one-time `reveal_url` that only opens in its creator's browser session.
## Related
* [Create an MCP token](/platform-api/mcp-tokens/create) · [Update](/platform-api/mcp-tokens/update) · [Rotate](/platform-api/mcp-tokens/rotate)
* [Permissions](/mcp-server/permissions): full access vs scoped, and how each reacts to permission changes.
* [Tools](/mcp-server/tools): the names `tool_scope` accepts.
# Rotate MCP Token
Source: https://docs.nekt.com/platform-api/mcp-tokens/rotate
POST /api/v1/mcp/tokens/{id}/rotate-credentials/
Issue a new secret for an existing token.
Rotating invalidates the previous secret immediately. Every agent using it, including OAuth
connections built on it, stops working and must be reconnected with the new secret. The
description, scopes, and expiration date are kept.
Two rules worth knowing before you script this:
* **The expiration is preserved.** A rotation through an API key keeps the token's original
deadline rather than restarting the term, so rotating often cannot turn a 1-year token into a
perpetual one.
* **A token cannot rotate itself.** Rotating token A with token A's own credential returns `400`.
An assistant rotating a *different* token on your behalf is fine: that is what this is for.
Like creation, the response carries a [`reveal_url`](/platform-api/mcp-tokens/create#getting-the-secret-the-reveal-link)
and no `bearer_token`, so a person has to open the link to collect the new secret.
```bash theme={null}
curl --request POST \
--url "https://api.nekt.ai/api/v1/mcp/tokens/TOKEN_ID/rotate-credentials/" \
--header "x-api-key: YOUR_API_KEY"
```
# Update MCP Token
Source: https://docs.nekt.com/platform-api/mcp-tokens/update
PATCH /api/v1/mcp/tokens/{id}/
Rename a token or change what it reaches.
Editing is how you **widen** a scoped token: its selection does not grow when its creator gains
access, so adding a table or a tool is an explicit edit. Narrowing happens on its own: access the
creator loses is dropped from the token on its next call.
```bash theme={null}
curl --request PATCH \
--url "https://api.nekt.ai/api/v1/mcp/tokens/TOKEN_ID/" \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"tool_scope": ["list_tables", "execute_sql", "get_semantic_context"]}'
```
The expiration cannot be changed here; it is set when the token is created.
Each edit is recorded in the workspace activity, naming **both** the person behind the key and
the key itself, so an unattended change is not attributed to someone who was asleep.
# Add Group Members
Source: https://docs.nekt.com/platform-api/permission-groups/add-members
POST /api/v1/organization/permission-groups/{permission_group_pk}/users/
Put people in a group, and hand them everything it holds.
Adds users to a group without touching the people already in it. They gain every grant the group holds, on all three permission axes, immediately, with no permission call.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `users` | array of integers | Yes | User ids to add. Ids already in the group are ignored. |
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/users/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"users": [12, 42]}'
```
## Response
`204`, with no body. Read the membership back with [List Members](/platform-api/permission-groups/members) if you need to confirm it.
## Invited users can be added before they accept
This is the middle step of onboarding, and it is why an invitation provisions the user immediately. Invite, group, and the grants are waiting for them the moment they accept.
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": os.environ["NEKT_API_KEY"], "Content-Type": "application/json"}
invite = requests.post(
f"{BASE_URL}/api/v1/invitations/",
headers=headers,
json={"email": "analyst@example.com", "role": "member"},
)
invite.raise_for_status()
# They exist now, pending, and can hold access. Find their user id as a recipient.
recipients = requests.get(
f"{BASE_URL}/api/v1/permission-recipients/",
headers=headers,
params={"expand[]": "users", "search": "analyst@example.com"},
).json()
user_id = recipients["users"][0]["id"]
requests.post(
f"{BASE_URL}/api/v1/organization/permission-groups/GROUP_ID/users/",
headers=headers,
json={"users": [user_id]},
).raise_for_status()
```
Invite as `member`. Owners and Admins bypass data-level permissions entirely, so grouping them changes nothing.
## Errors
| Status | When |
| - | - |
| `400` | A user id belongs to a deactivated member. |
| `400` | The group is the All group, which contains everybody automatically. |
| `403` | Your API key was created by a Member. |
| `404` | No group with that id in your workspace. |
## Related
* [Remove Members](/platform-api/permission-groups/remove-members): the reverse.
* [Update a Group](/platform-api/permission-groups/update): replacing the whole membership instead.
* [Onboarding flow](/platform-api/invitations/overview): invite, group, and grant in one script.
# Create a Permission Group
Source: https://docs.nekt.com/platform-api/permission-groups/create
POST /api/v1/organization/permission-groups/
A new group, optionally with its members already in it.
Creates a permission group. A new group holds no grants; make them with the [Permissions API](/platform-api/permissions/overview) once the group exists.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `name` | string | Yes | Shown wherever the group appears. Pick something a colleague will recognize in a grant listing. |
| `description` | string | No | What the group is for. |
| `users` | array of integers | No | User ids to put in the group immediately. |
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/organization/permission-groups/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"name": "Analysts",
"description": "Read access to the Gold layer",
"users": [12, 42]
}'
```
`users` takes **user ids**, which are integers, not the UUIDs used to identify groups and resources. Get them from [List Recipients](/platform-api/permissions/catalog/recipients).
## Response
`201`, with the [permission group object](/platform-api/permission-groups/permission-group).
```json theme={null}
{
"id": "c1f0a9d4-5b62-4e18-9a73-2d8e4f6b0c51",
"name": "Analysts",
"description": "Read access to the Gold layer",
"all_group": false,
"created_at": "2026-09-25T10:14:02Z",
"updated_at": "2026-09-25T10:14:02Z",
"users": [12, 42]
}
```
## Create a team and grant it access
The two calls that set a team up. After this, onboarding anybody onto that team is one membership call.
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": os.environ["NEKT_API_KEY"], "Content-Type": "application/json"}
group = requests.post(
f"{BASE_URL}/api/v1/organization/permission-groups/",
headers=headers,
json={"name": "Analysts", "description": "Read access to the Gold layer"},
)
group.raise_for_status()
group_id = group.json()["id"]
grant = requests.post(
f"{BASE_URL}/api/v1/permissions/",
headers=headers,
json={
"assignments": [
{"permission_level": "viewer", "groups": [group_id], "layers": ["LAYER_ID"]}
]
},
)
grant.raise_for_status()
```
## Errors
| Status | When |
| - | - |
| `400` | A user id in `users` belongs to a deactivated member. |
| `403` | Your API key was created by a Member. Creating groups requires an Owner's or Admin's key. |
## Related
* [Assign Permissions](/platform-api/permissions/catalog/assign): giving the new group its access.
* [Add Members](/platform-api/permission-groups/add-members): filling it later.
* [Permission groups](/platform-api/permission-groups/overview): why a group instead of direct grants.
# Delete a Permission Group
Source: https://docs.nekt.com/platform-api/permission-groups/delete
DELETE /api/v1/organization/permission-groups/{id}/
Remove a group and every grant that reached people through it.
Deleting a group takes away the access it was giving. Everybody in it loses every grant they held **through** the group, on all three permission axes, immediately. Grants they hold in their own right are untouched. This cannot be undone: recreating the group means granting it again from nothing.
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/ \
--header "x-api-key: YOUR_API_KEY"
```
## Response
`204`, with no body.
## Check what you are about to remove
A group's grants are worth reading before deleting it, because the effect is spread across everybody in it.
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": os.environ["NEKT_API_KEY"]}
group_id = "GROUP_ID"
params = {"expand[]": ["users", "lakehouse_permissions", "object_permissions", "semantic_layer_permissions"]}
group = requests.get(
f"{BASE_URL}/api/v1/organization/permission-groups/{group_id}/",
headers=headers,
params=params,
).json()
print(f"{group['name']} - {len(group['users'])} member(s) would lose:")
for axis in ("lakehouse_permissions", "object_permissions", "semantic_layer_permissions"):
print(f" {axis}: {len(group.get(axis) or [])} grant(s)")
```
## Emptying instead of deleting
If the group might be needed again, [replacing its membership](/platform-api/permission-groups/update) with an empty list has the same effect on access and keeps the group and its grants intact.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"users": []}'
```
## Errors
| Status | When |
| - | - |
| `400` | The group is the All group, which cannot be deleted. |
| `403` | Your API key was created by a Member. |
| `404` | No group with that id in your workspace. |
## Related
* [Update a Group](/platform-api/permission-groups/update): emptying it instead.
* [Revoke Permission](/platform-api/permissions/catalog/revoke): removing one grant rather than the group.
# Retrieve a Permission Group
Source: https://docs.nekt.com/platform-api/permission-groups/details
GET /api/v1/organization/permission-groups/{id}/
One group, with its membership and the grants it holds.
Reads a single group. Expanded, this is the fastest way to answer "what does this team have access to", because a group's grants are listed on all three axes at once.
## Query parameters
| Parameter | Type | Description |
| - | - | - |
| `expand[]` | string | `users`, `user_count`, `lakehouse_permissions`, `object_permissions`, `semantic_layer_permissions`. See [Expanding Responses](/platform-api/expanding-responses). |
## Audit a team's access
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/?expand[]=users&expand[]=lakehouse_permissions&expand[]=object_permissions&expand[]=semantic_layer_permissions" \
--header "x-api-key: YOUR_API_KEY"
```
The three permission fields list only the grants held **directly by the group**. A member of the group may reach more than this through their own grants or another group: a person's effective access is the union of everything that reaches them.
## Response
A [permission group object](/platform-api/permission-groups/permission-group).
```json theme={null}
{
"id": "c1f0a9d4-5b62-4e18-9a73-2d8e4f6b0c51",
"name": "Analysts",
"description": "Read access to the Gold layer",
"all_group": false,
"created_at": "2026-03-02T14:20:55Z",
"updated_at": "2026-09-18T11:47:03Z",
"users": [12, 42]
}
```
## Errors
| Status | When |
| - | - |
| `404` | No group with that id in your workspace. |
## Related
* [List Groups](/platform-api/permission-groups/list): where the id comes from.
* [List Members](/platform-api/permission-groups/members): the membership on its own.
* [Permissions flow](/platform-api/permissions/overview): reading grants by recipient instead.
# List Permission Groups
Source: https://docs.nekt.com/platform-api/permission-groups/list
GET /api/v1/organization/permission-groups/
Every group in your workspace, including the All group.
Returns the groups in your workspace, paginated. This is where a group's `id` comes from when you want to grant to it or change who is in it.
## Query parameters
| Parameter | Type | Description |
| - | - | - |
| `search` | string | Match on group name or description. |
| `users` | integer | Only groups this user belongs to. Takes a user id, not a UUID. |
| `page` | integer | Page number. |
| `page_size` | integer | Results per page. |
| `expand[]` | string | `users`, `user_count`, `lakehouse_permissions`, `object_permissions`, `semantic_layer_permissions`. See [Expanding Responses](/platform-api/expanding-responses). |
`users` answers "which groups is this person in", which is the fastest way to audit somebody's inherited access before you change it.
## Response
A paginated list of [permission group objects](/platform-api/permission-groups/permission-group).
```json theme={null}
{
"count": 2,
"next": null,
"previous": null,
"results": [
{
"id": "b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13",
"name": "All",
"description": "Everyone in the workspace",
"all_group": true,
"created_at": "2026-01-14T09:02:11Z",
"updated_at": "2026-01-14T09:02:11Z",
"users": [7, 12, 42]
},
{
"id": "c1f0a9d4-5b62-4e18-9a73-2d8e4f6b0c51",
"name": "Analysts",
"description": "Read access to the Gold layer",
"all_group": false,
"created_at": "2026-03-02T14:20:55Z",
"updated_at": "2026-09-18T11:47:03Z",
"users": [12, 42]
}
]
}
```
Exactly one group has `all_group: true`, and it contains every member automatically. See [the All group](/platform-api/permission-groups/overview#the-all-group) for what you can and cannot do to it.
## Find a group by name
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": os.environ["NEKT_API_KEY"]}
response = requests.get(
f"{BASE_URL}/api/v1/organization/permission-groups/",
headers=headers,
params={"search": "Analysts"},
)
response.raise_for_status()
for group in response.json()["results"]:
print(f"{group['name']}: {group['id']}")
```
## Related
* [Create a Group](/platform-api/permission-groups/create): when the one you want does not exist.
* [List Members](/platform-api/permission-groups/members): who is in one.
* [List Recipients](/platform-api/permissions/catalog/recipients): the picker view, filtered by what can still be granted.
# List Group Members
Source: https://docs.nekt.com/platform-api/permission-groups/members
GET /api/v1/organization/permission-groups/{permission_group_pk}/users/
Who is in a group, including people who have not accepted yet.
Returns the members of one group. Everybody listed here holds every grant the group holds.
## Response
An array of [user objects](/platform-api/users/user). This endpoint is not paginated.
```json theme={null}
[
{
"id": 12,
"first_name": "Dana",
"last_name": "Reed",
"email": "dana@example.com",
"role": "member",
"is_active": true,
"is_pending": false
},
{
"id": 42,
"first_name": "",
"last_name": "",
"email": "analyst@example.com",
"role": "member",
"is_active": false,
"is_pending": true
}
]
```
**Invited people appear here.** `is_pending: true` with `is_active: false` is somebody who has been invited and has not accepted: they hold the group's grants already, and accepting is what lets them sign in and use them. See [Onboarding flow](/platform-api/invitations/overview).
Deactivated members are not listed. They were switched off deliberately, and their access is gone regardless of what group they were in.
## Related
* [Add Members](/platform-api/permission-groups/add-members): putting somebody in.
* [Remove Members](/platform-api/permission-groups/remove-members): taking them out.
* [Retrieve a Group](/platform-api/permission-groups/details): membership plus the grants the group holds.
# Permission groups
Source: https://docs.nekt.com/platform-api/permission-groups/overview
Grant access to a team once, and manage who is on the team.
A permission group is a named set of people that grants can be made to. Grant a group access to a layer, a secret, or a Semantic Layer folder, and everybody in the group has it, including anybody you add later.
This is what keeps access control proportional to the number of **teams** rather than the number of **people**. Onboard a new analyst by putting them in the Analysts group, and they inherit every grant that group holds, in one call, with no permission API involved.
Managing groups requires an [API key](https://app.nekt.ai/settings/api-keys) created by an **Owner** or an **Admin**. A key created by a Member can read groups but not change them. See [the ceiling rule](/platform-api/permissions/overview#the-ceiling-rule).
***
## A group grant is live, not a snapshot
This is the property worth building on. A grant made to a group is evaluated through the group's membership every time, so:
* **Adding** somebody to the group gives them everything the group holds, immediately.
* **Removing** them takes it all back, immediately.
Neither requires a permission call. You change the membership; the access follows.
A person's effective access is the **union** of every grant that reaches them: their own grants plus every group they are in. Groups add access; they never subtract it.
***
## Three ways to set membership
They all do the same thing. Pick whichever fits the shape of your script.
| When | Use |
| - | - |
| Creating the group with its people already known | `users` on [Create a Group](/platform-api/permission-groups/create) |
| Replacing the whole membership | `users` on [Update a Group](/platform-api/permission-groups/update) |
| Adding or removing a few, leaving the rest alone | [Add Members](/platform-api/permission-groups/add-members) and [Remove Members](/platform-api/permission-groups/remove-members) |
The difference that matters: `users` on the group **replaces** the membership, while the members endpoints **adjust** it. Sending `{"users": [7]}` to a group of twelve leaves one person in it.
***
## Pending invitees count
Somebody who has been invited but has not accepted can be put in a group straight away, and the group's grants attach to them at that moment. That is what lets you finish an onboarding script on the day the contract is signed. See [Onboarding flow](/platform-api/invitations/overview).
***
## The All group
Every workspace has one group named **All**, marked `all_group: true`, which automatically contains every member. You can grant to it like any other group, but you cannot change it:
| Operation | On the All group |
| - | - |
| Grant to it | Works |
| Change its description | Works |
| Add or remove members | `400` (membership is automatic) |
| Rename it | `400` |
| Delete it | `400` |
Use it for access everybody should have. Anything narrower belongs in a group you create.
***
## Typical flow
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/organization/permission-groups/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"name": "Analysts", "description": "Read access to the Gold layer"}'
```
Once, for the whole team. See [Permissions flow](/platform-api/permissions/overview).
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{"permission_level": "viewer", "groups": ["GROUP_ID"], "layers": ["LAYER_ID"]}
]
}'
```
No permission call. The grants from step two reach them the moment they are in.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/users/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"users": [42]}'
```
***
## Related
Granting to a group, on all three axes.
Invite, group, and grant before somebody accepts.
Every field a group returns.
The same groups, managed by hand.
# Permission group object
Source: https://docs.nekt.com/platform-api/permission-groups/permission-group
# Remove Group Members
Source: https://docs.nekt.com/platform-api/permission-groups/remove-members
DELETE /api/v1/organization/permission-groups/{permission_group_pk}/users/
Take people out of a group, and take back what it was giving them.
Removes users from a group, leaving everybody else in it. They lose every grant they held **through** the group, immediately. Grants they hold in their own right are untouched.
The ids go in the **request body**, not the URL: this endpoint removes several people at once and addresses the group, not one member of it.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `users` | array of integers | Yes | User ids to remove. Ids not in the group are ignored. |
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/users/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"users": [42]}'
```
## Response
`204`, with no body.
## Offboarding somebody from a team
Removing them from every group takes back everything they inherited, and is the part of offboarding that is easy to miss when you only revoke direct grants.
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": os.environ["NEKT_API_KEY"], "Content-Type": "application/json"}
user_id = 42
groups = requests.get(
f"{BASE_URL}/api/v1/organization/permission-groups/",
headers=headers,
params={"users": user_id},
).json()
for group in groups["results"]:
if group["all_group"]:
continue # Membership is automatic; it goes when the account is deactivated.
requests.delete(
f"{BASE_URL}/api/v1/organization/permission-groups/{group['id']}/users/",
headers=headers,
json={"users": [user_id]},
).raise_for_status()
print(f"removed from {group['name']}")
```
This removes inherited access only. Grants made directly to the person survive; see [Revoke Permission](/platform-api/permissions/catalog/revoke) for those.
## Errors
| Status | When |
| - | - |
| `400` | The group is the All group, whose membership is automatic. |
| `403` | Your API key was created by a Member. |
| `404` | No group with that id in your workspace. |
## Related
* [Add Members](/platform-api/permission-groups/add-members): the reverse.
* [List Groups](/platform-api/permission-groups/list): with `users`, every group somebody is in.
* [Delete a Group](/platform-api/permission-groups/delete): when the whole team is going away.
# Update a Permission Group
Source: https://docs.nekt.com/platform-api/permission-groups/update
PATCH /api/v1/organization/permission-groups/{id}/
Rename a group, or replace its membership wholesale.
Changes a group's name, description, or membership. Send only the fields you are changing.
## Request body
| Parameter | Type | Description |
| - | - | - |
| `name` | string | The group's name. |
| `description` | string | What the group is for. |
| `users` | array of integers | **Replaces** the entire membership. |
`users` is a replacement, not an addition. Sending `{"users": [7]}` to a group of twelve leaves one person in it, and the other eleven lose every grant that reached them through the group. To add or remove a few people without touching the rest, use [Add Members](/platform-api/permission-groups/add-members) and [Remove Members](/platform-api/permission-groups/remove-members).
## Rename a group
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/organization/permission-groups/GROUP_ID/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"name": "Data Analysts"}'
```
Renaming changes nothing about access. Grants point at the group's id.
## Replace a membership
Useful when your own system is the source of truth for a team and you are syncing it across, rather than tracking individual joins and leaves.
```python theme={null}
import os
import requests
BASE_URL = "https://api.nekt.ai"
headers = {"x-api-key": os.environ["NEKT_API_KEY"], "Content-Type": "application/json"}
# Whoever is on the team right now, per your HR system.
current_team = [12, 42, 57]
response = requests.patch(
f"{BASE_URL}/api/v1/organization/permission-groups/GROUP_ID/",
headers=headers,
json={"users": current_team},
)
response.raise_for_status()
```
Anybody dropped from the list loses the group's grants immediately, and anybody added gains them, with no permission call either way.
## Response
`200`, with the updated [permission group object](/platform-api/permission-groups/permission-group).
## Errors
| Status | When |
| - | - |
| `400` | A user id in `users` belongs to a deactivated member. |
| `400` | You are renaming the All group, or setting its `users`. Its membership is automatic. |
| `403` | Your API key was created by a Member. |
| `404` | No group with that id in your workspace. |
`PUT` is also available and replaces every writable field at once. `PATCH` is what you usually want.
## Related
* [Add Members](/platform-api/permission-groups/add-members): adjusting membership instead of replacing it.
* [The All group](/platform-api/permission-groups/overview#the-all-group): what cannot be changed.
# Assign Permissions
Source: https://docs.nekt.com/platform-api/permissions/catalog/assign
POST /api/v1/permissions/
Grant, change, and revoke Catalog access for users and groups in a single call.
This endpoint is not a plain create. It is a **declarative bulk upsert**: you describe the access you want, and Nekt works out which grants to create, which to change, and which to remove.
Each entry in `assignments` is a **cross product**. Every recipient in `users` and `groups` is paired with every resource in `layers`, `folders`, `tables`, and `volumes`, and each resulting pair gets `permission_level`. Two users and three tables in one assignment is six grants.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `assignments` | array of objects | Yes | One or more assignment blocks. Must not be empty. |
| `notify_members` | boolean | No | Accepted for compatibility. See the note below. |
| `message` | string | No | Accepted for compatibility. See the note below. |
Each object in `assignments`:
| Parameter | Type | Required | Description |
| - | - | - | - |
| `permission_level` | `viewer`, `editor`, `manager`, or `null` | Yes | The level to apply. **`null` revokes** any existing grant for the pair. |
| `users` | array of integers | No | Recipient user ids. Must be active or pending members of your workspace. Owners and Admins are accepted, but the grant has no effect on them. |
| `groups` | array of UUIDs | No | Recipient permission groups. |
| `layers` | array of UUIDs | No | Target layers. |
| `folders` | array of UUIDs | No | Target folders. |
| `tables` | array of UUIDs | No | Target tables. |
| `volumes` | array of UUIDs | No | Target volumes. |
Recipient ids come from [List Permission Recipients](/platform-api/permissions/catalog/recipients). Resource ids come from [List Layers](/platform-api/catalog/list-layers), the folders, tables, and volumes endpoints.
`notify_members` and `message` are accepted by this endpoint but **do not send email**. They are carried over from an earlier version of the API. Grants take effect regardless; recipients are simply not notified.
## Response
Returns `201 Created`:
```json theme={null}
{
"created": [ { "id": "…", "permission_level": "viewer", "user": 42, "table": "…" } ],
"updated": [],
"revoked": []
}
```
Each list holds full [permission objects](/platform-api/permissions/catalog/permission). A pair that already had the level you asked for appears in none of them: the call is idempotent.
## Grant one level on one table
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "viewer",
"users": [42],
"tables": ["a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02"]
}
]
}'
```
## Onboard a group across a whole layer
Granting on a layer covers the folders, tables, and volumes inside it. You do not need to enumerate them.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "editor",
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"layers": ["e4d5c6b7-a8b9-40c1-d2e3-f4a5b6c7d8e9"]
}
]
}'
```
## Grant and revoke in one request
Different levels need separate assignment blocks. This is how you move someone up on one resource and off another atomically.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "editor",
"users": [42],
"tables": ["a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02"]
},
{
"permission_level": null,
"users": [42],
"tables": ["f3e4d5c6-b7a8-49e0-1f2a-3b4c5d6e7f80"]
}
]
}'
```
Two assignments that target the same recipient and the same resource with **different** levels are rejected with `400` naming the conflicting positions. Repeating the same pair at the same level is fine and simply ignored.
## Python: mirror a group's access onto a new member
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
NEW_USER = 42
REFERENCE_GROUP = "b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"
headers = {"x-api-key": API_KEY, "Content-Type": "application/json"}
# The list is paginated (20 per page by default, 100 at most), so follow `next`.
grants = []
url = f"{BASE_URL}/api/v1/permissions/"
params = {"group": REFERENCE_GROUP, "page_size": 100}
while url:
page = requests.get(url, headers=headers, params=params).json()
grants.extend(page["results"])
url, params = page["next"], None
# One assignment block per level, so each block carries a single permission_level.
by_level = {}
for grant in grants:
block = by_level.setdefault(grant["permission_level"], {"layers": [], "folders": [], "tables": [], "volumes": []})
for resource in ("layer", "folder", "table", "volume"):
if grant[resource]:
block[f"{resource}s"].append(grant[resource])
requests.post(
f"{BASE_URL}/api/v1/permissions/",
headers=headers,
json={
"assignments": [
{"permission_level": level, "users": [NEW_USER], **resources}
for level, resources in by_level.items()
],
},
)
```
## Errors
| Status | When |
| - | - |
| `400` | Two assignments conflict on the same recipient and resource, a recipient's membership is deactivated, or a referenced resource is not visible to the key's creator. |
| `403` | Your workspace is not on Growth or Custom, or the key's ceiling does not allow the grant. See [Permissions flow](/platform-api/permissions/overview). |
## Related
* [Permissions flow](/platform-api/permissions/overview): the end-to-end walkthrough, ceiling rules, and error semantics.
* [List Permission Recipients](/platform-api/permissions/catalog/recipients): where recipient ids come from.
* [Assign Object Permissions](/platform-api/permissions/objects/assign): the same shape for secrets and live connections.
# Retrieve Permission
Source: https://docs.nekt.com/platform-api/permissions/catalog/details
GET /api/v1/permissions/{id}/
# List Permissions
Source: https://docs.nekt.com/platform-api/permissions/catalog/list
GET /api/v1/permissions/
# Permission object
Source: https://docs.nekt.com/platform-api/permissions/catalog/permission
A single Catalog grant: one recipient, one resource, one level.
A Catalog grant is always **one recipient on one resource at one level**. There is no row that means "these three tables": [Assign Permissions](/platform-api/permissions/catalog/assign) expands a bulk request into individual grants, and each one is an object of this shape.
| Field | Type | Notes |
| - | - | - |
| `id` | UUID | |
| `permission_level` | `viewer` \| `editor` \| `manager` | The only mutable field. |
| `user` | integer | Set when the recipient is a person. Expandable. |
| `group` | UUID | Set when the recipient is a permission group. Expandable. |
| `layer` | UUID | Expandable. |
| `folder` | UUID | Expandable. |
| `table` | UUID | Expandable. |
| `volume` | UUID | Expandable. |
| `granted_by` | integer | The user who created the grant or last changed its level. For an API key, its creator. Expandable. |
| `created_at` | datetime | |
| `updated_at` | datetime | |
**Exactly one of `user` / `group` is set, and exactly one of `layer` / `folder` / `table` / `volume`.** The rest are `null`. That is a database constraint, not a convention, so you can branch on it safely.
Any field marked expandable can be inflated into the full object with `?expand[]=`; see [Expanding Responses](/platform-api/expanding-responses).
Field types and formats are also rendered on the operation pages, for example under the response of [List Permissions](/platform-api/permissions/catalog/list).
# List Permission Recipients
Source: https://docs.nekt.com/platform-api/permissions/catalog/recipients
GET /api/v1/permission-recipients/
The users and groups that can receive a Catalog grant.
Assigning a permission requires the integer id of a user or the UUID of a permission group, and this is the endpoint that provides them, filtered to who can still be granted on a given resource. To read or manage groups themselves, use [Permission groups](/platform-api/permission-groups/overview).
## Only members are returned
The response contains **active and pending Members**. A pending member is someone who has been [invited](/platform-api/invitations/invite) but has not yet accepted: you can grant them access ahead of time, and it takes effect once they join. Owners and Admins are deliberately absent: they bypass data-level permissions entirely, so a grant to them would change nothing. If you are looking for a colleague and cannot find them here, check their role first.
Deactivated members are absent too. They were switched off on purpose, and a grant to them would be access nobody asked to restore.
## Query parameters
| Parameter | Type | Description |
| - | - | - |
| `layer` | UUID | Recipients who do **not** already hold a direct grant on this layer. |
| `folder` | UUID | Recipients who do **not** already hold a direct grant on this folder. |
| `table` | UUID | Recipients who do **not** already hold a direct grant on this table. |
| `volume` | UUID | Recipients who do **not** already hold a direct grant on this volume. |
| `search` | string | Match users by name, username, or email, and groups by name or description. |
| `expand[]` | string | `users`, `groups`, or both. See [Expanding Responses](/platform-api/expanding-responses). |
Passing a resource **excludes recipients holding a direct grant on that exact resource**. Access
inherited through a group or a parent layer or folder does not exclude them. This is picker
semantics: the list answers "who can still be added", never "who has access". To read existing
access, use [List Permissions](/platform-api/permissions/catalog/list) instead. Without a resource
parameter you get every eligible member and group.
A resource id that does not exist, or that your API key cannot see, returns `400` rather than an empty list, so a typo fails loudly instead of looking like "nobody available".
## Response
By default the endpoint returns bare id lists:
```json theme={null}
{
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"users": [42]
}
```
Pass `expand[]` to get full objects instead, which is what you want when you need to match people by email:
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/permission-recipients/?expand[]=users&expand[]=groups" \
--header "x-api-key: YOUR_API_KEY"
```
## Find a person by email, then grant
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
TARGET_EMAIL = "analyst@example.com"
TABLE_ID = "a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02"
headers = {"x-api-key": API_KEY, "Content-Type": "application/json"}
recipients = requests.get(
f"{BASE_URL}/api/v1/permission-recipients/",
headers=headers,
params={"expand[]": "users", "search": TARGET_EMAIL},
).json()
if not recipients["users"]:
raise SystemExit(f"{TARGET_EMAIL} is not an eligible grant recipient in this workspace.")
requests.post(
f"{BASE_URL}/api/v1/permissions/",
headers=headers,
json={
"assignments": [
{
"permission_level": "viewer",
"users": [recipients["users"][0]["id"]],
"tables": [TABLE_ID],
}
]
},
)
```
## Related
* [Assign Permissions](/platform-api/permissions/catalog/assign): what to do with the ids.
* [List Object Permission Recipients](/platform-api/permissions/objects/recipients): the equivalent for secrets and live connections.
# Revoke Permission
Source: https://docs.nekt.com/platform-api/permissions/catalog/revoke
DELETE /api/v1/permissions/{id}/
Remove a single grant.
Revoking is a soft delete. The grant stops taking effect immediately and disappears from [List Permissions](/platform-api/permissions/catalog/list), but the row is retained with the actor and timestamp for the audit trail.
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/permissions/6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e/ \
--header "x-api-key: YOUR_API_KEY"
```
Returns `204 No Content`.
Revoking a Catalog grant also removes the cloud-level data access derived from it, asynchronously and normally within seconds, on workspaces where cloud-level data-access enforcement is active. The member then loses the ability to query the affected tables, not just the ability to see them in the app.
## Who can call it
An API key created by an Owner or Admin can revoke any grant. A key created by a Member can only revoke `viewer` and `editor` grants, and only on a resource where that Member holds `manager`.
## Related
* [Assign Permissions](/platform-api/permissions/catalog/assign): revoke in bulk by sending `permission_level: null`.
# Update Permission
Source: https://docs.nekt.com/platform-api/permissions/catalog/update
PATCH /api/v1/permissions/{id}/
Change the level of a single existing grant.
`permission_level` is the only writable field. The recipient and the resource are fixed; `granted_by` is updated to whoever changes the level. To move a grant to a different resource or recipient, revoke it and assign a new one.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/permissions/6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"permission_level": "editor"}'
```
[Assign Permissions](/platform-api/permissions/catalog/assign) can do the same thing in bulk and without knowing the grant id: it matches on recipient and resource. Use this endpoint when you already hold the id, for example from [List Permissions](/platform-api/permissions/catalog/list).
## Who can call it
An API key created by an Owner or Admin can change any grant. A key created by a Member can only move a grant between `viewer` and `editor`, and only on a resource where that Member holds `manager`. Anything else returns `403`.
## Related
* [Revoke Permission](/platform-api/permissions/catalog/revoke): remove the grant instead of changing it.
* [Permissions flow](/platform-api/permissions/overview): the ceiling rules in full.
# Assign Object Permissions
Source: https://docs.nekt.com/platform-api/permissions/objects/assign
POST /api/v1/organization/object-permissions/
Grant, change, and revoke access to secrets and live connections in a single call.
The same declarative bulk upsert as [Assign Permissions](/platform-api/permissions/catalog/assign), applied to the resources that live outside the Catalog: **secrets** and **live connections**. Each assignment is a cross product of recipients and resources, and `permission_level: null` revokes.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `assignments` | array of objects | Yes | One or more assignment blocks. Must not be empty. |
| `notify_members` | boolean | No | Accepted for compatibility. See the note below. |
| `message` | string | No | Accepted for compatibility. See the note below. |
`notify_members` and `message` are accepted by this endpoint but **do not send email**. Grants take effect regardless; recipients are simply not notified.
Each object in `assignments`:
| Parameter | Type | Required | Description |
| - | - | - | - |
| `permission_level` | `viewer`, `editor`, `manager`, or `null` | Yes | The level to apply. `null` revokes. |
| `users` | array of integers | No | Recipient user ids. |
| `groups` | array of UUIDs | No | Recipient permission groups. |
| `secrets` | array of UUIDs | No | Target secrets. |
| `live_connections` | array of UUIDs | No | Target live connections. |
| `tools` | array of strings | No | **Live connections only.** Restrict the grant to a subset of the connection's available tools. Omit for the full set; `[]` grants no tools. |
## What the levels mean here
Secrets and live connections do not use the Catalog meaning of the levels. Secret grants are enforced once secret permissions are enabled for your workspace.
| Level | Secret | Live connection |
| - | - | - |
| `viewer` | See the metadata and reference the secret in Queries and Notebooks. | Use the connection's tools. |
| `editor` | Also edit the description and rotate the value. | Also edit the description and tags. Nothing else. |
| `manager` | Also manage access and delete the secret. | Also change config, status, slug, tools, and setup links, and delete. |
The value of a secret is never readable through the API at any level, including `manager`. Only pipeline and notebook runtimes resolve it.
## Share a secret with a group
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/organization/object-permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "viewer",
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"secrets": ["c9d0e1f2-a3b4-45c6-87d8-e9f0a1b2c3d4"]
}
]
}'
```
## Scope a live connection to specific tools
`tools` narrows what the recipient may call. It is validated against each connection's **current available set**: a name the connection does not expose is rejected with `Connection '' has no available tools named: …`.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/organization/object-permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "viewer",
"users": [42],
"live_connections": ["d1e2f3a4-b5c6-47d8-99e0-f1a2b3c4d5e6"],
"tools": ["search_contacts", "get_contact"]
}
]
}'
```
Three rules govern `tools`, and all three return `400` when broken:
* It requires at least one live connection in the same assignment. A block with `tools` and only `secrets` is an error.
* It cannot be combined with `manager`, since a manager always holds the full set.
* Every name must be currently available on **every** connection in the same assignment.
Omitting `tools` is not the same as listing every tool. Omitted means "whatever is available", and it keeps tracking the connection as tools are added or removed. An explicit list is a fixed subset, and `tools: []` grants no tools at all. Re-assigning an existing scoped grant without `tools` resets it to the full set.
Removing a tool from a live connection's available set strips it from every grant that named it. Adding the tool back later does **not** restore those grants: you have to reassign them.
## Errors
| Status | When |
| - | - |
| `400` | A `tools` rule was broken, a recipient's membership is deactivated, or a referenced resource is not visible to the key's creator. |
| `403` | The key's ceiling does not allow the grant. See [Permissions flow](/platform-api/permissions/overview). |
## Related
* [Permissions flow](/platform-api/permissions/overview): the end-to-end walkthrough and the ceiling rules.
* [List Object Permission Recipients](/platform-api/permissions/objects/recipients): where recipient ids come from.
* [Secrets](/workspace/secrets): what a secret is and how levels behave in the app.
# Retrieve Object Permission
Source: https://docs.nekt.com/platform-api/permissions/objects/details
GET /api/v1/organization/object-permissions/{id}/
Grants CRUD for the "Others" axis (secrets), mirroring LakehousePermissionsViewSet.
The recipients listing lives in object_permission_recipients.py, mirroring the
lakehouse permission.py / permission_recipients.py split.
# List Object Permissions
Source: https://docs.nekt.com/platform-api/permissions/objects/list
GET /api/v1/organization/object-permissions/
Browse grants on secrets and live connections.
Filter with `resource_type=secret` or `resource_type=live_connection` to look at one axis at a time, or with `secret`, `live_connection`, `user`, `group`, and `permission_level` to narrow further. `search` matches a secret's key or a live connection's slug.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/organization/object-permissions/?resource_type=secret&expand[]=user&expand[]=secret" \
--header "x-api-key: YOUR_API_KEY"
```
Revoked grants are never returned.
# Object permission object
Source: https://docs.nekt.com/platform-api/permissions/objects/object-permission
A single grant on a secret or a live connection.
The shape of a grant on the axis that sits outside the Catalog. Same skeleton as the [Catalog permission object](/platform-api/permissions/catalog/permission), plus one field the Catalog does not have.
| Field | Type | Notes |
| - | - | - |
| `id` | UUID | |
| `permission_level` | `viewer` \| `editor` \| `manager` | |
| `tool_scope` | array of strings, or `null` | **Live connections only.** `null` means the connection's full available set, and keeps tracking it as tools change. |
| `user` | integer | Set when the recipient is a person. Expandable. |
| `group` | UUID | Expandable. |
| `secret` | UUID | Expandable. |
| `live_connection` | UUID | Expandable. |
| `granted_by` | integer | The user who created the grant or last changed it. For an API key, its creator. Expandable. |
| `created_at` | datetime | |
| `updated_at` | datetime | |
Exactly one of `user` / `group` is set, and exactly one of `secret` / `live_connection`.
A `manager` grant never carries a `tool_scope`, since a manager holds the full set by definition. Reading `tool_scope: null` on a `manager` row therefore tells you nothing about intent.
# List Object Permission Recipients
Source: https://docs.nekt.com/platform-api/permissions/objects/recipients
GET /api/v1/organization/object-permission-recipients/
The users and groups that can receive a secret or live connection grant.
The counterpart of [List Permission Recipients](/platform-api/permissions/catalog/recipients) for the secrets and live-connection axis, with the same contract: **active and pending Members** (Owners and Admins excluded), ids by default, full objects with `expand[]`, and picker semantics. Passing a resource excludes whoever holds a direct grant on that exact resource. Access received through a group does not exclude them.
## Query parameters
| Parameter | Type | Description |
| - | - | - |
| `secret` | UUID | Recipients who do **not** already hold a direct grant on this secret. |
| `live_connection` | UUID | Recipients who do **not** already hold a direct grant on this live connection. |
| `search` | string | Match users by name, username, or email, and groups by name or description. |
| `expand[]` | string | `users`, `groups`, or both. See [Expanding Responses](/platform-api/expanding-responses). |
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/organization/object-permission-recipients/?expand[]=users" \
--header "x-api-key: YOUR_API_KEY"
```
```json theme={null}
{
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"users": [42]
}
```
# Revoke Object Permission
Source: https://docs.nekt.com/platform-api/permissions/objects/revoke
DELETE /api/v1/organization/object-permissions/{id}/
Remove a single grant on a secret or live connection.
Revoking is a soft delete: the grant stops applying at once and leaves the listing, and the row is kept with the actor and timestamp for the audit trail. Returns `204 No Content`.
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/organization/object-permissions/6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e/ \
--header "x-api-key: YOUR_API_KEY"
```
Revoking a secret grant removes the member's ability to reference that secret in new Queries and Notebooks. Transformations that already list it as an input keep the reference; they are not rewritten.
# Update Object Permission
Source: https://docs.nekt.com/platform-api/permissions/objects/update
PATCH /api/v1/organization/object-permissions/{id}/
Change the level or the tool scope of a single grant.
Two fields are writable: `permission_level`, and `tool_scope` for live-connection grants.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/organization/object-permissions/6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"tool_scope": ["search_contacts", "get_contact"]}'
```
Raising a live-connection grant to `manager` clears its `tool_scope`. A manager sees the connection's full available tool set by definition, so the two cannot coexist: sending both in the same request returns `400`.
Setting `tool_scope` on a secret grant returns `400`: the field applies to live connections only.
## Related
* [Permissions flow](/platform-api/permissions/overview): where tool scoping fits in the wider model.
# Permissions flow
Source: https://docs.nekt.com/platform-api/permissions/overview
Grant and revoke access to data, secrets, and live connections with an API key.
The Permissions API lets you drive access control from your own systems instead of the app: provisioning a new analyst from your HR tool, mirroring a group's access onto a new hire, or revoking everything the moment someone leaves.
Access in Nekt has three axes. They are separate endpoints because they scope different kinds of thing, but they share one request shape, so learning one teaches all three.
| Axis | Resources | Endpoint |
| - | - | - |
| **Catalog** | Layers, folders, tables, and volumes | `/api/v1/permissions/` |
| **Secrets and live connections** | Secrets, live connections | `/api/v1/organization/object-permissions/` |
| **Semantic Layer** | Catalog root, folders, documents | `/api/v1/semantic-layer/permissions/` |
All three share the same three levels (`viewer`, `editor`, `manager`) and the same `assignments` payload. What a level *means* differs per resource: see [Permissions](/workspace/permissions) for the Catalog axis and [Assign Object Permissions](/platform-api/permissions/objects/assign) for secrets and live connections.
Data-level permissions apply on **Growth and Custom** plans, and the plan gate is not symmetric across the axes. Below Growth, `/api/v1/permissions/` and `/api/v1/semantic-layer/permissions/` return `403`. On secrets and live connections, grants can be written but do not restrict anything: every member still sees everything until the plan enforces. The recipients endpoints answer on every plan.
***
## Before you start
Permissions are granted to **Members**. Owners and Admins bypass data-level access entirely, so there is nothing to grant them, which is why they never appear in the recipients endpoints.
You will need an [API key](https://app.nekt.ai/settings/api-keys), and it matters who created it. Read the ceiling rule below before you build anything.
***
## The ceiling rule
An API key is not an all-powerful credential. **It acts as the person who created it**, carrying their id and their role.
A key created by a **Member** can only grant `viewer` and `editor`, and only on resources where that Member holds `manager`. It cannot create or remove `manager` grants at all. A key created by an **Owner** or **Admin** has no such limit. This ceiling applies on Growth and Custom. Below Growth, grants do not restrict anything, so a Member's key can write secret and live connection grants without it.
This is the same rule the app applies to that person, with no separate concept for machines, so the fastest way to reason about what your key can do is to ask what its creator can do in the UI. It also means a key does not outlive its creator's authority: demote the person and the key narrows with them.
If you are automating workspace-wide provisioning, create the key as an Owner or Admin. If you are letting a team self-serve within their own domain, a Member's key is the right ceiling.
***
## The flow
Grants target a user by integer id or a permission group by UUID. The recipients endpoint is where user ids come from; group ids also come from [List Groups](/platform-api/permission-groups/list).
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/permission-recipients/?expand[]=users&search=analyst@example.com" \
--header "x-api-key: YOUR_API_KEY"
```
An empty `users` list usually means the person is an Owner or Admin, or is not a member of the workspace (active or pending), not that the search failed.
Catalog resources come from [List Layers](/platform-api/catalog/list-layers) and the folders, tables, and volumes endpoints. Granting on a **layer** covers everything inside it, so prefer the coarsest resource that expresses your intent.
One call can create, change, and revoke at once. Each assignment block pairs every recipient with every resource and applies one level.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "viewer",
"users": [42],
"layers": ["e4d5c6b7-a8b9-40c1-d2e3-f4a5b6c7d8e9"]
}
]
}'
```
The response tells you exactly what changed:
```json theme={null}
{ "created": [ … ], "updated": [], "revoked": [] }
```
List the grants back, filtered by recipient. `indirect_user` includes grants the person receives through a group, which is what you usually want when auditing what someone can actually reach.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/permissions/?indirect_user=42&expand[]=table&expand[]=layer" \
--header "x-api-key: YOUR_API_KEY"
```
Two ways, and they differ in what you need to know beforehand. Send `permission_level: null` through the assign endpoint to revoke by recipient and resource, or `DELETE` the grant if you already hold its id.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": null,
"users": [42],
"layers": ["e4d5c6b7-a8b9-40c1-d2e3-f4a5b6c7d8e9"]
}
]
}'
```
Revoking is a soft delete: the grant stops applying immediately and leaves the listing, and the row is retained with the actor and timestamp for the audit trail.
***
## How an assignment resolves
Three behaviors surprise people, and all three are deliberate.
**It is an upsert, not a create.** You describe the access you want; Nekt computes the difference against what exists. Sending the same request twice is safe: the second call reports nothing created, updated, or revoked.
**Each block is a cross product.** Two users and three tables in one assignment produce six grants. To apply different levels, use separate blocks.
**`null` is a verb.** `permission_level: null` revokes the pair rather than being a no-op or an error.
`notify_members` and `message` are accepted on every axis but **no email is sent**. Recipients are not notified of new or changed grants.
Two blocks that name the same recipient and the same resource with **different** levels are rejected with `400`, naming the conflicting positions. Repeating the same pair at the same level is fine.
***
## Group grants versus direct grants
A recipient can be a user or a permission group, and a person's effective access is the **union** of every grant that reaches them. A group grant is not a snapshot: adding someone to the group later gives them that access, and removing them takes it away, with no permission call at all.
Managing a stable team through a group and reserving direct grants for exceptions keeps the number of API calls proportional to the number of teams rather than the number of people.
***
## Errors
| Status | Meaning | What to check |
| - | - | - |
| `400` | The request describes something impossible. | Conflicting blocks, a deactivated member, or a `tools` rule on the other axis. A resource the key's creator cannot see also returns `400`, as if it did not exist. |
| `403` | The plan or the ceiling refuses. | Your organization is on Free or Starter (Catalog and Semantic Layer), or the key's creator lacks `manager` on the resource, or is trying to touch a `manager` grant as a Member. |
| `404` | The grant id does not exist, or is already revoked. | Revoked grants are excluded from reads, so a second `DELETE` returns `404`. |
On assign, the `403` body names the rule that refused, for example `Member users cannot assign manager permissions.` Read the body before assuming the key is broken. Update, revoke, and the plan gate return the generic `You do not have permission to perform this action.`
***
## Reference
Grant, change, and revoke Catalog access in bulk.
The users and groups that can receive a grant.
The same flow for secrets and live connections.
The same flow for catalogs, folders, and documents.
Grant a team once, and manage who is on the team.
What each level allows, module by module.
# Assign Semantic Layer Permissions
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/assign
POST /api/v1/semantic-layer/permissions/
Grant, change, and revoke access to Semantic Layer catalogs, folders, and documents.
The same declarative bulk upsert as the other two axes, applied to the Semantic Layer. Each assignment is a cross product of recipients and resources, and `permission_level: null` revokes.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `assignments` | array of objects | Yes | One or more assignment blocks. Must not be empty. |
| `notify_members` | boolean | No | Accepted for compatibility. See the note below. |
| `message` | string | No | Accepted for compatibility. See the note below. |
Each object in `assignments`:
| Parameter | Type | Required | Description |
| - | - | - | - |
| `permission_level` | `viewer`, `editor`, `manager`, or `null` | Yes | The level to apply. `null` revokes. |
| `users` | array of integers | No | Recipient user ids. |
| `groups` | array of UUIDs | No | Recipient permission groups. |
| `catalogs` | array of UUIDs | No | Target the Semantic Layer root. |
| `folders` | array of UUIDs | No | Target folders. |
| `documents` | array of UUIDs | No | Target documents. |
`notify_members` and `message` are accepted by this endpoint but **do not send email**. Grants take effect regardless; recipients are simply not notified.
## The three resource levels
`catalogs`, `folders`, and `documents` are the same hierarchy the Semantic Layer shows in the app, from coarsest to finest. A grant on the catalog root reaches everything. A folder grant applies to the folder's whole subtree, and gives Viewer on every ancestor folder and on the root. Prefer the coarsest resource that expresses the intent: it survives documents being added later.
Resource ids come from [Read the Semantic Layer](/platform-api/permissions/semantic-layer/resources).
## Grant a group access to one folder
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/semantic-layer/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "viewer",
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"folders": ["a1b2c3d4-e5f6-4718-9a0b-1c2d3e4f5061"]
}
]
}'
```
## Open the whole Semantic Layer to a group
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/semantic-layer/permissions/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"assignments": [
{
"permission_level": "viewer",
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"catalogs": ["7d8e9f0a-1b2c-43d4-85e6-f708192a3b4c"]
}
]
}'
```
## Response
```json theme={null}
{ "created": [ … ], "updated": [], "revoked": [] }
```
## Errors
| Status | When |
| - | - |
| `400` | Two assignments name the same recipient and resource at different levels (the error lists the positions), a recipient's membership is deactivated, or a referenced id is not visible to your API key. |
| `403` | The key's creator is a Member and lacks Manager on a named resource, assigns `manager`, or touches an existing Manager grant. Also returned on Free and Starter, where this API is not available. See [Permissions flow](/platform-api/permissions/overview). |
## Related
* [Permissions flow](/platform-api/permissions/overview): the ceiling rules and the upsert semantics in full.
* [List Recipients](/platform-api/permissions/semantic-layer/recipients): where recipient ids come from.
* [Read the Semantic Layer](/platform-api/permissions/semantic-layer/resources): where resource ids come from.
# Retrieve Semantic Layer Permission
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/details
GET /api/v1/semantic-layer/permissions/{id}/
Grants CRUD, mirroring LakehousePermissionsViewSet.
# List Semantic Layer Permissions
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/list
GET /api/v1/semantic-layer/permissions/
Browse grants on the Semantic Layer.
Filter by `catalog`, `folder`, `document`, `user`, `group`, or `permission_level`.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/semantic-layer/permissions/?expand[]=user&expand[]=document" \
--header "x-api-key: YOUR_API_KEY"
```
Revoked grants are never returned.
The Semantic Layer permissions API is available on Growth and Custom. On Free and Starter, permissions are not enforced and this endpoint returns `403`.
# List Semantic Layer Folders
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/list-folders
GET /api/v1/semantic-layer/folders/
The folders you can see and grant on.
By default only top-level folders are returned; pass `?flat=true` to list every folder you can see, at any depth. Results are paginated (20 per page) and already filtered to what your API key's creator can reach: anything returned is a folder you may name in an [assignment](/platform-api/permissions/semantic-layer/assign).
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/semantic-layer/folders/?flat=true" \
--header "x-api-key: YOUR_API_KEY"
```
A folder grant applies to the folder's whole subtree, and gives Viewer on every ancestor folder and on the root, so this is usually the level you want rather than granting document by document.
Read-only for API keys, whatever the creator's role. Creating, renaming, or deleting a folder returns `403`; moving and sorting items are not available to API keys and return `401`. Manage the folder tree in the app.
## Related
* [Read the Semantic Layer](/platform-api/permissions/semantic-layer/resources): the three read endpoints and how they fit together.
* [List Semantic Layer Documents](/platform-api/semantic-layer/list): the level below.
# List Semantic Layer Permission Recipients
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/recipients
GET /api/v1/semantic-layer/permission-recipients/
The users and groups that can still be added to a Semantic Layer resource.
Same contract as the other two axes: **active and pending Members** (Owners and Admins excluded), ids by default, full objects with `expand[]`, and picker semantics: a recipient who holds a direct grant on the exact resource you name is omitted, so the list never offers a duplicate. Access inherited through a group or a parent folder does not exclude a recipient. User ids are integers; group ids are UUIDs. This endpoint answers on every plan.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/semantic-layer/permission-recipients/?expand[]=users" \
--header "x-api-key: YOUR_API_KEY"
```
```json theme={null}
{
"groups": ["b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"],
"users": [42]
}
```
Owners and Admins are absent by design: they bypass the Semantic Layer permission axis entirely, so a grant to them would change nothing.
# Read the Semantic Layer
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/resources
GET /api/v1/semantic-layer/catalog/
Discover the catalog, folder, and document ids a grant can name.
Assigning a Semantic Layer permission needs the UUID of a catalog, folder, or document. Three read endpoints provide them, and an API key may call all three.
| Endpoint | Returns | |
| - | - | - |
| `GET /api/v1/semantic-layer/catalog/` | The root of the tree, the coarsest resource a grant can name. | this page |
| `GET /api/v1/semantic-layer/folders/` | Top-level folders you can see, paginated. Pass `?flat=true` for folders at every depth. | [reference](/platform-api/permissions/semantic-layer/list-folders) |
| `GET /api/v1/semantic-layer/documents/` | Documents you can see, paginated. Filter with `?folder=`, or `?folder=root` for documents outside any folder. | [reference](/platform-api/semantic-layer/list) |
Each list is already filtered to what your key's creator can reach, so anything it returns is a resource you may name in an assignment. The playground on this page exercises the catalog root; the other two have their own pages.
An API key may also **write documents**: create and edit, bounded by the standing of the person who created
the key (see [Create a document](/platform-api/semantic-layer/create)). Two things stay closed to a key:
**deleting** a document, and every write on the **folder tree**. Deleting a document and creating, renaming,
or deleting a folder return `403`; moving and sorting items return `401`. Use the app for those.
## Walk the tree, then grant
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
GROUP = "b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"
headers = {"x-api-key": API_KEY, "Content-Type": "application/json"}
def all_folders():
url = f"{BASE_URL}/api/v1/semantic-layer/folders/?flat=true&page_size=100"
while url:
page = requests.get(url, headers=headers).json()
yield from page["results"]
url = page["next"]
finance = next(f for f in all_folders() if f["name"] == "Finance")
requests.post(
f"{BASE_URL}/api/v1/semantic-layer/permissions/",
headers=headers,
json={
"assignments": [
{"permission_level": "viewer", "groups": [GROUP], "folders": [finance["id"]]}
]
},
)
```
`/api/v1/context-documents/` is a legacy alias of the documents endpoint, kept while internal clients migrate. It is not part of the published API: build against `/api/v1/semantic-layer/documents/`.
## Related
* [List Semantic Layer Folders](/platform-api/permissions/semantic-layer/list-folders) and [List Semantic Layer Documents](/platform-api/semantic-layer/list): the other two reads, with their own playgrounds.
* [Assign Semantic Layer Permissions](/platform-api/permissions/semantic-layer/assign): what to do with the ids.
* [Create a Semantic Layer Document](/platform-api/semantic-layer/create): the write side of this tree.
* [Semantic Layer](/semantic-layer/overview): what the module is and how it is used in the app.
# Revoke Semantic Layer Permission
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/revoke
DELETE /api/v1/semantic-layer/permissions/{id}/
Remove a single grant on a catalog, folder, or document.
Revoking is a soft delete: the grant is revoked immediately and leaves the listing, and the row is kept with the actor and timestamp for the audit trail. Access derived from it (on the subtree and the ancestors) is cleared asynchronously, normally within seconds. Returns `204 No Content`.
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/semantic-layer/permissions/6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e/ \
--header "x-api-key: YOUR_API_KEY"
```
# Semantic Layer permission object
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/semantic-layer-permission
A single grant on a Semantic Layer catalog, folder, or document.
The shape of a grant on the Semantic Layer axis.
| Field | Type | Notes |
| - | - | - |
| `id` | UUID | |
| `permission_level` | `viewer` \| `editor` \| `manager` | The only mutable field. |
| `user` | integer | Expandable. |
| `group` | UUID | Expandable. |
| `catalog` | UUID | The Semantic Layer root. Not expandable. |
| `folder` | UUID | Expandable. |
| `document` | UUID | Expandable. |
| `granted_by` | integer | User id. Expandable. |
| `created_at` | datetime | |
| `updated_at` | datetime | |
Exactly one of `user` / `group` is set, and exactly one of `catalog` / `folder` / `document`.
The three resources are the hierarchy the Semantic Layer shows in the app, coarsest first: a grant on `catalog` reaches everything, and a grant on `folder` applies to the folder's whole subtree and gives Viewer on every ancestor folder and on the root. Ids come from [Read the Semantic Layer](/platform-api/permissions/semantic-layer/resources).
# Update Semantic Layer Permission
Source: https://docs.nekt.com/platform-api/permissions/semantic-layer/update
PATCH /api/v1/semantic-layer/permissions/{id}/
Change the level of a single existing grant.
`permission_level` is the only writable field. The recipient and the resource are fixed at assignment time.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/semantic-layer/permissions/6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"permission_level": "editor"}'
```
# Cancel Run
Source: https://docs.nekt.com/platform-api/runs/cancel
POST /api/v1/runs/{id}/cancel/
# Get Run
Source: https://docs.nekt.com/platform-api/runs/details
GET /api/v1/runs/{id}/
# Get Runs
Source: https://docs.nekt.com/platform-api/runs/list
GET /api/v1/runs/
# Get Run Logs
Source: https://docs.nekt.com/platform-api/runs/logs
GET /api/v1/runs/{id}/logs/
# Run object
Source: https://docs.nekt.com/platform-api/runs/run
# Create Semantic Layer Document
Source: https://docs.nekt.com/platform-api/semantic-layer/create
POST /api/v1/semantic-layer/documents/
Write a document into the Semantic Layer with an API key.
An API key writes as **the person who created it**, checked on every request. So a key may create a
document exactly where its creator may:
| Plan | Who may create |
| - | - |
| Growth, Custom | Owners and admins anywhere; a member needs **editor** on the destination, the folder, or the root when `folder` is omitted. |
| Free, Starter | Owners, admins, and members, anywhere. Semantic Layer permissions are not enforced on these plans. |
Demote the creator and every key they minted narrows on the next call: there is nothing to revoke
separately.
```bash theme={null}
curl --request POST \
--url "https://api.nekt.ai/api/v1/semantic-layer/documents/" \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"name": "MRR definition",
"content": "MRR counts active subscriptions in @table::silver.subscriptions, excluding trials.",
"folder": "b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"
}'
```
Mentions are parsed out of `content` for you and returned in the read-only `mentions` field; the
document is indexed for semantic search right after it is written.
Whatever its creator's role, an API key cannot write the folder tree, so `folder` has to name one
that already exists: create folders in the app. Deleting a document is not available to a key
either and returns `403`.
## Related
* [Update a document](/platform-api/semantic-layer/update).
* [List Semantic Layer Documents](/platform-api/semantic-layer/list): to find folder and document ids.
* [Assign Semantic Layer Permissions](/platform-api/permissions/semantic-layer/assign): to give the key's creator editor access somewhere.
# Semantic Layer document object
Source: https://docs.nekt.com/platform-api/semantic-layer/document
A document is a markdown note that teaches Nekt, and any assistant reading through MCP, what your
data means: metric definitions, business rules, caveats a column name cannot carry.
Two fields deserve a word:
| Field | |
| - | - |
| `content` | The markdown body. Mentions written as `@type::identifier` (for example `@table::silver.orders`) are parsed out of it server-side. |
| `mentions` | Read-only. The parsed result of the above: send `content`, never this. |
`folder` places the document in the tree; omit it (or send `null`) to leave it at the root.
## Related
* [Create a document](/platform-api/semantic-layer/create) and [Update a document](/platform-api/semantic-layer/update).
* [List Semantic Layer Documents](/platform-api/semantic-layer/list): the read, with its filters.
* [Semantic Layer](/semantic-layer/overview): what the module is and how it is used in the app.
# List Semantic Layer Documents
Source: https://docs.nekt.com/platform-api/semantic-layer/list
GET /api/v1/semantic-layer/documents/
Browse the documents an API key can reach.
Paginated, most recently updated first, and already filtered to what your API key's creator can
reach, so anything it returns is a document you may also read in full, and a document missing from
it is one that person cannot see.
| Filter | Effect |
| - | - |
| `?folder=` | Only documents inside that folder. |
| `?folder=root` | Only documents that sit outside any folder. `?folder=` empty is indistinguishable from the parameter being absent, which is why `root` is a literal. |
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/semantic-layer/documents/?folder=root&expand[]=created_by" \
--header "x-api-key: YOUR_API_KEY"
```
Use it to find the `id` to [read in full](/platform-api/semantic-layer/retrieve) or
[edit](/platform-api/semantic-layer/update), the `folder` to
[create](/platform-api/semantic-layer/create) into, or the ids to name in a
[permission assignment](/platform-api/permissions/semantic-layer/assign).
## Related
* [Retrieve a document](/platform-api/semantic-layer/retrieve): one document, with its content.
* [Read the Semantic Layer](/platform-api/permissions/semantic-layer/resources): this list plus the catalog root and the folders, as the three reads that feed a grant.
# Retrieve Semantic Layer Document
Source: https://docs.nekt.com/platform-api/semantic-layer/retrieve
GET /api/v1/semantic-layer/documents/{id}/
One document, with its full content.
The listing carries every field already, so reach for this when you have an id in hand, from a
[list](/platform-api/semantic-layer/list), from a grant, or from something you created earlier.
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/semantic-layer/documents/DOCUMENT_ID/" \
--header "x-api-key: YOUR_API_KEY"
```
A document your API key's creator cannot see answers **404**, not `403`: an id you cannot read is
indistinguishable from one that does not exist, and saying otherwise would confirm that a document
with that id exists in the workspace.
## Related
* [Document object](/platform-api/semantic-layer/document): every field, and which are read-only.
* [Update a document](/platform-api/semantic-layer/update).
# Update Semantic Layer Document
Source: https://docs.nekt.com/platform-api/semantic-layer/update
PATCH /api/v1/semantic-layer/documents/{id}/
Edit or replace a document's name, content, or folder with an API key.
Same rule as creating: the key edits as its creator. On Growth and Custom a member needs **editor**
on the document; on Free and Starter permissions are not enforced, so every member's key may edit any
document.
Changing `folder` moves the document. On Growth and Custom a member also needs **editor** on the
destination, the folder, or the root when `folder` is `null`. A move resets the document's manual
sort position.
Sending `content` re-parses the document's mentions and reindexes it for semantic search. A document
its creator cannot see answers `404`, not `403`: an id you cannot read is indistinguishable from one
that does not exist.
```bash theme={null}
curl --request PATCH \
--url "https://api.nekt.ai/api/v1/semantic-layer/documents/DOCUMENT_ID/" \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"content": "MRR excludes trials AND paused subscriptions."}'
```
## Replace a document
`PUT /api/v1/semantic-layer/documents/{id}/` applies the same rules as `PATCH`, with one difference:
`name` is required in the body. `content` and `folder` are optional, and an omitted field keeps its
current value.
```bash theme={null}
curl --request PUT \
--url "https://api.nekt.ai/api/v1/semantic-layer/documents/DOCUMENT_ID/" \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{"name": "MRR definition", "content": "MRR excludes trials and paused subscriptions."}'
```
`DELETE` is not available to an API key, whatever its creator's role, and returns `403`. Remove
documents from the app.
## Related
* [Create a document](/platform-api/semantic-layer/create).
* [Document object](/platform-api/semantic-layer/document): the fields and which of them are read-only.
# Bulk Delete Sources
Source: https://docs.nekt.com/platform-api/sources/bulk-delete
POST /api/v1/sources/bulk-delete/
Delete several sources at once with a single request.
Deletes several sources in one call. The request body takes a list of pipeline **slugs**, not source IDs.
This endpoint uses `POST` rather than `DELETE` because OpenAPI does not document a request body on `DELETE`. It deletes; it does not create anything.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `sources` | array of strings | Yes | Slugs of the sources to delete. |
## Response
Returns `200 OK` with a summary of what the action did, rather than the deleted objects.
```json theme={null}
{
"message": "Successfully deleted 2 sources"
}
```
The operation is all-or-nothing: nothing is deleted unless **every** listed source passes the same checks a single [Delete Source](/platform-api/sources/delete) call would apply. There is no undelete endpoint, so deletion cannot be reversed through the Platform API.
## Example
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/bulk-delete/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"sources": ["facebook-ads", "google-ads"]
}'
```
## Python example: delete every source carrying a given tag
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": API_KEY,
"Content-Type": "application/json",
}
sources = requests.get(
f"{BASE_URL}/api/v1/sources/",
headers=headers,
params={"expand[]": "tags"},
).json()
deprecated_slugs = [
s["slug"]
for s in sources["results"]
if any(tag["name"] == "deprecated" for tag in s.get("tags", []))
]
response = requests.post(
f"{BASE_URL}/api/v1/sources/bulk-delete/",
headers=headers,
json={"sources": deprecated_slugs},
)
print(response.json()["message"])
```
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header. MCP tokens cannot delete resources and are rejected with `403 Forbidden`.
## Related
* [Delete Source](/platform-api/sources/delete): remove a single source.
* [List Sources](/platform-api/sources/list): collect the slugs to send.
# Create Source
Source: https://docs.nekt.com/platform-api/sources/create
POST /api/v1/sources/
Learn how to programmatically create a data source using the Platform API.
Creating a source via the API involves three main steps: validating the connector configuration, waiting for validation to complete, and then creating the source with the validated configuration.
## Overview
The source creation process follows this flow:
```mermaid theme={null}
sequenceDiagram
participant Client
participant API
participant Connector validation (AWS/GCP)
Client->>API: POST /sources/connectors/{slug}/validate/
API->>Connector validation (AWS/GCP): Start validation
API-->>Client: Return validation ID (status: in_progress)
Connector validation (AWS/GCP)-->>API: Validation complete
loop Poll for status
Client->>API: GET /connector-validations/{id}/
API-->>Client: Return validation status
end
Note over Client,API: When status is "success"
Client->>API: POST /sources/
API-->>Client: Return created source
```
## Prerequisites
Before creating a source, you'll need:
1. An [API key](/platform-api/introduction#create-an-api-key) for authentication
2. The connector slug for the source type you want to create (e.g., `facebook-ads`, `google-ads`)
3. The output layer ID where the extracted data will be stored (use the [List Layers](/platform-api/catalog/list-layers) endpoint to find available layers)
4. The connector-specific configuration (credentials, account IDs, etc.)
To find available connectors and their required configuration fields, use the [List Source Connectors](/platform-api/connectors/source) endpoint with the `expand[]` parameter:
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/sources/connectors/?expand[]=current_version.config_template" \
--header "x-api-key: YOUR_API_KEY"
```
This returns the connector details including the `config_template` field, which contains the JSON schema describing all required and optional configuration fields. For example, the Facebook Ads connector requires:
```json theme={null}
{
"slug": "facebook-ads",
"current_version": {
"config_template": {
"type": "object",
"required": ["access_token", "account_id", "start_date"],
"properties": {
"access_token": {
"type": "string",
"title": "Access token",
"description": "The token to authenticate against the API service."
},
"account_id": {
"type": "string",
"title": "Account ID",
"description": "The unique ID number given to your ad account."
},
"start_date": {
"type": "string",
"format": "date",
"title": "Start date",
"description": "Records created or updated after the start date will be extracted from the source."
}
// ... additional optional fields
}
}
}
}
```
## Step 1: Validate the Connector Configuration
Before creating a source, you must validate that the connector configuration is correct and can successfully connect to the data source. This validation process also returns the available streams that can be extracted.
### Request
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/connectors/{connector_slug}/validate/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"config": {
// Connector-specific configuration
}
}'
```
### Example: Facebook Ads
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/connectors/facebook-ads/validate/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"config": {
"account_id": "YOUR_AD_ACCOUNT_ID",
"start_date": "2024-01-01",
"enable_advanced_reports": false,
"ad_insights_report_batch_size": 30,
"report_definition": {
"action_attribution_windows_view": "1d_view",
"action_attribution_windows_click": "7d_click",
"action_report_time": "mixed",
"lookback_window": 7
},
"creative_fields_mode": "basic",
"api_version": "v20.0",
"access_token": "YOUR_ACCESS_TOKEN"
}
}'
```
### Response
The validation endpoint returns a validation object with an initial status of `in_progress`:
```json theme={null}
{
"id": "a96414ab-8d0c-42ac-ae66-1df176bcd3c1",
"created_at": "2025-12-02T15:08:26.883724-03:00",
"updated_at": "2025-12-02T15:08:28.433163-03:00",
"status": "in_progress",
"streams": null,
"error_message": null,
"parsed_error": null,
"started_at": "2025-12-02T15:08:26.883446-03:00",
"ended_at": null,
"connector_version": "c965965c-119f-4b85-97ae-dac6b039fbe1"
}
```
## Step 2: Poll for Validation Status
The validation process runs asynchronously. You need to poll the validation status endpoint until the status changes from `in_progress` to either `success` or `failed`.
### Request
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/connector-validations/{validation_id}/ \
--header "x-api-key: YOUR_API_KEY"
```
### Successful Response
When validation succeeds, the response includes the available streams:
```json theme={null}
{
"id": "a96414ab-8d0c-42ac-ae66-1df176bcd3c1",
"status": "success",
"streams": [
{
"stream": "campaigns",
"sync_type": ["FULL_SYNC", "INCREMENTAL"],
"primary_keys": ["id", "updated_time"],
"replication_key": "updated_time",
"table_name_suggestion": "campaigns",
"fields": [
{"name": "id", "type": ["string", "null"], "format": null},
{"name": "name", "type": ["string", "null"], "format": null},
{"name": "objective", "type": ["string", "null"], "format": null}
// ... more fields
]
},
{
"stream": "ads",
"sync_type": ["FULL_SYNC", "INCREMENTAL"],
"primary_keys": ["id", "updated_time"],
"replication_key": "updated_time",
"table_name_suggestion": "ads",
"fields": [
{"name": "id", "type": ["string", "null"], "format": null},
{"name": "name", "type": ["string", "null"], "format": null}
// ... more fields
]
}
// ... more streams
],
"error_message": null,
"parsed_error": null
}
```
### Failed Response
If validation fails, the response includes error details:
```json theme={null}
{
"id": "a96414ab-8d0c-42ac-ae66-1df176bcd3c1",
"status": "failed",
"streams": null,
"error_message": "Authentication failed",
"parsed_error": "The access token provided is invalid or has expired. Please generate a new access token."
}
```
If validation fails, check the `parsed_error` field for a human-readable error message. Common issues include invalid credentials, expired tokens, or insufficient permissions.
## Step 3: Create the Source
Once validation succeeds, use the validation ID and stream information to create the source.
### Request
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"description": "Source created via API",
"connector_slug": "facebook-ads",
"connector_config": {
// Same config used in validation
},
"connector_validation": "VALIDATION_ID",
"output_layer": "OUTPUT_LAYER_ID",
"trigger": {
"type": "manual"
},
"streams": [
{
"enabled": true,
"stream_name": "ads",
"table_name": "facebook_ads_ads",
"primary_keys": ["id", "updated_time"],
"replication_key": "updated_time",
"sync_type": "INCREMENTAL",
"extract_all_fields": true,
"fields": null
}
]
}'
```
### Request Body Parameters
| Parameter | Type | Required | Description |
| - | - | - | - |
| `description` | string | No | A description for the source |
| `connector_slug` | string | Yes | The connector identifier (e.g., `facebook-ads`) |
| `connector_config` | object | Yes | The connector-specific configuration (same as validation) |
| `connector_validation` | string | Yes | The validation ID from step 1 |
| `output_layer` | string | Yes | The ID of the layer where data will be stored |
| `trigger` | object | Yes | The trigger configuration for the source |
| `streams` | array | Yes | The streams to extract with their configuration |
| `draft` | boolean | No | Create the source as a draft to fill in configuration later. Defaults to `false`. |
| `callback_webhook` | object | No | Optional webhook Nekt POSTs to when a draft source is configured through a [setup link](/platform-api/sources/setup-link). Write-only; responses expose only `has_callback_webhook`. |
Need a client or third party to provide the credentials? Create the source as a **draft** and share a secure setup link instead of collecting credentials yourself. See [Configure a source with a setup link](/platform-api/sources/setup-link).
### Stream Configuration
Each stream in the `streams` array requires:
| Parameter | Type | Required | Description |
| - | - | - | - |
| `enabled` | boolean | Yes | Whether the stream is enabled |
| `stream_name` | string | Yes | The name of the stream from validation |
| `table_name` | string | Yes | The name for the destination table |
| `primary_keys` | array | Yes | Primary key columns for the table |
| `replication_key` | string | No | Column used for incremental sync |
| `sync_type` | string | Yes | Either `INCREMENTAL` or `FULL_SYNC` |
| `extract_all_fields` | boolean | Yes | Whether to extract all available fields |
| `fields` | array | No | Specific fields to extract (if not extracting all) |
### Trigger Types
The `trigger` object supports the following types:
```json theme={null}
// Manual trigger - run only when triggered manually or via API
{
"type": "manual"
}
// Scheduled trigger - run on a cron schedule
{
"type": "scheduled",
"cron": "0 6 * * *",
"timezone": "America/Sao_Paulo"
}
```
### Response
```json theme={null}
{
"id": "d2c1b472-902c-477a-b300-e7c3fd72609b",
"slug": "facebook-ads-UcuH",
"deleted": false,
"active": true,
"status": "preparing",
"connector_config": {
"account_id": "YOUR_AD_ACCOUNT_ID",
"start_date": "2024-01-01",
"access_token": "EAA*****lgV"
},
"description": "Source created via API",
"created_at": "2025-12-02T15:09:15.323919-03:00",
"updated_at": "2025-12-02T15:09:15.323933-03:00",
"output_layer": "d661c642-646d-460c-bd75-4cd66d52e760"
}
```
The source is created with status `preparing`. Once the source is ready, you can trigger an extraction using the [Trigger Source](/platform-api/sources/trigger) endpoint.
## Complete Python Example
Here's a complete Python example that demonstrates the entire source creation flow:
```python theme={null}
import time
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": API_KEY,
"Content-Type": "application/json"
}
# Connector configuration
connector_slug = "facebook-ads"
connector_config = {
"account_id": "YOUR_AD_ACCOUNT_ID",
"start_date": "2024-01-01",
"enable_advanced_reports": False,
"ad_insights_report_batch_size": 30,
"report_definition": {
"action_attribution_windows_view": "1d_view",
"action_attribution_windows_click": "7d_click",
"action_report_time": "mixed",
"lookback_window": 7
},
"creative_fields_mode": "basic",
"api_version": "v20.0",
"access_token": "YOUR_ACCESS_TOKEN"
}
# Step 1: Validate the connector configuration
print("Starting validation...")
validate_url = f"{BASE_URL}/api/v1/sources/connectors/{connector_slug}/validate/"
validation_response = requests.post(
validate_url,
headers=headers,
json={"config": connector_config}
)
validation = validation_response.json()
validation_id = validation["id"]
print(f"Validation started with ID: {validation_id}")
# Step 2: Poll for validation status
validation_url = f"{BASE_URL}/api/v1/connector-validations/{validation_id}/"
while validation["status"] == "in_progress":
time.sleep(5)
validation = requests.get(validation_url, headers=headers).json()
print(f"Validation status: {validation['status']}")
# Check if validation succeeded
if validation["status"] != "success":
print(f"Validation failed: {validation.get('parsed_error')}")
exit(1)
# Print available streams
print("\nAvailable streams:")
for stream in validation["streams"]:
print(f" - {stream['stream']} (suggested table: {stream['table_name_suggestion']})")
# Step 3: Create the source
print("\nCreating source...")
source_payload = {
"description": "Facebook Ads source created via API",
"connector_slug": connector_slug,
"connector_config": connector_config,
"connector_validation": validation_id,
"output_layer": "YOUR_OUTPUT_LAYER_ID",
"trigger": {
"type": "manual"
},
"streams": [
{
"enabled": True,
"stream_name": "ads",
"table_name": "facebook_ads_ads",
"primary_keys": ["id", "updated_time"],
"replication_key": "updated_time",
"sync_type": "INCREMENTAL",
"extract_all_fields": True,
"fields": None
},
{
"enabled": True,
"stream_name": "campaigns",
"table_name": "facebook_ads_campaigns",
"primary_keys": ["id", "updated_time"],
"replication_key": "updated_time",
"sync_type": "INCREMENTAL",
"extract_all_fields": True,
"fields": None
}
]
}
create_response = requests.post(
f"{BASE_URL}/api/v1/sources/",
headers=headers,
json=source_payload
)
source = create_response.json()
print(f"\nSource created successfully!")
print(f" ID: {source['id']}")
print(f" Slug: {source['slug']}")
print(f" Status: {source['status']}")
```
### Expected Output
```
Starting validation...
Validation started with ID: 24e97049-67ab-4d2a-9803-58dde9e43d2e
Validation status: in_progress
Validation status: in_progress
Validation status: in_progress
Validation status: in_progress
Validation status: in_progress
Validation status: in_progress
Validation status: in_progress
Validation status: success
Available streams:
- adaccounts (suggested table: adaccounts)
- adimages (suggested table: adimages)
- adlabels (suggested table: adlabels)
- ads (suggested table: ads)
- adsets (suggested table: adsets)
- adsinsights (suggested table: adsinsights)
- advideos (suggested table: advideos)
- campaigns (suggested table: campaigns)
- creatives (suggested table: creatives)
- customaudiences (suggested table: customaudiences)
- customconversions (suggested table: customconversions)
Creating source...
Source created successfully!
ID: 3eb73ff3-e952-44e7-a16a-40345df7ac15
Slug: facebook-ads-msfK
Status: preparing
```
## Next Steps
After creating your source:
1. **Trigger an extraction**: Use the [Trigger Source](/platform-api/sources/trigger) endpoint to start the first data extraction
2. **Monitor the run**: Use the [Runs](/platform-api/runs/list) endpoints to track extraction progress
3. **View your data**: Once extraction completes, your data will be available in the specified output layer
## Error Handling
Common errors when creating sources:
| Error | Cause | Solution |
| - | - | - |
| `Validation ID not found` | Invalid or expired validation ID | Run a new validation |
| `Invalid connector configuration` | Missing required fields | Check connector documentation for required fields |
| `Output layer not found` | Invalid output layer ID | Verify the layer ID exists in your workspace |
| `Stream not available` | Stream name doesn't match validation | Use exact stream names from validation response |
# Delete Source
Source: https://docs.nekt.com/platform-api/sources/delete
DELETE /api/v1/sources/{slug}/
Delete a single source by its slug.
Deletes the source identified by `slug`. The identifier is the pipeline **slug**, not the source ID: the same value used by [Get Source](/platform-api/sources/details) and [Update Source](/platform-api/sources/update).
A successful call returns `204 No Content` with an empty body.
## Example
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/sources/facebook-ads/ \
--header "x-api-key: YOUR_API_KEY"
```
There is no undelete endpoint. Deleting a source cannot be reversed through the Platform API. Double-check the slug before sending the request.
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header. MCP tokens cannot delete resources and are rejected with `403 Forbidden`.
## Related
* [Bulk Delete Sources](/platform-api/sources/bulk-delete): remove several sources in a single request.
* [List Sources](/platform-api/sources/list): find the slug you want to delete.
# Get Source
Source: https://docs.nekt.com/platform-api/sources/details
GET /api/v1/sources/{slug}/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# List Sources
Source: https://docs.nekt.com/platform-api/sources/list
GET /api/v1/sources/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Validate Connector
Source: https://docs.nekt.com/platform-api/sources/revalidate
POST /api/v1/sources/{slug}/revalidate/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Configure a source with a setup link
Source: https://docs.nekt.com/platform-api/sources/setup-link
Programmatically create a draft source, invite a third party to fill in credentials through a setup link, and get notified via a callback webhook.
This flow lets you build a source configuration experience **on top of the Platform API**: create a source in draft, hand a secure link to a client or third party so they can fill in the connector credentials (without a Nekt account), and receive a webhook when they finish, then validate and finalize the source.
It mirrors the in-app [Share a setup link](/sources/share-setup-link) feature, end to end over the API.
## Overview
```mermaid theme={null}
sequenceDiagram
participant Client as Your app
participant API as Nekt API
participant ThirdParty as Third party
participant Hook as Your callback URL
Client->>API: POST /sources/ (draft=true, callback_webhook)
API-->>Client: Draft source
Client->>API: POST /sources/{slug}/setup-tokens/
API-->>Client: { token, expires_at }
Client->>ThirdParty: Share https://app.nekt.ai/scl/{token}
ThirdParty->>API: Fill in connector config (via the link)
API->>Hook: POST callback_webhook (source.setup_filled)
Client->>API: POST /sources/{slug}/revalidate/
loop Poll
Client->>API: GET /connector-validations/{id}/
API-->>Client: status
end
Client->>API: PATCH /sources/{slug}/ (draft=false)
API-->>Client: Source ready
```
## Prerequisites
* An [API key](/platform-api/introduction#create-an-api-key) for authentication.
* The connector slug for the source type (e.g. `facebook-ads`). See [List Source Connectors](/platform-api/connectors/source).
## Step 1: Create a draft source with a callback webhook
Create the source with `draft: true`. Only `connector_slug` is required for a draft: you (or a third party) fill in the rest later.
Optionally attach a **`callback_webhook`**: Nekt POSTs to it once the third party finishes filling in the configuration through the setup link. Use the optional custom header to authenticate the call on your side.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"draft": true,
"connector_slug": "facebook-ads",
"description": "Client Facebook Ads",
"callback_webhook": {
"url": "https://your-app.example.com/hooks/nekt",
"header_name": "X-Webhook-Secret",
"header_value": "your-shared-secret"
}
}'
```
### `callback_webhook` object
| Field | Type | Required | Description |
| - | - | - | - |
| `url` | string | Yes | HTTPS URL Nekt POSTs to on setup completion. Must be a public `https://` endpoint. |
| `header_name` | string | No | Optional custom header name to send with the callback. |
| `header_value` | string | No | Value for the custom header. Required if `header_name` is set. Stored encrypted and never returned. |
`callback_webhook` is **write-only**. Responses never echo it back; they expose a read-only boolean `has_callback_webhook` so you can confirm one is configured. To remove a configured webhook, send `"callback_webhook": null` on an update.
For security, the URL must use `https` and resolve to a public host. URLs pointing at `localhost`, loopback, private (RFC 1918), or link-local addresses are rejected.
## Step 2: Create a setup token
Generate a time-limited token scoped to this draft source, then build the link as `https://app.nekt.ai/scl/{token}` and share it with the third party.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/{source_slug}/setup-tokens/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{ "expires_in_seconds": 86400 }'
```
```json theme={null}
{
"id": "0d2c1b47-2902-4c47-8b30-0e7c3fd72609",
"token": "Qm9vR2k...redactedOnList",
"expires_at": "2026-06-18T12:00:00Z",
"created_at": "2026-06-17T12:00:00Z"
}
```
| Field | Type | Required | Description |
| - | - | - | - |
| `expires_in_seconds` | integer | No | Token lifetime. Defaults to 24 hours (`86400`). |
The full `token` is returned only on creation. When listing tokens it is redacted. To revoke a link, delete the token: `DELETE /api/v1/sources/{source_slug}/setup-tokens/{id}/`.
## Step 3: The third party fills in the configuration
The recipient opens `https://app.nekt.ai/scl/{token}` and enters the connector credentials (or authenticates via OAuth for connectors that support it). No Nekt account is required, and the link is scoped strictly to this draft source.
When they submit, Nekt POSTs to your `callback_webhook` (if configured):
```http theme={null}
POST https://your-app.example.com/hooks/nekt
Content-Type: application/json
X-Webhook-Secret: your-shared-secret
{
"event": "source.setup_filled",
"source": {
"id": "d2c1b472-902c-477a-b300-e7c3fd72609b",
"slug": "facebook-ads-UcuH",
"description": "Client Facebook Ads",
"draft": true,
"config_completed": false,
"missing_required_fields": ["access_token"]
}
}
```
| Field | Type | Description |
| - | - | - |
| `config_completed` | boolean | `true` when all required connector config fields are filled. |
| `missing_required_fields` | array | Dot-paths of required fields still missing (e.g. `report_definition.action_report_time`). Empty when `config_completed` is `true`. |
Use `config_completed` to decide whether you can proceed to validation, or whether you need to send another setup link to collect the remaining fields.
## Step 4: Validate the connector configuration
Start a validation for the draft source. This confirms the credentials work and discovers the available streams. The validation uses the configuration already saved on the draft (from the setup link), so you don't need to send a request body.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/{source_slug}/revalidate/ \
--header "x-api-key: YOUR_API_KEY"
```
You may optionally pass `{ "config": { ... } }` to merge extra fields into the saved configuration before validating.
The response returns a validation `id` with status `in_progress`. Poll it until the status is `success` or `failed`:
```bash theme={null}
curl --request GET \
--url https://api.nekt.ai/api/v1/connector-validations/{validation_id}/ \
--header "x-api-key: YOUR_API_KEY"
```
To stream the validation logs (useful for surfacing progress or diagnosing failures):
```bash theme={null}
curl --request GET \
--url "https://api.nekt.ai/api/v1/connector-validations/{validation_id}/logs/" \
--header "x-api-key: YOUR_API_KEY"
```
## Step 5: Finalize the source
Once validation succeeds, complete the source by setting `draft: false` along with the streams, output layer, and trigger. See [Update Source](/platform-api/sources/update) for the full set of fields.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/sources/{source_slug}/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"draft": false,
"output_layer": "OUTPUT_LAYER_ID",
"trigger": { "type": "manual" },
"add_streams": [
{
"enabled": true,
"stream_name": "ads",
"table_name": "facebook_ads_ads",
"primary_keys": ["id", "updated_time"],
"replication_key": "updated_time",
"sync_type": "INCREMENTAL",
"extract_all_fields": true,
"fields": null
}
]
}'
```
After finalizing, you can [Trigger Source](/platform-api/sources/trigger) to start the first extraction.
Setup tokens stop working once the source leaves draft. Any unused link for this source is invalidated when you finalize it.
## Customize the OAuth flow
For OAuth connectors (Facebook Ads, Google Analytics GA4, and similar), you can drive the entire setup over the API instead of sharing a setup link, with one exception: the **OAuth consent step**. The provider's login and "Authorize" screen must be opened in a browser by a human once per connected account.
This is a requirement of the provider (Meta, Google, etc.), not of Nekt. The consent step cannot be made fully server-to-server. Everything else (creating the source, generating the authorization URL, exchanging the code for a token, saving it, and validating) runs through the API.
The authorization code must be exchanged through Nekt's OAuth endpoints. The app's `client_id`/`client_secret` and the registered redirect URIs belong to Nekt, so you cannot exchange the code for a token on your own: the exchange has to go through Nekt's callback.
The steps below use **Facebook Ads** as the example. For GA4, swap `tap-facebook-ads` for `tap-google-analytics-ga4`.
Authenticate with your API key. Create the source with `draft: true` and the connector slug.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"draft": true,
"connector_slug": "facebook-ads"
}'
```
A human opens the initiate URL, which redirects to the provider's login and authorize screen. Pass your own return URL as `redirect_uri`.
```
GET https://api.nekt.ai/api/v1/tap-facebook-ads/oauth/initiate/?redirect_uri=YOUR_RETURN_URL
```
After the user authorizes, the provider redirects to your `redirect_uri` with `?code=...&state=...`. Store both values.
Call the callback endpoint with the `code`, `state`, and the **same** `redirect_uri` from step 2. This call uses a user token, not your API key.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tap-facebook-ads/oauth/callback/ \
--header "Content-Type: application/json" \
--data '{
"code": "...",
"state": "...",
"redirect_uri": "YOUR_RETURN_URL"
}'
```
The response returns the `access_token`, `refresh_token`, and related fields.
Authenticate with your API key. Patch the draft source with the `refresh_token` from the previous step.
```bash theme={null}
curl --request PATCH \
--url https://api.nekt.ai/api/v1/sources/{source_slug}/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"connector_config": {
"oauth_credentials": {
"refresh_token": "REFRESH_TOKEN_FROM_PREVIOUS_STEP"
}
}
}'
```
Nekt injects the `client_id`/`client_secret` at runtime, so you don't need to send them.
Start a validation, then poll it until the status is `success` (see [Step 4](#step-4-validate-the-connector-configuration) above).
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/sources/{source_slug}/revalidate/ \
--header "x-api-key: YOUR_API_KEY"
```
Patch the source with `draft: false` plus the streams, output layer, and trigger (see [Step 5](#step-5-finalize-the-source) above). You can then [Trigger Source](/platform-api/sources/trigger) to start the first extraction.
The exact endpoints and request/response shapes are available in the OpenAPI schema at [api.nekt.ai/api/schema/](https://api.nekt.ai/api/schema/).
# Create Setup Token
Source: https://docs.nekt.com/platform-api/sources/setup-tokens/create
POST /api/v1/sources/{source_slug}/setup-tokens/
# Delete Setup Token
Source: https://docs.nekt.com/platform-api/sources/setup-tokens/delete
DELETE /api/v1/sources/{source_slug}/setup-tokens/{id}/
# List Setup Tokens
Source: https://docs.nekt.com/platform-api/sources/setup-tokens/list
GET /api/v1/sources/{source_slug}/setup-tokens/
# Source object
Source: https://docs.nekt.com/platform-api/sources/source
# Trigger Source
Source: https://docs.nekt.com/platform-api/sources/trigger
POST /api/v1/sources/{slug}/trigger/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Get Source Trigger
Source: https://docs.nekt.com/platform-api/sources/triggers/details
GET /api/v1/sources/{source_slug}/triggers/{id}/
# List Source Triggers
Source: https://docs.nekt.com/platform-api/sources/triggers/list
GET /api/v1/sources/{source_slug}/triggers/
# Update Source
Source: https://docs.nekt.com/platform-api/sources/update
PATCH /api/v1/sources/{slug}/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Attach Tags
Source: https://docs.nekt.com/platform-api/tags/attach
POST /api/v1/tags/attach/
Attach one or more tags to sources, transformations, destinations, and other catalog resources.
The attach endpoint associates one or more tags with a set of resources. It accepts a list of tag `names`: any name that does not yet exist in your organization is created on the fly, and any name that already exists is reused. Every resource list in the request body is then attached to every tag in `names`.
This makes it possible to tag a single resource, apply several tags to a single resource, or apply tags to many resources in a single call.
The interactive API Playground on this page is generated from the OpenAPI schema and may only show the base `Tag` fields. The canonical request body is the one documented below (`names` plus the per-resource ID arrays), and the response is an **array** of tags, not a single object.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `names` | array of strings | Yes | Tag names to attach. Tags that don't exist are created automatically. |
| `sources` | array of UUIDs | No | Source IDs to tag. |
| `transformations` | array of UUIDs | No | Transformation IDs to tag (covers queries, notebooks, and histories). |
| `destinations` | array of UUIDs | No | Destination IDs to tag. |
| `visualizations` | array of UUIDs | No | Visualization IDs to tag. |
| `layers` | array of UUIDs | No | Layer IDs to tag. |
| `folders` | array of UUIDs | No | Folder IDs to tag. |
| `tables` | array of UUIDs | No | Table IDs to tag. |
| `volumes` | array of UUIDs | No | Volume IDs to tag. |
At least one resource list must contain at least one ID. Tag names provided in `names` but missing from the organization are created before the attachment happens.
## Response
The endpoint returns the full list of tags that were attached, both the ones that already existed and the ones that were created during the call.
```json theme={null}
[
{
"id": "6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e",
"name": "marketing",
"created_at": "2026-04-17T10:15:00.000000-03:00",
"updated_at": "2026-04-17T10:15:00.000000-03:00"
}
]
```
## Example: attach one tag to a single source
Use this pattern when you want to label an individual resource, for example marking a single Facebook Ads source as part of your marketing stack.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tags/attach/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"names": ["marketing"],
"sources": ["d2c1b472-902c-477a-b300-e7c3fd72609b"]
}'
```
## Example: attach multiple tags to one transformation
Multiple names in a single request attach every tag to every listed resource. Tags that do not yet exist are created before the attachment.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tags/attach/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"names": ["marketing", "weekly-refresh", "owned-by-revops"],
"transformations": ["a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02"]
}'
```
## Example: batch attach tags across sources, transformations, and destinations
A single request can target every resource type at once. Every tag in `names` is attached to every ID listed in the resource arrays.
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tags/attach/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"names": ["marketing", "production"],
"sources": [
"d2c1b472-902c-477a-b300-e7c3fd72609b",
"f3e4d5c6-b7a8-49e0-1f2a-3b4c5d6e7f80"
],
"transformations": [
"a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02",
"b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"
],
"destinations": [
"e4d5c6b7-a8b9-40c1-d2e3-f4a5b6c7d8e9"
]
}'
```
## Python example: tag every source that matches a pattern
The snippet below lists all sources, filters them by slug prefix, and attaches a tag to the whole batch in a single call.
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": API_KEY,
"Content-Type": "application/json",
}
sources = requests.get(f"{BASE_URL}/api/v1/sources/", headers=headers).json()
ads_source_ids = [s["id"] for s in sources["results"] if s["slug"].endswith("-ads")]
requests.post(
f"{BASE_URL}/api/v1/tags/attach/",
headers=headers,
json={
"names": ["ads"],
"sources": ads_source_ids,
},
)
```
## Related
* [Detach Tags](/platform-api/tags/detach): remove tags from the same resource types.
* [List Tags](/platform-api/tags/list): browse existing tags in your organization.
# Create Tag
Source: https://docs.nekt.com/platform-api/tags/create
POST /api/v1/tags/
# Detach Tags
Source: https://docs.nekt.com/platform-api/tags/detach
POST /api/v1/tags/detach/
Remove one or more tags from sources, transformations, destinations, and other catalog resources.
The detach endpoint removes the association between one or more existing tags and a set of resources. The tag itself is **not** deleted; only the links between the tag and the resources in the request body are removed. Tag names that do not exist in the organization are ignored silently.
The request body is identical to [Attach Tags](/platform-api/tags/attach): a list of tag `names` and one or more resource ID lists.
The interactive API Playground on this page is generated from the OpenAPI schema and may only show the base `Tag` fields. The canonical request body is the one documented below (`names` plus the per-resource ID arrays), and the response is an **array** of tags, not a single object.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `names` | array of strings | Yes | Tag names to detach. Names that don't exist in the organization are skipped. |
| `sources` | array of UUIDs | No | Source IDs to untag. |
| `transformations` | array of UUIDs | No | Transformation IDs to untag (covers queries, notebooks, and histories). |
| `destinations` | array of UUIDs | No | Destination IDs to untag. |
| `visualizations` | array of UUIDs | No | Visualization IDs to untag. |
| `layers` | array of UUIDs | No | Layer IDs to untag. |
| `folders` | array of UUIDs | No | Folder IDs to untag. |
| `tables` | array of UUIDs | No | Table IDs to untag. |
| `volumes` | array of UUIDs | No | Volume IDs to untag. |
Unlike `attach`, the `detach` endpoint does not create tags on the fly. If a name in `names` is not already present in your organization it is simply skipped.
## Response
The endpoint returns the list of tags that were matched by `names` and from which the resources were detached.
```json theme={null}
[
{
"id": "6f2a9c4b-2a10-4e7f-9c54-d2a9b1d28b0e",
"name": "marketing",
"created_at": "2026-04-17T10:15:00.000000-03:00",
"updated_at": "2026-04-17T11:02:00.000000-03:00"
}
]
```
## Example: detach one tag from a single source
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tags/detach/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"names": ["marketing"],
"sources": ["d2c1b472-902c-477a-b300-e7c3fd72609b"]
}'
```
## Example: detach multiple tags from one transformation
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tags/detach/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"names": ["weekly-refresh", "owned-by-revops"],
"transformations": ["a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02"]
}'
```
## Example: batch detach tags across sources, transformations, and destinations
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/tags/detach/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"names": ["production"],
"sources": [
"d2c1b472-902c-477a-b300-e7c3fd72609b",
"f3e4d5c6-b7a8-49e0-1f2a-3b4c5d6e7f80"
],
"transformations": [
"a7b4f0d2-5c91-4e34-9c1f-7e3a5f1c9d02",
"b8c5e1e3-6d02-4f45-ad20-8f4b6e2d0e13"
],
"destinations": [
"e4d5c6b7-a8b9-40c1-d2e3-f4a5b6c7d8e9"
]
}'
```
## Python example: remove a tag from every destination that currently carries it
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": API_KEY,
"Content-Type": "application/json",
}
destinations = requests.get(
f"{BASE_URL}/api/v1/destinations/",
headers=headers,
params={"expand[]": "tags"},
).json()
legacy_destination_ids = [
d["id"]
for d in destinations["results"]
if any(tag["name"] == "legacy" for tag in d.get("tags", []))
]
requests.post(
f"{BASE_URL}/api/v1/tags/detach/",
headers=headers,
json={
"names": ["legacy"],
"destinations": legacy_destination_ids,
},
)
```
## Related
* [Attach Tags](/platform-api/tags/attach): create the inverse association.
* [List Tags](/platform-api/tags/list): confirm which tags currently exist.
# Get Tag
Source: https://docs.nekt.com/platform-api/tags/details
GET /api/v1/tags/{id}/
# List Tags
Source: https://docs.nekt.com/platform-api/tags/list
GET /api/v1/tags/
# Tag object
Source: https://docs.nekt.com/platform-api/tags/tag
# Update Tag
Source: https://docs.nekt.com/platform-api/tags/update
PATCH /api/v1/tags/{id}/
# Get Token Access Log
Source: https://docs.nekt.com/platform-api/token-access-logs/details
GET /api/v1/token-access-logs/{id}/
One log line plus the request content recorded with it.
# Export Token Access Logs
Source: https://docs.nekt.com/platform-api/token-access-logs/export
POST /api/v1/token-access-logs/exports/
Build a Parquet file with every token access log of a period: each log's full detail, request body and headers included, plus what its Athena queries scanned and cost.
# Get Token Access Log Export
Source: https://docs.nekt.com/platform-api/token-access-logs/export-details
GET /api/v1/token-access-logs/exports/{id}/
Check an export's progress and get the download URL of its Parquet file.
# List Token Access Logs
Source: https://docs.nekt.com/platform-api/token-access-logs/list
GET /api/v1/token-access-logs/
Every request made with an API token or an MCP token, newest first.
# Token access log object
Source: https://docs.nekt.com/platform-api/token-access-logs/token-access-log
The shape of a single token access log line.
# Token access log export object
Source: https://docs.nekt.com/platform-api/token-access-logs/token-access-log-export
The shape of a token access log export.
# Bulk Delete Transformations
Source: https://docs.nekt.com/platform-api/transformations/bulk-delete
POST /api/v1/transformations/bulk-delete/
Delete several Queries, Notebooks, or Histories at once with a single request.
Deletes several transformations in one call. The request body takes a list of pipeline **slugs**, not transformation IDs.
Queries, Notebooks, and Histories are all served by the transformations endpoints, so a single request may mix all three. They differ only by `type`: `sql` for a Query, `python` or `pyspark` for a Notebook, and `scd_type_2` for a History.
This endpoint uses `POST` rather than `DELETE` because OpenAPI does not document a request body on `DELETE`. It deletes; it does not create anything.
## Request body
| Parameter | Type | Required | Description |
| - | - | - | - |
| `transformations` | array of strings | Yes | Slugs of the transformations to delete. |
## Response
Returns `200 OK` with a summary of what the action did, rather than the deleted objects.
```json theme={null}
{
"message": "Successfully deleted 2 transformations"
}
```
The operation is all-or-nothing: nothing is deleted unless **every** listed transformation passes the same checks a single [Delete Transformation](/platform-api/transformations/delete) call would apply. There is no undelete endpoint, so deletion cannot be reversed through the Platform API.
## Example
```bash theme={null}
curl --request POST \
--url https://api.nekt.ai/api/v1/transformations/bulk-delete/ \
--header "x-api-key: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
"transformations": ["silver-orders", "gold-revenue-daily"]
}'
```
## Python example: delete every transformation carrying a given tag
```python theme={null}
import requests
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.nekt.ai"
headers = {
"x-api-key": API_KEY,
"Content-Type": "application/json",
}
transformations = requests.get(
f"{BASE_URL}/api/v1/transformations/",
headers=headers,
params={"expand[]": "tags"},
).json()
sandbox_slugs = [
t["slug"]
for t in transformations["results"]
if any(tag["name"] == "sandbox" for tag in t.get("tags", []))
]
response = requests.post(
f"{BASE_URL}/api/v1/transformations/bulk-delete/",
headers=headers,
json={"transformations": sandbox_slugs},
)
print(response.json()["message"])
```
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header. MCP tokens cannot delete resources and are rejected with `403 Forbidden`.
## Related
* [Delete Transformation](/platform-api/transformations/delete): remove a single transformation.
* [List Transformations](/platform-api/transformations/list): collect the slugs to send.
# Delete Transformation
Source: https://docs.nekt.com/platform-api/transformations/delete
DELETE /api/v1/transformations/{slug}/
Delete a single Query, Notebook, or History through the Platform API transformations endpoint.
Deletes the transformation identified by `slug`. The identifier is the pipeline **slug**, not the transformation ID: the same value used by [Get Transformation](/platform-api/transformations/details) and [Update Transformation](/platform-api/transformations/update).
A successful call returns `204 No Content` with an empty body.
Queries, Notebooks, and Histories are all served by the transformations endpoints, so this one deletes any of the three. They differ only by `type`: `sql` for a Query, `python` or `pyspark` for a Notebook, and `scd_type_2` for a History.
## Example
```bash theme={null}
curl --request DELETE \
--url https://api.nekt.ai/api/v1/transformations/silver-orders/ \
--header "x-api-key: YOUR_API_KEY"
```
There is no undelete endpoint. Deleting a transformation cannot be reversed through the Platform API. Double-check the slug before sending the request.
## Authentication
This endpoint requires an [API key](/platform-api/introduction#create-an-api-key) in the `x-api-key` header. MCP tokens cannot delete resources and are rejected with `403 Forbidden`.
## Related
* [Bulk Delete Transformations](/platform-api/transformations/bulk-delete): remove several transformations in a single request.
* [List Transformations](/platform-api/transformations/list): find the slug you want to delete.
# Get Transformation
Source: https://docs.nekt.com/platform-api/transformations/details
GET /api/v1/transformations/{slug}/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Get Transformations
Source: https://docs.nekt.com/platform-api/transformations/list
GET /api/v1/transformations/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Transformation object
Source: https://docs.nekt.com/platform-api/transformations/transformation
# Trigger Transformation
Source: https://docs.nekt.com/platform-api/transformations/trigger
POST /api/v1/transformations/{slug}/trigger/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# Update Transformation
Source: https://docs.nekt.com/platform-api/transformations/update
PATCH /api/v1/transformations/{slug}/
Opt-in visibility of soft-deleted rows on the ``list`` action, for admins/owners only.
Views combine ``include_deleted_requested()`` with their own ``get_manager()`` to start the
list queryset from ``all_objects`` when the caller passes ``?include_deleted=true``. The opt-in
is intentionally scoped to ``list`` — update/trigger/destroy keep using the default (deleted-
excluding) manager, so they can never operate on a soft-deleted resource.
Only Owners/Admins may see deleted items; a non-admin caller (including token principals with
``role=None``, e.g. scoped MCP tokens) asking for them gets a 403.
# User object
Source: https://docs.nekt.com/platform-api/users/user
# Visualization object
Source: https://docs.nekt.com/platform-api/visualizations/visualization
# Queries
Source: https://docs.nekt.com/queries/overview
Write and manage SQL queries to process, aggregate, and analyze your data.
Our Queries module allows you to manipulate data using SQL.
Queries in Nekt combine the power and familiarity of SQL with our intuitive interface to help you build and deploy data pipelines efficiently.
Whether you're aggregating, joining, filtering, or reshaping data, the SQL Explorer provides all the tools you need while maintaining the simplicity and readability that SQL is known for.
### Creating a Query
Nekt makes it simple to write, test, and save queries from the SQL Explorer. Whether you're writing simple filters or complex multi-table joins, the interface provides all the tools you need to build and validate your SQL code.
#### **Step 1: Access the SQL Explorer**
You can access the SQL Explorer in two ways:
1. Navigate to the **Explorer** page in Nekt.
2. Go to **Queries** > **Add Query** > **SQL**
Once you're in the SQL Explorer, you'll find a clean, intuitive interface where you can write and test SQL queries. The Explorer provides access to all your available data tables and views.
[Open SQL Explorer](https://app.nekt.ai/explorer)
***
#### **Step 2: Build Your SQL Query**
You have two options for building your query:
1. **Write SQL Manually:**
* Use the SQL editor to write your query from scratch
* Leverage auto-completion and syntax highlighting
* Access table schemas and column information directly in the interface
2. **Use AI Assistance:**
* Describe what you want to achieve in natural language
* Let our AI help you generate the appropriate SQL query
* Review and modify the generated query as needed
**Best Practices:**
* Start with simple queries and build complexity gradually
* Test your query results before creating a transformation
* Use the query history to reference and modify previous queries
***
#### **Step 3: Save as a Query**
Once your SQL is working as expected:
1. Click the **Save as Query** button in the Explorer
2. Configure the query settings:
* Set a name and description
* Choose when the query should run
* Configure any necessary parameters
3. Save the query to make it available for execution
The query will now be available in your Queries list, where you can:
* Edit the SQL directly
* Modify the schedule and settings
* Monitor execution history and performance
[View Queries](https://app.nekt.ai/queries)
***
### **Best Practices**
1. **Query Optimization:**
* Use appropriate indexes and joins
* Limit the data processed to only what's necessary
* Consider query performance and resource usage
2. **Maintainability:**
* Use clear, descriptive names for queries
* Document complex SQL with comments
* Break down complex logic into smaller, manageable queries
3. **Testing:**
* Always test queries with sample data first
* Verify results match your expectations
* Check for edge cases and data quality issues
4. **Monitoring:**
* Review query execution logs
* Monitor query performance
* Set up alerts for failed queries
***
### **Need Help?**
If you need assistance with SQL queries, our team is here to help. You can:
* Use the AI assistant in the Explorer for query generation and debugging
* Contact our support team for additional guidance
## API & MCP
List, update, trigger, and delete transformations (Queries and Notebooks).
`create_query`, `get_code`, `update_transformation`, and `run_pipeline`.
# Versioning
Source: https://docs.nekt.com/queries/versioning
Version control your queries with GitHub and deploy from commits.
GitHub Versioning lets you manage code for Notebooks and Queries through Git: link a Nekt resource to a GitHub repository, and every push to the configured branch (e.g. **main**) automatically triggers a redeploy in Nekt. You get version control, collaboration via pull requests, and a clear audit trail.
This requires a [workspace-wide GitHub connection](/workspace/github): an owner or admin connects your organization in **Settings → GitHub** before you can link repositories to individual Notebooks or Queries.
Once connected, you link a repo per resource (when creating or editing, or via the **Versioning** column in the list). After that, no manual "deploy" step is needed: Nekt stays in sync with your branch.
## Prerequisite: Connect GitHub (workspace)
Before linking repositories, an owner or admin must [connect your GitHub organization](/workspace/github) in **Settings → GitHub**. After that, you can link repos to individual Queries.
## Prepare your repository
Use a GitHub repository to store your Query SQL. It can be empty. A dedicated repo per Query (or a shared repo with one file per query) keeps things clear.
## Link the repository to a Query
In the **Queries** list, use the **Versioning** column:
1. Click **Connect repository** for the Query you want to version.
2. In the dialog, select the GitHub repository (and branch, if offered) and confirm.
You can link or change the linked repo at any time from the same Versioning cell. If your workspace has no GitHub connection yet, the cell will prompt you to ask an admin to configure it in Settings.
## Automatic deployments
Once a Query is linked, any push to the configured branch (typically **main**) automatically redeploys that Query in Nekt. The deployed SQL in Nekt stays in sync with the branch; no manual update needed.
## API & MCP
Not available in the Platform API yet.
`connect_transformations_to_github` and `list_github_repositories`.
# Common questions
Source: https://docs.nekt.com/resources/faq
Common questions about Nekt
You probably haven’t had any successfully completed runs yet. When a source is created, the tables are already identified and created in the Catalog, but the data only becomes available once a successful run has occurred. Make sure you’ve triggered a manual run or wait for the scheduled automatic trigger. Once the run is complete, your data should be visible in the Catalog.
Access to sources, destinations, queries, notebooks, and histories is determined by table, folder, and layer access permissions.
To see a pipeline, a Member needs at least Viewer on every table it reads or writes. To edit, activate, or trigger it, they need Editor on the tables it writes to (to delete it, Manager). Input tables only need Viewer. You can read more about permissions in [Permissions](/workspace/permissions).
No! We extract data exactly as it is sent from the APIs.
So the way you see the data in Nekt is the same as you would if you used any tool to hit the endpoint and view the result. The only modifications you might notice are: column names are always standardized to snake\_case, and a new column called nekt\_sync\_at is added for internal processing.
A Full Sync ignores previously extracted data and makes a copy of all data from the source, exactly as it exists at the time of extraction.
Incremental Sync is more efficient and only fetches the data that has changed since the last extraction. Read more in [Sync Types](/runs/sync-types).
The use of Incremental Sync depends on the existence of a field that identifies when each record was last modified.
Typically, it's a column called "last\_updated\_at" or something similar. If the API doesn’t return this field, or if you didn’t specify it correctly when creating the source, Incremental Sync won’t be possible.
If your workspace runs on **your own cloud** (AWS or GCP), you can estimate costs by evaluating the cost of each resource's initial runs and extrapolating to your expected usage.\
Read more about how to monitor this in [Cloud](/workspace/cloud), and feel free to reach out if you have any questions!
If you use **Nekt Express**, the infrastructure is hosted and managed by Nekt; there is no separate cloud provider bill for the workspace itself. Credits are discounted from your Nekt subscription plan based on usage (mostly for storage and processing).
Nekt follows industry best practices for security and privacy. Your data stays within your control: within your cloud environment when you use your own AWS or GCP, or within Nekt's secure, managed environment when you use **Nekt Express**. In both cases we minimize risk and maintain strong data governance.
No. Canceling your subscription won’t delete your data. It stays safely stored in your cloud environment (or in Nekt Express, if you use our hosted environment). Nekt doesn’t store your data in a way that removes your access. It remains in your chosen environment unless you choose to export or delete it.
Nekt credits are used to pay for the value delivered as Nekt manages and monitors your data pipelines (1 credit = 1 pipeline run). We don’t charge for rows, number of connectors, or user seats. Failed pipelines and initial data source loads are also free.
If you selected to sync all fields when connecting the source, Nekt considers that you want everything to be synced, including new columns. In this case, the "Auto-add fields" flag will be active (you can check it on the source details screen).
Whether the new fields will be populated for all rows or just for newly imported ones depends on the sync type:
* **Full sync**: the new column will be filled for all rows in the table.
* **Incremental sync**: the new column will only be filled for newly imported rows; existing rows will have `null` in that column unless a full sync is triggered manually.
If you have the permission to do it, you can delete not only tables, but also sources, destinations and visualizations. The only limitation is that you can not delete a resource that is being used elsewhere. So you can't delete a table if it is being used in a transformation, for example.
It is also important to note that deleting a source doesn't automatically delete the tables associated with it. If you want to delete the table as well, you have to do it individually.
The time it takes for a run to finish depends mostly on the amont of data being processed and how powerful is the configuration of the machine dedicated to it. You can add more resources to process a specific pipeline if you need it to be faster or to support a bigger amount of data, but keep in mind it has some impact on costs.
When you are adding a new table to the catalog, you can consider that this process has two steps, which might represent a run a bit longer than you were expecting. Read more about how the data extraction happens [here](/sources/overview#how-extraction-works).
Manipulating or transforming data is like following a recipe: you take raw ingredients (your source data) and turn them into a finished dish (clean, organized data tables ready for analysis). The possibilities are endless.
Nekt's Notebooks, Queries and History modules let you process data using Python, SQL, and a pre-built SCD Type 2 template. You can run complex calculations, join multiple data sources, implement business rules, and schedule automated workflows. The transformation engine supports both batch and real-time processing, allowing you to build data pipelines that match your specific requirements. Whether you need to clean data, calculate metrics, or create new datasets, you have full control over how your data is processed.
# Logs
Source: https://docs.nekt.com/runs/logs
Inspect run logs, understand failures, and tune retries and alerts.
Data pipelines can sometimes fail. Although this is inherent to working with data and external systems, Nekt provides logs and alerts to help you understand and fix issues quickly.
## Run logs
Each run has a **log** that records what happened during execution. Use logs to:
* See when the run started and finished.
* Understand which step failed (source, query, notebook, history, or destination).
* Read error messages and stack traces when something goes wrong.
From the **Runs** page, click into a run to open its details and view the full log. For transformations (queries, notebooks, histories), logs often contain error messages from your SQL or code that point directly to the problem.
## Alerts (email notifications)
Nekt lets you receive email notifications when runs fail so you can act quickly:
* **Per-resource alerts**: enable alerts for individual sources, queries, notebooks, histories, and destinations from their details pages.
* **Global alerts**: owners and admins can configure workspace-wide alerts in **Settings → Alerts** so one or more email addresses receive notifications for any failed pipeline.
For full details on how alerts work and how to manage them (including bulk actions), see [Alerts](/workspace/alerts).
***
## Common failure reasons
Here are some common reasons why a run might fail and how to deal with them.
### An adjustment is required
If something is wrong with a connector (for sources and destinations), a query or notebook, or any credentials used to access external tools, you will usually need to fix or update that configuration. The easiest way to understand the error is by checking the run logs.
Logs typically include a message explaining what happened and what needs to be changed. If there is nothing useful logged, contact our support team and we will investigate further.
We are updating all resources to provide meaningful and actionable error messages, but there are many scenarios and we might not have covered all of them yet. We're sorry for any inconvenience.
### Instability or temporary issues
Pipelines can also fail due to temporary issues (e.g. network instability, upstream API timeouts, or cloud service limits). Nekt lets you configure **retry parameters** for each source, query, notebook, history, or destination to handle these cases gracefully:
1. **Number of retries**: how many times to retry before marking the run as failed.
2. **Retry delay**: how long (in seconds) to wait between retry attempts.
3. **Max consecutive failures**: how many consecutive failures are allowed before inactivating the pipeline.
* **Use case**: this is ideal for preventing repeated failures during off-hours or weekends, saving resources and avoiding unnecessary alerts.
These parameters can be managed in the **Settings** tab of each source, query, notebook, history, and destination, giving you flexibility in how you balance reliability, noise, and cost.
## API & MCP
Fetch a run's logs over REST.
`get_pipeline_run_logs` returns errors and the AI failure analysis.
# Runs
Source: https://docs.nekt.com/runs/overview
Understand and monitor pipeline executions in Nekt.
Every time Nekt triggers the extraction of a **source**, the execution of a **query, notebook, or history**, or sends data to a **destination**, it creates a **run**. Runs are the unit of execution for your data pipelines.
You can monitor runs to make sure processes complete successfully and to diagnose issues when something goes wrong.
## What is a run?
A **run** is a single execution of a pipeline step, such as:
* Source extraction
* Query execution
* Notebook execution
* History execution
* Destination load
Each time a pipeline is executed (manually, on a schedule, or via a trigger), a new run is created. You can track its progress and inspect its logs if something fails.
## What can I do on the Runs page?
On the **Runs** page you will find:
* A list of recent runs with their **status** (e.g. succeeded, failed, running).
* The **pipeline name** and type (source, query, notebook, history, destination).
* Key metadata such as start time, duration, and trigger type.
You can filter the list by **status** and/or by **pipeline**, so you can quickly focus on failures or on specific pipelines you care about.
For details on reading logs, retry behavior, and alerts, see [Logs](/runs/logs).
## API & MCP
List runs, read run details and logs, and cancel a run.
`list_pipeline_runs`, `get_pipeline_status`, and `get_pipeline_run_logs`.
# Scheduling and triggers
Source: https://docs.nekt.com/runs/scheduling-and-triggers
Automate and control your data workflows with precision.
### What is a Trigger?
A **trigger** defines the conditions under which a pipeline should start running, giving you full control over when and how your data workflows are executed. Triggers determine the timing for extracting data from sources, running queries, notebooks, or histories, or delivering data to destinations, depending on the pipeline's purpose.
When configuring a trigger, consider how frequently you need your data to be updated or processed, whether daily, hourly, or even less. Triggers ensure your data stays up-to-date and aligned with your operational needs.
***
### Types of Triggers
Nekt provides three flexible types of triggers to accommodate various workflow requirements:
#### Schedule Trigger
Automate your pipeline execution on a set schedule. Define specific week days and times for the pipeline to run, ensuring data updates occur precisely when needed.
* **Best for:** Periodic updates like daily reports, weekly aggregations, or monthly data refreshes.
#### Event-Based Trigger
Chain your workflows by setting one pipeline as a reference for another. When the reference pipeline completes a successful run, the next pipeline automatically starts.
* **Best for:** Creating dependent workflows, such as running a query or notebook immediately after a data extraction finishes.
#### Manual Trigger
Take full control by manually starting the pipeline whenever you decide. This option is perfect for ad-hoc updates or when you want to validate changes before running the pipeline.
* **Best for:** Testing, troubleshooting, or running one-off data updates.
***
### Choosing the right Trigger
The type of trigger you choose depends on your workflow’s specific needs:
* **Need regular, predictable updates?** Use a **Schedule Trigger**.
* **Want workflows to run automatically in sequence?** Opt for an **Event-Based Trigger**.
* **Prefer control over execution?** Go with a **Manual Trigger**.
By selecting the appropriate trigger type, you can ensure your data pipelines align with your operational priorities and deliver data precisely when needed.
***
### Edit a trigger on an existing resource
You can change the trigger on an existing Source, Query, or Notebook without recreating the resource. The updated trigger applies to future runs only; past runs are unchanged.
Open the Source, Query, or Notebook you want to update.
Open the **Details** page for that resource.
In the **Trigger** section, click **Edit trigger**.
Select the trigger type you want:
* **Schedule**: Runs automatically on a defined schedule.
* **Event**: Runs when another pipeline completes or emits a triggering event.
* **Manual**: Runs only when you start it manually.
Save your changes. The new trigger configuration is used for all future executions.
Sources, Queries, and Notebooks expose **Edit trigger** on their Details page.
***
### Why Triggers matter
Triggers are a key part of making your data workflows efficient and reliable. They minimize manual intervention, reduce latency, and ensure your data is always available when you need it, whether for analysis, reporting, or decision-making. With Nekt, triggers empower you to streamline and automate your data operations, saving time and ensuring accuracy.
Here's an example of a common trigger configuration for a complete data workflow:
* Source `pipedrive-123` has a schedule-based trigger: runs every day at 6am.
* Query `query-456` has an event-based trigger: runs as soon as `pipedrive-123` finishes the extraction.
* Destination `destination-789` has an event-based trigger: runs as soon as `query-456` finishes running.
## API & MCP
Read and manage source and destination triggers.
`update_trigger` sets a manual, cron, or event trigger.
# What are the types of Data Sync?
Source: https://docs.nekt.com/runs/sync-types
Choose how your data is synchronized during extraction.
In Nekt, **data sync types** determine how data is retrieved from your source during extraction and synchronized with your Lakehouse. Whether you need to fetch only updated records, the entire current state, or track changes through logs, Nekt provides flexible options to meet your data integration needs.
### **Types of Data Sync**
#### **Incremental Sync**
With **Incremental Sync**, only new or updated data is fetched each time an extraction occurs.
* **Best for:** Workflows with frequent updates or large volumes of data, where efficiency is a priority.
* **Considerations:** Deleted data from the source will remain in the lake until a **Full Sync** occurs.
Schedule periodic full syncs to ensure data integrity when using Incremental Sync.
#### **Full Sync**
With **Full Sync**, the entire current state of the data source is fetched during each extraction.
* **Best for:** Scenarios where your data must always reflect the exact state of the source, including records that have been deleted.
* **Considerations:** This method requires more time and resources compared to Incremental Sync, especially for large datasets.
#### **CDC Log-Based Sync**
The **CDC (Change Data Capture) Log-Based Sync** tracks changes in the source data by reading logs to capture updates, deletions, and inserts in near real-time.
* **Best for:** Scenarios where high-frequency data updates and deletions need to be captured efficiently without repeatedly scanning the entire dataset.
* **Key Benefits:**
* Near real-time updates.
* Efficient handling of large datasets with minimal resource usage.
* Maintains a detailed history of changes for auditing and tracking purposes.
Ensure your data source supports CDC and has the necessary logging enabled for this sync type.
### **Additional Full Sync**
If you choose **Incremental Sync**, you can schedule periodic **Full Syncs** to ensure your data remains fully aligned with the source, including any deleted records.
#### **Why Schedule Additional Full Syncs?**
* **Advantages:** Keeps your data up-to-date with the source while leveraging the efficiency of Incremental Sync for most operations.
* **Challenges:** Full Syncs require more time and resources. For small datasets, this might not be an issue, but for larger datasets, it can become costly.
#### **Recommendations**
* **Frequency Matters:** Choose a frequency that balances resource usage with the importance of having fully synchronized data. Consider triggering pipelines only on working days and between working hours, for example.
* **Evaluate Your Needs:** If your workflow doesn’t depend on immediate updates for deleted data, a less frequent Full Sync might suffice.
### **Choosing the Right Sync Type**
When selecting a sync type for your data source, consider the following:
* **Incremental Sync:** Prioritize for large, frequently updated datasets where efficiency is key.
* **Full Sync:** Use when maintaining an exact reflection of the source, including deletions, is critical.
* **CDC Log-Based Sync:** Opt for real-time or near real-time updates and precise tracking of changes.
* **Additional Full Sync:** Combine Incremental Sync with periodic Full Syncs for a balanced approach.
By selecting the appropriate sync strategy, you can optimize both performance and data integrity, ensuring your data meets your organization’s needs.
# Context Layer
Source: https://docs.nekt.com/semantic-layer/overview
Teach your AI agents how your business runs so their answers match your definitions and rules.
The Context Layer is a library of business knowledge that grounds Nekt's AI in your organization's definitions. You write context documents that explain what your metrics mean, which records count as paying customers, how the stages of your funnel are defined; every agent that queries your data reads those documents before generating SQL.
Without a context layer, agents have to guess from raw table and column names. With one, they answer questions the way you would.
### Why context belongs next to your data
Your data is already in the Catalog, and context documents reference it directly through annotations. That turns scattered business knowledge into a structured map of how your definitions connect to real tables, fields, and pipelines, instead of a document set that describes data living somewhere else.
Centralizing that layer also means everyone reaching your data through Nekt works from the same definitions, whatever tool they use: Claude, Codex, Copilot, or any other agent. Same source, same rules, same answers. That consistency is what makes agent responses faster, more accurate, and repeatable across a team.
### What's in a context document
Each document combines two things:
* **Prose** that describes a metric, segment, rule, or definition in plain language.
* **Annotations** that link the prose to specific resources in your workspace, so the AI knows exactly which tables, fields, and pipelines the definition refers to.
Annotations can reference:
| Annotation | Links to |
| - | - |
| Table | A table in the Catalog |
| Field | A column inside a table |
| Layer | A storage layer in the Lakehouse |
| Query | A saved [Query](/queries/overview) |
| Notebook | A saved [Notebook](/notebooks/overview) |
| Source | A [Sources](/sources/overview) connector |
| Destination | A [Destinations](/destinations/overview) connector |
| Context document | Another document, for cross-references |
When an agent retrieves a document, the annotations bring in the resolved resource names, layers, and descriptions automatically, so the model gets both the business definition and the underlying schema in one place.
### Creating a context document
Open **Context Layer** from the sidebar. You'll see two views:
* **All documents**: every context document you have access to.
* **Templates**: starter prompts that help your agent draft your first documents.
Pick a template that matches what you want to document, or click **New document** to start from scratch. If you don't know where to begin, the *Don't know where to start?* template lets your agent explore your workspace and propose what to document first.
Explain the concept the way you would to a new analyst. Focus on intent (what counts, what doesn't, why) rather than how the SQL should be written.
Reference the tables, fields, layers, queries, notebooks, sources, or destinations the definition depends on. Annotations keep the document anchored to real resources as your warehouse evolves.
The document is indexed immediately. The next agent question that touches that topic will use it.
### Let an agent build your library
If you already have this knowledge written down somewhere, the fastest path is to hand it to an agent connected to your workspace through the [MCP Server](/mcp-server/overview).
Metric definitions, onboarding docs, wiki exports, spreadsheets of business rules. Anything that explains how your business works.
Attach them to a conversation with Claude, Codex, or any other assistant connected to your Nekt MCP Server.
The agent writes each document in the right format and adds annotations pointing at the tables and fields that already exist in your Catalog, so the result is shaped for agent consumption from the start.
You still own the definitions. Read what the agent produced, correct anything that drifted, and drop documents that don't earn their place.
If the MCP Server isn't connected yet, use that time to curate. Decide what is actually worth documenting and clean up the rest, so the upload is quick once the connection is live.
### Templates
Nekt ships with starter prompts that cover the definitions most teams need first:
How revenue is recognized and MRR is calculated, including multi-currency handling.
Which subscription statuses count as paying, and the internal orgs that never do.
What counts as an MQL, SAL, and SQL, and which lead sources are excluded.
How customers consume the product, unit conversions, and plan-limit signals.
The gotchas that apply to every analysis: exclusions, soft deletes, currencies.
Let your agent explore your workspace and propose what to document first.
### How agents use the Context Layer
When you ask a question through the [MCP Server](/mcp-server/overview) or any other AI integration, Nekt runs a semantic search over your context documents and feeds the most relevant ones into the SQL generation step. The agent reads the business rules first, then writes a query that respects them.
This means the same question, *"What was revenue last month?"*, produces different SQL for different organizations, because each organization's definition of revenue lives in its own context layer.
### Best practices
Prefer prose plus annotations over raw SQL. Describe what a metric *means*, not how to compute it. Schemas change; definitions don't.
Upload only what matters. Useful context is the smallest set of high-signal information. The same disorganized documentation that confuses a new analyst also confuses an agent, so filter before you upload rather than after.
Include examples. A record that counts, one that doesn't, a worked calculation. Concrete cases resolve ambiguity faster than more prose.
Keep documents focused. One document per concept is easier to retrieve precisely than a single document covering everything.
Cross-reference instead of repeating. If *active customers* is defined in one document, link to that document from others rather than re-explaining the rule.
Update documents when business rules change. A stale definition is worse than no definition: the agent will follow it confidently.
### Programmatic access
AI agents can also create, update, and delete context documents through the MCP Server. MCP writes run with the token creator's standing: on Growth and Custom, updating a document requires Editor on it and deleting requires Manager, and a scoped token is limited to its scope. See [`create_context_document`](/mcp-server/tools#create_context_document), [`update_context_document`](/mcp-server/tools#update_context_document), and [`delete_context_document`](/mcp-server/tools#delete_context_document) for the full schemas.
## API & MCP
List, retrieve, create, and update context documents.
`get_semantic_context` reads the layer; `create_context_document` and more curate it.
# 3C Plus as a data source
Source: https://docs.nekt.com/sources/3c-plus
Bring data from 3C Plus to your Lakehouse.
3C Plus is a Brazilian automatic dialer and call center platform. The connector extracts your company's call history, agent login sessions, daily agent productivity per campaign, and scheduled callbacks, along with the setup of your operation: agents, users, teams, campaigns and their mailing lists, qualifications (call outcomes), work breaks, inbound queues, and IVRs. With it you can analyze your campaigns' results and your team's productivity in your Lakehouse.
![]()
## Configuring 3C Plus as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the 3C Plus option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
3C Plus authenticates with a token. The connector reads 3C Plus reports, which 3C Plus only shares with users that have the **manager** (gestor) role, so the token must belong to a manager. You can use either:
* **A service token** (recommended): in 3C Plus, go to **Configurações > Integração > Tokens de serviço** and create a token with manager permissions.
* **The API token of a manager user**: in 3C Plus, go to **Configurações > Usuários**, open a user with the manager role, and copy the token from the advanced options.
A token of an agent, or a service token with the agent role, only reaches one agent's data and is refused by the connector. The API token of a user does not expire on its own, but it stops working if a manager changes it in 3C Plus.
The following configurations are available:
* **API Token**: the service token or the manager's API token. Required.
* **Start Date**: the earliest date from which calls, agent logins, agent daily metrics, and scheduled callbacks will be synced. Required. An earlier date makes the first sync longer, mainly for the agent daily metrics, which are read one day per campaign.
* **Include Mailing Data**: also extract, with each call, the contact's data from the mailing list (identifier, phone, and the list's own columns) into the `mailing_data` column. On by default.
* **Lookback Days**: how many days are re-read before the last sync point by the calls, agent logins, and agent daily metrics, to pick up the qualifications, logouts, and totals 3C Plus completes after a record first appears. The default is 1.
* **Time Zone**: the time zone of your 3C Plus account. 3C Plus filters and returns dates in this time zone, and the connector converts them to UTC. The default is America/Sao\_Paulo.
* **Requests Per Minute**: how fast the connector may call 3C Plus. The default is 120.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Calls, Agent Login History, and Agent Metrics Daily support INCREMENTAL. The other streams are FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from 3C Plus and their corresponding fields. API reference: [3C Plus API](https://api-docs.3c.plus).
3C Plus sends dates in your account's time zone; the connector converts every timestamp to UTC. In the Calls stream, durations come as text (`HH:MM:SS`) and also as a number of seconds in the `_seconds` columns. Lists and objects (mailing data, feedback, settings, team members) are stored as JSON text.
Call history report of the company, one row per call (`GET /calls`). Read day by day in the account's time zone; the last **Lookback Days** are re-read on every sync, because 3C Plus completes a call (qualification, feedback) after it first appears.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the call. |
| `sid` | String | Human-readable identifier of the call, unique per call. |
| `call_date` | Datetime | Timestamp of the call (UTC), used as the incremental replication key. |
| `campaign_id` | Integer | Campaign the call belongs to. |
| `campaign` | String | Name of the campaign. |
| `list` | String | Name of the mailing list the number came from. |
| `list_id` | Integer | Mailing list the number came from. |
| `queue_name` | String | Name of the receptive queue, for inbound calls. |
| `number` | String | Phone number called. |
| `phone` | String | Phone number of the contact, when sent separately. |
| `phone_type` | String | Type of the phone: mobile or landline. |
| `identifier` | String | Identifier of the contact in the mailing list. |
| `mode` | String | Dialing mode of the call. |
| `has_agent` | Boolean | Whether the call reached an agent. |
| `agent_id` | Integer | Agent who handled the call. |
| `agent` | String | Name of the agent who handled the call. |
| `qualification_id` | Integer | Qualification (call outcome) given. |
| `qualification` | String | Qualification (call outcome) given: its name, or its id on older payloads. |
| `qualification_note` | String | Note the agent left when qualifying. |
| `behavior` | Integer | Behavior code of the qualification. |
| `readable_behavior_text` | String | Readable behavior of the qualification. |
| `is_dmc` | Boolean | Whether the call was a decision maker contact (DMC). |
| `status_id` | String | Status code of the call. |
| `readable_status_text` | String | Readable status of the call. |
| `readable_amd_status_text` | String | Answering machine detection result. |
| `hangup_cause` | String | Hangup cause code. |
| `readable_hangup_cause_text` | String | Readable hangup cause. |
| `ended_by_agent` | Boolean | Whether the agent ended the call. |
| `speaking_time` | String | Time spent speaking (HH:MM:SS). |
| `speaking_time_seconds` | Integer | Time spent speaking, in seconds. |
| `speaking_with_agent_time` | String | Time the contact spoke with the agent (HH:MM:SS). |
| `speaking_with_agent_time_seconds` | Integer | Time the contact spoke with the agent, in seconds. |
| `acw_time` | String | After-call work time of the agent (HH:MM:SS). |
| `acw_time_seconds` | Integer | After-call work time, in seconds. |
| `ivr_time` | String | Time spent in the IVR (HH:MM:SS). |
| `ivr_time_seconds` | Integer | Time spent in the IVR, in seconds. |
| `billed_time` | String | Billed time of the call (HH:MM:SS). |
| `billed_time_seconds` | Integer | Billed time of the call, in seconds. |
| `billed_value` | Number | Billed value of the call (BRL). |
| `route_id` | Integer | Telephony route used by the call. |
| `route_name` | String | Name of the telephony route. |
| `route_host` | String | Host of the telephony route. |
| `recorded` | Boolean | Whether the call was recorded. |
| `recording` | String | API URL of the call recording. |
| `recording_amd` | String | API URL of the AMD recording. |
| `recording_transfer` | String | API URL of the transfer recording. |
| `recording_consult` | String | API URL of the consult recording. |
| `is_transferred` | Boolean | Whether the call was transferred. |
| `is_transfer` | Boolean | Whether the call is a transfer. |
| `is_consult` | Boolean | Whether the call is a consult (warm transfer). |
| `parent` | String | API URL of the parent call (transfers and consults). |
| `parent_id` | String | Identifier of the parent call. |
| `transfer` | String | API URL of the transfer call. |
| `transfer_id` | String | Identifier of the transfer call. |
| `consult` | String | API URL of the consult call. |
| `consult_id` | String | Identifier of the consult call. |
| `consults` | String | Consults generated by the call (JSON array). |
| `consult_ids` | String | Identifiers of the consults (JSON). |
| `mailing_data` | String | Contact data from the mailing list: identifier, phone and the list's own columns (JSON object). Empty when Include Mailing Data is off. |
| `feedback` | String | Manager feedback given on the call (JSON object). |
| `additional_fields` | String | Fields 3C Plus returned that have no column of their own (JSON object). |
Login sessions of agents in campaigns (`GET /agents/login_history`). A session that is still open has no logout yet and is completed by a later sync.
Primary key: `row_key`
| Field | Type | Description |
| :- | :- | :- |
| `row_key` | String | Surrogate key of the session: a hash of agent, campaign and login time. |
| `login_at` | Datetime | Timestamp the agent logged in (UTC), used as the incremental replication key. |
| `logout_at` | Datetime | Timestamp the agent logged out (UTC). |
| `is_logged_in` | Boolean | Whether the session was still open when read. |
| `duration_seconds` | Integer | Length of the session, in seconds. |
| `agent_id` | Integer | Agent of the session. |
| `agent_name` | String | Name of the agent. |
| `campaign_id` | Integer | Campaign the agent logged into. |
| `campaign_name` | String | Name of the campaign. |
Productivity totals of each agent in each campaign per day (`GET /campaigns/{id}/agents/metrics/total`, read one day at a time). Agents without any activity on a day are left out.
Every metric is an integer: durations in seconds, the rest counts. The totals add up -- `workload = speaking + acw + manual` and `total_logged = workload + idle + interval` -- and each `*_average` is its total divided by the matching count.
Primary key: `campaign_id`, `agent_id`, `date`
| Field | Type | Description |
| :- | :- | :- |
| `campaign_id` | Integer | Campaign of the metrics. |
| `agent_id` | Integer | Agent of the metrics. |
| `date` | Date | Day of the metrics (account time zone), used as the incremental replication key. |
| `agent_name` | String | Name of the agent. |
| `total_logged` | Integer | Time logged in, in seconds (workload + idle + interval). |
| `workload` | Integer | Time working, in seconds (speaking + acw + manual). |
| `speaking` | Integer | Time speaking on dialer calls, in seconds. |
| `speaking_average` | Integer | Average speaking time per dialer call, in seconds. |
| `acw` | Integer | Time in after-call work, in seconds. |
| `acw_average` | Integer | Average after-call work time per dialer call, in seconds. |
| `idle` | Integer | Time idle, waiting for a call, in seconds. |
| `idle_average` | Integer | Average length of an idle period, in seconds. |
| `interval` | Integer | Time on work breaks, in seconds. |
| `interval_average` | Integer | Average length of a work break, in seconds. |
| `manual` | Integer | Time in manual mode, in seconds. |
| `manual_average` | Integer | Average length of a manual-mode session, in seconds. |
| `manual_acw` | Integer | Time in manual after-call work, in seconds. |
| `manual_acw_average` | Integer | Average length of a manual after-call work session, in seconds. |
| `manual_calls_calling` | Integer | Time dialing in manual mode, in seconds. |
| `manual_calls_speaking` | Integer | Time speaking on manual-mode calls, in seconds. |
| `manual_calls_acw_calling` | Integer | Time dialing during manual after-call work, in seconds. |
| `manual_calls_acw_speaking` | Integer | Time speaking on calls made during manual after-call work, in seconds. |
| `positive_qualification` | Integer | Time speaking on calls qualified as positive, in seconds (dialer and manual). |
| `positive_qualification_average` | Integer | Time speaking on calls qualified as positive per dialer call, in seconds. |
| `negative_qualification` | Integer | Time speaking on calls qualified as negative, in seconds (dialer and manual). |
| `negative_qualification_average` | Integer | Time speaking on calls qualified as negative per dialer call, in seconds. |
| `calls` | Integer | Dialer calls delivered to the agent. |
| `idle_quantity` | Integer | Idle periods of the agent. |
| `interval_quantity` | Integer | Work breaks taken by the agent. |
| `manual_calls` | Integer | Manual-mode sessions of the agent. |
| `manual_calls_made` | Integer | Calls dialed in manual mode. |
| `manual_calls_answered` | Integer | Manual-mode calls that were answered. |
| `manual_calls_acw` | Integer | Manual after-call work sessions of the agent. |
| `manual_calls_acw_made` | Integer | Calls dialed during manual after-call work. |
| `manual_calls_acw_answered` | Integer | Calls dialed during manual after-call work that were answered. |
Scheduled callbacks (`GET /schedules`), from the **Start Date** to one year ahead, so pending callbacks are included. Refreshed in full on every sync, since a schedule's status changes after it is created.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the schedule. |
| `date` | Datetime | Timestamp the callback is scheduled for (UTC). |
| `phone` | String | Phone number to call back. |
| `note` | String | Note left on the schedule. |
| `status` | String | Status of the schedule (e.g. pending). |
| `agent_id` | Integer | Agent the callback is assigned to. |
| `agent_name` | String | Name of the agent. |
| `campaign_id` | Integer | Campaign of the callback. |
| `campaign_name` | String | Name of the campaign. |
| `mailing_data` | String | Contact data from the mailing list, with the list's own columns (JSON object). |
Agents of the company, active and inactive (`GET /agents`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the agent. |
| `name` | String | Name of the agent. |
| `active` | Boolean | Whether the agent is active. |
| `extension_id` | Integer | Identifier of the agent's telephony extension. |
| `extension_number` | Integer | Extension number the agent uses in the 3C Plus telephony system. |
| `extension_type` | String | Type of the extension. |
| `profile_url` | String | API URL of the agent's profile. |
Users of the company, active and inactive: managers, supervisors, and agents (`GET /users`). The API token, extension password, and confirmation code 3C Plus keeps for each user are never extracted.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the user. |
| `name` | String | Name of the user. |
| `email` | String | Email address of the user. |
| `active` | Boolean | Whether the user is active. |
| `confirmed` | Boolean | Whether the user confirmed the account. |
| `telephony_id` | String | Identifier of the user in the telephony system. |
| `webphone` | Boolean | Whether the user has access to the web phone. |
| `user_document` | String | Document (CPF) of the user. |
| `last_login` | Datetime | Timestamp of the user's last login (UTC). |
| `role_id` | Integer | Identifier of the user's role. |
| `role_name` | String | Internal name of the role (e.g. agent, manager). |
| `role_readable_name` | String | Display name of the role. |
| `extension_id` | Integer | Identifier of the user's extension. |
| `extension_number` | Integer | Extension number of the user. |
| `extension_type` | String | Type of the extension. |
| `team_ids` | Array | Identifiers of the teams the user belongs to. |
| `settings` | String | User preferences: time zone, language, date and hour formats (JSON object). |
| `teams` | String | Teams the user belongs to (JSON array). |
| `campaigns` | String | Campaigns the user takes part in (JSON array). |
| `receptive_queues` | String | Receptive queues the user answers (JSON array). |
| `user_updated_data` | String | Additional profile data 3C Plus keeps for the user, when available (JSON object). |
| `created_at` | Datetime | Timestamp the user was created (UTC). |
| `updated_at` | Datetime | Timestamp of the user's last update (UTC). |
Teams of agents and their supervisors (`GET /teams`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the team. |
| `name` | String | Name of the team. |
| `color` | String | Color of the team (hex). |
| `whatsapp` | Boolean | Whether the team uses the WhatsApp integration. |
| `qualification_list_id` | Integer | Qualification list used by the team. |
| `whatsapp_quick_message_list_id` | Integer | Quick message list used by the team on WhatsApp. |
| `agent_ids` | Array | Identifiers of the team's agents. |
| `supervisor_ids` | Array | Identifiers of the team's supervisors. |
| `agents` | String | Agents of the team (JSON array). |
| `supervisor` | String | Supervisors of the team (JSON array). |
Dialer campaigns, running and paused (`GET /campaigns`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the campaign. |
| `name` | String | Name of the campaign. |
| `start_time` | String | Time of day the campaign starts dialing. |
| `end_time` | String | Time of day the campaign stops dialing. |
| `paused` | Boolean | Whether the campaign is paused. |
| `is_on_active_time` | Boolean | Whether the campaign is inside its active hours. |
| `is_predictive` | Boolean | Whether the campaign uses the predictive dialer. |
| `allows_manual` | Boolean | Whether agents can dial manually. |
| `amd_enabled` | Boolean | Whether answering machine detection (AMD) is enabled. |
| `acw_timeout` | Integer | After-call work (ACW) timeout in seconds; 0 means no timeout. |
| `limit_call_time` | Integer | Maximum call duration in seconds. |
| `limit_call_per_agent` | Number | Call limit per agent. |
| `exit_manual_mode` | Integer | Seconds before an agent leaves manual mode. |
| `caller_id` | String | Caller ID shown on outbound calls. |
| `asr` | Number | Answer-seizure ratio (ASR) configured for the campaign. |
| `ura_limit` | Integer | Limit of calls held in the IVR (URA). |
| `active_list_notify` | Boolean | Whether to notify when a mailing list is active. |
| `copy_identifier` | Boolean | Whether the connected call's identifier is copied. |
| `ivr_after_call_id` | Integer | IVR played after the call. |
| `route_limit_exceeded` | Boolean | Whether the route limit was exceeded. |
| `route_landline_id` | Integer | Route used for landline calls. |
| `route_mobile_id` | Integer | Route used for mobile calls. |
| `route_group_landline_id` | Integer | Route group used for landline calls. |
| `route_group_mobile_id` | Integer | Route group used for mobile calls. |
| `filter_calls` | Boolean | Whether call filtering is enabled. |
| `should_complete_failed_call` | Boolean | Whether failed calls are marked as completed. |
| `update_mailing_data` | Boolean | Whether mailing data is updated during calls. |
| `company_id` | Integer | Identifier of the 3C Plus company. |
| `extension_id` | Integer | Identifier of the campaign's extension. |
| `extension_number` | Integer | Extension number of the campaign. |
| `work_break_group_id` | Integer | Work break group used by the campaign. |
| `work_break_group_name` | String | Name of the work break group. |
| `dialer_settings` | String | Dialer configuration (JSON object). |
| `route_landline` | String | Landline route details (JSON object). |
| `route_mobile` | String | Mobile route details (JSON object). |
| `campaign_information` | String | Runtime counters 3C Plus attaches to the campaign, when sent (JSON object). |
Mailing lists of each campaign with their dialing progress (`GET /campaigns/{id}/lists`).
Primary key: `campaign_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `campaign_id` | Integer | Campaign the list belongs to. |
| `id` | Integer | Unique identifier of the mailing list. |
| `name` | String | Name of the mailing list. |
| `total` | Integer | Total phones in the list. |
| `dial` | Integer | Phones still to dial. |
| `redial` | Integer | Phones to redial. |
| `dialed` | Integer | Phones already dialed. |
| `dialed_percentage` | Number | Percentage of the phones already dialed. |
| `completed` | Integer | Calls completed on the list. |
| `completed_percentage` | Number | Percentage of completed calls. |
| `answered` | Integer | Calls answered on the list. |
| `answered_percentage` | Number | Percentage of answered calls. |
| `abandoned` | Integer | Calls abandoned on the list. |
| `asr` | Number | Answer-seizure ratio (ASR) of the list. |
| `asr_percentage` | Number | ASR of the list as a percentage. |
| `weight` | Integer | Weight of the list in the campaign. |
| `weight_percentage` | Number | Weight of the list as a percentage. |
| `dial_percentage` | Number | Share of the campaign's dialing on the list. |
| `redial_percentage` | Number | Share of the campaign's redialing on the list. |
| `created_at` | Datetime | Timestamp the list was created (UTC). |
Lists of qualifications (call outcomes) (`GET /qualification_lists`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the qualification list. |
| `name` | String | Name of the qualification list. |
| `type` | Integer | Type of the qualification list. |
| `campaign_ids` | Array | Campaigns that use the list. |
| `campaigns` | String | Campaigns that use the list (JSON array). |
Qualifications (call outcomes) of each list (`GET /qualification_lists/{id}/qualifications`).
Primary key: `qualification_list_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `qualification_list_id` | Integer | Qualification list the qualification belongs to. |
| `id` | Integer | Unique identifier of the qualification. |
| `name` | String | Name of the qualification (call outcome). |
| `behavior` | Integer | Behavior code of the qualification. |
| `readable_behavior` | String | Readable description of the behavior. |
| `color` | String | Color of the qualification (hex). |
| `emoji` | String | Emoji of the qualification. |
| `conversion` | Boolean | Whether the qualification counts as a conversion. |
| `dmc` | Boolean | Whether it counts as a decision maker contact (DMC). |
| `unknown` | Boolean | Whether it marks the contact as unknown. |
| `allow_schedule` | Boolean | Whether a callback can be scheduled. |
| `allow_schedule_to_another_number` | Boolean | Whether a callback can be scheduled to a different number. |
| `should_insert_blacklist` | Boolean | Whether the number is added to the blacklist. |
| `blacklist_blocking_days` | Integer | Days the number stays blacklisted. |
| `days_limit` | Integer | Days limit to schedule a callback. |
| `campaign_id` | Integer | Campaign the qualification is tied to, if any. |
Groups of pause types (`GET /work_break_group`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the work break group. |
| `name` | String | Name of the work break group. |
Pause types of each work break group (`GET /work_break_group/{id}/intervals`).
Primary key: `work_break_group_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `work_break_group_id` | Integer | Work break group of the interval. |
| `id` | Integer | Unique identifier of the interval. |
| `name` | String | Name of the pause (e.g. lunch, bathroom). |
| `minutes` | Integer | Duration allowed for the pause, in minutes. |
| `color` | String | Color of the interval (hex). |
Inbound queues (`GET /receptive_queues`). Skipped when the account does not use inbound calls.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the queue. |
| `name` | String | Name of the queue. |
| `wait_time` | Integer | Maximum waiting time in the queue, in seconds. |
| `acw_timeout` | Integer | After-call work timeout, in seconds. |
| `priority` | Integer | Priority of the queue. |
| `qualification_list_id` | Integer | Qualification list used by the queue. |
| `audio_path` | String | Path of the queue's audio. |
| `limit_call_time` | Integer | Maximum call duration, in seconds. |
| `ivr_after_call_id` | Integer | IVR played after the call. |
| `extension_id` | Integer | Identifier of the queue's extension. |
| `extension_number` | Integer | Extension number of the queue. |
| `agent_ids` | Array | Agents that answer the queue. |
| `agents` | String | Agents that answer the queue (JSON array). |
Inbound IVRs (`GET /receptive_ivr`). Skipped when the account does not use inbound calls.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the IVR. |
| `name` | String | Name of the IVR. |
| `timeout` | Integer | Timeout of the IVR, in seconds. |
| `original_audio_name` | String | Original name of the IVR's audio file. |
| `audio_path` | String | API path of the IVR's audio. |
| `redirect_id` | String | Destination the IVR redirects to. |
Tags of the company (`GET /tags`). Skipped when not available for the account.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the tag. |
| `name` | String | Name of the tag. |
| `color` | String | Color of the tag (hex). |
SMS mailing lists and their sending progress (`GET /mailing_list_sms`). Skipped when the account does not use SMS.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the SMS mailing list. |
| `name` | String | Name of the SMS mailing list. |
| `original_name` | String | Name of the file the list was imported from. |
| `company_id` | Integer | Identifier of the 3C Plus company. |
| `message` | String | Text of the SMS sent to the list. |
| `schedule` | Datetime | Timestamp the sending is scheduled for (UTC). |
| `status` | Integer | Status code of the list. |
| `paused` | Boolean | Whether the sending is paused. |
| `importing` | Boolean | Whether the list is still being imported. |
| `total` | Integer | Total contacts in the list. |
| `sent` | Integer | Messages sent. |
| `success` | Integer | Messages delivered successfully. |
| `failure` | Integer | Messages that failed. |
| `not_sent` | Integer | Messages not sent yet. |
| `created_at` | Datetime | Timestamp the list was created (UTC). |
# 3D Analytics as a data source
Source: https://docs.nekt.com/sources/3d-analytics
Bring data from 3D Analytics to Nekt.
3D Analytics is a platform for advanced data visualization and analysis using 3D models. It enables organizations to
explore complex datasets in three dimensions, providing deeper insights and interactive visual representations for better
decision-making.
## 1. Add your 3D Analytics access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the 3D Analytics (JSON) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
3. You need to insert the ID of the project you want to extract data from and select the type of API service: **MetaKosmos 3D** or **PulpoAR**. This will set the appropriate request parameters according to the selected service.
4. The Start Date is the date of the first analytics record you want to sync. A new analytics record will be extracted for each subsequent day until it reaches the current date.
5. Click **Next**.
## 2. Select your 3D Analytics data streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them. For this connector, we only support a single incremental Analytics stream.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your 3D Analytics data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your 3D Analytics data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Done** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
3D Analytics connector documentation as plain markdown, for use in AI agent contexts.
# Active Campaign as a data source
Source: https://docs.nekt.com/sources/active-campaign
Bring data from Active Campaign to Nekt.
ActiveCampaign is a customer experience automation platform that provides email marketing, marketing automation, and CRM tools to help businesses connect and engage with their customers.
## Configuring Active Campaign as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Active Campaign option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Active Campaign **URL** and **API Key** for this connection. You can find it in your Active Campaign account settings, under the "Developer" tab. Once you have it, add the account access and click **Next**.
**Advanced settings:**
* **Start Date** (optional): The earliest record date to sync.
* **Extract contact custom fields** (optional): When enabled, makes an additional call per contact to include custom field values. Useful when you have many contacts and custom fields; custom fields do not support incremental replication.
* **Extract deal custom fields** (optional): When enabled, makes an additional call per deal to include custom field values. Useful when you have many deals and custom fields; custom fields do not support incremental replication.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Active Campaign and their corresponding fields. **Deals** and **Deal Groups** appear only when your account has access to the Deals API; otherwise they are omitted from the stream list.
Stream for managing deals, representing sales opportunities with contacts.
**Deals Full vs. Partial Access:**
When configuring the Deals stream, you have two access levels:
* **Partial Access:** Syncs a limited set of fields, ideal for a quick overview. The replication key is not available, so only full syncs are supported.
* **Full Access:** Syncs all available deal fields, providing comprehensive data for in-depth analysis. The `mdate` (modification date) is used as the replication key, allowing for incremental syncs.
In order to have full access, the authenticated user needs to have `Manage pipeline` permission.
**Key Fields (Full Access):**
* `id` - Unique identifier for the deal.
* `title` - The title or name of the deal.
* `owner` - The ID of the user who owns the deal.
* `contact` - The ID of the primary contact associated with the deal.
* `organization` - The ID of the organization associated with the deal.
* `stage` - The ID of the current stage in the pipeline.
* `value` - The monetary value of the deal.
* `currency` - The currency of the deal's value.
* `status` - The current status of the deal (0=Open, 1=Won, 2=Lost).
* `cdate` - The date the deal was created.
* `mdate` - The date the deal was last modified.
**Additional Fields (Full Access):**
`hash`, `group`, `description`, `percent`, `nextdate`, `nexttaskid`, `winProbability`, `winProbabilityMdate`, `activitycount`, `nextdealid`, `edate`, `isDisabled`, `account`, `customerAccount`, and `links` to related entities.
Stream for managing contacts, the individuals you communicate with.
**Key Fields:**
* `id` - Unique identifier for the contact.
* `email` - The primary email address of the contact.
* `firstName` - The first name of the contact.
* `lastName` - The last name of the contact.
* `phone` - The phone number of the contact.
* `orgid` - The ID of the organization the contact belongs to.
* `cdate` - The date the contact was created.
* `udate` - The date the contact was last updated.
**Additional Fields:**
`orgname`, `segmentio_id`, `bounced_hard`, `bounced_soft`, `bounced_date`, `ip`, `ua`, `hash`, `socialdata_lastcheck`, `email_local`, `email_domain`, `sentcnt`, `rating_tstamp`, `gravatar`, `deleted`, `anonymized`, `adate`, `edate`, `deleted_at`, `created_utc_timestamp`, `updated_utc_timestamp`, `created_timestamp`, `updated_timestamp`, `created_by`, `updated_by`, `mpp_tracking`, `last_click_date`, `last_open_date`, `last_mpp_open_date`, `best_send_hour`, `scoreValues`, `accountContacts`, `links`, and `organization`.
If `extract_contact_custom_fields` is enabled, it also includes `customFields`.
Stream for managing deal groups, which are pipelines containing stages.
**Key Fields:**
* `id` - Unique identifier for the deal group.
* `title` - The name of the deal group (pipeline).
* `color` - The color code for the deal group.
* `order` - The display order of the deal group.
* `cdate` - The date the deal group was created.
* `udate` - The date the deal group was last updated.
**Additional Fields:**
`group`, `width`, `dealOrder`, `cardRegion1`, `cardRegion2`, `cardRegion3`, `cardRegion4`, `cardRegion5`, and `links` to related entities.
Stream for managing marketing campaigns.
**Key Fields:**
* `id` - Unique identifier for the campaign.
* `name` - The name of the campaign.
* `type` - The type of campaign (e.g., 'email', 'sms').
* `status` - The status of the campaign.
* `cdate` - The date the campaign was created.
* `sdate` - The date the campaign was sent.
**Metrics:**
* `opens` - Total number of opens.
* `uniqueopens` - Number of unique opens.
* `linkclicks` - Total number of link clicks.
* `uniquelinkclicks` - Number of unique link clicks.
* `unsubscribes` - Number of unsubscribes.
**Additional Fields:**
A wide range of fields related to campaign settings, tracking, and performance metrics like `userid`, `segmentid`, `send_amt`, `total_amt`, `hardbounces`, `softbounces`, and `links`.
Stream for managing custom fields available for contacts and deals.
**Key Fields:**
* `id` - Unique identifier for the custom field.
* `title` - The name of the custom field.
* `type` - The type of the field (e.g., 'text', 'dropdown').
* `descript` - A description of the field.
* `isrequired` - Whether the field is mandatory.
* `cdate` - The date the field was created.
* `udate` - The date the field was last updated.
**Additional Fields:**
`perstag`, `defval`, `show_in_list`, `rows`, `cols`, `visible`, `service`, `ordernum`, `options`, `relations`, and `links`.
## Data Model
The following diagram illustrates the relationships between the core data streams in Active Campaign.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Contacts("Contacts");
Deals("Deals");
DealGroups("Deal Groups (Pipelines)");
Campaigns("Campaigns");
end
subgraph "Customization"
Fields("Custom Fields");
end
Deals -- "contact" --> Contacts;
Deals -- "group" --> DealGroups;
Campaigns -- "Sends to" --> Contacts;
```
## Skills for agents
Active Campaign connector documentation as plain markdown, for use in AI agent contexts.
# Acuity Scheduling
Source: https://docs.nekt.com/sources/acuity
Connect your Acuity Scheduling data to Nekt.
Acuity Scheduling is an online appointment scheduling platform that allows businesses to manage bookings, calendars, and client information. This connector extracts data about appointments, calendars, appointment types, and products from your Acuity Scheduling account.
![]()
## Configuring Acuity Scheduling as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Acuity Scheduling** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
To connect Acuity Scheduling, you need:
* **User ID**: Your numeric Acuity Scheduling User ID.
* **API Key**: Your Acuity Scheduling API Key.
Both credentials are available in your Acuity Scheduling dashboard under **Integrations > API**.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis:
* **Appointments**
* **Calendars**
* **Appointment Types**
* **Products**
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated.
Optionally, you can define some additional settings:
* **Start Date** (optional): The earliest date to fetch appointments from. Only appointments on or after this date will be extracted.
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Acuity Scheduling and their corresponding fields:
Scheduled appointments with client details, pricing, calendar assignment, and status information. Fetches both active and canceled appointments. Supports incremental sync based on appointment datetime.
| Field | Type | Description |
| :- | :- | :- |
| `id` | integer | Unique appointment identifier |
| `firstName` | string | Client first name |
| `lastName` | string | Client last name |
| `phone` | string | Client phone number |
| `email` | string | Client email address |
| `date` | string | Formatted appointment date |
| `time` | string | Formatted start time |
| `endTime` | string | Formatted end time |
| `dateCreated` | string | Appointment creation date |
| `datetime` | datetime | ISO 8601 appointment datetime (replication key) |
| `price` | string | Appointment price |
| `paid` | string | Payment status (yes/no) |
| `amountPaid` | string | Amount paid |
| `type` | string | Appointment type name |
| `appointmentTypeID` | integer | Appointment type identifier |
| `addonIDs` | string | Associated addon IDs (JSON) |
| `classID` | integer | Class ID if applicable |
| `duration` | string | Duration in minutes |
| `calendar` | string | Calendar name |
| `calendarID` | integer | Calendar identifier |
| `canClientCancel` | boolean | Whether client can cancel |
| `canClientReschedule` | boolean | Whether client can reschedule |
| `location` | string | Appointment location |
| `certificate` | string | Certificate code if used |
| `confirmationPage` | string | Confirmation URL |
| `formsText` | string | Form summary text |
| `notes` | string | Appointment notes |
| `timezone` | string | Client timezone |
| `forms` | string | Intake form responses (JSON) |
| `labels` | string | Applied labels (JSON) |
| `noShow` | boolean | Marked as no-show |
| `scheduledBy` | string | Admin who scheduled it |
| `canceled` | boolean | Whether appointment is canceled |
Staff calendars with contact and location information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | integer | Unique calendar identifier |
| `name` | string | Calendar name |
| `email` | string | Associated email address |
| `replyTo` | string | Reply-to email address |
| `description` | string | Calendar description |
| `location` | string | Calendar location |
| `timezone` | string | Timezone identifier |
Service types and appointment configurations with pricing and duration.
| Field | Type | Description |
| :- | :- | :- |
| `id` | integer | Unique appointment type identifier |
| `active` | boolean | Whether the appointment type is active |
| `name` | string | Appointment type name |
| `description` | string | Appointment type description |
| `duration` | integer | Duration in minutes |
| `price` | string | Price in account currency |
| `category` | string | Category of the appointment type |
| `color` | string | Display color |
| `private` | boolean | Whether the type is private |
| `type` | string | Service type (service, class, series) |
| `classSize` | integer | Maximum class capacity |
| `paddingAfter` | integer | Buffer time after appointment (minutes) |
| `paddingBefore` | integer | Buffer time before appointment (minutes) |
| `calendarIDs` | array | Associated calendar identifiers |
Packages and product offerings with pricing and configuration.
| Field | Type | Description |
| :- | :- | :- |
| `id` | integer | Unique product identifier |
| `name` | string | Product name |
| `description` | string | Product description |
| `price` | string | Price in account currency |
| `type` | string | Product type (product, appointments, minutes) |
| `hidden` | boolean | Whether the product is hidden from the store |
| `expires` | integer | Days until package expiration |
| `appointmentTypeIDs` | array | Associated appointment type IDs |
| `appointmentTypeCounts` | string | Appointment counts by type (JSON) |
| `minutes` | integer | Minutes included in package |
# Data Model
The following diagram illustrates the basic relationships between the core data streams in Acuity Scheduling.
```mermaid theme={null}
erDiagram
APPOINTMENTS {
integer id PK
integer appointmentTypeID FK
integer calendarID FK
}
CALENDARS {
integer id PK
}
APPOINTMENT_TYPES {
integer id PK
}
PRODUCTS {
integer id PK
}
APPOINTMENTS }o--|| APPOINTMENT_TYPES : "is of type"
APPOINTMENTS }o--|| CALENDARS : "assigned to"
```
## Skills for agents
Acuity Scheduling connector documentation as plain markdown, for use in AI agent contexts.
# Adyen as a data source
Source: https://docs.nekt.com/sources/adyen
Bring data from Adyen to Nekt.
Adyen is a global payments platform that processes card, wallet and local payment methods, settles the funds and pays them out to your bank account. This connector brings your Adyen transaction, settlement and dispute data into your Nekt Catalog.
## Before you start
Adyen works differently from most connectors, and it is worth understanding before you begin.
Adyen has **no API that lists your transactions**. Everything transactional leaves the platform as a **report file** that Adyen generates in your Customer Area and serves over HTTPS. This connector downloads those files: it can only read a report that **already exists**. Adyen never generates one on request, and it offers no way to list which ones are available.
So there are two things you have to do in your Adyen Customer Area before the connector produces anything:
1. **Turn on automatic generation for each report you want**, in CSV.
2. **Create a Report user API credential** for Nekt to download them with.
If a report is not set to generate automatically, Adyen answers "not found" for it, exactly as it does for a day with no data. The connector reports in the run log which report and which account found nothing, but it cannot tell the two situations apart.
### 1. Turn on the reports
In your [Customer Area](https://ca-live.adyen.com/), for each report below: go to **Reports**, open the report, select **Manage report** > **Automatic (generate on a schedule)**, and choose the **CSV** file format.
| Report | Stream it feeds | What it answers | Suggested frequency |
| - | - | - | - |
| Received payment details | `received_payment_details` | Which payments came in, which were approved or refused, and what the acquirer answered | Daily |
| Payment accounting | `payment_accounting` | Every accounting event of every payment, with the fees charged at each stage | Daily |
| Settlement details | `settlement_details` | What was actually settled and paid out, and what each transaction cost | At batch close |
| Dispute transaction details | `dispute_transaction_details` | Chargebacks, notifications of fraud and requests for information | Daily |
| External settlement detail | `external_settlement_details` | Transactions settled by an external acquirer (only if you use one) | Daily |
You need the **Merchant Report** or **Generate key financial reports** [user role](https://docs.adyen.com/account/user-roles) to configure them.
Note the **account level** you generate each report at. Most reports can be generated for the company account, for individual merchant accounts, or both, and the connector has to look in the same place. By default it looks where Adyen publishes each report: per merchant account for the payment-oriented ones, and for the company account for the dispute report. The Settlement details report only exists per merchant account.
Adyen keeps generated reports for **two years**. Data older than that cannot be recovered through this connector; it has to be regenerated in the Customer Area first.
### 2. Create a Report user credential
In your Customer Area, go to **Developers** > **API credentials** > **Create new credential**:
1. Under **User type**, select **Report user**.
2. Under **Roles and Associated Accounts**, make sure the **Merchant Report Download** role is on, and give the credential access to the accounts you want to read.
3. Under **Authentication**, either create a password for **Basic Auth** (and note the user name shown next to it) or select **Generate New API Key**. Nekt accepts either one.
4. Select **Save**.
Copy the password or the API key immediately. Adyen shows each of them only once.
## Configuring Adyen as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Adyen option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **Company Account**: the name of your Adyen company account, exactly as it appears at the top of your Customer Area.
* **Merchant Accounts**: the names of the merchant accounts to read reports for, separated by commas. Required unless every report of yours is generated at company level.
* **Report User** and **Report User Password**: the Basic auth credentials of the Report user API credential you created above.
* **API Key**: the API key of that same credential, as an alternative to the user name and password. Fill in one or the other.
* **Environment**: `live` (default) or `test`. Live and test are separate Customer Areas with separate credentials, so the credentials must belong to the environment you select here.
The advanced settings let you match the connector to how your reports are generated:
* **Report account level**: (Default: `merchant`) where your reports are generated. The default reads each report at the level Adyen publishes it by default: the payment-oriented reports per merchant account, and the Dispute transaction details report for the company account. Choose `company` if you generate all of them for the company account, or `both` if you generate both; note that `both` extracts the same payment twice when a report exists at both levels.
* **Initial sync date**: (Default: 90 days ago) the earliest report date to read. The connector looks for one file per day from this date onwards, so a very early date makes the first sync long.
* **Lookback (days)**: (Default: 3) how many days before the last synced date every run reads again. Adyen generates the daily reports between 02:00 and 10:00 CET of the following day, so the lookback makes sure a report that was not ready during the previous run is still picked up.
* **Report time zone**: the IANA name of the time zone your reports are written in, for example `America/Sao_Paulo`. Adyen writes report timestamps in your account's reporting time zone with no offset, and names that zone in a separate column as an abbreviation that is ambiguous for several of its values. Set this and every timestamp is converted to UTC; leave it empty and timestamps are stored exactly as Adyen wrote them.
* **Report file format**: (Default: `csv`) the format your reports are generated in.
* **Initial settlement batch**: (Default: 1) the settlement batch number the first sync starts looking from. Adyen names the Settlement details report after the payout batch instead of a date, so set this to the oldest batch number still listed under **Reports** > **Settlement details** > **Generated reports**.
* **Settlement miss tolerance** and **Settlement probe limit**: how many consecutive missing batch reports end the walk, once a batch has been read (default 5) and before any batch has been read (default 500).
* **Requests per minute**: (Default: 240) an upper bound on how fast the connector downloads report files.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams whose reports you actually generate in Adyen.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Report rows have no unique identifier: the same `psp_reference` appears once per accounting event, and fee and payout entries carry no reference at all. The connector therefore declares no primary key, and the incremental sync is by report file: each run reads the files it has not read yet, plus the lookback window.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. A daily trigger scheduled after 10:00 CET matches Adyen's own report generation window.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Good to know
* **Your report columns are yours to choose.** Under **Settings** > **Report columns** in your Customer Area you decide which columns each report carries and in which order. The connector knows every column Adyen documents and reads rows by column name, so adding a column or reordering them is safe. A column that Adyen does not document cannot be extracted, and the run log names it when that happens.
* **Two reports are not covered.** The **Advancements detail report** and the **Aggregate settlement details report** are named after values that cannot be predicted (the advancement code, or the schedule you configured), and Adyen offers no way to list the generated files, so the connector cannot find them.
* **Timestamps are in your reporting time zone**, not UTC, unless you fill in the **Report time zone** setting. Each timestamp column has a paired column naming the zone it was written in.
# Streams and Fields
## Available streams
| Stream | Slug | Adyen report | Description |
| - | - | - | - |
| Settlement Details | `settlement_details` | Settlement details | Settled transactions, fees and payouts, per payout batch. |
| Payment Accounting | `payment_accounting` | Payment accounting | Every accounting event of every payment, with its fees. |
| Received Payment Details | `received_payment_details` | Received payment details | Every payment attempt, with the acquirer response that approved or refused it. |
| Dispute Transaction Details | `dispute_transaction_details` | Dispute transaction details | Chargebacks, notifications of fraud and requests for information. |
| External Settlement Details | `external_settlement_details` | External settlement detail | Transactions settled by an external acquirer. |
## Fields by stream
Below you'll find all available data streams from Adyen and their corresponding fields. A column only reaches your Catalog if it is switched on for that report in your Customer Area.
Settled transactions, transaction fees, deposit corrections, invoice deductions and payouts, as they appear in the Settlement details report of one payout batch. Adyen names this report after the batch number, so the connector reads one file per batch.
**Report columns:**
* `company_account` - The name of your company account as registered in the Adyen payments platform
* `merchant_account` - The name of the merchant account that was used to process the original payment request
* `psp_reference` - Adyen's 16-character unique reference associated with the transaction/the request. This value is globally unique; quote it when communicating with
* `merchant_reference` - The reference that was specified in the payment request
* `payment_method` - The payment method used for the transaction. For example: visa, mc, amex
* `creation_date` - The timestamp indicating when the payment was created on the Adyen payments platform. Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `timezone` - The time zone of the Creation Date. This is the ISO code for your time zone setting
* `creation_date_ams` - The Creation Date in Amsterdam time (CET or CEST). Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `type` - The Journal type of the entry
* `modification_reference` - Adyen's unique 16-character reference for the modification (capture, refund or chargeback) that produced the entry. For MerchantPayout entries it
* `gross_currency` - The three-character ISO currency code for the transaction currency
* `gross_debit_gc` - The amount submitted in the transaction request. For example, 100.00
* `gross_credit_gc` - The amount submitted in the transaction request. For example, 100.00
* `exchange_rate` - The exchange rate used for converting the gross amount into the net amount. This will be 1 if you process and settle in the same currency.For
* `net_currency` - The three-character ISO currency code for the settlement currency
* `net_debit_nc` - The net amount debited from the payable batch
* `net_credit_nc` - The net amount credited from the payable batch
* `commission_nc` - The commission fee withheld by the acquirer, in the settlement (net) currency. When the acquirer reports at interchange level this column is empty
* `markup_nc` - The fee charged by the acquiring bank. If the acquirer does not provide the transaction information at the interchange level, this field is empty
* `scheme_fees_nc` - The fee charged by, for example, Visa or Mastercard. If the acquirer does not provide the transaction information at the interchange level, this
* `interchange_nc` - The fee charged by the issuing bank. If the acquirer doesn't provide the transaction information at the interchange level, this field is empty
* `dcc_markup_nc` - The DCC markup (in Net Currency)
* `advanced_nc` - The installment amount that was advanced. This is the gross amount of the advancement fee (in Net Currency)
* `advancement_code` - The code of the advancement batch
* `advancement_batch` - The code of the advancement. An advancement code can occur in multiple batches
* `booking_type` - Identifies the transaction/installment that was accelerated or represented. Possible values: FIRST: Installment first/original presentment
* `payment_method_variant` - The sub-brand of the payment method (if applicable). For example: visaclassic, visadebit, mccorporate
* `batch_number` - The sequence number of the settlement
* `reserved1` - Reserved for future enhancements
* `reserved2` - Reserved for future enhancements
* `reserved3` - Reserved for future enhancements
* `reserved4` - Reserved for future enhancements
* `reserved5` - Reserved for future enhancements
* `reserved6` - Reserved for future enhancements
* `reserved7` - Reserved for future enhancements
* `reserved8` - Reserved for future enhancements
* `reserved9` - Reserved for future enhancements
* `reserved10` - Reserved for future enhancements
* `acquirer` - The name of the acquirer that processed the original payment. For example: Cielo, Redecard
* `modification_merchant_reference` - The optional modification reference for modifications, if specified in the transaction
* `split_settlement` - Contains detailed information about split settlements in the form of key value pairs
* `booking_date` - The timestamp indicating when the event was entered into Adyen's accounting system. The format is `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26
* `booking_date_timezone` - The time zone of the Booking Date. This is the ISO code for your time zone setting. You can view the time zone setting for your company account in
* `booking_date_ams` - The timestamp indicating when the event was entered into Adyen's accounting system, in Amsterdam time (CET or CEST). Format: \`YYYY-MM-DD
* `additionaltype` - Additional journal type that gives more information about the entry
* `issuer_country` - The ISO country code of the issuer
* `shopper_country` - The ISO country code of the shopper, if available
* `clearing_network` - The clearing network. For example: Maestro, MasterCard, MoneySend, VISA, VISA Returns, VISA OCT
* `terminal_id` - Point-of-sale terminal identifier, for example P400Plus-123456789
* `tender_reference` - The tender reference of the point-of-sale payment
* `merchant_order_reference` - A reference that allows you to link multiple transactions to each other for reporting purposes. See merchantOrderReference
* `metadata` - Contents from the metadata object related to the payment, in the form of key value pairs
* `pos_transaction_date` - The transaction date for a point-of-sale transaction. Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `pos_transaction_date_timezone` - The time zone of the Pos Transaction Date
* `store` - The identifier assigned to the store in your point-of-sale account
* `dispute_reference` - Adyen's unique 16 character reference for the dispute
* `register_booking_type` - Custom column concatenating the record type and the booking type of the entry
* `arn` - The Acquirer Reference Number, a unique number that tags a credit card transaction when it goes from your bank (the acquiring bank) to the card
* `shopper_reference` - The shopper ID or reference that was specified in the original transaction
* `payment_transaction_group` - A custom value that corresponds with a merchant-specific transaction group. This value is for specific use cases, and can be configured on request
* `settlement_flow` - The Adyen flow into which the transaction was booked
* `split_payment_data` - Split payment data for the payment, if applicable
* `authorisation_code` - The authorisation code returned for a successful payment
* `card_number` - Obfuscated card number showing the card BIN, an obfuscated section and the last four digits. For example: 4111111111
* `authorised_date` - The timestamp indicating when the payment was authorised. Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `authorised_date_timezone` - The time zone of the Authorised Date
* `balance_currency` - The three-character ISO currency code for the balance account currency. This column is only relevant if your company has (split) payments that are
* `net_debit_bc` - The net debit (in Balance Currency) of the balance account. This column is only relevant if your company has (split) payments that are partly
* `net_credit_bc` - The net credit (in Balance Currency) of the balance account. This column is only relevant if your company has (split) payments that are partly
* `payment_fees_nc` - The total amount (in Net Currency) in fees, commissions, and interchange costs for the transaction
* `global_card_brand` - The brand of the card used for the payment
* `gratuity_amount` - The additional gratuity amount on the payment
* `surcharge_amount` - The surcharge amount on the payment
* `funds_destination` - This column is only relevant if you are upgrading your Adyen for Platforms integration from the classic Platforms integration. This column shows
* `balance_platform_debit` - The total amount debited to the Balance Platform for a split payment. This column is only relevant if your company has (split) payments that are
* `balance_platform_credit` - The total amount credited to the Balance Platform for a split payment. This column is only relevant if your company has (split) payments that are
* `installments` - For installment transactions, the installment this entry belongs to, written as '6/9' for the 6th of 9 installments
**Added by the connector:**
* `report_account` - Adyen account the report file was downloaded for
* `report_scope` - Account level the file came from: `MerchantAccount` or `Company`
* `report_file_name` - Name of the report file the row was read from
* `report_batch_number` - Settlement batch number of the file, used as the incremental replication key
One row per accounting event of every payment (Received, Authorised, Refused, Captured, Settled, Refunded, Chargeback, PaidOut and the rest), with the fee columns that go with each stage. This is the stream that ties a payment to its cost.
**Report columns:**
* `company_account` - The name of your company account as registered in the Adyen payments platform
* `merchant_account` - The name of the merchant account that was used to process the original payment request. The report may contain line items for multiple merchant
* `psp_reference` - Adyen's 16-character unique reference associated with the transaction/the request. This value is globally unique; quote it when communicating with
* `merchant_reference` - The reference that was specified in the payment request
* `payment_method` - The payment method used for the transaction. For example: visa, mc, amex
* `booking_date` - The timestamp indicating when the event was entered into Adyen's accounting system. The format is `YYYY-MM-DD HH:MM:SS`. For example: 2021-07-26
* `timezone` - The time zone of the Booking Date column. This is the ISO code for your time zone setting
* `main_currency` - The three-character ISO code for the currency of the main amount
* `main_amount` - The amount recorded when the transaction reaches the Record Type. For example, the main amount of a `Settled` transaction is the amount after
* `record_type` - The accounting record type of the entry, which is the stage of the payment lifecycle it records. For example: Received, Authorised, Refused,
* `payment_currency` - The three-character ISO code for the currency that was used to process the payment
* `received_pc` - The amount (in Payment Currency) debited or credited on the Received accounting register
* `authorised_pc` - The amount (in Payment Currency) debited or credited on the Authorised accounting register
* `captured_pc` - The amount (in Payment Currency) debited or credited on the Captured accounting register
* `settlement_currency` - The three-character ISO code for the currency that was used when settling the payment
* `payable_sc` - The amount (in Settlement Currency) debited or credited on the Payable accounting register. This will be paid out to your bank account
* `commission_sc` - The commission fee that was withheld by the acquirer (in Settlement Currency) If the acquirer provides the transaction information at the
* `markup_sc` - The fee charged by the acquiring bank (in Settlement Currency)
* `scheme_fees_sc` - The fee charged by, for example, Visa or Mastercard (in Settlement Currency)
* `interchange_sc` - The fee charged by the issuing bank (in Settlement Currency)
* `processing_fee_currency` - The three-character ISO code for the currency in which the processing (or gateway) fee is charged
* `processing_fee_fc` - The amount (in Processing Fee Currency) charged on the Processing Fee accounting register
* `user_name` - The user (interactive or webservice user) who performed the action resulting in the accounting record. If the record was created by an automated
* `payment_method_variant` - The sub-brand of the payment method (if applicable). For example: visaclassic, visadebit, mccorporate
* `modification_merchant_reference` - The optional modification reference for modifications, if specified in the transaction
* `metadata` - The contents of the metadata object of the payment, in the form of key-value pairs
* `merchant_order_reference` - A reference that allows you to link multiple transactions to each other for reporting purposes. See merchantOrderReference
* `reserved3` - Reserved for future enhancements
* `reserved4` - Reserved for future enhancements
* `reserved5` - Reserved for future enhancements
* `reserved6` - Reserved for future enhancements
* `reserved7` - Reserved for future enhancements
* `reserved8` - Reserved for future enhancements
* `reserved9` - Reserved for future enhancements
* `reserved10` - Reserved for future enhancements
* `issuer_country` - The two-character ISO 3166-1 alpha-2 country code where the card was issued
* `shopper_country` - The two-character ISO country code of the shopper, if available
* `payable_batch` - The sequence number of the payout batch
* `unique_terminal_id` - Point-of-sale terminal identifier, for example P400Plus-123456789
* `booking_date_ams` - The timestamp indicating when the event was entered into Adyen's accounting system, in Amsterdam time (CET or CEST). Format: \`YYYY-MM-DD
* `creation_date` - The timestamp indicating when the payment was created on the Adyen payments platform. Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `creation_date_ams` - The Creation Date in Amsterdam time (CET or CEST). Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `payout_date` - The timestamp indicating the payout date, when available. Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `gratuity_amount` - The additional gratuity amount on the payment
* `icsf_details` - The detailed specification of each applicable interchange and scheme fee for the payment, if available. The data is included in this columns in
* `card_number` - Obfuscated card number showing the card BIN, an obfuscated section, and the last four digits. For example: 4111111111
* `acquirer` - The name of the acquirer that processed the original payment. For example: Cielo, Redecard
* `acquirer_account` - The acquirer account that was used for the transaction
* `mid` - The Merchant ID from the acquirer that was used to process the payment
* `acquirer_auth_code` - The Acquirer Authorisation Code. Can be included and used to match Auth Code on the merchant gateway side with the Auth Code in this report
* `acquirer_reference` - The acquirer's unique ID for the payment, if available
* `arn` - The Acquirer Reference Number, a unique number that tags a credit card transaction when it goes from your bank (The Acquiring Bank) through to the
* `original_amount` - The amount of the transaction
* `installments_pc` - The amount of the installment, in payment currency (PC)
* `advanced_pc` - The installment amount that was advanced. This is the gross amount of the advancement fee, in payment currency (PC)
* `advancement_code` - The code of the advancement. An advancement code can contain multiple advancement batches
* `advancement_batch` - The code of the advancement batch
* `booking_type` - Identifies the transaction/installment that was accelerated or represented. Possible values: FIRST: Installment first/original presentment
* `advancement_fee_percent` - The calculated fee for the advancement of an installment, as a percentage
* `installments` - For installment transactions, the installment this entry belongs to, written as '6/9' for the 6th of 9 installments
* `store` - The store identifier, as configured in point-of-sale
* `issuer_name` - Name of the issuing bank, if available
* `risk_premium` - For payments where the Record Type is Received, the value of this field indicates if the payment triggered RevenueProtect checks (resulting in
* `register_booking_type` - Custom column concatenating the record type and the booking type of the entry
* `modification_psp_reference` - In most instances, this is Adyen's unique 16 character code child reference that is associated with the modification request, for example,
* `shopper_reference` - The shopper ID or reference that was specified in the original transaction
* `shopper_name` - The name of the cardholder, as specified in the transaction, if available
* `payment_account_reference` - The Payment Account Reference is a unique identifier that links a network token with the underlying primary account number (PAN)
* `tender_reference` - The tender reference of the point-of-sale payment
* `dcc_markup_sc` - The DCC markup (in Settlement Currency)
* `global_card_brand` - The brand of the card used for the payment
* `surcharge_amount` - The surcharge amount on the payment
* `funding_source` - The funding source of the card used for the payment. Possible values: CREDIT, DEBIT, DEFERRED\_DEBIT, PREPAID, PREPAID\_RELOADABLE,
* `payment_requester_type` - Contains the value `autorescue` if there was an auto rescue attempt for the payment
* `network_token_used` - If true, a network token was used to authorize the payment
* `network_token_available` - If true, a network token was available for the payment
* `split_payment_data` - Split payment data for the payment, if applicable
* `exchange_rate` - The exchange rate for the payment
**Added by the connector:**
* `report_account` - Adyen account the report file was downloaded for
* `report_scope` - Account level the file came from: `MerchantAccount` or `Company`
* `report_file_name` - Name of the report file the row was read from
* `report_date` - Date the report file covers, used as the incremental replication key
One row per payment attempt, authorised or refused, with the acquirer's answer (`acquirer_response` and `raw_acquirer_response`), the risk score, the 3D Secure result and the shopper and card details.
**Report columns:**
* `company_account` - The name of your company account as registered in the Adyen payments platform
* `merchant_account` - The name of the merchant account that was used to process the original payment request. The report may contain line items for multiple merchant
* `psp_reference` - Adyen's 16-character unique reference associated with the transaction/the request. This value is globally unique; quote it when communicating with
* `merchant_reference` - The reference that was specified in the payment request
* `payment_method` - The payment method used for the transaction. For example: visa, mc, mc\_applepay, amex
* `creation_date` - The timestamp indicating when the payment was created on the Adyen payments platform. Format: `YYYY-MM-DD HH:MM:SS`. For example: 2012-07-26 19:58:55
* `time_zone` - The time zone of the Creation Date. This is the ISO code for your time zone setting
* `currency` - The three-character ISO currency code
* `amount` - The amount of the payment (authorisation amount). Always a positive value, even if the payment was canceled before being captured
* `type` - The record type. For this report, the value is always Payment
* `risk_scoring` - The total risk score of the payment
* `shopper_interaction` - The transaction type. Possible values: Ecommerce, ContAuth, POS, MOTO
* `shopper_name` - The name of the cardholder, as specified in the transaction, if available
* `shopper_pan` - The Primary Account Number (PAN). In case of card payments, the last four digits of the card number. For SEPA, this field shows the IBAN and BIC
* `shopper_ip` - The IP address (IPV4 or IPV6) of the shopper as specified in the original transaction, if available
* `shopper_country` - The ISO country code of the shopper, if available. If not available, this is computed based on the shopper's IP address
* `issuer_name` - The name of the issuing bank, if available
* `issuer_id` - The unique identifier of the issuer. In case of card payments this will be the issuer BIN (first six digits of the card number)
* `issuer_city` - The city of the issuer, if available
* `issuer_country` - The ISO country code of the issuer
* `acquirer_response` - The normalized response from the acquirer. Not necessarily the final status of the payment. When a payment is canceled before being captured, the
* `authorisation_code` - The authorisation code returned for a successful payment
* `shopper_email` - The email address of the shopper, as specified in the original transaction, if available
* `shopper_reference` - The shopper ID or reference that was specified in the original transaction
* `three_d_directory_response` - For payments processed with 3D Secure (Secure Code/Verified by Visa), the 3D Directory response. Values: Y, N, U, E
* `three_d_authentication_response` - For payments processed with 3D Secure (Secure Code/Verified by Visa), and directory response Y, this holds the authentication response. Values: Y,
* `cvc2_response` - The result passed back by the issuer after verifying the CVC2, or CVV2/CID in case of Visa/Amex
* `avs_response` - If Address Verification (AVS) was performed, the result of the AVS Check
* `billing_street` - The street part of the billing address
* `billing_house_number_name` - The house number or name part of the billing address
* `billing_city` - The city part of the billing address
* `billing_postal_code_zip` - The ZIP code or postal code part of the billing address
* `billing_state_province` - The state or province of the billing address
* `billing_country` - The ISO country code of the billing address
* `delivery_street` - The street part of the delivery address
* `delivery_house_number_name` - The house number or name part of the billing address
* `delivery_city` - The city part of the delivery address
* `delivery_postal_code_zip` - The ZIP code or postal code part of the delivery address
* `delivery_state_province` - The state or province of the delivery address
* `delivery_country` - The ISO country code of the delivery address
* `acquirer_reference` - The reference number for the transaction provided by the acquirer
* `payment_method_variant` - The payment method sub-brand, or the main payment method if not available (for example mccredit or visagold)
* `raw_acquirer_response` - The raw response we receive from the acquirer, where available. When a payment is canceled before being captured, the raw acquirer response will
* `original_amount` - The original amount of the payment
* `cashback_amount` - The additional cashback amount on the payment
* `pos_charity_amount` - The additional charity amount on the payment. Applies to point-of-sale transactions
* `pos_gratuity_amount` - The additional gratuity amount on the payment. Applies to point-of-sale transactions
* `pos_cashback_amount` - The additional cashback amount on the payment. Applies to point-of-sale transactions
* `pos_discount_amount` - The additional discount amount on the payment. Applies to point-of-sale transactions
* `pos_surcharge_amount` - The surcharge amount on the payment
* `pos_payment_device` - The device that was used to make the payment
* `shopper_phone_number` - The phone number of the shopper, as specified in the original transaction, if available
* `alternative_acquirer_reference` - The alternative acquirer reference of the acquirer that was used for the transaction
* `acquirer` - Name of the acquirer that processed the original payment
* `acquirer_account` - The acquirer account that was used for the transaction
* `installments` - For installment transactions, the installment this entry belongs to, written as '6/9' for the 6th of 9 installments
* `dcc` - Whether Dynamic Currency Conversion was used for the payment. Possible values: 'DCC accepted', 'DCC declined' (the shopper paid in the merchant
* `bcmc_iban_bic` - The Bancontact IBAN or BIC
* `bin_funding_source` - The funding source derived from the card BIN. Possible values: CREDIT, DEBIT, DEFERRED\_DEBIT, PREPAID, PREPAID\_RELOADABLE, PREPAID\_NONRELOADABLE
* `device_fingerprint` - The device fingerprint of the shopper's device, as specified in the original transaction, if available
* `device` - The device that was used to make the payment
* `browser` - The browser that was used to make the payment
* `merchant_order_reference` - A reference that allows you to link multiple transactions to each other for reporting purposes. See merchantOrderReference
* `payment_requester_type` - Contains the value `autorescue` if there was an auto rescue attempt for the payment
* `network_token_used` - If true, a network token was used to authorise the payment
* `network_token_available` - If true, a network token was available for use for the payment
* `airline_data` - Airline data related to the payment, if available
* `risk_profile` - The name of the risk profile that was evaluated during the transaction
* `risk_profile_id` - The ID of the risk profile that was evaluated during the transaction
* `risk_profile_reference` - The reference number of the risk profile that was evaluated during the transaction
* `pos_transaction_date` - The transaction date for a point-of-sale transaction
* `pos_store` - The identifier assigned to the store in your point-of-sale account
* `pos_entry_mode` - How the card details were captured at the point of sale. For example: MKE, MSR, Bar Code Read, OCR Coding Read, Chip, CLTS Chip, RFID, CTLS MSR or
* `pos_cvm_performed` - A two-digit hexadecimal code indicating the type of card verification that was used
* `pos_library_implementation` - The type of library implementation, for example Android, iOS, C or .NET
* `pos_library_version` - Version number of the library used, or standalone if the payment terminal is not integrated with the POS system
* `pos_terminal_version` - The version of the POS terminal application
* `ares_transstatus` - The transaction status of the 3D Secure 2 authentication request. This is the final result of the authentication, unless the value is C, which
* `rreq_transstatus` - The result of the 3D Secure challenge flow that is sent to the 3D Secure directory server. This is the final result of the authentication. This
* `three_ds_version` - The version of the 3D Secure protocol used for the payment
* `liability_shift` - Indicates if a liability shift took place for the payment. Possible values: TRUE or FALSE
* `merchant_category_code` - The Merchant Category Code
* `retry_json` - Retry information about the payment
* `pos_store_and_forward_indicator` - Indicates if this was a store-and-forward transaction. Possible values: TRUE or FALSE
* `pos_cash_register_name` - The name of the POS system
* `pos_tender_reference` - The tender reference of the point-of-sale payment
* `split_payment_data` - Split payment data for the payment, if applicable
* `realtime_account_updater_status` - The status of the Real Time Account Updater request
* `original_card_bin` - The card BIN of the card that was used to process the original payment request
* `network_transaction_reference` - The network transaction reference of the payment
* `card_number` - Obfuscated card number showing the card BIN, an obfuscated section and the last four digits. For example: 4111111111
* `metadata` - The contents of the metadata object of the payment, in the form of key value pairs
* `global_card_brand` - The brand of the card used for the payment
* `recurring_detail_reference` - The recurring detail reference used for the payment
* `reserved2` - Reserved for future enhancements
* `reserved3` - Reserved for future enhancements
**Added by the connector:**
* `report_account` - Adyen account the report file was downloaded for
* `report_scope` - Account level the file came from: `MerchantAccount` or `Company`
* `report_file_name` - Name of the report file the row was read from
* `report_date` - Date the report file covers, used as the incremental replication key
Chargebacks, second chargebacks, notifications of fraud and requests for information, with the scheme reason codes and the dates of the dispute life cycle.
**Report columns:**
* `company_account` - Name of your company account as registered in the Adyen payments platform
* `merchant_account` - Name of your merchant account used to process the original payment request
* `psp_reference` - Adyen's unique 16 character reference for the payment. This is the PSP reference for the original authorisation request
* `merchant_reference` - Reference number provided when initiating the payment request
* `payment_method` - Payment method type of the payment which was processed. For example: visa, mc, amex
* `record_date` - Timestamp of when the dispute event took place as entered into Adyen's accounting system. Format: `YYYY-MM-DD HH:MM:SS` For example: 2012-07-26
* `record_date_timezone` - Time zone of the Record Date. This is the ISO code for your time zone setting
* `dispute_currency` - Disputes do not have to be in the original transaction currency, but can be in the currency of the cardholder. Also disputes can be of a lower
* `dispute_amount` - The amount of the disputed payment. For example, 100.00
* `record_type` - The accounting record type of the entry, which is the stage of the payment lifecycle it records. For example: Received, Authorised, Refused,
* `dispute_psp_reference` - Adyen's unique 16 character reference for the dispute
* `dispute_reason` - Dispute reason (free format) provided by the bank or payment method
* `rfi_scheme_code` - Payment method scheme/institution which issued the code in the following column. For example: visa, mc, amex
* `rfi_reason_code` - Request for Information reason code as provided by the bank or payment method. The mapping is defined by the scheme/institution. This is blank if
* `cb_scheme_code` - Payment method scheme/institution which issued the code in the following column. For example: visa, mc, amex
* `cb_reason_code` - Chargeback reason code as provided by the bank or payment method. The mapping is defined by the scheme/institution. This may be blank if there is
* `nof_scheme_code` - Payment method scheme/institution which issued the Notification of Fraud (NoF) code in the NoF Reason Code column. For example: visa, mc, amex
* `nof_reason_code` - Notification of Fraud reason code as provided by the bank or payment method. The mapping is defined by the scheme/institution
* `payment_date` - Date of the original payment (authorisation)
* `payment_date_timezone` - Time zone of the Payment Date. This is the ISO code for your time zone setting
* `payment_currency` - Three character ISO code for the currency which was used for processing the payment. This may differ from the dispute currency in rare cases
* `payment_amount` - Authorisation amount of the original payment
* `dispute_date` - Date the dispute was first entered into the system. This can be the same date as the RFI
* `dispute_date_timezone` - Time zone of the Dispute Date. This is the ISO code for your time zone setting
* `dispute_arn` - Acquirer Reference Number, a unique number that tags a dispute when it goes from your bank account (The Acquiring Bank)
* `user_name` - User (interactive or webservice user) who performed the action resulting in the accounting record. If the record was created by an automated
* `risk_scoring` - Total risk scoring value of the original payment
* `shopper_interaction` - Transaction type. Possible values: Ecommerce, ContAuth, POS, Moto
* `shopper_name` - Name of the cardholder as provided during the original transaction (if available)
* `shopper_email` - Email address of the shopper/consumer as provided during the original transaction (if available)
* `shopper_reference` - Supplied customer ID/reference as supplied in the original transaction
* `shopper_pan` - Account number of the shopper/consumer. In case of card payments, the last four digits of the card number
* `iban` - Shopper's bank account IBAN number
* `bic` - Shopper's bank account BIC number
* `shopper_ip` - IP address (IPV4 or IPV6) of the shopper/consumer as provided during the original transaction (if available)
* `shopper_country` - ISO country code of the shopper (if available)
* `issuer_country` - ISO country code of the issuer (if available)
* `issuer_id` - Unique identifier of the issuer. In case of card payments this will be the issuer BIN (first 6 digits of the card number)
* `three_d_directory_response` - Payments which were processed using 3D Secure (Secure Code / Verified by Visa), the directory response. Possible values: Y, N, U, E
* `three_d_authentication_response` - Payments which were processed using 3D Secure (Secure Code / Verified by Visa) and had directory response 'Y', the authentication response
* `cvc2_response` - Result passed back by the issuer after verifying the CVC2 (or CVV2/CID in case of Visa/Amex)
* `avs_response` - Result of the AVS check, if performed
* `dispute_auto_defended` - Set to `true`, if chargeback was auto-defended by Adyen. Empty if it was not
* `dispute_end_date` - Date and time of day when dispute defense period ended
* `dispute_end_date_timezone` - Time zone of the Dispute End Date. This is the ISO code for your time zone setting
* `reserved4` - Reserved for future enhancements
* `reserved5` - Reserved for future enhancements
* `acquirer` - Name of the Acquirer that processed the original payment
* `acquirer_account` - The Acquirer account that was used for the transaction
* `issuer_region_program` - The issuer region in which the transaction was made
* `liability_shift` - Indicates if there was a liability shift for the transaction
* `metadata` - Contents from the metadata object related to the payment, in the form of key value pairs
* `data_ed` - Contents from the `Data Ed` object related to the dispute
* `rdr` - Indicates Rapid Dispute Resolution (RDR) chargebacks (Visa only)
* `second_chargeback` - Indicator that marks if a second chargeback is present
* `shopper_phone_number` - Phone number of the shopper or consumer as provided during the original transaction (if available)
* `shopper_statement` - The shopper statement (descriptor) posted in the payment request
* `status` - The most recent chargeback status
**Added by the connector:**
* `report_account` - Adyen account the report file was downloaded for
* `report_scope` - Account level the file came from: `MerchantAccount` or `Company`
* `report_file_name` - Name of the report file the row was read from
* `report_date` - Date the report file covers, used as the incremental replication key
Transactions settled by an external acquirer rather than by Adyen. Empty unless your account processes through an external acquirer.
**Report columns:**
* `company_account` - Name of your company account within the Adyen payments platform
* `merchant_account` - Name of your merchant account that was used to process the original payment request. The report may contain line items for multiple merchant accounts
* `psp_reference` - Unique 16-character reference provided by Adyen for the original authorisation request. For unmatched transactions, this field can contain a
* `merchant_reference` - Reference number that you provided when initiating the payment request
* `payment_method` - Payment method type of the processed payment. For example: `visa`, `mc`, or `amex`
* `creation_date` - The timestamp of when the payment was received by the Adyen payments platform. Format: `YYYY-MM-DD HH:MM:SS` For example: `2012-07-26 19:58:55`
* `timezone` - The time zone of the Creation Date. This is the ISO code for your time zone setting
* `type` - Journal type of the entry
* `modification_reference` - Adyen's unique 16-character reference for the modification (capture, refund or chargeback) that produced the entry. For MerchantPayout entries it
* `gross_currency` - Three-character ISO currency code that was used for processing the payment (transaction currency)
* `gross_debit_gc` - Amount submitted in the transaction request. For example, `100.00`
* `gross_credit_gc` - Amount submitted in the transaction request. For example, `100.00`
* `exchange_rate` - Exchange rate used for converting the gross amount into the net amount. This is 1 if you process and settle in the same currency
* `net_currency` - Three-character ISO currency code that was used for processing the payment (settlement currency)
* `net_debit_nc` - Amount submitted in the transaction request minus acquirer fees. For example: `100.00`
* `net_credit_nc` - Amount submitted in the transaction request minus acquirer fees. For example: `100.00`
* `commission_nc` - The commission fee withheld by the acquirer, in the settlement (net) currency. When the acquirer reports at interchange level this column is empty
* `markup_nc` - Fee charged by the acquirer. This field is empty if the acquirer doesn't provide transaction information at the interchange level
* `scheme_fees_nc` - Fee which is charged by Visa or Mastercard. This field is empty if the acquirer doesn't provide transaction information at the interchange level
* `interchange_nc` - Fee charged by the issuing bank. This field is empty if the acquirer doesn't provide transaction information at the interchange level
* `payment_method_variant` - Payment method sub-type of the processed payment. For example: `visa classic`, `mc`, or `amex`
* `acquirer` - Name of the acquirer that processed the original payment. For example: `Cielo` or `Redecard`
* `mid` - Merchant ID from the acquirer that was used for processing the payment
* `arn` - Acquirer Reference Number, a unique number that tags a credit card transaction when it goes from the acquirer to the card scheme at the issuer
* `acquirer_reference` - The acquirer's unique ID for the payment, if provided
* `modification_merchant_reference` - Your optional reference for the payment modification
* `reserved7` - Where installment transactions are applicable, this shows the installment corresponding to this entry. For example, `6/9` indicates that this
* `reserved8` - Reserved for future enhancements
* `reserved9` - Reserved for future enhancements
* `reserved10` - Reserved for future enhancements
* `store_code` - Identifier for the store where a transaction occurs
* `merchant_order_reference` - A reference that allows you to link multiple transactions to each other for reporting purposes. See merchantOrderReference
* `acquirer_auth_code` - Authorisation code returned by the acquirer for a successful payment. You can match it to the Authorisation Code on the merchant gateway.This code
* `card_bin` - First six digits of the card number
* `card_number_summary` - Last four digits of the card number
* `submerchant_identifier` - An identifier for the Merchant of Record when a transaction is submitted through a payment facilitator. The payment facilitator must send this
* `metadata` - The contents of the metadata object of the payment, in the form of key:value pairs
**Added by the connector:**
* `report_account` - Adyen account the report file was downloaded for
* `report_scope` - Account level the file came from: `MerchantAccount` or `Company`
* `report_file_name` - Name of the report file the row was read from
* `report_date` - Date the report file covers, used as the incremental replication key
# AfterShip as a data source
Source: https://docs.nekt.com/sources/aftership
Bring data from AfterShip to Nekt.
AfterShip is a shipment tracking platform that helps e-commerce businesses track packages across 1,100+ carriers worldwide. It provides real-time tracking updates, delivery estimates, and analytics to improve post-purchase customer experience and operational efficiency.
![]()
## Configuring AfterShip as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the AfterShip option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an AfterShip API key to authenticate. The following configurations are available:
* **API Key**: Your AfterShip API key used to authenticate API requests.
* **Start Date**: The earliest date from which records will be synced based on the last modification date.
Go to [AfterShip](https://www.aftership.com/) and log in to your account.
1. Click on your profile icon in the top-right corner
2. Select **Settings**
3. In the left sidebar, click on **API Keys** under the "Integrations" section
1. Click **Generate API Key**
2. Give your API key a descriptive name (e.g., "Nekt Integration")
3. Click **Generate**
4. Copy the generated API key immediately
**Important:** The API key is only shown once. Copy it immediately and store it securely. If you lose it, you'll need to generate a new one.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new or updated data based on the `updated_at` field.
* Full table: every time the extraction happens, we'll get the current state of all data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from AfterShip and their corresponding fields:
Stream containing shipment tracking information with full tracking history.
**Key Fields:**
* `id` - Unique identifier for the tracking
* `legacy_id` - Legacy tracking ID
* `tracking_number` - The carrier tracking number
* `slug` - Courier code (e.g., "ups", "fedex", "dhl")
* `active` - Whether the tracking is currently active
* `created_at` - When the tracking was created
* `updated_at` - When the tracking was last updated (replication key)
**Order Information:**
* `order_id` - Associated order ID
* `order_number` - Order number
* `order_id_path` - Order IDs path
* `order_date` - Date of the order
* `order_tags[]` - Tags associated with the order
* `order_promised_delivery_date` - Promised delivery date
**Origin Location:**
* `origin_country_region` - Origin country/region code
* `origin_state` - Origin state
* `origin_city` - Origin city
* `origin_postal_code` - Origin postal code
* `origin_raw_location` - Raw origin location string
**Destination Location:**
* `destination_country_region` - Destination country/region code
* `destination_state` - Destination state
* `destination_city` - Destination city
* `destination_postal_code` - Destination postal code
* `destination_raw_location` - Raw destination location string
* `courier_destination_country_region` - Courier-reported destination
**Delivery Status:**
* `tag` - Current tracking status tag (e.g., "Delivered", "InTransit", "OutForDelivery")
* `subtag` - Detailed status subtag
* `subtag_message` - Human-readable status message
* `on_time_status` - On-time delivery status
* `on_time_difference` - Difference from promised delivery time (in days)
* `signed_by` - Name of person who signed for delivery
* `return_to_sender` - Whether package was returned to sender
**Shipment Details:**
* `shipment_type` - Type of shipment
* `shipment_package_count` - Number of packages
* `shipment_pickup_date` - Pickup date
* `shipment_delivery_date` - Actual delivery date
* `shipping_method` - Shipping method used
* `shipment_tags[]` - Tags for the shipment
* `shipment_weight.value` - Weight value
* `shipment_weight.unit` - Weight unit (kg, lb, etc.)
* `transit_time` - Transit time in days
**Estimated Delivery:**
* `courier_estimated_delivery_date.estimated_delivery_date` - Courier's estimated delivery
* `courier_estimated_delivery_date.estimated_delivery_date_min` - Minimum estimate
* `courier_estimated_delivery_date.estimated_delivery_date_max` - Maximum estimate
* `aftership_estimated_delivery_date` - AfterShip's predicted delivery date
* `custom_estimated_delivery_date` - Custom estimated delivery date
* `first_estimated_delivery.datetime` - First estimated delivery datetime
* `first_estimated_delivery.source` - Source of first estimate
* `latest_estimated_delivery.datetime` - Latest estimated delivery datetime
* `latest_estimated_delivery.revise_reason` - Reason for revision
**Checkpoints (Tracking History):**
* `checkpoints[]` - Array of tracking events:
* `checkpoints[].checkpoint_time` - Timestamp of the event
* `checkpoints[].tag` - Status tag at checkpoint
* `checkpoints[].subtag` - Detailed subtag
* `checkpoints[].subtag_message` - Status message
* `checkpoints[].message` - Checkpoint description
* `checkpoints[].location` - Full location string
* `checkpoints[].city` - City
* `checkpoints[].state` - State
* `checkpoints[].country_region` - Country/region code
* `checkpoints[].country_region_name` - Country/region name
* `checkpoints[].postal_code` - Postal code
* `checkpoints[].coordinate.latitude` - Latitude
* `checkpoints[].coordinate.longitude` - Longitude
* `checkpoints[].slug` - Courier slug
* `checkpoints[].source` - Data source
* `checkpoints[].created_at` - When checkpoint was created
* `checkpoints[].events[]` - Events at this checkpoint
**Customer Information:**
* `customers[]` - Array of customers:
* `customers[].name` - Customer name
* `customers[].email` - Customer email
* `customers[].phone_number` - Phone number
* `customers[].language` - Preferred language
* `customers[].role` - Role (buyer or receiver)
**Notifications:**
* `subscribed_emails[]` - Subscribed email addresses
* `subscribed_smses[]` - Subscribed SMS numbers
**Delivery Attempts:**
* `first_attempted_at` - First delivery attempt timestamp
* `failed_delivery_attempts` - Number of failed attempts
* `delivery_type` - Type of delivery
* `delivery_location_type` - Type of delivery location
* `signature_requirement` - Signature requirement info
* `pickup_location` - Pickup location (for pickup deliveries)
* `pickup_note` - Pickup note
**Tracking Links:**
* `courier_tracking_link` - Direct link to carrier tracking
* `courier_redirect_link` - Courier redirect link
* `aftership_tracking_url` - AfterShip tracking page URL
* `aftership_tracking_order_url` - AfterShip order tracking URL
**Additional Fields:**
* `title` - Tracking title (usually order number or product name)
* `note` - Notes about the tracking
* `source` - Source of the tracking data
* `language` - Language of the tracking record
* `tracked_count` - Number of times tracked
* `tracking_account_number` - Carrier account number
* `tracking_key` - Tracking key
* `tracking_ship_date` - Ship date (YYYY-MM-DD)
* `courier_connection_id` - Courier connection ID
* `location_id` - Location ID
* `carbon_emissions` - Estimated carbon emissions
* `custom_fields.item_names` - Custom field for item names
* `first_mile` - First mile information
* `last_mile` - Last mile information
# Data Model
The following diagram illustrates the structure of the AfterShip data. The Trackings stream is the main entity containing all shipment tracking information including nested checkpoints and customer data.
```mermaid theme={null}
erDiagram
Trackings {
string id PK
string tracking_number
string slug
string order_id
string order_number
datetime created_at
datetime updated_at
string tag
string subtag
boolean active
string origin_country_region
string destination_country_region
datetime shipment_delivery_date
string signed_by
array checkpoints
array customers
}
Checkpoints {
datetime checkpoint_time
string tag
string subtag
string message
string city
string state
string country_region
string postal_code
number latitude
number longitude
}
Customers {
string name
string email
string phone_number
string role
}
Trackings ||--o{ Checkpoints : "contains"
Trackings ||--o{ Customers : "has"
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating AfterShip data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Delivery Performance Overview
Track delivery performance across carriers and identify trends in on-time delivery.
**Business Value:**
* Monitor carrier performance and reliability
* Identify delivery delays and their causes
* Optimize carrier selection based on performance data
```sql theme={null}
WITH delivery_metrics AS (
SELECT
slug AS carrier,
tag AS status,
COUNT(*) AS total_shipments,
AVG(transit_time) AS avg_transit_days,
SUM(CASE WHEN on_time_status = 'on_time' THEN 1 ELSE 0 END) AS on_time_count,
SUM(CASE WHEN tag = 'Delivered' THEN 1 ELSE 0 END) AS delivered_count
FROM
nekt_raw.aftership_trackings
WHERE
created_at >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
slug,
tag
)
SELECT
carrier,
status,
total_shipments,
ROUND(avg_transit_days, 1) AS avg_transit_days,
on_time_count,
ROUND(on_time_count * 100.0 / NULLIF(delivered_count, 0), 1) AS on_time_rate_pct
FROM
delivery_metrics
ORDER BY
total_shipments DESC
```
```sql theme={null}
WITH delivery_metrics AS (
SELECT
slug AS carrier,
tag AS status,
COUNT(*) AS total_shipments,
AVG(transit_time) AS avg_transit_days,
SUM(CASE WHEN on_time_status = 'on_time' THEN 1 ELSE 0 END) AS on_time_count,
SUM(CASE WHEN tag = 'Delivered' THEN 1 ELSE 0 END) AS delivered_count
FROM
`nekt_raw.aftership_trackings`
WHERE
DATE(created_at) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
slug,
tag
)
SELECT
carrier,
status,
total_shipments,
ROUND(avg_transit_days, 1) AS avg_transit_days,
on_time_count,
ROUND(SAFE_DIVIDE(on_time_count * 100.0, delivered_count), 1) AS on_time_rate_pct
FROM
delivery_metrics
ORDER BY
total_shipments DESC
```
| carrier | status | total\_shipments | avg\_transit\_days | on\_time\_count | on\_time\_rate\_pct |
| - | - | - | - | - | - |
| ups | Delivered | 1,245 | 3.2 | 1,156 | 92.9 |
| fedex | Delivered | 892 | 2.8 | 845 | 94.7 |
| dhl | InTransit | 456 | 4.1 | - | - |
| usps | Delivered | 378 | 4.5 | 312 | 82.5 |
### 2. Shipment Status Distribution
Analyze current shipment statuses to understand pipeline and identify potential issues.
**Business Value:**
* Monitor shipments in transit
* Identify stuck or problematic shipments
* Track exception rates by carrier
```sql theme={null}
SELECT
tag AS status,
subtag AS detailed_status,
COUNT(*) AS shipment_count,
ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER (), 1) AS percentage
FROM
nekt_raw.aftership_trackings
WHERE
active = true
GROUP BY
tag,
subtag
ORDER BY
shipment_count DESC
```
```sql theme={null}
SELECT
tag AS status,
subtag AS detailed_status,
COUNT(*) AS shipment_count,
ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER (), 1) AS percentage
FROM
`nekt_raw.aftership_trackings`
WHERE
active = true
GROUP BY
tag,
subtag
ORDER BY
shipment_count DESC
```
| status | detailed\_status | shipment\_count | percentage |
| - | - | - | - |
| InTransit | InTransit\_PickedUp | 523 | 35.2 |
| InTransit | InTransit\_Departed | 312 | 21.0 |
| OutForDelivery | OutForDelivery\_ToBeDelivered | 178 | 12.0 |
| Delivered | Delivered\_Signed | 156 | 10.5 |
| Exception | Exception\_Delayed | 89 | 6.0 |
### 3. Destination Analysis
Analyze shipment destinations to understand geographic distribution and regional performance.
**Business Value:**
* Identify top shipping destinations
* Analyze regional delivery performance
* Plan logistics and carrier partnerships by region
```sql theme={null}
SELECT
destination_country_region AS country,
destination_state AS state,
COUNT(*) AS total_shipments,
AVG(transit_time) AS avg_transit_days,
SUM(CASE WHEN tag = 'Delivered' THEN 1 ELSE 0 END) AS delivered,
SUM(CASE WHEN tag = 'Exception' THEN 1 ELSE 0 END) AS exceptions
FROM
nekt_raw.aftership_trackings
WHERE
destination_country_region IS NOT NULL
AND created_at >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
destination_country_region,
destination_state
ORDER BY
total_shipments DESC
LIMIT 20
```
```sql theme={null}
SELECT
destination_country_region AS country,
destination_state AS state,
COUNT(*) AS total_shipments,
AVG(transit_time) AS avg_transit_days,
SUM(CASE WHEN tag = 'Delivered' THEN 1 ELSE 0 END) AS delivered,
SUM(CASE WHEN tag = 'Exception' THEN 1 ELSE 0 END) AS exceptions
FROM
`nekt_raw.aftership_trackings`
WHERE
destination_country_region IS NOT NULL
AND DATE(created_at) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
destination_country_region,
destination_state
ORDER BY
total_shipments DESC
LIMIT 20
```
| country | state | total\_shipments | avg\_transit\_days | delivered | exceptions |
| - | - | - | - | - | - |
| US | California | 456 | 2.8 | 423 | 12 |
| US | Texas | 312 | 3.1 | 287 | 8 |
| US | New York | 289 | 2.5 | 271 | 5 |
| US | Florida | 234 | 3.4 | 212 | 11 |
| CA | Ontario | 178 | 4.2 | 156 | 7 |
## Implementation Notes
### Incremental Sync Support
* The Trackings stream supports incremental sync using the `updated_at` field as the replication key
* Only trackings modified since the last sync will be fetched, improving extraction performance
### Nested Data Structures
* The `checkpoints` array contains the full tracking history with timestamps, locations, and status updates
* The `customers` array may contain multiple customers (buyer and receiver)
* Use JSON functions or UNNEST to analyze nested checkpoint and customer data
### Tracking Status Tags
AfterShip uses standardized tags to represent shipment status:
* `Pending` - Tracking created, no updates yet
* `InfoReceived` - Carrier received shipment information
* `InTransit` - Shipment is in transit
* `OutForDelivery` - Out for delivery
* `AttemptFail` - Delivery attempt failed
* `Delivered` - Successfully delivered
* `AvailableForPickup` - Available at pickup location
* `Exception` - Exception occurred (delay, customs, etc.)
* `Expired` - Tracking expired with no delivery confirmation
## Skills for agents
AfterShip connector documentation as plain markdown, for use in AI agent contexts.
# Agendor as a data source
Source: https://docs.nekt.com/sources/agendor
Bring data from Agendor to Nekt.
Agendor is a comprehensive customer relationship management (CRM) platform designed for Brazilian businesses. It provides tools for lead management, deal tracking, contact management, and sales pipeline management with a focus on the Brazilian market and compliance with local regulations.
![]()
## Configuring Agendor as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Agendor option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Agendor API Token for this connection. You can find it in your [Agendor account settings](https://app.agendor.com.br/integracoes) under Menu > Integrações. Once you have it, add the account access and the connector configuration.
The following configurations are available:
* **Auth token**: The token to authenticate against the API service.
* **Start Date**: The earliest date from which records will be synced. This should be in DD-MM-YYYY format. Records created or updated after this date will be extracted from the source.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Users | `users` | Users of the Agendor account. |
| Categories | `categories` | Organization categories configured in the account. |
| Sectors | `sectors` | Sectors (segments) configured in the account. |
| Funnels | `funnels` | Sales funnels configured in the account. |
| Products | `products` | Products registered in the account. |
| Product Categories | `product_categories` | Product categories in Agendor. |
| People | `people` | People (contacts) registered in the account. |
| Organizations | `organizations` | Organizations (companies) registered in the account. |
| Deals | `deals` | Deals (opportunities) and their attributes. |
| Tasks | `tasks` | Tasks and activities linked to deals, people and organizations. |
| Deal Stages | `deal_stages` | Stages that make up each funnel. |
| Lead Origins | `lead_origins` | Registered lead origins (sources). |
| Loss Reasons | `loss_reasons` | Registered reasons for losing a deal. |
| Organizations Custom Fields | `organizations_custom_fields` | Custom field definitions for organizations. |
| People Custom Fields | `people_custom_fields` | Custom field definitions for people. |
| Deals Custom Fields | `deals_custom_fields` | Custom field definitions for deals. |
## Fields by stream
Below you'll find all available data streams from Agendor and their corresponding fields:
Stream for managing team members and user accounts in Agendor.
**Key Fields:**
* `id` - Unique identifier for the user
* `name` - Full name of the user
* `accountId` - Account ID
* `createdAt` - When the user was created
* `isBlocked` - Whether the user is blocked
**Account Information:**
* `account` - Account details including features and capabilities
* `account.features` - Available account features (elasticsearch, new\_funnel, etc.)
**Contact Information:**
* `contact` - Contact details including:
* `email` - Email address
* `work` - Work phone number
* `mobile` - Mobile phone number
* `fax` - Fax number
* `ramal` - Extension number
* `radio` - Radio contact
* `whatsapp` - WhatsApp number
* `facebook` - Facebook profile
* `twitter` - Twitter profile
* `linkedIn` - LinkedIn profile
* `skype` - Skype contact
* `instagram` - Instagram profile
**Team Management:**
* `teams` - Array of teams the user belongs to, including team ID and leadership status
Core stream for deal management and sales pipeline tracking.
**Key Fields:**
* `id` - Unique identifier for the deal
* `title` - Title of the deal
* `description` - Deal description
* `value` - Monetary value of the deal
* `ranking` - Deal ranking (1-5)
* `startTime` - Deal start time
* `endTime` - Deal end time
* `wonAt` - When the deal was won
* `lostAt` - When the deal was lost
* `createdAt` - When the deal was created
* `updatedAt` - When the deal was last updated
**Related Entities:**
* `organization` - Related organization details
* `person` - Related person/contact details
* `author` - User who created the deal
* `owner` - User who owns the deal
* `dealStage` - Current stage in the sales funnel
* `dealStatus` - Current status of the deal
* `lossReason` - Reason for deal loss (if applicable)
**Deal Structure:**
* `parentDealId` - Parent deal ID for deal hierarchies
* `accountId` - Account ID
* `products` - Array of products associated with the deal
* `allowedUsers` - Users with access to this deal
**Custom Fields:**
* `customFields` - Custom field values specific to your organization
Stream for managing company and organization records.
**Key Fields:**
* `id` - Unique identifier for the organization
* `name` - Organization name
* `legalName` - Legal name of the organization
* `email` - Organization email address
* `cnpj` - Brazilian tax identification number
* `description` - Organization description
* `ranking` - Organization ranking
* `website` - Organization website
* `createdAt` - When the organization was created
* `updatedAt` - When the organization was last updated
**Visual Assets:**
* `logo` - Logo URLs in different sizes (full, thumb, thumb\_large)
**Contact Information:**
* `contact` - Complete contact details including phone numbers, social media, and messaging platforms
**Address Information:**
* `address` - Complete address including:
* `country` - Country
* `state` - State/Province
* `city` - City
* `district` - District/Neighborhood
* `streetName` - Street name
* `streetNumber` - Street number
* `postalCode` - Postal/ZIP code
* `additionalInfo` - Additional address information
**Classification:**
* `category` - Organization category
* `leadOrigin` - Lead origin source
* `sector` - Business sector
**Related Data:**
* `products` - Products associated with the organization
* `people` - People/contacts associated with the organization
* `author` - User who created the organization
* `ownerUser` - User who owns the organization
* `allowedUsers` - Users with access to this organization
**Custom Fields:**
* `customFields` - Custom field values specific to your organization
Stream for managing contact information and person records.
**Key Fields:**
* `id` - Unique identifier for the person
* `name` - Full name of the person
* `email` - Email address
* `cpf` - Brazilian individual tax identification number
* `role` - Person's role
* `ranking` - Person ranking
* `description` - Person description
* `birthday` - Birthday
* `avatar` - Avatar URL
* `createdAt` - When the person was created
* `updatedAt` - When the person was last updated
**Related Organization:**
* `organization` - Organization this person belongs to
**Contact Information:**
* `contact` - Complete contact details including phone numbers, social media, and messaging platforms
**Address Information:**
* `address` - Complete address including country, state, city, district, street details, and postal code
**Classification:**
* `category` - Person category
* `leadOrigin` - Lead origin source
**Related Data:**
* `products` - Products associated with the person
* `author` - User who created the person
* `ownerUser` - User who owns the person
* `allowedUsers` - Users with access to this person
**Custom Fields:**
* `customFields` - Custom field values specific to your organization
Stream for managing activities, tasks, and appointments.
**Key Fields:**
* `id` - Unique identifier for the task
* `text` - Task description
* `type` - Task type
* `dueDate` - Task due date
* `finishedAt` - When the task was completed
* `createdAt` - When the task was created
* `updatedAt` - When the task was last updated
**Assignment:**
* `user` - User assigned to the task
* `assignedUsers` - Array of users assigned to the task
* `finishedBy` - User who completed the task
**Related Entities:**
* `organization` - Related organization
* `person` - Related person/contact
* `deal` - Related deal
**Metrics:**
* `metrics` - Array of task metrics including metric ID, name, and value
Stream for managing sales funnels and their configuration.
**Key Fields:**
* `id` - Unique identifier for the funnel
* `name` - Funnel name
* `initials` - Funnel initials
* `color` - Funnel color
* `sequence` - Funnel sequence order
* `default` - Whether this is the default funnel
**Deal Stages:**
* `dealStages` - Array of deal stages in this funnel, including:
* `id` - Stage ID
* `name` - Stage name
* `sequence` - Stage sequence within the funnel
Stream for managing categories used to classify organizations and people.
**Key Fields:**
* `id` - Unique identifier for the category
* `name` - Category name
Stream for managing business sectors used to classify organizations.
**Key Fields:**
* `id` - Unique identifier for the sector
* `name` - Sector name
Stream for managing products and services offered.
**Key Fields:**
* `id` - Unique identifier for the product
* `name` - Product name
* `code` - Product code
* `price` - Product price
* `active` - Whether the product is active
* `createdAt` - When the product was created
Stream for managing product categories in Agendor.
**Key Fields:**
* `id` - Product category ID
* `name` - Product category name
# Data Model
The following diagram illustrates the relationships between the core data streams in Agendor. The arrows indicate the links between different entities within your CRM structure.
```mermaid theme={null}
erDiagram
ORGANIZATIONS ||--o{ DEALS : "has"
ORGANIZATIONS ||--o{ PEOPLE : "employs"
PEOPLE ||--o{ DEALS : "associated with"
FUNNELS ||--o{ DEALS : "contains"
DEALS ||--o{ TASKS : "has"
USERS ||--o{ TASKS : "assigned to"
USERS ||--o{ DEALS : "owns"
USERS ||--o{ ORGANIZATIONS : "owns"
```
## Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Authentication errors | Wrong API Token | Verify your API Token in your Agendor account under Menu > Integrações. Ensure there are no leading or trailing spaces in the token input. |
| No data / empty table | Start Date in the future or no records in range | Check the **Start Date** configuration. Confirm that there are created or updated records in the requested date range within your Agendor account. |
| Missing recent records | Incremental sync not completed | For incremental syncs, make sure the extraction has successfully finished. If a full refresh is needed, trigger an Additional Full Sync. |
## Skills for agents
Agendor connector documentation as plain markdown, for use in AI agent contexts.
# Ahgora as a data source
Source: https://docs.nekt.com/sources/ahgora
Bring data from TOTVS Ahgora to Nekt.
Ahgora (TOTVS RH Ponto Eletrônico) is a Brazilian workforce management platform for time and attendance, covering electronic time punches, work schedules, shifts, and employee leaves. This connector extracts your data through the official Ahgora API, landing employees, time punches, leaves, schedules, and shifts in your Catalog.
## Configuring Ahgora as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Ahgora option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Ahgora uses HTTP Basic authentication with a dedicated **API integration user**. This user must be linked to a Time Automation profile (*Perfil de Automação de Ponto*) with the API access permissions enabled, such as *API Integração Batidas* and the read permissions for the data you want to extract (employees, punches, leaves, schedules, and shifts). If you don't have an integration user yet, ask your Ahgora administrator or TOTVS/Ahgora support to provision one.
Back in Nekt, provide:
* **Username**: the username of the Ahgora API integration user.
* **Password**: the password of the Ahgora API integration user.
Optionally, under advanced settings you can define:
* **Start Date**: the earliest date to sync punches and leaves from. When not set, it defaults to January 1st, 2024.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
| Stream | Slug | Description |
| - | - | - |
| Employees | `employees` | All employees registered in the account, with contract and product-usage information. |
| Punches | `punches` | Time punches (batidas de ponto) of all employees. Synced incrementally in date windows. |
| Leaves | `leaves` | Employee leaves (afastamentos), synced incrementally by last update. |
| Schedules | `schedules` | Work schedules (escalas) configured in the account. |
| Shifts | `shifts` | Work shifts (turnos) configured in the account. |
Fields holding nested structures (for example `financial_data` and `job_title_history` on employees) are delivered as **JSON strings**. Parse them downstream (for example in a Query) when you need the individual attributes.
## Fields by stream
Below you'll find all available data streams from Ahgora and their corresponding fields:
All employees registered in the Ahgora account. **Sync:** Full table. **Primary key:** `register`.
* `register` - Employee's unique registration number (matrícula)
* `name` - Full name of the employee
* `pis` - PIS number
* `cpf` - CPF number
* `rg` - RG (identity document) number
* `email` - Email address
* `phone_number` - Phone number
* `gender` - Gender
* `birth_date` - Birth date
* `hire_date` - Hire date
* `termination_date` - Termination date, if applicable
* `location` - Location path within the company structure
* `work_permit` - Work permit (CTPS)
* `job_title` - Current job title
* `department` - Department
* `cost_center` - Cost center
* `work_regime` - Work regime
* `internal_code` - Internal code in the company's systems
* `leadership_register` - Registration number of the direct leader
* `leadership_name` - Name of the direct leader
* `leadership_email` - Email of the direct leader
* `workload` - Contractual workload
* `schedule_type` - Type of work schedule
* `branch_office_index` - Index of the branch office
* `branch_office_history` - History of branch offices (JSON string)
* `financial_data` - Financial data such as monthly workload, hour cost, additionals and benefits (JSON string)
* `job_title_history` - History of job titles (JSON string)
* `uses_punch_collector` - Whether the employee punches on a physical collector (REP)
* `uses_online_punch` - Whether the employee uses the online punch
* `uses_mobile_punch` - Whether the employee uses the mobile app punch
* `uses_time_sheet` - Whether the employee uses the timesheet product
* `uses_smartgate` - Whether the employee uses the Smartgate access control product
* `uses_vacations` - Whether the employee uses the vacations product
* `uses_rostering` - Whether the employee uses the rostering (escalas) product
Time punches (batidas de ponto) of all employees. **Sync:** Incremental on `punch_time`, fetched in 30-day windows from the start date. **Primary key:** `employee_id`, `punch_time`.
* `employee_id` - Unique identifier of the employee who registered the punch
* `employee_pis` - PIS number of the employee
* `punch_time` - Date and time the punch was registered (incremental replication key)
* `type` - Type of the punch (e.g. MANUAL)
* `serial_number` - Serial number of the device where the punch was registered
* `device_name` - Name of the device where the punch was registered
* `latitude` - Latitude where the punch was registered, when available
* `longitude` - Longitude where the punch was registered, when available
* `reason` - Reason associated with the punch, when available
* `punches` - Additional punch information returned by the API
Employee leaves (afastamentos). **Sync:** Incremental on `updated_at`. **Primary key:** `id`.
* `id` - Unique identifier of the leave
* `employee_registers` - Registration numbers of the employees covered by the leave
* `start` - Start date of the leave
* `end` - End date of the leave
* `reason` - Description of the leave reason
* `reason_code` - Code of the leave reason
* `blocks_employee_access` - Whether the leave blocks the employee's access during its period
* `ignores_absence_payment` - Whether absence payment is ignored
* `ignores_dsr_discount` - Whether the paid weekly rest (DSR) discount is ignored
* `ignores_treatment_on_worked_hours` - Whether treatments on worked hours are ignored
* `internal_code` - Internal code in the company's systems
* `is_compensating_for_holiday` - Whether the leave compensates for a holiday
* `treat_worked_hours_as` - How worked hours during the leave are treated
* `updated_at` - Timestamp of the last update (incremental replication key)
* `updated_by` - User who last updated the leave
Work schedules (escalas) configured in the account. **Sync:** Full table. **Primary key:** `key`.
* `key` - Unique identifier of the schedule
* `name` - Name of the schedule
* `description` - Description of the schedule
* `code` - Code of the schedule
* `workload` - Workload of the schedule
* `type` - Type of the schedule (e.g. SEMANAL)
* `ignore_workload_and_schedule_type_from_employee` - Whether the schedule ignores the workload and schedule type configured on the employee
* `shifts` - Keys of the shifts that compose the schedule
* `negative_hours_monthly_tolerance` - Monthly tolerance for negative hours
* `paid_weekly_rest` - Paid weekly rest (DSR) configuration (JSON string)
* `punch_on_leave_treatment` - Treatment applied to punches registered while on leave
* `punch_on_nightly_leave_treatment` - Treatment applied to punches registered while on nightly leave
* `nightly_account_code` - Account code used for nightly hours
Work shifts (turnos) configured in the account. **Sync:** Full table. **Primary key:** `key`.
* `key` - Unique identifier of the shift
* `name` - Name of the shift
* `description` - Description of the shift
* `uses_rostering` - Whether the shift is used in rostering (escalas)
* `day_starts_at` - Time at which the shift's day starts
* `starts_on_previous_day` - Whether the shift starts on the previous calendar day
* `office_hours` - Expected office hours of the shift
* `has_no_office_hours` - Whether the shift has no fixed office hours
* `flexible_break_hours` - Flexible break hours of the shift
* `extra_hours` - Extra hour thresholds configured for the shift
* `extra_office_hours_treatment` - Treatment applied to extra hours worked within office hours
* `extra_hours_treatment` - Treatment applied to extra hours worked
* `extra_nightly_hours_treatment` - Treatment applied to extra hours worked at night
* `uses_hours_compensation` - Whether the shift uses an hours compensation bank (banco de horas)
* `applies_factor_after_compensation` - Whether the extra-hours factor is applied after compensation
* `negative_compensation_treatment` - Treatment applied to negative compensated hours
* `positive_compensation_treatment` - Treatment applied to positive compensated hours
* `negative_compensation_tolerance` - Tolerance for negative compensated hours
* `positive_compensation_tolerance` - Tolerance for positive compensated hours
* `intershift_break_treatment` - Treatment applied to breaks between shifts (interjornada)
* `intrashift_break_treatment` - Treatment applied to breaks within the shift (intrajornada)
* `extra_intra_shift_breat_treatment` - Treatment applied to extra breaks within the shift
* `in_itinere_entry_bonus` - In itinere bonus applied at entry
* `in_itinere_exit_bonus` - In itinere bonus applied at exit
* `punch_on_leave_treatment` - Treatment applied to punches registered while on leave
* `punch_on_nightly_leave_treatment` - Treatment applied to punches registered while on nightly leave
* `periods_treatments` - Treatments applied to specific periods of the shift (JSON string)
* `early_entry_tolerance` - Tolerance for early entry punches
* `late_entry_tolerance` - Tolerance for late entry punches
* `early_exit_tolerance` - Tolerance for early exit punches
* `late_exit_tolerance` - Tolerance for late exit punches
# Ahrefs as a data source
Source: https://docs.nekt.com/sources/ahrefs
Bring data from Ahrefs to your Lakehouse.
Ahrefs is an SEO toolset that indexes the web to report on backlinks, organic and paid search performance, keyword research, rank tracking, and site audits. This connector reads the Ahrefs API v3 and brings those reports into your Lakehouse.
## Configuring Ahrefs as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Ahrefs option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an Ahrefs API key. In Ahrefs, go to **Account settings > API keys** and create one. Only workspace owners and admins can create API keys, and each key expires one year after it is created.
The Ahrefs API is available on eligible paid plans, and most reports consume **API units** from your monthly allowance. Read the cost section below before configuring the source.
The following configurations are available:
* **API Key**: your Ahrefs API key.
* **Targets**: the domains or URLs (for example `example.com`) to pull the Site Explorer reports for. One request is made per target per report. Leave this empty to skip every Site Explorer stream.
* **Keywords**: the keywords to pull the Keywords Explorer reports for. Leave this empty to skip the keyword streams.
* **Country**: (Default: `br`) the two-letter country code (ISO 3166-1 alpha-2) the reports are scoped to. It applies to every Site Explorer and Keywords Explorer report that accepts a country.
* **Row Limit per Report**: (Default: `1000`) the maximum number of rows fetched per report per target. Ahrefs bills API units per returned row, so this is the main lever on how much a sync costs.
Optionally, under advanced settings:
* **Start Date**: the earliest date to pull the historical reports from. It is only used on the first sync; afterwards each history stream resumes from its own bookmark. Defaults to one year ago.
* **Target Scope**: (Default: `subdomains`) how much of each target to report on: `exact` for that URL only, `prefix` for the URL and everything under it, `domain` for the domain without subdomains, or `subdomains` for the domain and its subdomains.
* **Protocol**: (Default: `both`) the protocol of the target to report on: `both`, `http`, or `https`.
* **History Granularity**: (Default: `daily`) the time interval used to group the historical reports: `daily`, `weekly`, or `monthly`.
* **Search Volume Mode**: (Default: `monthly`) how search volume is calculated: `monthly` for the latest month, or `average` for the average of the last 12 months.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## API units and sync cost
Ahrefs meters most of its API. A request costs `max(50, per_row_cost x rows)` API units, where `per_row_cost` is the sum of the unit cost of every column requested (1 unit by default, and 5 or 10 units for premium metrics such as search volume, traffic, and keyword difficulty). The field descriptions below name the premium columns explicitly.
Three things control how much a sync costs:
* **Row Limit per Report** caps how many rows each report returns. Reports are ordered by their most meaningful metric, so the rows kept are the most relevant ones.
* **Stream selection** decides which reports run at all. The Rank Tracker, project management, and subscription streams are free; the Site Audit issues report costs a flat 50 units per project.
* **Column selection** is passed through to Ahrefs. Columns you deselect are not requested, so they are not billed.
The units actually consumed appear in the run logs, and the `subscription_limits_and_usage` stream records your remaining monthly allowance on every sync.
You can also cap the monthly usage of an individual API key in Ahrefs under **Account settings > API keys**, which is a useful safety net while you tune the configuration.
## Limitations
* **No pagination.** Ahrefs API v3 accepts a row limit but offers no way to request a next page, so each report returns its top rows only. Raise the row limit if you need more, keeping the unit cost in mind.
* **Rate limit.** Ahrefs allows 60 requests per minute. The connector paces itself below that limit, so syncs with many targets take proportionally longer.
* **Reports outside your plan.** Endpoints your subscription does not cover are skipped with a warning instead of failing the run.
* **Adding a target later.** History streams keep one bookmark per stream rather than one per target. If you add a target to an existing source, run a full sync to backfill its history.
# Streams and Fields
Below you'll find all available data streams from Ahrefs and their corresponding fields:
### Site Explorer: snapshots
Domain Rating and Ahrefs Rank of each target, as of today.
**Primary key:** `target` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `ahrefs_rank` - The strength of your target's backlink profile compared to the other websites in our database, with rank #1 being the strongest.
* `domain_rating` - The strength of your target's backlink profile compared to the other websites in our database on a 100-point logarithmic scale.
Top-line organic and paid search metrics of each target, as of today.
**Primary key:** `target` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `org_cost` - The estimated value of your target's monthly organic search traffic, in USD cents. Costs 10 API units per row.
* `org_keywords` - The total number of keywords that your target ranks for in the top 100 organic search results.
* `org_keywords_1_3` - The total number of keywords that your target ranks for in the top 3 organic search results.
* `org_traffic` - The estimated number of monthly visitors that your target gets from organic search. Costs 10 API units per row.
* `paid_cost` - The estimated cost of your target's monthly paid search traffic, in USD cents. Costs 10 API units per row.
* `paid_keywords` - The total number of keywords that your target ranks for in paid search results.
* `paid_pages` - The total number of pages from a target ranking in paid search results.
* `paid_traffic` - The estimated number of monthly visitors that your target gets from paid search. Costs 10 API units per row.
Organic and paid search metrics of each target broken down by country.
**Primary key:** `target`, `country` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `country` - The country reported by Ahrefs.
* `org_cost` - The estimated value of your target's monthly organic search traffic, in USD cents. Costs 10 API units per row.
* `org_keywords` - The total number of keywords that your target ranks for in the top 100 organic search results.
* `org_keywords_1_3` - The total number of keywords that your target ranks for in the top 3 organic search results.
* `org_traffic` - The estimated number of monthly visitors that your target gets from organic search. Costs 10 API units per row.
* `paid_cost` - The estimated cost of your target's monthly paid search traffic, in USD cents. Costs 10 API units per row.
* `paid_keywords` - The total number of keywords that your target ranks for in paid search results.
* `paid_pages` - The total number of pages from a target ranking in the top 100 paid search results.
* `paid_traffic` - The estimated number of monthly visitors that your target gets from paid search. Costs 10 API units per row.
Live counts of backlinks and referring domains pointing at each target.
**Primary key:** `target` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `all_time` - The total number of links from other websites pointing to your target for all time.
* `all_time_refdomains` - The total number of unique domains linking to your target for all time. Costs 5 API units per row.
* `live` - The total number of links from other websites pointing to your target.
* `live_refdomains` - The total number of unique domains linking to your target. Costs 5 API units per row.
Counts of the outgoing links each target points at other sites.
**Primary key:** `target` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `linked_domains` - The number of unique root domains linked from the target.
* `linked_domains_dofollow` - The number of unique root domains linked via dofollow links from the target.
* `outgoing_links` - The number of external links from the target.
* `outgoing_links_dofollow` - The number of external dofollow links from the target.
Keywords each target ranks for in the top 100 organic results.
**Primary key:** `target`, `keyword`, `keyword_country` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `best_position` - The top position your target ranks for in the organic search results for a keyword.
* `best_position_diff` - The change in position between your selected dates.
* `best_position_has_thumbnail` - The top position has a thumbnail.
* `best_position_has_video` - The top position has a video.
* `best_position_kind` - The kind of the top position: organic, paid, or a SERP feature.
* `best_position_set` - The ranking group of the top position.
* `best_position_url` - The ranking URL in organic search results.
* `cpc` - Cost Per Click shows the average price that advertisers pay for each ad click in paid search results for a keyword, in USD cents.
* `entities` - Organizations, products, persons, works, events, and locations found in a keyword. Serialized as a JSON array of objects.
* `is_best_position_set_top_11_50` - The ranking group of the top position is 11-50.
* `is_best_position_set_top_3` - The ranking group of the top position is Top 3.
* `is_best_position_set_top_4_10` - The ranking group of the top position is 4-10.
* `is_branded` - User intent: branded. The user is searching for a specific brand or company name.
* `is_commercial` - User intent: commercial. The user is comparing products or services before making a purchase decision.
* `is_informational` - User intent: informational. The user is looking for information or an answer to a specific question.
* `is_local` - User intent: local. The user is looking for information relevant to a specific location or nearby services.
* `is_navigational` - User intent: navigational. The user is searching for a specific website or web page.
* `is_transactional` - User intent: transactional. The user is ready to complete an action, often a purchase.
* `keyword` - The keyword your target ranks for.
* `keyword_country` - The country of a keyword your target ranks for.
* `keyword_difficulty` - An estimation of how hard it is to rank in the top 10 organic search results for a keyword on a 100-point scale. Costs 10 API units per row.
* `keyword_language` - The language of the search query.
* `language` - The SERP language.
* `last_update` - The date when we last checked search engine results for a keyword.
* `serp_features` - The SERP features that appear in search results for a keyword.
* `serp_features_count` - The number of SERP features that appear in search results for a keyword.
* `serp_target_main_positions_count` - The number of target URLs ranking for a keyword excluding positions in Sitelinks, Top stories, Image packs, and posts on X (Twitter).
* `serp_target_positions_count` - The number of target URLs ranking for a keyword.
* `status` - The status of a page: the new page that just started to rank ('left'), the lost page that disappeared from search results ('right'), or no change ('both').
* `sum_paid_traffic` - An estimation of the number of monthly visits that your target gets from paid search for a keyword. Costs 10 API units per row.
* `sum_traffic` - An estimation of the number of monthly visitors that your target gets from organic search for a keyword. Costs 10 API units per row.
* `volume` - An estimation of the number of searches for a keyword over the latest month. Costs 10 API units per row.
* `volume_desktop_pct` - The percentage of the total search volume that comes from desktop devices.
* `volume_mobile_pct` - The percentage of the total search volume that comes from mobile devices.
* `words` - The number of words in a keyword.
Sites competing with each target for the same organic keywords.
**Primary key:** `target`, `competitor_domain` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `competitor_domain` - A competitor's domain of your target in 'domains' group mode.
* `competitor_url` - A competitor's URL of your target in pages' group mode.
* `domain_rating` - The strength of a domain's backlink profile compared to the others in our database on a 100-point scale.
* `group_mode` - To see competing pages instead, use the 'exact URL' target mode or 'path' target mode if your target doesn't have multiple pages.
* `keywords_common` - Organic keywords that both your target and a competitor are ranking for.
* `keywords_competitor` - Organic keywords that a competitor is ranking for, but your target isn't.
* `keywords_target` - Organic keywords that your target is ranking for, but a competitor isn't.
* `pages` - The total number of pages from a target ranking in search results.
* `pages_diff` - The change in pages between your selected dates.
* `share` - The percentage of common keywords out of the total number of keywords that your target and a competitor both rank for.
* `traffic` - An estimation of the number of monthly visits that a page gets from organic search over the latest month or over the latest known 12 months of data depending on the 'volume\_mode' parameter. Costs 10 API units per row.
* `traffic_diff` - The change in traffic between your selected dates.
* `value` - The estimated value of a page's monthly organic search traffic, in USD cents. Costs 10 API units per row.
* `value_diff` - The change in value between your selected dates.
Pages of each target that bring the most organic search traffic.
**Primary key:** `target`, `url` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `keywords` - The total number of keywords that your target ranks for in the top 100 organic search results.
* `keywords_diff` - The change in keywords between your selected dates.
* `keywords_diff_percent` - The change in keywords between your selected dates, in percents.
* `page_type` - Comma-separated list of AI-predicted hierarchical page type paths. Each value is a slash-prefixed path (e.g. /Article/How\_to).
* `referring_domains` - The number of unique domains linking to a page. Costs 5 API units per row.
* `status` - The status of a page: the new page that just started to rank ('left'), the lost page that disappeared from search results ('right'), or no change ('both').
* `sum_traffic` - An estimation of the monthly organic search traffic that a page gets from all the keywords that it ranks for. Costs 10 API units per row.
* `top_keyword` - The keyword that brings the most organic traffic to a page.
* `top_keyword_best_position` - The ranking position that a page holds for its top keyword.
* `top_keyword_best_position_diff` - The change in the top position between your selected dates.
* `top_keyword_best_position_kind` - The kind of the top position: organic, paid or a SERP feature.
* `top_keyword_best_position_title` - The title displayed for the page in its top keyword's SERP.
* `top_keyword_country` - The country in which a page ranks for its top keyword.
* `top_keyword_volume` - An estimation of the average monthly number of searches for the top keyword over the latest month or over the latest known 12 months of data depending on the 'volume\_mode' parameter. Costs 10 API units per row.
* `traffic_diff` - The change in traffic between your selected dates.
* `traffic_diff_percent` - The change in traffic between your selected dates, in percents.
* `ur` - URL Rating (UR) shows the strength of your target page's backlink profile on a 100-point logarithmic scale.
* `url` - The ranking page URL.
* `value` - The estimated value of a page's monthly organic search traffic, in USD cents. Costs 10 API units per row.
* `value_diff` - The change in traffic value between your selected dates.
* `value_diff_percent` - The change in traffic value between your selected dates, in percents.
Pages of each target that receive paid search traffic.
**Primary key:** `target`, `url` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `ads_count` - The number of unique ads with a page.
* `ads_count_diff` - The change in ads between your selected dates.
* `keywords` - The total number of keywords that your target ranks for in paid search results.
* `keywords_diff` - The change in keywords between your selected dates.
* `keywords_diff_percent` - The change in keywords between your selected dates, in percents.
* `referring_domains` - The number of unique domains linking to a page. Costs 5 API units per row.
* `status` - The status of a page: the new page that just started to rank in paid results ('left'), the lost page that disappeared from paid results ('right'), or no change ('both').
* `sum_traffic` - An estimation of the monthly paid search traffic that a page gets from all the keywords that it ranks for. Costs 10 API units per row.
* `top_keyword` - The keyword that brings the most paid traffic to a page.
* `top_keyword_best_position` - The ranking position that a page holds for its top keyword.
* `top_keyword_best_position_diff` - The change in the top position between your selected dates.
* `top_keyword_best_position_kind` - The kind of the top position: organic, paid or a SERP feature.
* `top_keyword_best_position_title` - The title displayed for the page in its top keyword's SERP.
* `top_keyword_country` - The country in which a page ranks for its top keyword.
* `top_keyword_volume` - An estimation of the average monthly number of searches for the top keyword over the latest month or over the latest known 12 months of data depending on the 'volume\_mode' parameter. Costs 10 API units per row.
* `traffic_diff` - The change in traffic between your selected dates.
* `traffic_diff_percent` - The change in traffic between your selected dates, in percents.
* `ur` - URL Rating (UR) shows the strength of your target page's backlink profile on a 100-point logarithmic scale.
* `url` - The ranking page URL.
* `value` - The estimated cost of a page's monthly paid search traffic, in USD cents. Costs 10 API units per row.
* `value_diff` - The change in traffic value between your selected dates.
* `value_diff_percent` - The change in traffic value between your selected dates, in percents.
Domains linking to each target, strongest first.
**Primary key:** `target`, `domain` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `dofollow_linked_domains` - The number of unique root domains with dofollow links linked from the referring domain.
* `dofollow_links` - The number of links from the referring domain to your target that don't have the 'nofollow' attribute.
* `dofollow_refdomains` - The number of unique domains with dofollow links to the referring domain. Costs 5 API units per row.
* `domain` - A referring domain that has at least one link to your target.
* `domain_rating` - The strength of a domain's backlink profile compared to the others in our database on a 100-point scale.
* `first_seen` - The date we first found a backlink to your target from the referring domain.
* `ip_source` - The referring domain IP address.
* `is_root_domain` - The domain name is a root domain name.
* `is_spam` - Indicates whether the backlink comes from a known spammy domain.
* `last_seen` - The date your target lost its last live backlink for the referring domain.
* `links_to_target` - The number of backlinks from the referring domain to your target.
* `lost_links` - The number of backlinks lost from the referring domain for the selected time period.
* `new_links` - The number of new backlinks found from the referring domain for the selected time period.
* `positions_source_domain` - The number of keywords that the referring domain ranks for in the top 100 positions.
* `traffic_domain` - The referring domain's estimated monthly organic traffic from search. Costs 10 API units per row.
Anchor texts used by the links pointing at each target.
**Primary key:** `target`, `anchor` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `anchor` - The clickable words in a link that point to a URL.
* `dofollow_links` - The number of links with a given anchor to your target that don't have the 'nofollow' attribute.
* `first_seen` - The date we first found a link with a given anchor to your target.
* `is_spam` - Indicates whether the backlink comes from a known spammy domain.
* `last_seen` - The date we discovered the last backlink with a given anchor was lost.
* `links_to_target` - The number of inbound backlinks your target has with a given anchor.
* `lost_links` - The number of backlinks with a given anchor lost during the selected time period.
* `new_links` - The number of new backlinks with a given anchor found during the selected time period.
* `refdomains` - The number of unique domains linking to your target with a given anchor. Costs 5 API units per row.
* `refpages` - The number of pages containing a link with a given anchor to your target.
* `top_domain_rating` - The highest Domain Rating (DR) counted out of all referring domains. DR shows the strength of a website's backlink profile compared to the others in our database on a 100-point scale.
Individual backlinks pointing at each target, strongest first.
**Primary key:** `target`, `url_from`, `url_to`, `anchor` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `ahrefs_rank_source` - The strength of the referring domain's backlink profile compared to the other websites in our database, with rank #1 being the strongest.
* `ahrefs_rank_target` - The strength of the target domain's backlink profile compared to the other websites in our database, with rank #1 being the strongest.
* `alt` - The alt attribute of the link.
* `anchor` - The clickable words in a link that point to a URL.
* `broken_redirect_new_target` - The new destination of a modified redirect.
* `broken_redirect_reason` - The reason the redirect was considered broken during the last crawl.
* `broken_redirect_source` - The redirecting URL that was modified, causing the redirect to become broken.
* `class_c` - The number of unique class\_c subnets linking to the referring page. Costs 5 API units per row.
* `discovered_status` - The reason the link was discovered during the last crawl: the page was crawled for the first time, the link was added to the page, or the link re-appeared after being removed.
* `domain_rating_source` - The strength of the referring domain's backlink profile compared to the others in our database on a 100-point scale.
* `domain_rating_target` - The strength of the referring domain's backlink profile compared to the others in our database on a 100-point scale.
* `drop_reason` - The reason we removed the link from our index.
* `encoding` - The character set encoding of the referring page HTML.
* `first_seen` - The date the referring page URL was first discovered.
* `first_seen_link` - The date we first found a backlink to your target on a given referring page.
* `http_code` - The return code from HTTP protocol returned during the referring page crawl.
* `http_crawl` - The link was discovered without executing javascript and rendering the page.
* `ip_source` - The referring domain IP address.
* `is_alternate` - The link with the rel='alternate' attribute.
* `is_canonical` - The link with the rel='canonical' attribute.
* `is_content` - The link was found in the biggest piece of content on the page.
* `is_dofollow` - The link has no special nofollow attribute.
* `is_form` - The link was found in a form HTML tag.
* `is_frame` - The link was found in an iframe HTML tag.
* `is_image` - The link is a regular link that has an image inside their href attribute.
* `is_lost` - The link currently does not exist anymore.
* `is_new` - The link was discovered on the last crawl.
* `is_nofollow` - The link or the referring page has the nofollow attribute set.
* `is_redirect` - The link pointing to your target via a redirect.
* `is_redirect_lost` - The redirected link currently does not exist anymore.
* `is_root_source` - The referring domain name is a root domain name.
* `is_root_target` - The target domain name is a root domain name.
* `is_rss` - The link was found in an RSS feed.
* `is_spam` - Indicates whether the backlink comes from a known spammy domain.
* `is_sponsored` - The link has the Sponsored attribute set in the referring page HTML.
* `is_text` - The link is a standard href hyperlink.
* `is_ugc` - The link has the User Generated Content attribute set in the referring page HTML.
* `js_crawl` - The link was discovered after executing javascript and rendering the page.
* `languages` - The languages listed in the referring page metadata or detected by the crawler to appear in the HTML.
* `last_seen` - The date we discovered that the link was lost.
* `last_visited` - The date we last verified a live link to your target page.
* `link_group_count` - The number of backlinks that were grouped together based on the aggregation parameter. This field cannot be used with aggregation 'all'.
* `link_type` - The kind of the backlink.
* `linked_domains_source_domain` - The number of unique root domains linked from the referring domain.
* `linked_domains_source_page` - The number of unique root domains linked from the referring page.
* `linked_domains_target_domain` - The number of unique root domains linked from the target domain.
* `links_external` - The number of external links from the referring page.
* `links_internal` - The number of internal links from the referring page.
* `lost_reason` - The reason the link was lost during the last crawl.
* `name_source` - The complete referring domain name, including subdomains.
* `name_target` - The complete target domain name, including subdomains.
* `noindex` - The referring page has the noindex meta attribute.
* `page_category_source` - Comma-separated list of AI-predicted hierarchical category paths for the referring page. Each value is a slash-prefixed path (e.g. /Business\_and\_Industrial/Advertising\_and\_Marketing/Marketing).
* `page_size` - The size in bytes of the referring page content.
* `page_type_source` - Comma-separated list of AI-predicted hierarchical page type paths for the referring page. Each value is a slash-prefixed path (e.g. /Article/How\_to).
* `port_source` - The network port of the referring page URL.
* `port_target` - The network port of the target page URL.
* `positions` - The number of keywords that the referring page ranks for in the top 100 positions.
* `powered_by` - Web technologies used to build and serve the referring page content.
* `redirect_code` - The HTTP status code of a referring page pointing to your target via a redirect.
* `redirect_kind` - The HTTP status codes returned by the target redirecting URL or redirect chain.
* `refdomains_source` - The number of unique referring domains linking to the referring page. Costs 5 API units per row.
* `refdomains_source_domain` - The number of unique referring domains linking to the referring domain. Costs 5 API units per row.
* `refdomains_target_domain` - The number of unique referring domains linking to the target domain. Costs 5 API units per row.
* `root_name_source` - The root domain name of the referring domain, not including subdomains.
* `root_name_target` - The root domain name of the target domain, not including subdomains.
* `snippet_left` - The snippet of text appearing just before the link.
* `snippet_right` - The snippet of text appearing just after the link.
* `source_page_author` - The author of the referring page.
* `source_page_publish_date` - the date we identified the page was published.
* `title` - The html title of the referring page.
* `tld_class_source` - The top level domain class of the referring domain.
* `tld_class_target` - The top level domain class of the target domain.
* `traffic` - The referring page's estimated monthly organic traffic from search. Costs 10 API units per row.
* `traffic_domain` - The referring domain's estimated monthly organic traffic from search. Costs 10 API units per row.
* `url_from` - The URL of the page containing a link to your target.
* `url_rating_source` - The strength of the referring page's backlink profile compared to the others in our database on a 100-point scale.
* `url_redirect` - A redirect chain the target URL of the link points to.
* `url_redirect_with_target` - The target URL of the link with its redirect chain.
* `url_to` - The URL the backlink points to.
Backlinks pointing at pages of each target that no longer resolve.
**Primary key:** `target`, `url_from`, `url_to`, `anchor` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `ahrefs_rank_source` - The strength of the referring domain's backlink profile compared to the other websites in our database, with rank #1 being the strongest.
* `ahrefs_rank_target` - The strength of the target domain's backlink profile compared to the other websites in our database, with rank #1 being the strongest.
* `alt` - The alt attribute of the link.
* `anchor` - The clickable words in a link that point to a URL.
* `class_c` - The number of unique class\_c subnets linking to the referring page. Costs 5 API units per row.
* `domain_rating_source` - The strength of the referring domain's backlink profile compared to the others in our database on a 100-point scale.
* `domain_rating_target` - The strength of the referring domain's backlink profile compared to the others in our database on a 100-point scale.
* `encoding` - The character set encoding of the referring page HTML.
* `first_seen` - The date the referring page URL was first discovered.
* `first_seen_link` - The date we first found a backlink to your target on a given referring page.
* `http_code` - The return code from HTTP protocol returned during the referring page crawl.
* `http_code_target` - The return code from HTTP protocol returned during the target page crawl.
* `http_crawl` - The link was discovered without executing javascript and rendering the page.
* `ip_source` - The referring domain IP address.
* `is_alternate` - The link with the rel='alternate' attribute.
* `is_canonical` - The link with the rel='canonical' attribute.
* `is_content` - The link was found in the biggest piece of content on the page.
* `is_dofollow` - The link has no special nofollow attribute.
* `is_form` - The link was found in a form HTML tag.
* `is_frame` - The link was found in an iframe HTML tag.
* `is_image` - The link is a regular link that has an image inside their href attribute.
* `is_nofollow` - The link or the referring page has the nofollow attribute set.
* `is_redirect` - The link pointing to your target via a redirect.
* `is_root_source` - The referring domain name is a root domain name.
* `is_root_target` - The target domain name is a root domain name.
* `is_rss` - The link was found in an RSS feed.
* `is_spam` - Indicates whether the backlink comes from a known spammy domain.
* `is_sponsored` - The link has the Sponsored attribute set in the referring page HTML.
* `is_text` - The link is a standard href hyperlink.
* `is_ugc` - The link has the User Generated Content attribute set in the referring page HTML.
* `js_crawl` - The link was discovered after executing javascript and rendering the page.
* `languages` - The languages listed in the referring page metadata or detected by the crawler to appear in the HTML.
* `last_seen` - The date we discovered that the link was lost.
* `last_visited` - The date we last re-crawled the referring page to verify the backlink is alive.
* `last_visited_target` - The date we last re-crawled the target page to verify that it is broken.
* `link_group_count` - The number of backlinks that were grouped together based on the aggregation parameter. This field cannot be used with aggregation 'all'.
* `link_type` - The kind of the backlink.
* `linked_domains_source_domain` - The number of unique root domains linked from the referring domain.
* `linked_domains_source_page` - The number of unique root domains linked from the referring page.
* `linked_domains_target_domain` - The number of unique root domains linked from the target domain.
* `links_external` - The number of external links from the referring page.
* `links_internal` - The number of internal links from the referring page.
* `name_source` - The complete referring domain name, including subdomains.
* `name_target` - The complete target domain name, including subdomains.
* `page_category_source` - Comma-separated list of AI-predicted hierarchical category paths for the referring page. Each value is a slash-prefixed path (e.g. /Business\_and\_Industrial/Advertising\_and\_Marketing/Marketing).
* `page_size` - The size in bytes of the referring page content.
* `page_type_source` - Comma-separated list of AI-predicted hierarchical page type paths for the referring page. Each value is a slash-prefixed path (e.g. /Article/How\_to).
* `port_source` - The network port of the referring page URL.
* `port_target` - The network port of the target page URL.
* `positions` - The number of keywords that the referring page ranks for in the top 100 positions.
* `powered_by` - Web technologies used to build and serve the referring page content.
* `redirect_code` - The HTTP status code of a referring page pointing to your target via a redirect.
* `redirect_kind` - The HTTP status codes returned by the target redirecting URL or redirect chain.
* `refdomains_source` - The number of unique referring domains linking to the referring page. Costs 5 API units per row.
* `refdomains_source_domain` - The number of unique referring domains linking to the referring domain. Costs 5 API units per row.
* `refdomains_target_domain` - The number of unique referring domains linking to the target domain. Costs 5 API units per row.
* `root_name_source` - The root domain name of the referring domain, not including subdomains.
* `root_name_target` - The root domain name of the target domain, not including subdomains.
* `snippet_left` - The snippet of text appearing just before the link.
* `snippet_right` - The snippet of text appearing just after the link.
* `source_page_author` - The author of the referring page.
* `title` - The html title of the referring page.
* `tld_class_source` - The top level domain class of the referring domain.
* `tld_class_target` - The top level domain class of the target domain.
* `traffic` - The referring page's estimated monthly organic traffic from search. Costs 10 API units per row.
* `traffic_domain` - The referring domain's estimated monthly organic traffic from search. Costs 10 API units per row.
* `url_from` - The URL of the page containing a link to your target.
* `url_rating_source` - The strength of the referring page's backlink profile compared to the others in our database on a 100-point scale.
* `url_redirect` - A redirect chain the target URL of the link points to.
* `url_redirect_with_target` - The target URL of the link with its redirect chain.
* `url_to` - The URL the backlink points to.
Pages of each target that attract the most referring domains.
**Primary key:** `target`, `url_to` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `dofollow_to_target` - The number of links to your target page that don't have the 'nofollow' attribute.
* `first_seen_link` - The date we first found a link to your target.
* `http_code_target` - The return code from HTTP protocol returned during the target page crawl.
* `is_spam` - Indicates whether the backlink comes from a known spammy domain.
* `languages_target` - The languages listed in the target page metadata or detected by the crawler to appear in the HTML.
* `last_seen` - The date your target page lost its last live link.
* `last_visited_source` - The date we last verified a live link to your target page.
* `last_visited_target` - The date we last crawled your target page.
* `links_to_target` - The number of inbound backlinks the target page has.
* `lost_links_to_target` - The number of backlinks lost during the selected time period.
* `new_links_to_target` - The number of new backlinks found during the selected time period.
* `nofollow_to_target` - The number of links to your target page that have the 'nofollow' attribute.
* `powered_by_target` - Web technologies used to build and serve the target page content.
* `redirects_to_target` - The number of inbound redirects to your target page.
* `refdomains_target` - The number of unique referring domains linking to the target page. Costs 5 API units per row.
* `target_redirect` - The target's redirect if any.
* `title_target` - The html title of the target page.
* `top_domain_rating_source` - The highest Domain Rating (DR) counted out of all referring domains. DR shows the strength of a website's backlink profile compared to the others in our database on a 100-point scale.
* `url_rating_target` - The strength of the target page's backlink profile compared to the others in our database on a 100-point scale.
* `url_to` - The URL the backlink points to.
Pages of each target that the Ahrefs crawler has visited.
**Primary key:** `target`, `url` - **Sync:** full table
**Fields:**
* `target` - The domain or URL the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `first_seen` - The timestamp when the URL was first seen by Ahrefs.
* `http_code` - The HTTP status code returned when the page was last crawled.
* `last_attempt` - The timestamp of the last crawl attempt.
* `last_crawled` - The timestamp of the last successful crawl.
* `title` - The page's meta title.
* `url` - The URL of the crawled page.
* `url_rating` - URL Rating of the page at the time of the last crawl, on a 100-point scale.
### Site Explorer: history
Domain Rating of each target over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `domain_rating` - The strength of your target page's backlink profile compared to the other websites in our database on a 100-point logarithmic scale.
URL Rating of each target over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `url_rating` - The strength of your target page's backlink profile compared to the other websites in our database on a 100-point logarithmic scale.
Organic and paid traffic and traffic value of each target over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `org_cost` - The estimated cost of your target's monthly organic search traffic, in USD cents. Costs 10 API units per row.
* `org_traffic` - The estimated number of monthly visitors that your target gets from organic search. Costs 10 API units per row.
* `paid_cost` - The estimated cost of your target's monthly paid search traffic, in USD cents. Costs 10 API units per row.
* `paid_traffic` - The estimated number of monthly visitors that your target gets from paid search. Costs 10 API units per row.
Number of domains linking to each target over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `refdomains` - The total number of unique domains linking to your target. Costs 5 API units per row.
Number of pages of each target ranking in organic search over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `pages` - The total number of pages from a target ranking in the top 100 organic search results.
Organic keyword counts of each target, split by ranking group, over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `top11_20` - The total number of keywords that your target ranks for in the top 11-20 organic search results.
* `top11_plus` - The total number of keywords that your target ranks for in the top 11+ organic search results.
* `top21_50` - The total number of keywords that your target ranks for in the top 21-50 organic search results.
* `top3` - The total number of keywords that your target ranks for in the top 3 organic search results.
* `top4_10` - The total number of keywords that your target ranks for in the top 4-10 organic search results.
* `top51_plus` - The total number of keywords that your target ranks for in the top 51+ organic search results.
Combined search volume of the keywords each target ranks for, over time.
**Primary key:** `target`, `date` - **Sync:** incremental on `date`
**Fields:**
* `target` - The domain or URL the report was requested for.
* `date` - The date reported by Ahrefs.
* `total_search_volume` - The total search volume of keywords for which your target ranks within the specified `top_positions` in the search results. Costs 10 API units per row.
### Keywords Explorer
Search volume, difficulty and click metrics of each configured keyword.
**Primary key:** `keyword`, `country` - **Sync:** full table
**Fields:**
* `country` - The two-letter country code (ISO 3166-1 alpha-2) the report was requested for.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `clicks` - The average monthly number of clicks on the search results that people make while searching for the target keyword.
* `cpc` - Cost Per Click shows the average price that advertisers pay for each ad click in paid search results for a keyword, in USD cents.
* `cps` - Clicks Per Search (or CPS) is the ratio of Clicks to Keyword Search volume. It shows how many different search results get clicked, on average, when people search for the target keyword in a given country.
* `difficulty` - An estimation of how hard it is to rank in the top 10 organic search results for a keyword on a 100-point scale. Costs 10 API units per row.
* `first_seen` - The date when we first checked search engine results for a keyword.
* `global_volume` - How many times per month, on average, people search for the target keyword across all countries in our database. Costs 10 API units per row.
* `intents` - Indicates the purpose behind the user's search query. Object fields: `informational`, `navigational`, `commercial`, `transactional`, `branded` or `local`. All the fields are of type `bool`, with possible values `true` or `false`. Costs 10 API units per row. Serialized as a JSON object.
* `keyword` - The keyword reported by Ahrefs.
* `parent_topic` - Parent Topic determines if you can rank for your target keyword while targeting a more general topic on your page instead. To identify the Parent Topic, we take the #1 ranking page for your keyword and find the keyword responsible for sending the most traffic to that page.
* `parent_volume` - The search volume of the parent topic. Costs 10 API units per row.
* `searches_pct_clicks_organic_and_paid` - The average monthly percentage of people who clicked on both organic and paid results while searching for the target keyword.
* `searches_pct_clicks_organic_only` - The average monthly percentage of people who clicked only on organic results while searching for the target keyword.
* `searches_pct_clicks_paid_only` - The average monthly percentage of people who clicked only on paid results while searching for the target keyword.
* `serp_features` - The enriched results on a search engine results page (SERP) that are not traditional organic results.
* `serp_last_update` - The date when we last checked search engine results for a keyword.
* `traffic_potential` - The sum of organic traffic that the #1 ranking page for your target keyword receives from all the keywords that it ranks for. Costs 10 API units per row.
* `volume` - An estimation of the average monthly number of searches for a keyword over the latest known 12 months of data. Costs 10 API units per row.
* `volume_desktop_pct` - The percentage of searches for a keyword performed on desktop devices.
* `volume_mobile_pct` - The percentage of searches for a keyword performed on mobile devices.
* `volume_monthly` - An estimation of the number of searches for a keyword over the latest month. This field may not be included in the `order_by` parameter. Costs 10 API units per row.
Monthly search volume of each configured keyword over time.
**Primary key:** `keyword`, `country`, `date` - **Sync:** incremental on `date`
**Fields:**
* `keyword` - The keyword the report was requested for.
* `country` - The two-letter country code (ISO 3166-1 alpha-2) the report was requested for.
* `date` - The date reported by Ahrefs.
* `volume` - An estimation of the number of searches for a keyword over a given month.
### Rank Tracker
Current ranking of every keyword tracked in each Rank Tracker project.
**Primary key:** `project_id`, `device`, `keyword`, `country` - **Sync:** full table
**Fields:**
* `project_id` - The unique identifier of the Ahrefs project the report belongs to.
* `device` - The device rankings were tracked on: desktop or mobile.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `best_position_has_thumbnail` - The top position (or target URL's, if set) has a thumbnail.
* `best_position_has_video_preview` - The top position (or target URL's, if set) has a video preview.
* `best_position_kind` - The kind of top position (or target URL's, if set): organic, paid, or a SERP feature.
* `clicks` - Clicks metric refers to the average monthly number of clicks on the search results that people make while searching for the target keyword. Some searches generate clicks on multiple results, while others might not end in any clicks at all.
* `clicks_per_search` - Clicks Per Search is the ratio of Clicks to Keyword Search volume. It shows how many different search results get clicked, on average, when people search for the target keyword in a given country.
* `cost_per_click` - Cost Per Click shows the average price that advertisers pay for each ad click in paid search results for a keyword.
* `country` - The country that a given keyword is being tracked in. A two-letter country code (ISO 3166-1 alpha-2).
* `created_at` - The date when a keyword was added to the project.
* `is_branded` - User intent: branded. The user is searching for a specific brand or company name.
* `is_commercial` - User intent: commercial. The user is comparing products or services before making a purchase decision.
* `is_informational` - User intent: informational. The user is looking for information or an answer to a specific question.
* `is_local` - User intent: local. The user is looking for information relevant to a specific location or nearby services.
* `is_navigational` - User intent: navigational. The user is searching for a specific website or web page.
* `is_transactional` - User intent: transactional. The user is ready to complete an action, often a purchase.
* `keyword` - The keyword your target ranks for.
* `keyword_difficulty` - An estimation of how hard it is to rank in the top 10 organic search results for a keyword on a 100-point scale.
* `keyword_has_data` - Will return `false` if the keyword is still processing and no SERP has been fetched yet.
* `keyword_is_frozen` - Indicates whether a keyword has exceeded the tracked keywords limit on your plan. Such keywords are 'frozen', meaning they do not have their rankings updated.
* `language` - The SERP language that a given keyword is being tracked for.
* `location` - The location (country, state/province, or city) that a given keyword is being tracked in.
* `parent_topic` - Parent Topic determines if you can rank for your target keyword while targeting a more general topic on your page instead. To identify the Parent Topic, we take the #1 ranking page for your keyword and find the keyword responsible for sending the most traffic to that page.
* `position` - The top position (or target URL's, if set) in organic search.
* `position_diff` - The change in top position (or target URL's, if set) between selected dates.
* `search_type_image` - Search type Image shows the percentage of searches for a keyword made for images, highlighting interest in visual content.
* `search_type_news` - Search type News shows the percentage of searches for a keyword made for news articles.
* `search_type_video` - Search type Video shows the percentage of searches for a keyword made for video, reflecting interest in video content.
* `search_type_web` - Search type Web shows the percentage of searches for a keyword made for general web content, indicating interest in a wide range of information.
* `serp_features` - The SERP features that appear in search results for a keyword.
* `serp_updated` - The date when we last checked search engine results for a keyword.
* `tags` - A list of tags assigned to a given keyword.
* `target_positions_count` - The number of target URLs ranking for a keyword.
* `traffic` - An estimation of the number of monthly visits that a page gets from organic search over the latest month or over the latest known 12 months of data depending on the 'volume\_mode' parameter.
* `traffic_diff` - The change in traffic between your selected dates.
* `url` - The top-ranking URL (or target URL, if set) in organic search.
* `volume` - An estimation of the average monthly number of searches for a keyword over the latest month or over the latest known 12 months of data depending on the 'volume\_mode' parameter.
* `volume_desktop_pct` - The percentage of the total search volume that comes from desktop devices.
* `volume_mobile_pct` - The percentage of the total search volume that comes from mobile devices.
Visibility of each competitor tracked in a Rank Tracker project.
**Primary key:** `project_id`, `device`, `competitor` - **Sync:** full table
**Fields:**
* `project_id` - The unique identifier of the Ahrefs project the report belongs to.
* `device` - The device rankings were tracked on: desktop or mobile.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `ai_overview_count` - The total number of tracked keywords for which your target ranks in an AI Overview.
* `average_position` - The average of your target's top organic positions across all tracked keywords.
* `competitor` - Competitor's URL.
* `discussions_count` - The total number of tracked keywords for which your target ranks in Discussions and forums.
* `featured_snippet_count` - The total number of tracked keywords for which your target ranks in a Featured snippet.
* `image_pack_count` - The total number of tracked keywords for which your target ranks in an Image pack.
* `knowledge_card_count` - The total number of tracked keywords for which your target ranks in a Knowledge card.
* `knowledge_panel_count` - The total number of tracked keywords for which your target ranks in a Knowledge panel.
* `pos_11_20` - The total number of tracked keywords for which your target's top organic position is within the 11th to 20th results.
* `pos_1_3` - The total number of tracked keywords for which your target's top organic position is within the top 3 results.
* `pos_21_50` - The total number of tracked keywords for which your target's top organic position is within the 21st to 50th results.
* `pos_4_10` - The total number of tracked keywords for which your target's top organic position is within the 4th to 10th results.
* `pos_51_plus` - The total number of tracked keywords for which your target's top organic position is the 51st or higher.
* `pos_no_rank` - The total number of tracked keywords where your target doesn't rank.
* `share_of_traffic_value` - The share of your target's organic search traffic value compared to the total organic search traffic value for all tracked keywords.
* `share_of_voice` - The share of your target's organic search traffic compared to the total organic search traffic for all tracked keywords.
* `sitelinks_count` - The total number of tracked keywords for which your target ranks in Sitelinks.
* `thumbnail_count` - The total number of tracked keywords for which your target ranks in a Thumbnail.
* `top_stories_count` - The total number of tracked keywords for which your target ranks in Top stories.
* `traffic` - The estimated number of monthly visits that your target gets from organic search for all tracked keywords.
* `traffic_value` - The estimated value of your target's monthly organic search traffic for all tracked keywords.
* `video_preview_count` - The total number of tracked keywords for which your target ranks in a Video preview.
* `videos_count` - The total number of tracked keywords for which your target ranks in Videos.
* `x_count` - The total number of tracked keywords for which your target ranks in an X (Twitter) widget.
### Site Audit
Health score and crawl summary of every Site Audit project.
**Primary key:** `project_id` - **Sync:** full table
**Fields:**
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `date` - The finish date and time of the last finished crawl, in GMT time zone.
* `health_score` - Reflects the proportion of internal URLs on your site that do not have errors, based on the last finished crawl. Excludes crawls that are starting, in progress, finalizing, or were skipped.
* `project_id` - The unique identifier of the project.
* `project_name` - The project name.
* `status` - The status of the most recent finished crawl. Possible values: `Completed`, `Stopped`, `Error`, `In_progress`.
* `target_mode` - The scope of the target. Possible values: `exact`, `prefix`, `domain`, `subdomains`.
* `target_protocol` - The protocol of the target. Possible values: `both`, `http`, `https`.
* `target_url` - The URL of the project's target.
* `total` - Number of total crawled internal URLs.
* `urls_with_errors` - Number of internal URLs with errors.
* `urls_with_notices` - Number of internal URLs with notices.
* `urls_with_warnings` - Number of internal URLs with warnings.
Issues found by the latest crawl of each Site Audit project.
**Primary key:** `project_id`, `issue_id` - **Sync:** full table
**Fields:**
* `project_id` - The unique identifier of the Ahrefs project the report belongs to.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `added` - Number of URLs that have the issue on the current date but did not have it on the previous date.
* `category` - The category of the issue. Possible values: `Internal pages`, `Indexability`, `Links`, `Redirects`, `Content`, `Social tags`, `Duplicates`, `Localization`, `Usability and performance`, `Images`, `JavaScript`, `CSS`, `Sitemaps`, `External pages`, `Other`.
* `change` - Difference in the number of affected URLs between the specified dates.
* `crawled` - Number of URLs currently affected by the issue.
* `importance` - The importance of the issue. Possible values: `Error`, `Warning`, `Notice`.
* `is_indexable` - True if the issue applies only to indexable pages.
* `issue_id` - The unique identifier of the issue.
* `missing` - Number of URLs that had the issue on the previous date but cannot be found on the current date.
* `name` - The name of the issue.
* `new` - Number of newly discovered URLs that have the issue on the current date.
* `removed` - Number of URLs that had the issue on the previous date but no longer have it on the current date.
### Project management
Rank Tracker projects the API key has access to.
**Primary key:** `project_id` - **Sync:** full table
**Fields:**
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `access` - The access level of the project. Possible values: `private`, `shared`.
* `folder` - Innermost folder that the project is in. Serialized as a JSON object.
* `keyword_count` - The number of keywords in the project.
* `mode` - The scope of the target. Possible values: `exact`, `prefix`, `domain`, `subdomains`.
* `owned_by` - The email of the project owner.
* `project_id` - The unique identifier of the project. You can find it in the URL of your Rank Tracker project in Ahrefs: `https://app.ahrefs.com/rank-tracker/overview/#project_id#`.
* `project_name` - The project name.
* `protocol` - The protocol of the target. Possible values: `both`, `http`, `https`.
* `url` - The URL of the project's target.
* `verified` - Whether the project is verified.
* `web_analytics_data_key` - Web Analytics Data Key.
Keywords tracked in each Rank Tracker project, with their location and tags.
**Primary key:** `project_id`, `keyword`, `location_id` - **Sync:** full table
**Fields:**
* `project_id` - The unique identifier of the Ahrefs project the report belongs to.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `keyword` - The keyword added to the project.
* `language` - The name of the language assigned to a given keyword.
* `language_code` - The code of the language assigned to a given keyword.
* `location` - The name of the location assigned to a given keyword.
* `location_id` - The ID of the location assigned to a given keyword.
* `tags` - A list of tags assigned to a given keyword.
Competitors configured for each Rank Tracker project.
**Primary key:** `project_id`, `url` - **Sync:** full table
**Fields:**
* `project_id` - The unique identifier of the Ahrefs project the report belongs to.
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `mode` - The scope of the target. Possible values: `exact`, `prefix`, `domain`, `subdomains`.
* `url` - The URL of the project's target.
### Account
API unit allowance and consumption of the workspace and of this API key.
**Primary key:** `snapshot_date` - **Sync:** full table
**Fields:**
* `snapshot_date` - The UTC date on which this snapshot was pulled from Ahrefs.
* `api_key_expiration_date` - Date on which this API key will expire and stop working.
* `subscription` - Ahrefs subscription plan.
* `units_limit_api_key` - Limit for the number of API units that can be consumed via this API key per billing month (null = unlimited).
* `units_limit_workspace` - Total number of API units available to the workspace.
* `units_usage_api_key` - Number of API units consumed by this API key in the current billing month.
* `units_usage_workspace` - Number of API units consumed by the workspace in the current billing month.
* `usage_reset_date` - Start date of the next billing period when the API units usage will be reset.
# Airtable as a data source
Source: https://docs.nekt.com/sources/airtable
Bring data from Airtable to Nekt.
Airtable is a cloud-based platform that combines the features of a database with the simplicity of a spreadsheet. It allows users to create custom databases, organize information, and collaborate on projects through an intuitive interface that supports various data types and relationships.
![]()
## 1. Add your Airtable access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "add source" button located on the top right of your screen. Then, select the Airtable option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access. Check Airtable's documentation to discover where you can find the required parameters for the connection. Make some decisions about tables to be sync and emojis.
3. Click **Next**.
## 2. Select your Airtable streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Airtable data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
2. Click **Next**.
## 4. Configure your Airtable data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Airtable connector documentation as plain markdown, for use in AI agent contexts.
# Alterdata as a data source
Source: https://docs.nekt.com/sources/alterdata
Bring data from Alterdata eContador to Nekt.
Alterdata eContador is a cloud-based accounting management platform used by accounting firms in Brazil. It provides modules for HR/payroll (DP), document management, certifications (CND), task tracking, and customer service, enabling seamless integration between accountants and their clients.
## Configuring Alterdata as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Alterdata option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Alterdata eContador API credentials. The following configurations are available:
* **API Key**: Your Bearer token for the Alterdata eContador API. This is a JWT token generated in eContador under Configuracoes > ePlugin > Gerar.
* **Start Date**: (Optional) The earliest date from which records will be synced for incremental streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Alterdata eContador and their corresponding fields:
Companies registered in the DP (HR/Payroll) module.
**Key Fields:**
* `id` - Internal company identifier
* `nome` - Company name
* `cpfcnpj` - CPF or CNPJ number
* `endereco` - Address
* `externoid` - External identifier code
* `ativa` - Whether the company is active
* `tipoMovimentoPermitido` - Allowed movement types
* `controlaTransferenciaTomadores` - Whether contractor transfers are controlled
**Sync Type:** Full Table
Employees registered in the DP module. Supports incremental sync.
**Key Fields:**
* `id` - Internal employee identifier
* `codigo` - Employee code
* `nome` - Full name
* `cpf` - CPF number
* `email` - Email address
* `status` - Employment status (e.g., Ativo)
**Personal Information:**
* `nascimento` - Date of birth
* `telefone` - Phone number
* `telefonecelular` - Mobile phone
* `identidade` - ID document number
* `orgaoemissor` - ID issuing authority
**Employment Details:**
* `admissao` - Admission date
* `demissao` - Termination date
* `retorno` - Return date
* `nomefuncao` - Job function name
* `salarioBase` - Base salary
* `afastamento` - Leave status
* `afastamentodescricao` - Leave description
* `matriculaESocial` - eSocial registration number
**Address:**
* `rua` - Street
* `numero` - Number
* `complemento` - Address complement
* `bairro` - Neighborhood
* `cidade` - City
* `cep` - ZIP code
**Banking:**
* `banco` - Bank code
* `nomeDoBanco` - Bank name
* `agencia` - Branch number
* `conta` - Account number
* `chavePix` - PIX key
* `pis` - PIS number
**Work Documents:**
* `carteiradetrabalho` - Work card number
* `serie` - Work card series
* `dataemissaocarteiradetrabalho` - Work card issue date
* `datavencimentocontratoexperiencia` - Experience contract expiration
* `dataprorrogacaocontratoexperiencia` - Experience contract extension
**Schedule:**
* `horarios` - Work schedule (JSON with entry, lunch, exit times and weekly rest days)
**Metadata:**
* `externoid` - External identifier
* `dataAtualizacao` - Last update timestamp (replication key)
**Sync Type:** Incremental (via `dataAtualizacao`)
Documents shared through the eContador document module. Fetched per company. Supports incremental sync.
**Key Fields:**
* `id` - Document identifier
* `titulo` - Document title
* `descricao` - Description
* `categoria` - Category name
* `valor` - Monetary value
* `status` - Document status (e.g., Recebido)
* `pago` - Whether the document has been paid
* `empresaId` - Company CNPJ
* `departamento` - Department
**Dates:**
* `criacao` - Creation date (replication key)
* `vencimento` - Due date
* `pagamento` - Payment date
* `expiracao` - Expiration date
**Sync Type:** Incremental (via `criacao`)
Document categories available in the eContador platform. Fetched per user.
**Key Fields:**
* `id` - Category identifier
* `descricao` - Category description
* `personalizada` - Whether it is a custom category
* `pagavel` - Whether documents in this category are payable
* `login` - Associated user login
**Sync Type:** Full Table
CND (Certidao Negativa de Debito) certifications. Fetched per company. Supports incremental sync.
**Key Fields:**
* `id` - Certification identifier
* `empresaId` - Company CNPJ
* `abrangencia` - Scope/coverage
* `mimeType` - File MIME type
* `emissao` - Issue date (replication key)
* `vencimento` - Expiration date
* `status` - Certification status
* `tipoCertidaoId` - Reference to certification type
**Sync Type:** Incremental (via `emissao`)
Reference table of certification types.
**Key Fields:**
* `id` - Type identifier
* `nome` - Type name
**Sync Type:** Full Table
Users registered in the eContador identification module.
**Key Fields:**
* `id` - User identifier (CPF/CNPJ)
* `nome` - Full name
* `email` - Email address
* `telefone` - Phone number
**Sync Type:** Full Table
# Amazon Advertising as a data source
Source: https://docs.nekt.com/sources/amazon-ads
Bring data from Amazon Advertising to Nekt.
Amazon Advertising is Amazon's advertising platform for sellers, vendors and agencies to run Sponsored Products, Sponsored Brands and Sponsored Display campaigns across Amazon's marketplaces. This source brings your **campaign management** structure (campaigns, ad groups, ads, keywords and targeting) into Nekt.
## Configuring Amazon Advertising as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Amazon Advertising option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Amazon Advertising data. Click on the **Amazon Authorization** button and log in with an Amazon account that has access to the advertising profiles you want to extract data from.
The following configurations are available:
* **Where are your advertising accounts located?**: Select the region of your Amazon Advertising accounts (North America, Europe or Far East) so we can determine the correct API URL.
* **Profile IDs**: *(Advanced, optional)* A list of specific advertising profile IDs to sync. Leave it empty to sync every profile the authorized account can access.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data.
* Full table: every time the extraction happens, we'll get the current state of the data, recommended for campaign-management streams since campaign structure changes in place.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Amazon Advertising exposes data per **profile** (advertising account). Every campaign-management record includes a `profile_id` column identifying the owning profile. Identifiers are returned as strings, and nested or variable objects (budgets, bidding settings, targeting expressions) are provided as JSON strings.
Below you'll find all available data streams from Amazon Advertising and their corresponding fields:
Advertising profiles (accounts) the authorized user can access. This stream is the source of the profile scope used by every other stream.
**Key Fields:**
* `profile_id` - Advertising profile (account) identifier
* `country_code` - Marketplace country code (e.g. US, BR)
* `currency_code` - Currency used for the profile
* `daily_budget` - Default daily budget for the profile
* `timezone` - Profile timezone
* `account_id` - Seller/vendor account id
* `account_marketplace_string_id` - Marketplace string id
* `account_name` - Account display name
* `account_type` - Account type: seller, vendor or agency
* `account_sub_type` - Account sub-type, when present
* `account_valid_payment_method` - Whether a valid payment method exists
Portfolios group campaigns under a shared budget.
**Key Fields:**
* `profile_id` - Owning advertising profile id
* `portfolio_id` - Portfolio identifier
* `name` - Portfolio name
* `state` - ENABLED, PAUSED or ARCHIVED
* `in_budget` - Whether the portfolio is within budget
* `budget` - Budget object as a JSON string
* `budget_controls` - Budget controls object as a JSON string
Sponsored Products campaigns.
**Key Fields:**
* `profile_id`, `campaign_id`, `name`, `state`
* `targeting_type` - AUTO or MANUAL targeting
* `start_date`, `end_date`, `portfolio_id`
* `budget` - Budget object as a JSON string
* `dynamic_bidding` - Dynamic bidding config as a JSON string
* `serving_status` - Delivery serving status
* `extended_data` - Remaining extendedData (timestamps, status details) as a JSON string
* `tags` - Campaign tags as a JSON string
**Key Fields:** `profile_id`, `ad_group_id`, `campaign_id`, `name`, `state`, `default_bid`, `serving_status`, `extended_data`
**Key Fields:** `profile_id`, `ad_id`, `campaign_id`, `ad_group_id`, `state`, `sku`, `asin`, `custom_text`, `serving_status`, `extended_data`
**Key Fields:** `profile_id`, `keyword_id`, `campaign_id`, `ad_group_id`, `keyword_text`, `match_type` (EXACT, PHRASE, BROAD), `state`, `bid`
Targeting clauses for Sponsored Products.
**Key Fields:** `profile_id`, `target_id`, `campaign_id`, `ad_group_id`, `state`, `expression_type` (AUTO/MANUAL), `bid`, `expression` and `resolved_expression` (JSON strings)
**Key Fields:** `profile_id`, `keyword_id`, `campaign_id`, `ad_group_id`, `keyword_text`, `match_type`, `state`
Campaign-level negative keywords.
**Key Fields:** `profile_id`, `keyword_id`, `campaign_id`, `keyword_text`, `match_type`, `state`
**Key Fields:** `profile_id`, `target_id`, `campaign_id`, `ad_group_id`, `state`, `expression` and `resolved_expression` (JSON strings)
Sponsored Brands campaigns.
**Key Fields:**
* `profile_id`, `campaign_id`, `name`, `state`
* `budget`, `budget_type` (DAILY/LIFETIME)
* `bidding` - Bidding config as a JSON string
* `start_date`, `end_date`, `portfolio_id`, `brand_entity_id`
* `goal`, `cost_type` (CPC/VCPM), `is_multi_ad_groups_enabled`, `product_location`
* `serving_status`, `extended_data`, `tags`
**Key Fields:** `profile_id`, `ad_group_id`, `campaign_id`, `name`, `state`, `serving_status`, `extended_data`
**Key Fields:** `profile_id`, `ad_id`, `campaign_id`, `ad_group_id`, `name`, `state`, `ad_type`, `creative` and `landing_page` (JSON strings), `serving_status`, `extended_data`
**Key Fields:** `profile_id`, `keyword_id`, `campaign_id`, `ad_group_id`, `keyword_text`, `match_type`, `state`, `bid`
**Key Fields:** `profile_id`, `keyword_id`, `campaign_id`, `ad_group_id`, `keyword_text`, `match_type`, `state`
**Key Fields:** `profile_id`, `target_id`, `campaign_id`, `ad_group_id`, `state`, `bid`, `expressions` (JSON string)
**Key Fields:** `profile_id`, `target_id`, `campaign_id`, `ad_group_id`, `state`, `expressions` (JSON string)
Sponsored Display campaigns.
**Key Fields:**
* `profile_id`, `campaign_id`, `name`, `state`
* `tactic` (T00020 contextual / T00030 audiences)
* `budget`, `budget_type`, `cost_type` (cpc/vcpm)
* `start_date`, `end_date`, `portfolio_id`, `delivery_profile`, `bid_optimization`
* `serving_status`, `creation_date`, `last_updated_date`
**Key Fields:** `profile_id`, `ad_group_id`, `campaign_id`, `name`, `state`, `default_bid`, `bid_optimization`, `serving_status`, `creation_date`, `last_updated_date`
**Key Fields:** `profile_id`, `ad_id`, `campaign_id`, `ad_group_id`, `state`, `asin`, `sku`, `serving_status`, `creation_date`, `last_updated_date`
**Key Fields:** `profile_id`, `target_id`, `campaign_id`, `ad_group_id`, `state`, `bid`, `expression_type` (auto/manual), `expression` (JSON string), `serving_status`, `creation_date`, `last_updated_date`
**Key Fields:** `profile_id`, `target_id`, `campaign_id`, `ad_group_id`, `state`, `expression` (JSON string), `serving_status`, `creation_date`, `last_updated_date`
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| Rate limit from the Amazon Advertising API | Too many requests in the same window. | Reduce concurrency by scheduling sources that share the account at different times, then re-run. |
| A report comes back with a cancelled status | Amazon cancelled the report generation on its side. | Re-run the pipeline. If the cancellation repeats for the same period, narrow the window. |
| Permission error | The authorizing user's role does not include access to the advertising account. | Grant the user access in Amazon Advertising and re-authorize. |
# Amazon Selling Partner as a data source
Source: https://docs.nekt.com/sources/amazon-sp
Bring data from Amazon Selling Partner to Nekt.
Amazon Selling Partner (SP-API) is Amazon's suite of REST-based APIs that gives sellers and vendors programmatic access to their Amazon marketplace data. The SP-API provides access to order reports, product listings, and sales & traffic analytics across all Amazon marketplaces.
## Configuring Amazon Selling Partner as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Amazon Selling Partner option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Amazon Selling Partner data. Click on the **Amazon Authorization** button and log in with your Amazon seller account. Grant the necessary permissions for the marketplace(s) you want to extract data from.
The following configurations are available:
* **Region**: Select the appropriate region for your marketplace:
* North America (US, Canada, Mexico, Brazil)
* Europe (UK, France, Germany, Italy, Spain, Netherlands, etc.)
* Far East (Japan, Australia, Singapore, India)
* **Marketplace IDs**: The Amazon marketplace IDs you want to sync data from (e.g., `ATVPDKIKX0DER` for Amazon US, `A2Q3Y263D00KWC` for Amazon Brazil).
Find the complete list of marketplace IDs on this link: [https://developer-docs.amazon.com/sp-api/docs/marketplace-ids](https://developer-docs.amazon.com/sp-api/docs/marketplace-ids)
| Region | Country | Marketplace Name | Marketplace ID |
| :- | :- | :- | :- |
| North America | United States | Amazon.com | ATVPDKIKX0DER |
| North America | Canada | Amazon.ca | A2EUQ1WTGCTBG2 |
| North America | Mexico | Amazon.com.mx | A1AM78C64UM0Y8 |
| North America | Brazil | Amazon.com.br | A2Q3Y263D00KWC |
| Europe | United Kingdom | Amazon.co.uk | A1F83G8C2ARO7P |
| Europe | Germany | Amazon.de | A1PA6795UKMFR9 |
| Europe | France | Amazon.fr | A13V1IB3VIYZZH |
| Europe | Italy | Amazon.it | APJ6JRA9NG5V4 |
| Europe | Spain | Amazon.es | A1RKKUPIHCS9HS |
| Europe | Netherlands | Amazon.nl | A1805IZSGTT6HS |
| Europe | Sweden | Amazon.se | A2NODRKZP88ZB9 |
| Europe | Poland | Amazon.pl | A1C3SOZRARQ6R3 |
| Europe | Belgium | Amazon.com.be | AMEN7PMS3EDWL |
| Far East | Japan | Amazon.co.jp | A1VC38T7YXB528 |
| Far East | Singapore | Amazon.sg | A19VAU5U5O7RUS |
| Far East | Australia | Amazon.com.au | A39IBJ37TRP1C6 |
| Far East | India | Amazon.in | A21TJRUUN4KGV |
| Middle East | United Arab Emirates | Amazon.ae | A2VIGQ35RCS4UG |
| Middle East | Turkey | Amazon.com.tr | A33AVAJ2PDY3EV |
| Middle East | Saudi Arabia | Amazon.sa | A17E79C6D8DWNP |
| Other | Egypt | Amazon.eg | ARBP9OOSHTCHU |
* **Start Date**: The earliest date from which records will be synced. If not provided, the connector defaults to the last 30 days of data.
Make sure your Amazon Seller account has the required permissions to request reports via the SP-API. The connector uses the Reports API to extract orders, listings, and sales & traffic data.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Amazon Selling Partner and their corresponding fields:
Complete order data including buyer information, item details, pricing, taxes, and shipping, fetched via the flat-file all-orders report (`GET_FLAT_FILE_ALL_ORDERS_DATA_BY_LAST_UPDATE_GENERAL`). Primary key is the composite `(amazonOrderId, sku)`.
| Field | Type | Description |
| :- | :- | :- |
| `amazonOrderId` | String | Amazon order identifier |
| `merchantOrderId` | String | Merchant order identifier |
| `purchaseDate` | DateTime | Date the order was placed |
| `lastUpdatedDate` | DateTime | Date the order was last updated (replication key) |
| `orderStatus` | String | Order status |
| `fulfillmentChannel` | String | Fulfillment channel (AFN for FBA, MFN for merchant-fulfilled) |
| `salesChannel` | String | Sales channel |
| `orderChannel` | String | Order channel |
| `shipServiceLevel` | String | Ship service level |
| `productName` | String | Product name |
| `sku` | String | Seller SKU |
| `asin` | String | Amazon Standard Identification Number |
| `itemStatus` | String | Item status |
| `quantity` | Integer | Quantity ordered |
| `currency` | String | Currency code |
| `itemPrice` | String | Item price |
| `itemTax` | String | Item tax |
| `shippingPrice` | String | Shipping price |
| `shippingTax` | String | Shipping tax |
| `giftWrapPrice` | String | Gift wrap price |
| `giftWrapTax` | String | Gift wrap tax |
| `itemPromotionDiscount` | String | Item promotion discount |
| `shipPromotionDiscount` | String | Shipping promotion discount |
| `shipCity` | String | Ship-to city |
| `shipState` | String | Ship-to state |
| `shipPostalCode` | String | Ship-to postal code |
| `shipCountry` | String | Ship-to country |
| `promotionIds` | String | Promotion IDs |
| `isBusinessOrder` | String | Whether this is a business order |
| `purchaseOrderNumber` | String | Purchase order number |
| `priceDesignation` | String | Price designation |
Current snapshot of all active merchant listings with product details, pricing, inventory, and fulfillment information.
| Field | Type | Description |
| :- | :- | :- |
| `itemName` | String | Listing title |
| `itemDescription` | String | Listing description |
| `listingId` | String | Listing identifier |
| `sellerSku` | String | Seller SKU (primary key) |
| `price` | String | Listing price |
| `quantity` | Integer | Available quantity |
| `openDate` | DateTime | Date the listing was opened |
| `imageUrl` | String | Main image URL |
| `itemIsMarketplace` | String | Whether the item is listed on the marketplace |
| `productIdType` | String | Product ID type (e.g. ASIN, UPC) |
| `zshopShippingFee` | String | zShop shipping fee |
| `itemNote` | String | Seller note on the item |
| `itemCondition` | String | Item condition |
| `asin1` | String | Primary ASIN |
| `asin2` | String | Secondary ASIN |
| `asin3` | String | Tertiary ASIN |
| `willShipInternationally` | String | Whether the item ships internationally |
| `expeditedShipping` | String | Expedited shipping option |
| `productId` | String | Product identifier |
| `addDelete` | String | Add or delete flag |
| `pendingQuantity` | Integer | Quantity pending fulfillment |
| `fulfillmentChannel` | String | Fulfillment channel (AFN for FBA, MFN for merchant-fulfilled) |
| `merchantShippingGroup` | String | Merchant shipping group |
| `status` | String | Listing status |
Daily sales and traffic metrics including units ordered, revenue, refunds, sessions, page views, and conversion data, with B2B breakdowns. Amazon retains approximately 2 years of data for this report (730-day lookback cap).
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | Date (YYYY-MM-DD, replication key) |
| `unitsOrdered` | Integer | Total units ordered |
| `unitsOrderedB2B` | Integer | B2B units ordered |
| `totalOrderItems` | Integer | Total order items |
| `totalOrderItemsB2B` | Integer | B2B total order items |
| `orderedProductSalesAmount` | Number | Ordered product sales amount |
| `orderedProductSalesCurrencyCode` | String | Ordered product sales currency |
| `orderedProductSalesB2BAmount` | Number | B2B ordered product sales amount |
| `orderedProductSalesB2BCurrencyCode` | String | B2B ordered product sales currency |
| `unitsRefunded` | Integer | Units refunded |
| `refundRate` | Number | Refund rate |
| `claimsGranted` | Integer | Claims granted |
| `claimsAmountAmount` | Number | Claims amount |
| `claimsAmountCurrencyCode` | String | Claims amount currency |
| `unitsShipped` | Integer | Units shipped |
| `ordersShipped` | Integer | Orders shipped |
| `sessions` | Integer | Total sessions |
| `browserSessions` | Integer | Browser sessions |
| `mobileAppSessions` | Integer | Mobile app sessions |
| `sessionPercentage` | Number | Session percentage |
| `pageViews` | Integer | Total page views |
| `browserPageViews` | Integer | Browser page views |
| `mobileAppPageViews` | Integer | Mobile app page views |
| `pageViewsPercentage` | Number | Page views percentage |
| `buyBoxPercentage` | Number | Buy box percentage |
| `unitSessionPercentage` | Number | Unit session percentage (conversion rate) |
| `unitSessionPercentageB2B` | Number | B2B unit session percentage |
| `averageOfferCount` | Integer | Average offer count |
| `averageParentItems` | Integer | Average parent items |
| `feedbackReceived` | Integer | Feedback received |
| `negativeFeedbackReceived` | Integer | Negative feedback received |
| `receivedNegativeFeedbackRate` | Number | Received negative feedback rate |
Financial events stream via the Amazon SP-API Finances API. Fetches fee breakdowns (including commissions) per order-item. Primary key is the composite `(amazonOrderId, sellerSku, orderItemId, feeType)`.
| Field | Type | Description |
| :- | :- | :- |
| `amazonOrderId` | String | Amazon order ID. |
| `sellerSku` | String | Seller SKU. |
| `orderItemId` | String | Order item ID. |
| `marketplaceId` | String | Marketplace ID. |
| `postedDate` | DateTime | Date the event was posted (replication key). |
| `feeType` | String | Fee type (e.g. Commission, FBAPerUnitFulfillmentFee). |
| `feeAmount` | Number | Fee amount (negative = deduction). |
| `feeCurrency` | String | Fee currency code. |
| `chargeType` | String | Charge type (e.g. Principal, ShippingCharge). |
| `chargeAmount` | Number | Charge amount. |
| `chargeCurrency` | String | Charge currency code. |
| `quantityShipped` | Integer | Quantity shipped. |
Consolidated view of FBA inventory including sales velocity, shipment statuses, estimated storage and long-term storage fees, and aged/excess units, fetched via the `GET_FBA_INVENTORY_PLANNING_DATA` report. Snapshot report (full table sync). Primary key is `sku`.
| Field | Type | Description |
| :- | :- | :- |
| `snapshotDate` | Date | Snapshot date. |
| `sku` | String | Seller SKU. |
| `fnsku` | String | Fulfillment network SKU. |
| `asin` | String | ASIN. |
| `productName` | String | Product name. |
| `condition` | String | Item condition. |
| `available` | Integer | Available (sellable) quantity. |
| `pendingRemovalQuantity` | Integer | Quantity pending removal. |
| `invAge0To90Days` | Integer | Units aged 0-90 days. |
| `invAge91To180Days` | Integer | Units aged 91-180 days. |
| `invAge181To270Days` | Integer | Units aged 181-270 days. |
| `invAge271To365Days` | Integer | Units aged 271-365 days. |
| `invAge365PlusDays` | Integer | Units aged 365+ days. |
| `currency` | String | Currency code. |
| `qtyToBeChargedLtsf6Mo` | Integer | Qty to be charged LTSF at 6 months. |
| `projectedLtsf6Mo` | Number | Projected LTSF charge at 6 months. |
| `qtyToBeChargedLtsf12Mo` | Integer | Qty to be charged LTSF at 12 months. |
| `estimatedLtsfNextCharge` | Number | Estimated LTSF next charge. |
| `unitsShippedT7` | Integer | Units shipped last 7 days. |
| `unitsShippedT30` | Integer | Units shipped last 30 days. |
| `unitsShippedT60` | Integer | Units shipped last 60 days. |
| `unitsShippedT90` | Integer | Units shipped last 90 days. |
| `alert` | String | Alert flag. |
| `yourPrice` | Number | Your price. |
| `salesPrice` | Number | Sales price. |
| `lowestPriceNewPlusShipping` | Number | Lowest new price plus shipping. |
| `lowestPriceUsed` | Number | Lowest used price. |
| `recommendedAction` | String | Recommended action. |
| `sellThrough` | Number | Sell-through rate. |
| `itemVolume` | Number | Item volume. |
| `volumeUnitMeasurement` | String | Volume unit of measurement. |
| `storageType` | String | Storage type (Standard/Oversize). |
| `storageVolume` | Number | Storage volume. |
| `marketplace` | String | Marketplace identifier. |
| `productGroup` | String | Product group/category. |
| `salesRank` | Integer | Sales rank. |
| `daysOfSupply` | Number | Days of supply. |
| `estimatedExcessQuantity` | Integer | Estimated excess quantity. |
| `weeksOfCoverT30` | Number | Weeks of cover based on 30-day sales. |
| `weeksOfCoverT90` | Number | Weeks of cover based on 90-day sales. |
| `featuredofferPrice` | Number | Featured offer (buy box) price. |
| `salesShippedLast7Days` | Number | Sales shipped last 7 days. |
| `salesShippedLast30Days` | Number | Sales shipped last 30 days. |
| `salesShippedLast60Days` | Number | Sales shipped last 60 days. |
| `salesShippedLast90Days` | Number | Sales shipped last 90 days. |
| `invAge0To30Days` | Integer | Units aged 0-30 days. |
| `invAge31To60Days` | Integer | Units aged 31-60 days. |
| `invAge61To90Days` | Integer | Units aged 61-90 days. |
| `invAge181To330Days` | Integer | Units aged 181-330 days. |
| `invAge331To365Days` | Integer | Units aged 331-365 days. |
| `estimatedStorageCostNextMonth` | Number | Estimated storage cost next month. |
| `inboundQuantity` | Integer | Total inbound quantity. |
| `inboundWorking` | Integer | Inbound working quantity. |
| `inboundShipped` | Integer | Inbound shipped quantity. |
| `inboundReceived` | Integer | Inbound received quantity. |
| `totalReservedQuantity` | Integer | Total reserved quantity. |
| `unfulfillableQuantity` | Integer | Unfulfillable quantity. |
| `weeksOfCover` | Number | Weeks of cover. |
| `weeksOfCoverEstimatedExcessQuantity` | Integer | Weeks of cover estimated excess quantity. |
| `recommendedOrderQuantity` | Integer | Recommended order quantity. |
| `recommendedOrderDate` | Date | Recommended order date. |
| `inventoryAgeSnapshotDate` | Date | Inventory age snapshot date. |
| `inventorySupplyAtFba` | Integer | Inventory supply at FBA. |
| `reservedFcTransfer` | Integer | Reserved for FC transfer. |
| `reservedFcProcessing` | Integer | Reserved for FC processing. |
| `reservedCustomerOrder` | Integer | Reserved for customer orders. |
| `totalDaysOfSupplyIncludingOpenShipments` | Number | Total days of supply including open shipments. |
Per-SKU inventory data including fulfillable, inbound, reserved, researching, and unfulfillable quantities, fetched via the FBA Inventory API. Primary key is the composite `(sellerSku, marketplaceId)`.
| Field | Type | Description |
| :- | :- | :- |
| `asin` | String | Amazon Standard Identification Number. |
| `fnSku` | String | Fulfillment network SKU. |
| `sellerSku` | String | Seller SKU. |
| `condition` | String | Item condition. |
| `productName` | String | Product title. |
| `totalQuantity` | Integer | Total quantity across all statuses. |
| `lastUpdatedTime` | DateTime | Last time quantity was updated (replication key). |
| `marketplaceId` | String | Marketplace identifier. |
| `fulfillableQuantity` | Integer | Units available for order fulfillment. |
| `inboundWorkingQuantity` | Integer | Units in inbound shipment being worked on. |
| `inboundShippedQuantity` | Integer | Units in inbound shipment that have been shipped. |
| `inboundReceivingQuantity` | Integer | Units in inbound shipment being received at FC. |
| `reservedTotalQuantity` | Integer | Total reserved quantity. |
| `reservedPendingCustomerOrderQuantity` | Integer | Reserved for customer orders. |
| `reservedPendingTransshipmentQuantity` | Integer | Reserved for FC transfers. |
| `reservedFcProcessingQuantity` | Integer | Reserved for FC processing. |
| `researchingTotalQuantity` | Integer | Total researching quantity. |
| `researchingQuantityInShortTerm` | Integer | Researching 1-10 days. |
| `researchingQuantityInMidTerm` | Integer | Researching 11-20 days. |
| `researchingQuantityInLongTerm` | Integer | Researching 21+ days. |
| `unfulfillableTotalQuantity` | Integer | Total unfulfillable quantity. |
| `customerDamagedQuantity` | Integer | Customer damaged units. |
| `warehouseDamagedQuantity` | Integer | Warehouse damaged units. |
| `distributorDamagedQuantity` | Integer | Distributor damaged units. |
| `carrierDamagedQuantity` | Integer | Carrier damaged units. |
| `defectiveQuantity` | Integer | Defective units. |
| `expiredQuantity` | Integer | Expired units. |
| `reservedFutureSupplyQuantity` | Integer | Reserved future supply quantity. |
| `futureSupplyBuyableQuantity` | Integer | Future supply buyable quantity. |
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Amazon Selling Partner data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Sales Performance Overview
Analyze daily sales performance including revenue, units sold, and refund trends.
**Business Value:**
* Track revenue trends over time
* Monitor refund rates to identify product quality issues
* Compare B2B vs. B2C sales performance
```sql theme={null}
SELECT
date,
unitsOrdered,
orderedProductSalesAmount,
orderedProductSalesCurrencyCode,
unitsRefunded,
refundRate,
unitsShipped,
ordersShipped,
CASE
WHEN unitsOrdered > 0 THEN ROUND(unitsRefunded * 100.0 / unitsOrdered, 2)
ELSE 0
END AS calculated_refund_pct
FROM
nekt_raw.amazon_sp_sales_and_traffic_report
WHERE
date >= CURRENT_DATE - INTERVAL '30' DAY
ORDER BY
date DESC
```
```sql theme={null}
SELECT
date,
unitsOrdered,
orderedProductSalesAmount,
orderedProductSalesCurrencyCode,
unitsRefunded,
refundRate,
unitsShipped,
ordersShipped,
CASE
WHEN unitsOrdered > 0 THEN ROUND(SAFE_DIVIDE(unitsRefunded * 100.0, unitsOrdered), 2)
ELSE 0
END AS calculated_refund_pct
FROM
`nekt_raw.amazon_sp_sales_and_traffic_report`
WHERE
date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
ORDER BY
date DESC
```
| date | unitsOrdered | orderedProductSalesAmount | orderedProductSalesCurrencyCode | unitsRefunded | refundRate | unitsShipped | ordersShipped | calculated\_refund\_pct |
| - | - | - | - | - | - | - | - | - |
| 2026-03-10 | 342 | 18,456.90 | USD | 8 | 2.34 | 328 | 298 | 2.34 |
| 2026-03-09 | 298 | 15,230.45 | USD | 5 | 1.68 | 290 | 265 | 1.68 |
| 2026-03-08 | 410 | 22,890.10 | USD | 12 | 2.93 | 395 | 350 | 2.93 |
| 2026-03-07 | 275 | 14,100.00 | USD | 3 | 1.09 | 270 | 245 | 1.09 |
### 2. Order Fulfillment Analysis
Track order distribution by status and fulfillment channel to monitor operational health.
**Business Value:**
* Monitor order processing efficiency across FBA and merchant-fulfilled channels
* Identify bottlenecks in fulfillment
* Track cancellation rates by channel
```sql theme={null}
WITH order_stats AS (
SELECT
fulfillmentChannel,
orderStatus,
COUNT(*) AS order_count,
SUM(CAST(itemPrice AS DOUBLE)) AS total_revenue
FROM
nekt_raw.amazon_sp_orders_report
WHERE
purchaseDate >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
fulfillmentChannel, orderStatus
),
total AS (
SELECT SUM(order_count) AS total_orders FROM order_stats
)
SELECT
os.fulfillmentChannel,
os.orderStatus,
os.order_count,
ROUND(os.order_count * 100.0 / t.total_orders, 2) AS percentage,
ROUND(os.total_revenue, 2) AS total_revenue
FROM
order_stats os, total t
ORDER BY
os.order_count DESC
```
```sql theme={null}
WITH order_stats AS (
SELECT
fulfillmentChannel,
orderStatus,
COUNT(*) AS order_count,
SUM(CAST(itemPrice AS FLOAT64)) AS total_revenue
FROM
`nekt_raw.amazon_sp_orders_report`
WHERE
purchaseDate >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
fulfillmentChannel, orderStatus
),
total AS (
SELECT SUM(order_count) AS total_orders FROM order_stats
)
SELECT
os.fulfillmentChannel,
os.orderStatus,
os.order_count,
ROUND(SAFE_DIVIDE(os.order_count * 100.0, t.total_orders), 2) AS percentage,
ROUND(os.total_revenue, 2) AS total_revenue
FROM
order_stats os, total t
ORDER BY
os.order_count DESC
```
| fulfillmentChannel | orderStatus | order\_count | percentage | total\_revenue |
| - | - | - | - | - |
| AFN | Shipped | 1,245 | 52.30 | 89,234.50 |
| MFN | Shipped | 520 | 21.84 | 31,245.00 |
| AFN | Pending | 312 | 13.11 | 22,523.00 |
| MFN | Cancelled | 156 | 6.55 | 9,856.00 |
| AFN | Cancelled | 148 | 6.22 | 8,432.00 |
### 3. Traffic-to-Sales Conversion
Measure how effectively your store traffic converts into orders using session and conversion data.
**Business Value:**
* Identify trends in traffic and conversion rates
* Compare browser vs. mobile app performance
* Optimize listing strategies based on Buy Box percentage
```sql theme={null}
SELECT
date,
sessions,
browserSessions,
mobileAppSessions,
pageViews,
unitsOrdered,
unitSessionPercentage AS conversion_rate,
buyBoxPercentage,
CASE
WHEN sessions > 0 THEN ROUND(pageViews * 1.0 / sessions, 2)
ELSE 0
END AS pages_per_session
FROM
nekt_raw.amazon_sp_sales_and_traffic_report
WHERE
date >= CURRENT_DATE - INTERVAL '30' DAY
ORDER BY
date DESC
```
```sql theme={null}
SELECT
date,
sessions,
browserSessions,
mobileAppSessions,
pageViews,
unitsOrdered,
unitSessionPercentage AS conversion_rate,
buyBoxPercentage,
CASE
WHEN sessions > 0 THEN ROUND(SAFE_DIVIDE(pageViews * 1.0, sessions), 2)
ELSE 0
END AS pages_per_session
FROM
`nekt_raw.amazon_sp_sales_and_traffic_report`
WHERE
date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
ORDER BY
date DESC
```
| date | sessions | browserSessions | mobileAppSessions | pageViews | unitsOrdered | conversion\_rate | buyBoxPercentage | pages\_per\_session |
| - | - | - | - | - | - | - | - | - |
| 2026-03-10 | 4,523 | 1,890 | 2,633 | 8,245 | 342 | 7.56 | 92.30 | 1.82 |
| 2026-03-09 | 3,980 | 1,650 | 2,330 | 7,120 | 298 | 7.49 | 91.80 | 1.79 |
| 2026-03-08 | 5,120 | 2,100 | 3,020 | 9,800 | 410 | 8.01 | 93.10 | 1.91 |
| 2026-03-07 | 3,560 | 1,480 | 2,080 | 6,450 | 275 | 7.72 | 90.50 | 1.81 |
## Implementation Notes
The connector primarily uses the Amazon SP-API Reports API rather than direct API endpoints for reports. Reports are created asynchronously: the connector submits a report request, polls until it's ready (up to 15 minutes per chunk: 60 attempts × 15 seconds), then downloads and parses the result. Downloaded documents are automatically decompressed when Amazon returns them in GZIP format.
If a report comes back with a `CANCELLED` status, the connector treats it as an empty result rather than a failure. This is expected behavior, as Amazon automatically cancels reports when there is no data available for the requested marketplace and date range.
This approach avoids the strict rate limits of the Orders API (0.0056 req/s) and enables efficient bulk data extraction.
*Note: The **Financial Events** and **FBA Inventory Summaries** streams differ by using the Finances API and FBA Inventory API respectively, not the Reports API.*
* **Orders, Merchant Listings, and FBA Manage Inventory Health** reports are returned as TSV files encoded in ISO-8859-1.
* **Sales and Traffic** reports are returned as JSON encoded in UTF-8.
The connector supports all Amazon SP-API regional endpoints:
* **North America**: US, Canada, Mexico, Brazil
* **Europe**: UK, France, Germany, Italy, Spain, Netherlands, Turkey, UAE, India, etc.
* **Far East**: Japan, Australia, Singapore
Make sure to select the correct API URL and marketplace IDs for your seller account's region.
The connector supports incremental sync for:
* **Orders Report**: Uses `lastUpdatedDate` as the replication key; data is fetched in 30-day chunks.
* **Sales and Traffic Report**: Uses `date` as the replication key; data is fetched in 30-day chunks. Amazon retains approximately 2 years of historical data, so the connector clamps the start date to a maximum of 730 days in the past regardless of the configured `Start Date`.
* **Financial Events**: Uses `postedDate` as the replication key; data is fetched in 30-day chunks with a maximum lookback of 730 days.
* **FBA Inventory Summaries**: Uses `lastUpdatedTime` as the replication key; data is fetched in 30-day chunks with a maximum lookback of 540 days.
The **Merchant Listings Report** and **FBA Manage Inventory Health** are always a full-table sync since they represent snapshots of your current active listings and inventory.
The connector automatically manages OAuth access token refresh to prevent authentication failures:
* Access tokens are refreshed proactively 15 minutes before expiration to avoid race conditions during long-running extractions.
The connector handles transient SP-API errors automatically:
* **429 QuotaExceeded**: Retried up to 5 times. The `Retry-After` header is respected when present; otherwise exponential backoff is used (starting at 60 seconds, doubling each attempt).
* **Stale connections**: If the server closes an idle connection during a long polling sleep, the connector reopens the session and retries transparently.
For time-based reports (Orders, Sales & Traffic, Financial Events, and FBA Inventory Summaries), the connector splits requests into 30-day chunks to stay within SP-API limits and avoid timeouts on large date ranges. The chunking is handled automatically; you only need to set the desired `Start Date`.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `429 QuotaExceeded` | You hit the Amazon API rate limit. | Reduce concurrency and re-run. Extraction resumes from where it stopped. |
| A report comes back with a cancelled status | Amazon cancelled the report generation on its side. | Re-run the pipeline. If the cancellation repeats for the same period, narrow the window. |
| Permission error on the SP-API | The user's role on the account does not include the required scopes. | Grant the corresponding roles in Seller Central and re-authorize. |
## Skills for agents
Amazon Selling Partner connector documentation as plain markdown, for use in AI agent contexts.
# Amazon Vendor as a data source
Source: https://docs.nekt.com/sources/amazon-vendor
Bring data from Amazon Vendor Central to Nekt.
Amazon Vendor Central is the platform used by first-party vendors, the brands that sell their products *to* Amazon, which then resells them. Its retail analytics reports are exposed through the Amazon Selling Partner API (SP-API) and cover sales, traffic, inventory health, product margin, and Amazon's own demand forecast, all broken down by ASIN.
Vendor Central and Seller Central are different programs. If you sell directly to customers on the Amazon marketplace, you are a seller: use the [Amazon Selling Partner](/sources/amazon-sp) source instead. A vendor account has no seller orders, merchant listings, or FBA inventory, and a seller account has none of the vendor reports below.
## Configuring Amazon Vendor as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Amazon Vendor option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Amazon Vendor Central data. Click on the **Amazon Authorization** button and log in with your **Vendor Central** account. The authorization grants access to the vendor group tied to those credentials, and therefore to every vendor code in that group.
The authorization must be granted from Vendor Central, not from Seller Central. A Seller Central authorization is accepted by Amazon but returns no vendor data, and every stream will fail as unauthorized.
The following configurations are available:
* **Region**: Select the region where your vendor account is registered:
* North America (US, Canada, Mexico, Brazil)
* Europe (UK, France, Germany, Italy, Spain, Netherlands, etc.)
* Far East (Japan, Australia, Singapore)
* **Marketplace ID**: The Amazon marketplace your vendor account belongs to (e.g., `ATVPDKIKX0DER` for Amazon US, `A2Q3Y263D00KWC` for Amazon Brazil). A vendor account belongs to a single marketplace.
Find the complete list of marketplace IDs on this link: [https://developer-docs.amazon.com/sp-api/docs/marketplace-ids](https://developer-docs.amazon.com/sp-api/docs/marketplace-ids)
| Region | Country | Marketplace Name | Marketplace ID |
| :- | :- | :- | :- |
| North America | United States | Amazon.com | ATVPDKIKX0DER |
| North America | Canada | Amazon.ca | A2EUQ1WTGCTBG2 |
| North America | Mexico | Amazon.com.mx | A1AM78C64UM0Y8 |
| North America | Brazil | Amazon.com.br | A2Q3Y263D00KWC |
| Europe | United Kingdom | Amazon.co.uk | A1F83G8C2ARO7P |
| Europe | Germany | Amazon.de | A1PA6795UKMFR9 |
| Europe | France | Amazon.fr | A13V1IB3VIYZZH |
| Europe | Italy | Amazon.it | APJ6JRA9NG5V4 |
| Europe | Spain | Amazon.es | A1RKKUPIHCS9HS |
| Europe | Netherlands | Amazon.nl | A1805IZSGTT6HS |
| Europe | Sweden | Amazon.se | A2NODRKZP88ZB9 |
| Europe | Poland | Amazon.pl | A1C3SOZRARQ6R3 |
| Europe | Belgium | Amazon.com.be | AMEN7PMS3EDWL |
| Far East | Japan | Amazon.co.jp | A1VC38T7YXB528 |
| Far East | Singapore | Amazon.sg | A19VAU5U5O7RUS |
| Far East | Australia | Amazon.com.au | A39IBJ37TRP1C6 |
| Far East | India | Amazon.in | A21TJRUUN4KGV |
| Middle East | United Arab Emirates | Amazon.ae | A2VIGQ35RCS4UG |
| Middle East | Turkey | Amazon.com.tr | A33AVAJ2PDY3EV |
| Middle East | Saudi Arabia | Amazon.sa | A17E79C6D8DWNP |
| Other | Egypt | Amazon.eg | ARBP9OOSHTCHU |
* **Distributor view**: Which side of the business the sales and inventory numbers describe.
* **Manufacturing**: your view as the manufacturer, including what Amazon ordered from you. This is the usual choice.
* **Sourcing**: Amazon's view of what it sourced from you. Ordered revenue and ordered units are not reported in this view.
* **Selling program**: The Amazon program your vendor account participates in: **Retail**, **Business**, or **Fresh**. Amazon publishes the inventory and forecasting reports for Retail and Fresh only, so those two streams fall back to Retail when Business is selected. The run log records it when that happens.
* **Start Date**: The earliest date from which records will be synced. Amazon keeps roughly 4 years of daily vendor data, so earlier dates are automatically adjusted to the oldest date Amazon still serves.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Amazon publishes vendor metrics with up to 72 hours of delay, so a daily trigger is usually enough.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
All streams carry one record per ASIN per period. Amazon also returns account-wide totals in each report; those are not extracted, since they are the sum of the per-ASIN rows and can be recomputed with a `GROUP BY`.
Daily sales performance per ASIN, from the `GET_VENDOR_SALES_REPORT` report. Primary key is the composite `(asin, startDate)`.
Which metrics Amazon fills in depends on your configuration: ordered revenue and ordered units are only reported for the Manufacturing distributor view, and customer returns only for the Retail and Business selling programs.
| Field | Type | Description |
| :- | :- | :- |
| `startDate` | Date | First day of the reported period (replication key) |
| `endDate` | Date | Last day of the reported period |
| `asin` | String | Amazon Standard Identification Number |
| `customerReturns` | Integer | Quantity of units returned by customers |
| `orderedRevenueAmount` | Number | Ordered product sales in the period, adjusted for returns |
| `orderedRevenueCurrencyCode` | String | Currency of the ordered revenue |
| `orderedUnits` | Integer | Number of units ordered by Amazon customers |
| `shippedCogsAmount` | Number | Revenue based on the cost of goods sold, adjusted for returns |
| `shippedCogsCurrencyCode` | String | Currency of the shipped cost of goods sold |
| `shippedRevenueAmount` | Number | Revenue based on units shipped in the period, adjusted for returns |
| `shippedRevenueCurrencyCode` | String | Currency of the shipped revenue |
| `shippedUnits` | Integer | Number of units shipped to Amazon customers |
Daily detail page traffic per ASIN, from the `GET_VENDOR_TRAFFIC_REPORT` report. Combined with the Sales Report, this is what makes conversion rate computable. Primary key is the composite `(asin, startDate)`.
| Field | Type | Description |
| :- | :- | :- |
| `startDate` | Date | First day of the reported period (replication key) |
| `endDate` | Date | Last day of the reported period |
| `asin` | String | Amazon Standard Identification Number |
| `glanceViews` | Integer | Number of customer views of the product detail page |
Daily inventory health per ASIN, from the `GET_VENDOR_INVENTORY_REPORT` report: what Amazon received from you, what it still holds, and how healthy that stock is. Primary key is the composite `(asin, startDate)`.
| Field | Type | Description |
| :- | :- | :- |
| `startDate` | Date | First day of the reported period (replication key) |
| `endDate` | Date | Last day of the reported period |
| `asin` | String | Amazon Standard Identification Number |
| `sourceableProductOutOfStockRate` | Number | Out of stock rate across the products Amazon can source, as a share of detail page views |
| `procurableProductOutOfStockRate` | Number | Out of stock rate across procurable products, as a share of detail page views |
| `receiveFillRate` | Number | Purchase order units received by Amazon divided by the units confirmed by the vendor |
| `uft` | Number | Percentage of time the product is available to be shipped from a fulfillment center, versus the selection Amazon intended to carry |
| `vendorConfirmationRate` | Number | Units confirmed by the vendor divided by the units Amazon asked for |
| `netReceivedInventoryCostAmount` | Number | Cost of inventory received, minus the cost of inventory returned to the vendor |
| `netReceivedInventoryCostCurrencyCode` | String | Currency of the net received inventory cost |
| `netReceivedInventoryUnits` | Integer | Units received minus units returned to the vendor |
| `openPurchaseOrderUnits` | Integer | Units in confirmed purchase orders that Amazon has not received yet |
| `averageVendorLeadTimeDays` | Number | Average days between purchase order submission and receipt at a fulfillment center |
| `sellThroughRate` | Number | Units shipped (less returns) over units held at the start of the period plus units received |
| `unfilledCustomerOrderedUnits` | Integer | Units ordered or pre-ordered by customers that have not shipped |
| `sellableOnHandInventoryCostAmount` | Number | Cost of the sellable inventory on the last day of the period |
| `sellableOnHandInventoryCostCurrencyCode` | String | Currency of the sellable on-hand inventory cost |
| `sellableOnHandInventoryUnits` | Integer | Sellable units on the last day of the period |
| `unsellableOnHandInventoryCostAmount` | Number | Cost of the unsellable inventory on the last day of the period |
| `unsellableOnHandInventoryCostCurrencyCode` | String | Currency of the unsellable on-hand inventory cost |
| `unsellableOnHandInventoryUnits` | Integer | Unsellable units on the last day of the period |
| `aged90PlusDaysSellableInventoryCostAmount` | Number | Cost of sellable inventory at least 90 days old |
| `aged90PlusDaysSellableInventoryCostCurrencyCode` | String | Currency of the aged 90+ days sellable inventory cost |
| `aged90PlusDaysSellableInventoryUnits` | Integer | Sellable units at least 90 days old |
| `unhealthyInventoryCostAmount` | Number | Cost of the excess inventory, based on forecasted demand |
| `unhealthyInventoryCostCurrencyCode` | String | Currency of the unhealthy inventory cost |
| `unhealthyInventoryUnits` | Integer | Excess inventory units, based on forecasted demand |
Daily net pure product margin per ASIN, from the `GET_VENDOR_NET_PURE_PRODUCT_MARGIN_REPORT` report. Primary key is the composite `(asin, startDate)`.
| Field | Type | Description |
| :- | :- | :- |
| `startDate` | Date | First day of the reported period (replication key) |
| `endDate` | Date | Last day of the reported period |
| `asin` | String | Amazon Standard Identification Number |
| `netPureProductMargin` | Number | Amazon's profitability on the product: (product revenue - product cost of goods sold - vendor funding) / product revenue |
Amazon's demand forecast per ASIN, from the `GET_VENDOR_FORECASTING_REPORT` report. Unlike the other reports this one looks forward, so each run pulls the current snapshot, covering several future periods per ASIN. Snapshots are kept side by side (keyed by `forecastGenerationDate`), so you can compare what Amazon predicted against what actually happened. Primary key is the composite `(asin, forecastGenerationDate, startDate)`.
| Field | Type | Description |
| :- | :- | :- |
| `forecastGenerationDate` | Date | Date on which Amazon generated this forecast |
| `asin` | String | Amazon Standard Identification Number |
| `startDate` | Date | First day of the forecasted period |
| `endDate` | Date | Last day of the forecasted period |
| `meanForecastUnits` | Number | Average number of units Amazon predicts will be ordered |
| `p70ForecastUnits` | Number | Units forecast with 70% probability of being ordered |
| `p80ForecastUnits` | Number | Units forecast with 80% probability of being ordered |
| `p90ForecastUnits` | Number | Units forecast with 90% probability of being ordered |
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Amazon Vendor data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Shipped Revenue and Margin by ASIN
Rank your catalog by what Amazon actually shipped, and check the margin Amazon makes on each product.
**Business Value:**
* Identify the products driving revenue
* Spot ASINs where Amazon's margin is thin, which is what buyers cite in cost negotiations
* Track customer returns against shipped units
```sql theme={null}
SELECT
s.asin,
SUM(s.shippedUnits) AS shipped_units,
ROUND(SUM(s.shippedRevenueAmount), 2) AS shipped_revenue,
ROUND(SUM(s.shippedCogsAmount), 2) AS shipped_cogs,
SUM(s.customerReturns) AS customer_returns,
ROUND(AVG(m.netPureProductMargin), 4) AS avg_net_pure_product_margin
FROM
nekt_raw.amazon_vendor_sales_report s
LEFT JOIN nekt_raw.amazon_vendor_net_pure_product_margin_report m
ON m.asin = s.asin AND m.startDate = s.startDate
WHERE
s.startDate >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
s.asin
ORDER BY
shipped_revenue DESC
```
```sql theme={null}
SELECT
s.asin,
SUM(s.shippedUnits) AS shipped_units,
ROUND(SUM(s.shippedRevenueAmount), 2) AS shipped_revenue,
ROUND(SUM(s.shippedCogsAmount), 2) AS shipped_cogs,
SUM(s.customerReturns) AS customer_returns,
ROUND(AVG(m.netPureProductMargin), 4) AS avg_net_pure_product_margin
FROM
`nekt_raw.amazon_vendor_sales_report` s
LEFT JOIN `nekt_raw.amazon_vendor_net_pure_product_margin_report` m
ON m.asin = s.asin AND m.startDate = s.startDate
WHERE
s.startDate >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
s.asin
ORDER BY
shipped_revenue DESC
```
| asin | shipped\_units | shipped\_revenue | shipped\_cogs | customer\_returns | avg\_net\_pure\_product\_margin |
| - | - | - | - | - | - |
| B123456789 | 4,210 | 189,450.00 | 121,300.00 | 63 | 0.3120 |
| B987654321 | 2,980 | 96,120.50 | 70,880.00 | 41 | 0.2140 |
| B456789012 | 1,455 | 54,300.00 | 33,900.00 | 12 | 0.3760 |
### 2. Detail Page Conversion by ASIN
Combine traffic and sales to see which product pages convert the views they get.
**Business Value:**
* Find high-traffic, low-conversion pages where content or pricing is the bottleneck
* Prioritize A+ content and imagery work by potential impact
* Measure the effect of a listing change on conversion
```sql theme={null}
SELECT
t.asin,
SUM(t.glanceViews) AS glance_views,
SUM(s.orderedUnits) AS ordered_units,
CASE
WHEN SUM(t.glanceViews) > 0
THEN ROUND(SUM(s.orderedUnits) * 100.0 / SUM(t.glanceViews), 2)
ELSE 0
END AS conversion_rate_pct
FROM
nekt_raw.amazon_vendor_traffic_report t
LEFT JOIN nekt_raw.amazon_vendor_sales_report s
ON s.asin = t.asin AND s.startDate = t.startDate
WHERE
t.startDate >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
t.asin
HAVING
SUM(t.glanceViews) > 100
ORDER BY
glance_views DESC
```
```sql theme={null}
SELECT
t.asin,
SUM(t.glanceViews) AS glance_views,
SUM(s.orderedUnits) AS ordered_units,
ROUND(SAFE_DIVIDE(SUM(s.orderedUnits) * 100.0, SUM(t.glanceViews)), 2) AS conversion_rate_pct
FROM
`nekt_raw.amazon_vendor_traffic_report` t
LEFT JOIN `nekt_raw.amazon_vendor_sales_report` s
ON s.asin = t.asin AND s.startDate = t.startDate
WHERE
t.startDate >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
t.asin
HAVING
SUM(t.glanceViews) > 100
ORDER BY
glance_views DESC
```
| asin | glance\_views | ordered\_units | conversion\_rate\_pct |
| - | - | - | - |
| B123456789 | 84,320 | 4,560 | 5.41 |
| B987654321 | 51,780 | 3,020 | 5.83 |
| B456789012 | 22,140 | 640 | 2.89 |
### 3. Stock Risk Against Amazon's Forecast
Compare the stock Amazon holds today with what it forecasts it will sell, to anticipate stockouts and excess.
**Business Value:**
* Flag ASINs whose on-hand cover is below Amazon's own demand forecast
* Catch aging and unhealthy inventory before it turns into chargebacks
* Bring an objective number to purchase order negotiations
```sql theme={null}
WITH latest_inventory AS (
SELECT
asin,
sellableOnHandInventoryUnits,
openPurchaseOrderUnits,
unhealthyInventoryUnits,
aged90PlusDaysSellableInventoryUnits,
ROW_NUMBER() OVER (PARTITION BY asin ORDER BY startDate DESC) AS rn
FROM
nekt_raw.amazon_vendor_inventory_report
),
next_4_weeks AS (
SELECT
asin,
SUM(meanForecastUnits) AS forecast_units
FROM
nekt_raw.amazon_vendor_forecasting_report
WHERE
forecastGenerationDate = (SELECT MAX(forecastGenerationDate) FROM nekt_raw.amazon_vendor_forecasting_report)
AND startDate <= CURRENT_DATE + INTERVAL '28' DAY
GROUP BY
asin
)
SELECT
i.asin,
i.sellableOnHandInventoryUnits,
i.openPurchaseOrderUnits,
f.forecast_units,
i.sellableOnHandInventoryUnits + i.openPurchaseOrderUnits - f.forecast_units AS projected_gap,
i.aged90PlusDaysSellableInventoryUnits,
i.unhealthyInventoryUnits
FROM
latest_inventory i
JOIN next_4_weeks f ON f.asin = i.asin
WHERE
i.rn = 1
ORDER BY
projected_gap ASC
```
```sql theme={null}
WITH latest_inventory AS (
SELECT
asin,
sellableOnHandInventoryUnits,
openPurchaseOrderUnits,
unhealthyInventoryUnits,
aged90PlusDaysSellableInventoryUnits,
ROW_NUMBER() OVER (PARTITION BY asin ORDER BY startDate DESC) AS rn
FROM
`nekt_raw.amazon_vendor_inventory_report`
),
next_4_weeks AS (
SELECT
asin,
SUM(meanForecastUnits) AS forecast_units
FROM
`nekt_raw.amazon_vendor_forecasting_report`
WHERE
forecastGenerationDate = (SELECT MAX(forecastGenerationDate) FROM `nekt_raw.amazon_vendor_forecasting_report`)
AND startDate <= DATE_ADD(CURRENT_DATE(), INTERVAL 28 DAY)
GROUP BY
asin
)
SELECT
i.asin,
i.sellableOnHandInventoryUnits,
i.openPurchaseOrderUnits,
f.forecast_units,
i.sellableOnHandInventoryUnits + i.openPurchaseOrderUnits - f.forecast_units AS projected_gap,
i.aged90PlusDaysSellableInventoryUnits,
i.unhealthyInventoryUnits
FROM
latest_inventory i
JOIN next_4_weeks f ON f.asin = i.asin
WHERE
i.rn = 1
ORDER BY
projected_gap ASC
```
| asin | sellableOnHandInventoryUnits | openPurchaseOrderUnits | forecast\_units | projected\_gap | aged90PlusDaysSellableInventoryUnits | unhealthyInventoryUnits |
| - | - | - | - | - | - | - |
| B456789012 | 120 | 300 | 980 | -560 | 0 | 0 |
| B987654321 | 1,540 | 0 | 1,700 | -160 | 210 | 0 |
| B123456789 | 4,300 | 1,200 | 2,900 | 2,600 | 890 | 640 |
## Implementation Notes
Vendor Central and Seller Central data both live on the Amazon Selling Partner API, but on disjoint endpoints. A vendor sells to Amazon, so it has no seller orders, no merchant listings, and no FBA inventory; those are covered by the [Amazon Selling Partner](/sources/amazon-sp) source. The authorization is separate as well: this connector requires consent granted from Vendor Central, and a seller refresh token will be rejected by every stream.
All streams use the SP-API Reports API. Reports are generated asynchronously: the connector submits a report request, polls until it's ready (up to 45 minutes per window), then downloads and parses the JSON document, decompressing it when Amazon returns GZIP.
Generation time varies with how recent the window is. A report covering an older period is usually ready in about 30 seconds, while one ending a few days ago can take more than 15 minutes. This is Amazon's processing time, not the connector waiting unnecessarily.
Amazon caps how much a single daily report may span at 15 days, so each sync walks the requested range in 14-day windows. Requesting a long history therefore creates many reports, and the first sync of several years of data takes a while.
Amazon does not produce every vendor report for every account, typically because the account is not enrolled in the program a given report covers. When that happens, the affected stream is skipped with a warning in the run log explaining what Amazon reported, and every other stream is extracted normally. One unavailable report never blocks the rest of your tables.
A stream that fails partway through stops there rather than skipping ahead to a later period, so its table can never end up with a silent gap in the middle. The skipped days are retried on the next run.
The one error that stops the whole run is a rejected authorization, since no stream can work without it. In that case, reconnect the source from a Vendor Central account.
The Sales, Traffic, Inventory, and Net Pure Product Margin streams use `startDate` as the replication key.
Amazon revises recent vendor metrics after the fact, as returns are processed and costs adjusted. To keep already-loaded rows in sync instead of freezing them at their first value, every incremental run rewinds its bookmark by 30 days and re-reads that window. Re-extracted rows carry the same primary key, so they update in place rather than duplicating.
The Forecasting Report is a forward-looking snapshot with no date range, so it is always a full extraction. Each run's snapshot is preserved through the `forecastGenerationDate` in its primary key.
* **Publication lag**: Amazon takes up to 72 hours to publish a day's metrics, so extraction windows stop three days short of today. When Amazon is running later than that, it refuses the entire report rather than returning the days it does have; the stream then stops at the edge of published data and the remaining days arrive on the next run. Either way, recent days are filled in through the 30-day rewind above.
* **Retention**: roughly 4 years (1460 days) of daily data. A `Start Date` earlier than that is automatically clamped, and the run log states the adjusted date.
* **Selling program**: the Inventory and Forecasting reports exist for Retail and Fresh only. When the source is configured as Business, those two streams are extracted for Retail and the run log records the fallback.
Amazon refills the report creation quota roughly once per minute, so long backfills are throttled by design. The connector handles this automatically:
* **429 QuotaExceeded**: retried up to 5 times. The `Retry-After` header is respected when present, otherwise exponential backoff is used, starting at 60 seconds and doubling each attempt. Waits are reported in the run log, which is why a large first sync may look slow.
* **Stale connections**: if the server closes an idle connection during a long polling sleep, the connector reopens the session and retries transparently.
* **Access tokens** are refreshed 15 minutes before expiration to avoid failures in the middle of a long extraction.
## Skills for agents
Amazon Vendor connector documentation as plain markdown, for use in AI agent contexts.
# Amplitude as a data source
Source: https://docs.nekt.com/sources/amplitude
Bring data from Amplitude analytics charts and raw events to Nekt.
Amplitude is a product analytics platform that helps teams understand user behavior, measure product engagement, and drive growth through data-driven insights. It provides powerful analytics, experimentation, and personalization capabilities.
![]()
## Configuring Amplitude as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Amplitude option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Amplitude API credentials for this connection. Once you have them, add the account access, then click **Next**.
* **API Key**: Your Amplitude API key for authentication. You can find it in your Amplitude workspace under **Settings** > **Organization settings** > **API Keys**.
* **Secret Key**: Your project-specific secret key. Find it under **Settings** > **Projects** > **\[Your Project]** > **General** > **Secret key**.
* **Chart IDs** *(optional)*: The IDs of the Amplitude charts you want to extract data from.
* **Start Date**: The start date for raw event export, in `YYYY-MM-DD` format. If the events stream is selected and this is not configured, it will default to 30 days ago.
* **Event Export Window (Days)**: Number of days per API request when exporting raw events. Lower values reduce memory usage for high-volume projects. Defaults to 365 (the API maximum).
**API Key:**
1. Log in to your Amplitude workspace
2. Go to **Settings** (gear icon in the top-right)
3. Navigate to **Organization settings** > **API Keys**
4. Copy your API key from this page
**Secret Key:**
1. In Amplitude, go to **Settings**
2. Navigate to **Projects** in the left sidebar
3. Select your project
4. Go to the **General** tab
5. Scroll down to find the **Secret key** field
6. Copy the secret key
Keep your API key and secret key secure. These credentials provide access to your Amplitude data.
To find the ID of an Amplitude chart:
1. Open your Amplitude workspace and navigate to the chart you want to sync
2. Look at the URL in your browser's address bar
3. The chart ID is the alphanumeric string in the URL after `/chart/` or in the `id` parameter
**Example URLs:**
```
https://analytics.amplitude.com/org-name/chart/abc123def
```
The chart ID is: `abc123def`
```
https://analytics.amplitude.com/org-name/workspace/project-name/analyze/chart?id=xyz789
```
The chart ID is: `xyz789`
**Tips:**
* You can extract data from multiple charts by providing multiple Chart IDs
* Each chart will create a separate stream in your data warehouse
* Make sure you have access to the charts you want to sync
Once you're done, click **Next**.
### 2. Select streams
The Amplitude connector provides two types of streams:
* **Chart streams**: Dynamically created based on your configured Chart IDs. Each chart creates a separate stream named `chart_{chart_id}`, plus a `charts_metadata` stream.
* **Events stream**: Raw event data exported from Amplitude, containing every tracked user event with full detail.
You can use chart streams, the events stream, or both. Chart IDs are only required if you want to extract chart data.
Choose which streams you want to sync and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**:
* For **chart streams**: **FULL\_TABLE** sync is recommended since chart data represents aggregated metrics that should reflect the current state.
* For the **events stream**: **INCREMENTAL** sync is recommended to efficiently fetch only new events since the last extraction.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Common configurations for Amplitude:
* **Daily**: Most common, captures daily updates to your analytics
* **Every 12 hours**: For more frequent monitoring of key metrics
* **Weekly**: For less frequently changing aggregated reports
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
The Amplitude Export API has a data delay of approximately 2 hours. Events will only be available for extraction after Amplitude finishes processing them.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
The Amplitude connector provides chart data streams, a charts metadata stream, and a raw events stream.
Each Chart ID you configure creates a separate stream containing the chart's data in an unpivoted format. This makes it easy to analyze and aggregate metrics across different dimensions and segments.
**Stream Naming:** `chart_{chart_id}` where `{chart_id}` is your Amplitude chart identifier.
| Field | Type | Description |
| :- | :- | :- |
| `chart_id` | String | The Amplitude chart ID. |
| `dimension_name` | String | The name of the dimension being measured (e.g., "Country", "Device Type", "Event Name"). |
| `dimension_value` | String | The value for this dimension (e.g., "United States", "iOS", "Page View"). |
| `segment` | String | The segment or cohort this data represents (e.g., "All Users", "New Users", date labels for time series). |
| `value` | Number | The numeric metric value for this dimension/segment combination. |
**Unpivoted Data Structure:** Chart data is unpivoted, meaning each row represents a single data point for a specific dimension value and segment combination. This structure makes it flexible for analysis across different slices of your data.
**Example data:**
| chart\_id | dimension\_name | dimension\_value | segment | value |
| :- | :- | :- | :- | -: |
| abc123 | Country | United States | 2024-01-15 | 15420 |
| abc123 | Country | United States | 2024-01-16 | 16234 |
| abc123 | Country | Canada | 2024-01-15 | 3421 |
| abc123 | Country | Canada | 2024-01-16 | 3567 |
A single stream containing metadata about all your configured charts. This stream provides context about what each chart measures.
| Field | Type | Description |
| :- | :- | :- |
| `chart_id` | String | The Amplitude chart ID. |
| `title` | String | The title/name of the chart as shown in Amplitude. |
| `metric` | String | The metric being measured (e.g., "Uniques", "Event Totals", "Average"). |
| `event` | String | The event being tracked (if applicable). |
| `dimension` | String | The dimension the chart is grouped by. |
| `filters` | String | Any filters applied to the chart. |
This metadata stream helps you understand what each chart represents and can be joined with chart data streams using the `chart_id` field.
Raw event data exported from Amplitude via the [Export API](https://amplitude.com/docs/apis/analytics/export). Each record represents a single tracked event with full user, device, and session context.
This stream supports **incremental sync** using the `event_time` field as the bookmark, so subsequent extractions only fetch new events.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier for the event. |
| `event_type` | String | The name of the event (e.g., "Page Viewed", "Button Clicked"). |
| `event_time` | DateTime | The time the event occurred (UTC). Used as the replication key for incremental sync. |
| `server_received_time` | DateTime | When Amplitude's servers received the event. |
| `processed_time` | DateTime | When Amplitude finished processing the event. |
| `client_upload_time` | DateTime | When the client uploaded the event. |
| `server_upload_time` | DateTime | When the server uploaded the event. |
| `client_event_time` | DateTime | The event time as reported by the client. |
| `user_id` | String | The user ID associated with the event. |
| `device_id` | String | The device ID associated with the event. |
| `amplitude_id` | Integer | Amplitude's internal user identifier. |
| `session_id` | Integer | The session ID for the event. |
| `event_id` | Integer | A counter for events within a session. |
| `app` | Integer | The Amplitude app/project ID. |
| `platform` | String | The platform (e.g., "Web", "iOS", "Android"). |
| `os_name` | String | The operating system or browser name. |
| `os_version` | String | The operating system or browser version. |
| `device_type` | String | The type of device (e.g., "Mac", "Windows", "iPhone"). |
| `device_family` | String | The device family (e.g., "Mac OS X", "Windows"). |
| `device_carrier` | String | The mobile carrier. |
| `library` | String | The Amplitude SDK library and version. |
| `version_name` | String | The app version name. |
| `start_version` | String | The app version when the user was first seen. |
| `language` | String | The user's language setting. |
| `ip_address` | String | The user's IP address. |
| `city` | String | The user's city. |
| `country` | String | The user's country. |
| `region` | String | The user's region/state. |
| `dma` | String | The designated market area. |
| `location_lat` | Number | Latitude of the user's location. |
| `location_lng` | Number | Longitude of the user's location. |
| `paying` | Boolean | Whether the user is a paying user. |
| `sample_rate` | Number | The sample rate applied to the event. |
| `amplitude_attribution_ids` | String | Amplitude attribution IDs. |
| `amplitude_event_type` | String | Amplitude's internal event type classification. |
| `insert_id` | String | A unique insert identifier for deduplication. |
| `insert_key` | String | A unique insert key. |
| `schema` | Integer | The schema version of the event. |
| `device_brand` | String | The device brand. |
| `device_manufacturer` | String | The device manufacturer. |
| `device_model` | String | The device model. |
| `adid` | String | The Android advertising ID. |
| `idfa` | String | The iOS identifier for advertisers. |
| `source_id` | String | The source ID of the event. |
| `partner_id` | String | The partner ID for attribution. |
| `data_type` | String | The type of data record (e.g., "event"). |
| `is_attribution_event` | Boolean | Whether this is an attribution event. |
| `user_creation_time` | DateTime | When the user was first seen. |
| `plan` | String (JSON) | Tracking plan information as a JSON string. |
| `global_user_properties` | String (JSON) | Global user properties as a JSON string. |
| `event_properties` | String (JSON) | Event-specific properties as a JSON string. |
| `user_properties` | String (JSON) | User properties at the time of the event as a JSON string. |
| `group_properties` | String (JSON) | Group properties as a JSON string. |
| `groups` | String (JSON) | Group membership as a JSON string. |
| `data` | String (JSON) | Additional event metadata as a JSON string. |
**JSON String Fields:** Fields like `event_properties`, `user_properties`, and `group_properties` are stored as JSON strings because their structure varies per event. You can parse them in your transformations to extract specific properties.
# Data Model
The Amplitude data model consists of unpivoted chart data streams, a metadata stream, and a raw events stream.
```mermaid theme={null}
erDiagram
CHART_STREAM {
string chart_id PK
string dimension_name
string dimension_value PK
string segment PK
number value
}
CHARTS_METADATA {
string chart_id PK
string title
string metric
string event
string dimension
string filters
}
EVENTS {
string uuid PK
string event_type
datetime event_time
string user_id
string device_id
integer session_id
string platform
string country
string event_properties
string user_properties
}
CHART_STREAM }o--|| CHARTS_METADATA : "chart_id"
```
**Multiple Chart Streams:** Each configured chart ID creates its own stream (`chart_{chart_id}`). The diagram above shows the structure that applies to all chart streams. The events stream is independent and contains raw event-level data.
## Implementation Notes
### Dynamic Stream Discovery
* **One stream per chart**: Each Chart ID creates a separate stream named `chart_{chart_id}`
* **Plus metadata stream**: The `charts_metadata` stream contains information about all configured charts
* **Plus events stream**: The `events` stream is always available for selection regardless of chart configuration
* **Configure specific charts**: Only the Chart IDs you specify in the configuration are extracted
* **No automatic discovery**: Unlike some connectors, you must explicitly provide Chart IDs: the connector does not discover all available charts. When you add a new chart ID, you have to manually click on 'Add streams' to configure the table where this data will be stored.
### Data Structure
* **Unpivoted format**: Chart data is unpivoted (also called "narrow" or "long" format) with one row per dimension-segment-value combination
* **Flexible dimensions**: The `dimension_name` and `dimension_value` fields can represent any dimension your chart uses (country, device, event name, etc.)
* **Segment variations**: The `segment` field can represent time periods, user cohorts, or other groupings depending on your chart configuration
* **Numeric values**: All metric values are stored as numbers in the `value` field
* **JSON string fields**: Dynamic fields (`event_properties`, `user_properties`, `group_properties`, `global_user_properties`, `groups`, `data`, `plan`) in the events stream are serialized as JSON strings to accommodate their dynamic schemas per event.
### Authentication
* **HTTP Basic Auth**: Uses your API key as the username and secret key as the password
* **Project-specific**: The secret key is specific to each Amplitude project
* **Organization-level API key**: The API key is at the organization level
### Sync Strategy
* **Charts: Full table sync**: Chart data represents aggregated metrics and should be refreshed completely on each sync
* **Events: Incremental sync**: The events stream uses `event_time` as a bookmark to fetch only new events on each extraction. The first sync starts from the configured `start_date` (or defaults to 30 days ago).
### Handling Large Event Volumes
The Amplitude Export API has limits on response size and request duration. If you encounter errors:
| Error | Cause | Solution |
| :- | :- | :- |
| **400: Response exceeds 4 GB** | Too many events in the time window | Reduce `event_export_window_days` to a smaller value (e.g., 7 or 1) |
| **504: Request timed out** | Data volume too large for the time window | Reduce `event_export_window_days` to a smaller value |
The tap will log a clear error message with the current window size and instructions to adjust it.
For very large-scale extraction, Amplitude recommends using their native Snowflake, BigQuery, or S3 integrations instead of the API. Consider this if your daily event volume consistently exceeds millions of events.
### Best Practices
1. **Select relevant charts**: Only configure Chart IDs for the metrics you need to reduce API calls and data volume
2. **Use metadata stream**: Join chart data with the `charts_metadata` stream to add context to your analysis
3. **Set an appropriate start date**: For the events stream, choose a start date that balances historical coverage with initial sync time
4. **Tune the export window**: If you have high event volume, start with a smaller `event_export_window_days` (e.g., 7) and increase if needed
5. **Schedule appropriately**: Amplitude analytics typically update daily, so daily sync frequency is usually sufficient. Remember the \~2 hour export delay for raw events.
6. **Parse JSON fields**: Use SQL transformations to extract specific properties from the JSON string fields in the events stream
### Troubleshooting
| Issue | Possible Cause | Solution |
| :- | :- | :- |
| Authentication errors | Incorrect API key or secret key | Verify credentials in Amplitude settings, ensure you're using the correct project's secret key |
| No data extracted | Invalid Chart IDs | Verify Chart IDs by checking URLs in Amplitude, ensure charts exist and you have access |
| Empty streams | Chart has no data | Check that the chart contains data in Amplitude before syncing |
| Events export fails with 400 | Response exceeds 4 GB | Reduce `event_export_window_days` in your configuration |
| Events export fails with 504 | Request timed out | Reduce `event_export_window_days` in your configuration |
| No new events exported | Within the 2-hour delay window | Wait for Amplitude to finish processing recent events |
| Unexpected data structure | Chart configuration changed | Re-extract the data, the structure reflects the current chart configuration in Amplitude |
## Skills for agents
Amplitude connector documentation as plain markdown, for use in AI agent contexts.
# AnyMarket as a data source
Source: https://docs.nekt.com/sources/anymarket
Bring data from AnyMarket to Nekt.
AnyMarket is a leading e-commerce integration hub platform in Brazil that connects sellers to multiple marketplaces from a single platform. It enables businesses to manage products, inventory, orders, and shipments across various sales channels including Mercado Livre, Amazon, Shopee, Magalu, and many others.
![]()
## Configuring AnyMarket as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the AnyMarket option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your AnyMarket API Token for this connection. You can generate or find this token in your AnyMarket account settings under API integrations.
The following configurations are available:
* **Token**: Your AnyMarket API authentication token. This is required to access your account data.
* **Start Date**: The earliest date from which records will be synced. Records created or updated after this date will be extracted from the source.
If no start date is provided, the connector will extract data from the past 120 days.
* **Lookback Window**: (Default: 30 days) The number of days to look back and refetch records based on the creation date. This helps ensure data completeness for recently modified records.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. AnyMarket provides comprehensive order data that includes marketplace information, buyer details, payments, items, and shipping information.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from AnyMarket and their corresponding fields:
Stream containing comprehensive order data from all connected marketplaces.
**Key Fields:**
* `id` - Unique order identifier
* `accountName` - Account name
* `marketPlace` - Marketplace name (e.g., Mercado Livre, Amazon, Shopee)
* `marketPlaceId` - Marketplace identifier
* `marketPlaceNumber` - Marketplace order number
* `partnerId` - Partner identifier
* `subChannel` - Subchannel
* `subChannelNormalized` - Normalized subchannel
* `createdAt` - Order creation timestamp
* `lastUpdate` - Last update timestamp
* `status` - Order status
* `transmissionStatus` - Transmission status
* `marketPlaceStatus` - Marketplace order status
* `marketPlaceStatusComplement` - Marketplace status complement
* `deliveryStatus` - Delivery status
* `fulfillment` - Whether order is fulfilled
* `orderTypeName` - Order type name
**Financial Fields:**
* `discount` - Order discount
* `freight` - Freight cost
* `sellerFreight` - Freight cost for seller
* `interestValue` - Interest value
* `gross` - Gross value
* `total` - Total order value
**Cancellation Fields:**
* `cancelDate` - Date of cancellation
* `cancelDetails` - Cancellation details
* `cancellationCode` - Cancellation code
* `cancellationOrigin` - Cancellation origin
**Payment Fields:**
* `paymentDate` - Date of payment
* `documentPaymentInstitution` - Payment institution document
* `documentIntermediator` - Intermediator document
* `intermediateRegistrationId` - Intermediate registration ID
**Shipping & Logistics:**
* `shippingId` - Shipping identifier
* `shippingOptionId` - Shipping option identifier
* `logisticType` - Logistic type
* `marketPlaceShipmentStatus` - Marketplace shipment status
* `marketPlaceShipmentSubstatus` - Marketplace shipment substatus
* `printTagDate` - Print tag date
**URLs & Messages:**
* `marketPlaceUrl` - Marketplace URL
* `transmissionMessage` - Transmission message
**Nested Objects:**
**`invoice`** - Invoice details:
* `accessKey` - Invoice access key (NFe)
* `series` - Invoice series
* `number` - Invoice number
* `date` - Invoice date
* `cfop` - CFOP code
* `companyStateTaxId` - Company state tax ID
* `linkNfe` - NFe link
* `invoiceLink` - Invoice link
* `extraDescription` - Extra description
**`shipping`** - Shipping address:
* `address` - Full address
* `street` - Street name
* `number` - Street number
* `neighborhood` - Neighborhood
* `city` - City
* `state` - State
* `stateNameNormalized` - Normalized state name
* `country` - Country
* `countryAcronymNormalized` - Normalized country acronym
* `countryNameNormalized` - Normalized country name
* `zipCode` - ZIP code
* `comment` - Address comment
* `reference` - Address reference
* `receiverName` - Receiver name
* `promisedShippingTime` - Promised shipping time
* `promisedDispatchTime` - Promised dispatch time
**`billingAddress`** - Billing address:
* Same fields as `shipping` plus:
* `shipmentUserDocument` - User document
* `shipmentUserDocumentType` - Document type
* `shipmentUserName` - User name
**`anymarketAddress`** - AnyMarket normalized address:
* `address`, `street`, `number`, `neighborhood`
* `city`, `state`, `stateAcronymNormalized`
* `country`, `zipCode`, `comment`, `reference`
* `receiverName`, `promisedShippingTime`
**`buyer`** - Buyer information:
* `name` - Buyer name
* `email` - Email address
* `phone` - Phone number
* `cellPhone` - Cell phone number
* `document` - Document number
* `documentNumberNormalized` - Normalized document number
* `documentType` - Document type (CPF/CNPJ)
* `marketPlaceId` - Marketplace buyer ID
* `dateOfBirth` - Date of birth
* `companyStateTaxId` - Company state tax ID
**`payments`** - Payment information (array):
* `method` - Payment method
* `paymentMethodNormalized` - Normalized payment method
* `paymentDetailNormalized` - Normalized payment detail
* `status` - Payment status
* `value` - Payment value
* `installments` - Number of installments
* `marketplaceId` - Marketplace payment ID
* `gatewayFee` - Gateway fee
* `marketplaceFee` - Marketplace fee
* `documentPaymentInstitution` - Payment institution document
* `cardOperator` - Card operator
* `orderAuthorizationCardCode` - Card authorization code
**`items`** - Order items (array):
* `product` - Product details:
* `id` - Product ID
* `title` - Product title
* `externalIdProduct` - External product ID
* `sku` - SKU details:
* `id` - SKU ID
* `title` - SKU title
* `ean` - EAN code
* `partnerId` - Partner ID
* `externalId` - External SKU ID
* `amount` - Quantity
* `unit` - Unit price
* `gross` - Gross value
* `total` - Total value
* `discount` - Item discount
* `marketPlaceId` - Marketplace item ID
* `orderItemId` - Order item ID
* `idInMarketPlace` - ID in marketplace
* `officialStoreId` - Official store ID
* `officialStoreName` - Official store name
* `listingType` - Listing type
* `freeShipping` - Free shipping flag
* `isCatalog` - Catalog flag
* `idInMarketplaceCatalogOrigin` - Catalog origin ID
* `gatewayFee` - Gateway fee
* `marketplaceFee` - Marketplace fee
* `itemIdInMarketPlace` - Item ID in marketplace
* `shippings` - Item shipping methods (array):
* `id` - Shipping ID
* `shippingCarrierNormalized` - Normalized carrier
* `shippingCarrierTypeNormalized` - Normalized carrier type
* `shippingtype` - Shipping type
* `customization` - Customization details:
* `customizationType` - Type
* `customizationQuantity` - Quantity
* `customizationPrice` - Price
* `customizationValue` - Value
* `stocks` - Stock details (array):
* `amount` - Stock amount
* `stockLocalId` - Stock location ID
* `stockName` - Stock name
**`tracking`** - Tracking information:
* `carrier` - Carrier name
* `carrierDocument` - Carrier document
* `number` - Tracking number
* `url` - Tracking URL
* `date` - Tracking date
* `shippedDate` - Shipped date
* `estimateDate` - Estimated delivery date
* `deliveredDate` - Delivered date
* `bufferingDate` - Buffering date
**`pickup`** - Pickup details:
* `id` - Pickup ID
* `description` - Description
* `partnerId` - Partner ID
* `marketplaceId` - Marketplace ID
* `receiverName` - Receiver name
**`quoteReconciliation`** - Quote reconciliation:
* `quoteId` - Quote ID
* `price` - Quote price
**`metadata`** - Order metadata (array):
* `key` - Metadata key
* `value` - Metadata value
**`discountMetadata`** - Discount metadata (array):
* `type` - Discount types (array)
**Account Fields:**
* `idAccount` - Account ID
## Skills for agents
AnyMarket connector documentation as plain markdown, for use in AI agent contexts.
# Apify as a data source
Source: https://docs.nekt.com/sources/apify
Bring data from Apify to Nekt.
Apify is a web scraping and automation platform that allows you to build, run, and scale web scrapers, crawlers, and automation tools called Actors. It stores results in datasets that can be accessed via API, making it easy to collect and process data from any website.
![]()
## Configuring Apify as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Apify option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Apify API token for authentication. You can find your API token in the Apify Console under **Settings > API & Integrations**.
The following configurations are available:
* **API Key**: Your Apify API token used for authentication. This is required.
* **Dataset IDs** *(optional)*: A list of dataset IDs to extract. You can find dataset IDs in the Apify Console under **Storage > Datasets**, or in the output tab of your Actor runs. Note that most Apify datasets (especially those created by Actor runs) are unnamed and won't appear in generic listings, so you must provide their IDs explicitly.
* **Actor IDs** *(optional)*: A list of Actor IDs to extract run data from. The tap will incrementally sync succeeded runs and their dataset items. You can find Actor IDs in the Apify Console under **Actors**, or use the format `username~actor-name`. This is ideal when you have scheduled Actors running periodically and want to automatically sync their results without manually tracking dataset IDs.
You must provide at least one of **Dataset IDs** or **Actor IDs**. You can also use both at the same time.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data.
* Full table: every time the extraction happens, we'll get the current state of the data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Apify and their corresponding fields:
Lists all datasets in your Apify account. This stream supports incremental sync based on the `modifiedAt` field.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the dataset |
| `name` | String | Name of the dataset |
| `createdAt` | Date-time | Timestamp when the dataset was created |
| `modifiedAt` | Date-time | Timestamp when the dataset was last modified |
| `accessedAt` | Date-time | Timestamp when the dataset was last accessed |
| `itemCount` | Integer | Total number of items in the dataset |
| `cleanItemCount` | Integer | Number of clean (deduplicated) items |
| `actId` | String | ID of the Actor that created the dataset |
| `actRunId` | String | ID of the Actor run that created the dataset |
| `stats` | String | Dataset statistics as a JSON string |
| `fields` | String | Dataset field definitions as a JSON string |
Retrieves all items (records) stored in each dataset. Each item is serialized as a JSON string in the `data` field, preserving the original structure from the Actor run. This stream supports incremental sync based on the `run_started_at` field.
| Field | Type | Description |
| :- | :- | :- |
| `dataset_id` | String | ID of the parent dataset |
| `actor_id` | String | ID of the Actor that generated the dataset |
| `run_id` | String | ID of the Actor run |
| `run_started_at` | Date-time | Timestamp when the Actor run started |
| `_row_index` | Integer | Sequential index of the item within the dataset |
| `data` | String | The dataset item serialized as a JSON string |
Lists all succeeded runs for each configured Actor, with incremental sync based on the `startedAt` field. Each run includes metadata such as cost, schedule info, and a reference to its default dataset.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the Actor run |
| `actId` | String | ID of the Actor |
| `userId` | String | ID of the user who triggered the run |
| `actorTaskId` | String | ID of the Actor task (if run via a task) |
| `status` | String | Run status (always SUCCEEDED for this stream) |
| `startedAt` | Date-time | Timestamp when the run started |
| `finishedAt` | Date-time | Timestamp when the run finished |
| `buildId` | String | ID of the Actor build used |
| `buildNumber` | String | Semantic version of the build |
| `usageTotalUsd` | Number | Total cost of the run in USD |
| `defaultKeyValueStoreId` | String | ID of the run's default key-value store |
| `defaultDatasetId` | String | ID of the run's default dataset |
| `defaultRequestQueueId` | String | ID of the run's default request queue |
| `meta` | Object | Run metadata including origin, scheduleId, scheduledAt, clientIp, and userAgent |
Retrieves all items from each Actor run's default dataset. This is a child stream of Actor Runs, meaning it automatically fetches dataset items for every new run synced. Each item is serialized as a JSON string in the `data` field. This stream supports incremental sync based on the `run_started_at` field.
| Field | Type | Description |
| :- | :- | :- |
| `dataset_id` | String | ID of the dataset (from the Actor run) |
| `actor_id` | String | ID of the Actor that produced this data |
| `run_id` | String | ID of the Actor run |
| `run_started_at` | Date-time | Timestamp when the parent Actor run started (used as replication key for incremental sync) |
| `_row_index` | Integer | Sequential index of the item within the dataset |
| `data` | String | The dataset item serialized as a JSON string |
Retrieves detailed metadata and statistics for each dataset. This stream supports incremental sync based on the `modifiedAt` field.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the dataset |
| `name` | String | Name of the dataset |
| `createdAt` | Date-time | Timestamp when the dataset was created |
| `modifiedAt` | Date-time | Timestamp when the dataset was last modified |
| `accessedAt` | Date-time | Timestamp when the dataset was last accessed |
| `itemCount` | Integer | Total number of items in the dataset |
| `cleanItemCount` | Integer | Number of clean (deduplicated) items |
| `actId` | String | ID of the Actor that created the dataset |
| `actRunId` | String | ID of the Actor run that created the dataset |
# Data Model
The following diagram illustrates the relationships between the core data streams in Apify.
```mermaid theme={null}
erDiagram
DATASETS {
string id PK
string actId
string actRunId
}
ACTOR_RUNS {
string id PK
string actId
string defaultDatasetId FK
}
DATASET_ITEMS {
string dataset_id FK
integer _row_index PK
}
ACTOR_RUN_DATASET_ITEMS {
string dataset_id FK
string run_id FK
integer _row_index PK
}
DATASET_STATISTICS {
string id PK
}
ACTOR_RUNS ||--|| DATASETS : generates
DATASETS ||--o{ DATASET_ITEMS : contains
DATASETS ||--|| DATASET_STATISTICS : has
ACTOR_RUNS ||--o{ ACTOR_RUN_DATASET_ITEMS : has
```
## Skills for agents
Apify connector documentation as plain markdown, for use in AI agent contexts.
# Apple App Store as a data source
Source: https://docs.nekt.com/sources/apple-app-store
Bring data from the Apple App Store to Nekt.
The Apple App Store is Apple's distribution platform for iOS, iPadOS, macOS, tvOS, watchOS, and visionOS apps. Through the App Store Connect API, it exposes how people discover, download, use, and pay for your apps, along with the sales and financial reports behind your payouts.
## Configuring Apple App Store as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Apple App Store option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Apple authenticates every request with a key you generate in App Store Connect. Create it before starting the setup:
In [App Store Connect](https://appstoreconnect.apple.com), go to **Users and Access**, select the **Integrations** tab, and choose **App Store Connect API** in the left column. Make sure the **Team Keys** tab is selected: individual keys cannot read sales and financial reports.
Click **Generate API Key**, give it a name, and select the **Admin** role under Access. Apple only lets a key with the Admin role turn on analytics reports, which is what makes the report tables fill in automatically.
Click **Download API Key** to save the `.p8` file. Apple allows this download only once; if the file is lost, you have to generate a new key.
The following configurations are available:
* **Key ID**: The identifier shown in the Active column next to your key on the Integrations page.
* **Issuer ID**: Shown near the top of the same Integrations page. It is the same for every team key of your account.
* **Private key (.p8)**: The full contents of the `.p8` file you downloaded, including the BEGIN and END lines. Open the file in a text editor and paste everything.
* **Vendor Number**: Found in App Store Connect under **Payments and Financial Reports**. It is only needed for the Sales Reports and Finance Reports streams; every App Store analytics stream works without it.
* **Start Date**: The earliest date from which records will be synced.
* **Load historical analytics data**: (Default: enabled) Asks Apple once for a snapshot of all available analytics history. Leave it on for a new source: without it, analytics reports only cover the period after the source was created.
Once you're done, click **Next**.
Apple takes 24 to 48 hours to produce the first set of analytics reports after a source is created. Your first run will succeed with empty report tables, and the data appears on a later run. Sales Reports and Finance Reports are not affected and load immediately.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Most analytics reports come in two versions, **Standard** and **Detailed**. They describe the same events: Detailed adds attribution columns (source info, campaign, page title) and, in exchange, applies stricter privacy thresholds that can omit low-volume rows. Pick one per report: selecting both puts two overlapping aggregations of the same events in your catalog, and any straightforward SUM across them counts the same activity twice.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Apple publishes analytics data once a day, so a daily trigger matches how often new data actually exists.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How Apple limits the data
These are Apple's own rules, not Nekt's, and they shape what you can expect to see:
* **Analytics history starts on January 1, 2024.** Subscription State and Subscription Event reports start on January 1, 2025.
* **Report data expires after 35 days.** Apple deletes a report instance 35 days after producing it. Keeping your source on a daily trigger means the data is always collected well inside that window.
* **Daily sales reports are kept for one year.** The source does not look further back than that.
* **A day is complete two to five days later**, depending on the report. Apple restates a day when late events arrive, and the source re-reads the most recent day on every run so those corrections update the existing rows instead of duplicating them.
* **Privacy thresholds apply.** Usage reports only count users who opted in to share data with Apple and developers, rows representing fewer than five users or devices are omitted, and a small amount of statistical noise is added. Low-volume apps and brand-new features may show no data at all.
* **Rate limit**: Apple allows roughly 3,500 requests per hour per API key. The connector tracks the remaining budget and slows down on its own when it runs low.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Apps | `apps` | Every app in your App Store Connect account, with bundle ID, SKU and primary locale. |
| Customer Reviews | `customer_reviews` | Ratings and written reviews left on the App Store, per app and territory. |
| Analytics Report Requests | `analytics_report_requests` | The report requests active for each app. Shows whether Apple stopped generating reports. |
| Analytics Reports | `analytics_reports` | The reports Apple generates for each request, with name and category. |
| Analytics Report Instances | `analytics_report_instances` | The daily instances of each report, showing how far back data currently reaches. |
| App Store Downloads | `app_store_downloads_standard` / `app_store_downloads_detailed` | Downloads from the App Store, split by download type, source, device and territory. |
| App Store Purchases | `app_store_purchases_standard` / `app_store_purchases_detailed` | Paid app and in-app purchase revenue, attributed to download sources and product pages. |
| App Store Discovery and Engagement | `app_store_discovery_and_engagement_standard` / `app_store_discovery_and_engagement_detailed` | Impressions, page views and taps across product pages, store sheets and in-app event pages. |
| App Store Installations and Deletions | `app_store_installs_and_deletions_standard` / `app_store_installs_and_deletions_detailed` | How often people install and delete the app. |
| App Sessions | `app_sessions_standard` / `app_sessions_detailed` | How often people open the app and how long they spend in it. |
| App Crashes | `app_crashes` | Crash counts by app version and device type. |
| Subscription State | `app_store_subscription_state_standard` / `app_store_subscription_state_detailed` | Daily snapshot of active and churned subscriptions, free trials and billing issues. |
| Subscription Events | `app_store_subscription_events_standard` / `app_store_subscription_events_detailed` | Subscription lifecycle changes: conversions to paid, renewals, and churn. |
| App Store Pre-Orders | `app_store_pre_orders_standard` / `app_store_pre_orders_detailed` | Pre-orders placed and canceled for the app. |
| Sales Reports | `sales_reports` | Daily Summary Sales report: units and estimated proceeds per SKU, country and device. Requires the Vendor Number. |
| Finance Reports | `finance_reports` | Consolidated financial report per fiscal month, in the currency you are actually paid in. Requires the Vendor Number. |
Analytics reports measure estimated proceeds in USD. Finance Reports are the only stream carrying settled amounts in your payout currency, which is why the two do not match exactly.
## Fields by stream
Below you'll find the main data streams from the Apple App Store and their corresponding fields:
Every app registered in your App Store Connect account.
**Key Fields:**
* `id` - App Store Connect resource ID of the app, used to link every other stream
* `name` - Name of the app as it appears on the App Store
* `bundle_id` - Bundle ID of the app, matching the one used in Xcode
* `sku` - Unique ID for the app that is not visible on the App Store
* `primary_locale` - Primary locale, used when localized information is unavailable in a territory
* `content_rights_declaration` - Whether the app contains third-party content
* `is_or_ever_was_made_for_kids` - Whether the app is, or has ever been, in the Made for Kids category
Ratings and written reviews customers left on the App Store.
**Key Fields:**
* `id` - Unique identifier of the review
* `app_id` - Resource ID of the reviewed app
* `app_name` - Name of the reviewed app
* `rating` - Star rating the customer gave, 1 to 5
* `title` - Title the customer wrote for the review
* `body` - Review text the customer wrote
* `reviewer_nickname` - Nickname the customer used
* `territory` - App Store territory the review was written in
* `created_date` - When the review was created, used as the incremental replication key
How often your app is downloaded from the App Store.
**Key Fields:**
* `date` - Date the downloads occurred
* `app_name`, `app_apple_identifier` - The app the row refers to
* `download_type` - First-time download, redownload, manual update, auto-update or restore
* `app_version` - App version being downloaded
* `device`, `platform_version` - Device and OS version
* `source_type` - Where the user discovered the app (App Store search, browse, app referrer, web referrer)
* `page_type` - Product page, in-app event page, store sheet or no page
* `territory` - App Store country or region
* `counts` - Total number of downloads
* `source_info`, `campaign`, `page_title` - Attribution detail, Detailed stream only
Paid app and in-app purchase revenue on the App Store.
**Key Fields:**
* `date` - Date the purchase occurred
* `purchase_type` - Type of purchase made
* `content_name`, `content_apple_identifier` - What was purchased
* `payment_method` - Payment type used to charge the customer
* `purchases` - Count of purchases; negative values are refunds
* `proceeds_in_usd` - Estimated proceeds: customer price minus taxes and Apple's commission
* `sales_in_usd` - Estimated sales
* `paying_users` - Unique paying users; not summable across rows
* `app_download_date` - When the app was downloaded, if within the previous 30 days
* `territory`, `device`, `platform_version` - Where and on what the purchase happened
How users interact with your app on the App Store itself.
**Key Fields:**
* `date` - Date the event occurred
* `event` - Type of event that occurred
* `engagement_type` - User action, if any, on the impression or page
* `page_type` - Page type associated with the event
* `source_type` - Where the user discovered the app
* `counts` - Total number of events
* `unique_counts` - Total number of unique users who performed the event
Installs and deletions, among users who opted in to share usage data.
**Key Fields:**
* `date` - Date the event occurred
* `event` - Type of usage event (installation or deletion)
* `download_type` - Type of download that preceded the event
* `app_version`, `device`, `platform_version` - What the event happened on
* `counts` - Total count of events
* `unique_devices` - Number of unique devices that generated the events
How often people open your app and how long they spend in it.
**Key Fields:**
* `date` - Date the sessions occurred
* `app_version`, `device`, `platform_version` - What the sessions ran on
* `sessions` - Number of sessions
* `total_session_duration` - Total duration of all sessions, in seconds
* `unique_devices` - Unique devices contributing to the sessions
* `territory` - App Store country or region
Crash counts by app version and device type.
**Key Fields:**
* `date` - Date the crashes occurred
* `app_version` - App version associated with the crash
* `device`, `platform_version` - Device and OS version
* `crashes` - Total number of crashes
* `unique_devices` - Number of unique devices on which the app crashed
Daily snapshot of your subscriptions.
**Key Fields:**
* `date` - Date the snapshot was generated for
* `state_metric`, `state_metric_grouping` - The subscription state being counted
* `subscription_name`, `subscription_identifier` - The subscription
* `subscription_duration`, `billing_frequency`, `billing_period` - Terms of the subscription
* `subscription_group`, `subscription_group_identifier` - Group the subscription belongs to
* `offer_type`, `offer_name`, `offer_pricing`, `offer_duration` - Redeemed offer
* `family_sharing` - Whether the subscription is shared with an iCloud family
* `cancellation_reason` - Why the subscription was canceled
* `counts` - Number of subscriptions
Changes across the subscription lifecycle.
**Key Fields:**
* `event_date` - Date the event occurred
* `event_sub_type`, `event_grouping` - The lifecycle event being counted
* `subscription_name`, `subscription_identifier` - The subscription
* `plan_change_type` - Upgrade, crossgrade or downgrade when users switch plans
* `previous_subscription_name`, `previous_subscription_identifier` - The plan switched away from
* `paid_service_days_recovered` - Paid days recovered through the billing grace period
* `cancellation_reason` - Why the subscription was canceled
* `counts` - Number of events
Pre-orders placed and canceled for your app.
**Key Fields:**
* `date` - Date the pre-order activity occurred
* `pre_order_start_date` - Date the app became available for pre-order
* `pre_order_end_date` - Last date the app is available for pre-order
* `pre_orders_placed` - Total number of pre-orders placed
* `pre_orders_canceled` - Total number of pre-orders canceled
* `territory`, `device`, `source_type` - Where the pre-order came from
The daily Summary Sales report from Sales and Trends. Requires the Vendor Number.
**Key Fields:**
* `report_date` - Day the report covers, used as the incremental replication key
* `sku`, `title`, `version` - The product sold
* `product_type_identifier` - Type of transaction, such as an initial download or an update
* `units` - Aggregated units; negative values are refunds
* `developer_proceeds_per_unit` - Amount received per unit
* `customer_price` - Price per unit billed to the customer
* `customer_currency`, `currency_of_proceeds` - Currencies involved
* `country_code` - App Store country or region of the sale
* `device`, `supported_platforms` - Where the purchase happened
* `subscription`, `period`, `preserved_pricing`, `proceeds_reason` - Subscription detail
The consolidated financial report per fiscal month, covering every region. Requires the Vendor Number.
**Key Fields:**
* `report_date` - Fiscal month the report covers, as YYYY-MM
* `start_date`, `end_date` - Period boundaries on Apple's fiscal calendar
* `vendor_identifier` - SKU of the app, or product ID of the in-app purchase
* `quantity` - Aggregated units sold
* `partner_share` - Proceeds received per unit
* `extended_partner_share` - Quantity multiplied by the partner share: the amount earned
* `partner_share_currency` - Currency of the amounts earned
* `sale_or_return` - S for a sale, R for a return
* `country_of_sale` - Country of the sale
* `customer_price`, `customer_currency` - What the customer paid
Every stream also carries an `additional_fields` column. It is normally empty and holds any column Apple adds to a report before the connector models it, so new data is never silently dropped.
# Appmax as a data source
Source: https://docs.nekt.com/sources/appmax
Bring data from Appmax to your Lakehouse.
Appmax is a Brazilian payment platform that combines gateway, anti-fraud and acquiring for digital businesses, processing credit card, Pix and boleto payments through its checkout. The connector extracts every order registered in your Appmax account (amounts, status, payment method, boleto and Pix details), with the buyer's data carried inside each order, so you can analyse your sales in your Lakehouse and join them with your marketing and product data.
## Configuring Appmax as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Appmax option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Appmax authenticates with an **API key** issued for your account. In the Appmax admin panel, go to **Integrations** and open the **API** section to generate or copy the key.
The key reaches every order in the account it belongs to. Treat it as a credential: generate it from your own Appmax account and avoid reusing a key shared with other integrations, so it can be revoked independently.
The following configurations are available:
* **API key**: the key copied from the Appmax panel, under Integrations. Required.
* **Orders per page**: how many orders are requested per page during the extraction. The default of 100 suits most accounts; lower it only if Appmax rejects the value.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
The buyer's data (name, email, phone and document number) arrives inside each order, in the **customer** field, as a JSON object. Appmax has no endpoint that lists customers on their own, so there is no separate Customers stream.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: the Appmax orders endpoint accepts no date filter, so Orders is replicated as full table: every run reads the complete order history and refreshes the table. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Because every run reads the full order history, an account with a large volume of orders takes longer as it grows. Daily extractions are usually enough for sales analysis.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Appmax and their corresponding fields.
Every order registered in the Appmax account, with its amounts, payment method, current status and the buyer who placed it. Payment-specific fields are filled according to the method used: boleto orders carry the boleto link and dates, and Pix orders carry the Pix payload and its expiration.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the order in Appmax, used as the primary key. |
| `customer_id` | Integer | Identifier of the customer who placed the order. |
| `status` | String | Current status of the order, in Portuguese, as Appmax spells it: 'pendente', 'autorizado', 'aprovado', 'integrado', 'cancelado', 'estornado', 'recusado\_por\_risco', 'pendente\_integracao', 'chargeback\_em\_tratativa', 'chargeback\_em\_disputa', 'chargeback\_perdido' or 'chargeback\_vencido'. |
| `payment_type` | String | Payment method used on the order, as returned by Appmax, for example 'CreditCard', 'Pix' or 'Billet'. |
| `total` | Number | Total amount of the order in reais, including shipping and installment interest and net of discounts. |
| `total_products` | Number | Sum of the products in the order, in reais, before discounts and shipping. |
| `discount` | Number | Discount applied to the order, in reais. |
| `freight_value` | Number | Shipping amount charged on the order, in reais. |
| `full_payment_amount` | Number | Amount effectively charged to the buyer in reais, including installment interest when the order was paid in installments. |
| `company_name` | String | Name of the store the order belongs to. |
| `issuer_message` | String | Message returned by the card issuer or the payment provider, explaining why a payment was declined or is pending. |
| `billet_url` | String | Link to the boleto PDF, for orders paid by boleto. |
| `billet_expiration_date` | Datetime | Date the boleto expires, for orders paid by boleto. |
| `billet_date_overdue` | Datetime | Date the boleto went overdue without being paid. |
| `pix_code` | String | Pix copy-and-paste payload generated for the order. |
| `pix_expiration_date` | Datetime | Moment the Pix charge expires, for orders paid by Pix. |
| `created_at` | Datetime | Moment the order was created, converted from Appmax's Brasilia local time to UTC. |
| `updated_at` | Datetime | Moment the order was last updated (a status change, a payment or a refund), converted from Appmax's Brasilia local time to UTC. |
| `customer` | String | Buyer of the order as a JSON object, carrying at least id, firstname, lastname, email, telephone and document number (CPF). |
# Arbo as a data source
Source: https://docs.nekt.com/sources/arbo
Bring data from Arbo to Nekt.
Arbo is a real estate platform that provides property management and listing services for real estate agencies. It offers tools for managing properties, condominiums, owners, and real estate transactions, enabling real estate professionals to efficiently organize and showcase their property portfolios.
## Configuring Arbo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Arbo option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Arbo API authentication token to access your data.
The following configuration is available:
* **Auth Token**: (Required) The API key to authenticate against the Arbo API service. Contact Arbo support to obtain your API authentication token.
* **Leads API Auth Token**: (Optional) When provided, the **Leads** stream becomes available. This token authenticates against the Arbo Leads API (separate endpoint). Use it if you want to sync lead data (contacts, status, broker, associated properties). If omitted, the Leads stream will not appear in the stream list.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Arbo and their corresponding fields:
Stream for managing property listings, including all property details, location information, pricing, and features.
**Identifiers:**
* `ref_id` - Unique identifier for the property (primary key)
* `codigo` - Property code
* `codigo_origem` - Original property code
**Status & Visibility:**
* `ativo` - Whether the property is active
* `publicado` - Whether the property is published
* `status_comercial` - Commercial status
* `origemcadastro` - Registration origin
**Property Information:**
* `titulo` - Property title
* `descricao` - Property description
* `categoria` - Property category
* `categoria_imovel` - Property category type
* `tipo_imovel` - Property type (Residencial, Comercial, etc.)
* `finalidade` - Property purpose (Venda, Aluguel, Venda e Locação)
* `mobiliado` - Whether the property is furnished
* `permuta` - Whether exchange is accepted
* `financiamento` - Whether financing is available
**Property Details:**
* `qtd_quartos` - Number of bedrooms
* `qtd_suites` - Number of suites
* `qtd_vagas` - Number of parking spaces
* `qtd_banheiro` - Number of bathrooms
* `area_total` - Total area
* `area_privativa` - Private area
**Address Information:**
* `end_condominio` - Condominium name
* `end_bairro` - Neighborhood
* `end_cidade` - City
* `end_estado` - State
* `end_logradouro` - Street address
* `end_complemento` - Address complement
* `end_numero` - Address number
* `end_cep` - ZIP code
* `latitude` - Latitude coordinate
* `longitude` - Longitude coordinate
**Pricing:**
* `valor_venda` - Sale price
* `valor_aluguel` - Rental price
* `valor_condominio` - Condominium fee
* `valor_iptu` - IPTU tax value
**Media:**
* `fotos` - Array of property photos, each containing:
* `url` - Photo URL
* `ordem` - Display order
* `principal` - Whether it's the main photo
* `marcadagua_url` - Watermark URL
* `sizes` - Object with `medium` and `small` image URLs
* `url_video` - Video URL
* `url_tour` - Virtual tour URL
**Relationships:**
* `condominio` - Condominium ID (foreign key to condominios)
* `prop_id` - Property ID (used for child stream proprietarios)
* `corretor` - Broker information object:
* `codigo` - Broker code
* `nome` - Broker name
* `imobiliaria` - Real estate agency information object:
* `nome` - Agency name
* `infos` - Array of info objects with `url` and `tipo`
* `telefones` - Array of phone numbers
**Features & Metadata:**
* `caracteristicas` - Array of property features/amenities
* `emp_fields` - Enterprise fields object with pricing and area filters:
* `emp_preco_min` - Minimum price
* `emp_preco_max` - Maximum price
* `emp_preco_min_loc` - Minimum rental price
* `emp_preco_max_loc` - Maximum rental price
* `emp_areautil_min` - Minimum usable area
* `emp_areautil_max` - Maximum usable area
* `emp_areaprivativa_min` - Minimum private area
* `emp_areaprivativa_max` - Maximum private area
* `emp_areatotal_min` - Minimum total area
* `emp_areatotal_max` - Maximum total area
* `emp_dorm_min` - Minimum bedrooms
* `emp_dorm_max` - Maximum bedrooms
* `emp_porcentagem_total` - Total percentage
**Timestamps:**
* `created_at` - Creation timestamp
* `updated_at` - Last update timestamp
Stream for managing condominium information, including location, characteristics, and photos.
**Identifiers:**
* `id` - Unique identifier for the condominium (primary key)
* `cond_cod` - Condominium code
* `cond_cod_origem` - Original condominium code
**Basic Information:**
* `cond_nome` - Condominium name
* `cond_tipo` - Condominium type
* `cond_ano` - Construction year (can be string or number)
* `cond_construt` - Builder/Constructor
* `cond_desc` - Description
* `cond_coment` - Comments
* `condominio_ativo` - Whether the condominium is active
**Address Information:**
* `cond_logradouro` - Street address
* `cond_numero` - Address number
* `cond_bairro` - Neighborhood
* `cond_ptref` - Reference point
* `cond_city` - City
* `cond_est` - State
* `cond_cep` - ZIP code
* `cond_latitude` - Latitude coordinate
* `cond_longitude` - Longitude coordinate
* `cond_raiomapa` - Map radius
**Media & Features:**
* `fotos` - Array of condominium photos, each containing:
* `url` - Photo URL
* `ordem` - Display order
* `principal` - Whether it's the main photo
* `marcadagua_url` - Watermark URL
* `sizes` - Object with `medium` and `small` image URLs
* `caracteristicas` - Array of condominium features/amenities
* `imobiliaria` - Real estate agency information (can be ID number or full object)
**Timestamps:**
* `createdat` - Creation timestamp
* `updatedat` - Last update timestamp
Stream for managing property owners. This is a child stream of Imoveis, meaning it requires a parent property record to fetch owner information.
**Identifiers:**
* `id` - Unique identifier for the owner (primary key)
* `usu_id` - User ID (property ID, references prop\_id from imoveis)
* `imb_id` - Real estate agency ID
**Owner Information:**
* `nome` - Owner name
* `email` - Owner email
* `rg` - RG (Brazilian ID)
* `cpf` - CPF (Brazilian tax ID)
**Contact Information:**
* `telefones` - Array of phone number objects, each containing:
* `id` - Phone number ID
* `numero` - Phone number
* `contato` - Contact name/description
**Timestamps:**
* `createdat` - Creation timestamp
* `updatedat` - Last update timestamp
Stream for retrieving cities with available properties.
**Identifiers:**
* `end_cidade` - City name (primary key)
**Metrics:**
* `count` - Number of properties in this city
Stream for retrieving neighborhoods with available properties.
**Identifiers:**
* `end_bairro` - Neighborhood name (part of composite primary key)
* `end_cidade` - City name (part of composite primary key)
**Metrics:**
* `count` - Number of properties in this neighborhood
Stream for retrieving property type categories.
**Identifiers:**
* `id` - Property type ID (primary key)
**Information:**
* `categoria` - Property type category
Stream for lead/contact data from the Arbo Leads API. This stream is **only available** when the optional **Leads API Auth Token** is configured in the source. Uses a separate API base URL and pagination (page size 100).
**Identifiers:**
* `id` - Lead ID (primary key)
**Lead Information:**
* `ativo` - Whether the lead is active
* `nome` - Lead name
* `email` - Lead email
* `telefone` - Lead phone number
**Broker & Assignment:**
* `corretor` - Broker information object:
* `id` - Broker ID
* `nome` - Broker name
* `email` - Broker email
* `telefone` - Broker phone number
* `telefones` - Array of phone entries (id, label)
* `time` - Assigned team object (id, nome)
**Lead Classification:**
* `midia` - Lead acquisition media/source (id, nome)
* `status` - Lead status (id, label)
* `negocio` - Business type, e.g. Locação, Venda (id, label)
**Associated Properties:**
* `imoveis` - Array of properties linked to the lead, each with:
* `id` - Property ID
* `codigo` - Property code
* `codigo_origem` - Original property code
**Timestamps:**
* `createdat` - Creation timestamp
* `updatedat` - Last update timestamp
# Data Model
The following diagram illustrates the relationships between the data streams in Arbo. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Imoveis("Imoveis");
Condominios("Condominios");
Proprietarios("Proprietarios");
Leads("Leads");
TiposImovel("Tipos Imovel");
end
subgraph "Reference Data"
Cidades("Cidades");
Bairros("Bairros");
end
Imoveis -- "condominio -> id" --> Condominios;
Proprietarios -- "usu_id -> prop_id" --> Imoveis;
Leads -- "imoveis[].id" --> Imoveis;
Bairros -- "end_cidade" --> Cidades;
Imoveis -- "categoria" --> TiposImovel;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Arbo data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Property Portfolio Overview
Get a comprehensive view of your property portfolio with key metrics including property counts, pricing distribution, and availability by location.
**Business Value:**
* Identify property distribution across cities and neighborhoods
* Analyze pricing trends and availability
* Track active vs. published properties
* Understand property type distribution
```sql theme={null}
SELECT
i.end_cidade AS city,
i.end_bairro AS neighborhood,
i.categoria AS category,
i.tipo_imovel AS property_type,
i.finalidade AS purpose,
COUNT(*) AS property_count,
COUNT(DISTINCT i.condominio) AS condominium_count,
AVG(i.valor_venda) AS avg_sale_price,
AVG(i.valor_aluguel) AS avg_rental_price,
SUM(CASE WHEN i.ativo = true THEN 1 ELSE 0 END) AS active_count,
SUM(CASE WHEN i.publicado = true THEN 1 ELSE 0 END) AS published_count,
AVG(i.area_total) AS avg_total_area,
AVG(i.qtd_quartos) AS avg_bedrooms
FROM
nekt_raw.arbo_imoveis i
WHERE
i.ativo = true
GROUP BY
1, 2, 3, 4, 5
ORDER BY
property_count DESC
```
```sql theme={null}
SELECT
i.end_cidade AS city,
i.end_bairro AS neighborhood,
i.categoria AS category,
i.tipo_imovel AS property_type,
i.finalidade AS purpose,
COUNT(*) AS property_count,
COUNT(DISTINCT i.condominio) AS condominium_count,
AVG(i.valor_venda) AS avg_sale_price,
AVG(i.valor_aluguel) AS avg_rental_price,
SUM(CASE WHEN i.ativo = true THEN 1 ELSE 0 END) AS active_count,
SUM(CASE WHEN i.publicado = true THEN 1 ELSE 0 END) AS published_count,
AVG(i.area_total) AS avg_total_area,
AVG(i.qtd_quartos) AS avg_bedrooms
FROM
`nekt_raw.arbo_imoveis` i
WHERE
i.ativo = true
GROUP BY
1, 2, 3, 4, 5
ORDER BY
property_count DESC
```
| city | neighborhood | category | property\_type | purpose | property\_count | condominium\_count | avg\_sale\_price | avg\_rental\_price | active\_count | published\_count | avg\_total\_area | avg\_bedrooms |
| - | - | - | - | - | - | - | - | - | - | - | - | - |
| Pelotas | Centro | Studio | Residencial | Venda | 45 | 12 | 245000.00 | NULL | 45 | 42 | 35.5 | 1.0 |
| Pelotas | Três Vendas | Apartamento | Residencial | Venda e Locação | 38 | 8 | 385000.00 | 2800.00 | 38 | 35 | 72.3 | 2.5 |
| Pelotas | Laranjal | Terreno | Residencial | Venda | 22 | 3 | 150000.00 | NULL | 22 | 20 | 250.0 | NULL |
### 2. Property-Owner Relationships
Analyze the relationship between properties and their owners to understand portfolio distribution and owner information.
**Business Value:**
* Identify owners with multiple properties
* Analyze owner contact information completeness
* Track property-owner associations
* Understand property distribution per owner
```sql theme={null}
SELECT
p.nome AS owner_name,
p.email AS owner_email,
p.cpf AS owner_cpf,
COUNT(DISTINCT p.usu_id) AS property_count,
STRING_AGG(DISTINCT i.categoria, ', ') AS property_categories,
STRING_AGG(DISTINCT i.end_cidade, ', ') AS cities,
AVG(i.valor_venda) AS avg_property_value,
SUM(ARRAY_LENGTH(p.telefones)) AS total_phone_numbers
FROM
nekt_raw.arbo_proprietarios p
LEFT JOIN nekt_raw.arbo_imoveis i ON p.usu_id = i.prop_id
GROUP BY
1, 2, 3
HAVING
COUNT(DISTINCT p.usu_id) > 0
ORDER BY
property_count DESC
```
```sql theme={null}
SELECT
p.nome AS owner_name,
p.email AS owner_email,
p.cpf AS owner_cpf,
COUNT(DISTINCT p.usu_id) AS property_count,
STRING_AGG(DISTINCT i.categoria, ', ') AS property_categories,
STRING_AGG(DISTINCT i.end_cidade, ', ') AS cities,
AVG(i.valor_venda) AS avg_property_value,
ARRAY_LENGTH(p.telefones) AS total_phone_numbers
FROM
`nekt_raw.arbo_proprietarios` p
LEFT JOIN `nekt_raw.arbo_imoveis` i ON p.usu_id = i.prop_id
GROUP BY
1, 2, 3, 8
HAVING
COUNT(DISTINCT p.usu_id) > 0
ORDER BY
property_count DESC
```
| owner\_name | owner\_email | owner\_cpf | property\_count | property\_categories | cities | avg\_property\_value | total\_phone\_numbers |
| - | - | - | - | - | - | - | - |
| João Silva | [joao@example.com](mailto:joao@example.com) | 123.456.789-00 | 5 | Studio, Apartamento | Pelotas | 285000.00 | 2 |
| Maria Santos | [maria@example.com](mailto:maria@example.com) | 987.654.321-00 | 3 | Terreno, Casa | Pelotas, Rio Grande | 320000.00 | 1 |
### 3. Condominium Analysis
Analyze condominium data to understand property distribution, features, and location patterns.
**Business Value:**
* Understand property concentration in condominiums
* Analyze condominium characteristics and features
* Track condominium locations and distribution
* Compare property values within condominiums
```sql theme={null}
SELECT
c.cond_nome AS condominium_name,
c.cond_city AS city,
c.cond_bairro AS neighborhood,
COUNT(DISTINCT i.ref_id) AS property_count,
AVG(i.valor_venda) AS avg_sale_price,
AVG(i.area_total) AS avg_area,
ARRAY_LENGTH(c.caracteristicas) AS feature_count,
ARRAY_LENGTH(c.fotos) AS photo_count,
c.condominio_ativo AS is_active
FROM
nekt_raw.arbo_condominios c
LEFT JOIN nekt_raw.arbo_imoveis i ON c.id = i.condominio
GROUP BY
1, 2, 3, 7, 8, 9
ORDER BY
property_count DESC
```
```sql theme={null}
SELECT
c.cond_nome AS condominium_name,
c.cond_city AS city,
c.cond_bairro AS neighborhood,
COUNT(DISTINCT i.ref_id) AS property_count,
AVG(i.valor_venda) AS avg_sale_price,
AVG(i.area_total) AS avg_area,
ARRAY_LENGTH(c.caracteristicas) AS feature_count,
ARRAY_LENGTH(c.fotos) AS photo_count,
c.condominio_ativo AS is_active
FROM
`nekt_raw.arbo_condominios` c
LEFT JOIN `nekt_raw.arbo_imoveis` i ON c.id = i.condominio
GROUP BY
1, 2, 3, 7, 8, 9
ORDER BY
property_count DESC
```
## Implementation Notes
### Leads stream
* The **Leads** stream is only available when **Leads API Auth Token** is set in the source configuration. It uses a separate Arbo Leads API endpoint and token.
* Leads API uses the same pagination pattern (page, lastPage) with a page size of 100. Pagination fields may be returned as strings or numbers; the tap normalizes them for comparison.
### Data Quality Considerations
* The `proprietarios` stream is a child stream of `imoveis` and requires a parent property record with a valid `prop_id`
* Property prices (`valor_venda`, `valor_aluguel`) may be NULL for properties not available for sale/rent
* Some fields like `cond_ano` (construction year) may be returned as strings or numbers depending on the API response
* The `imobiliaria` field in condominios can be either an ID number or a full object, and is converted to a string format for consistency
### API Limits & Performance
* The main Arbo API and the Leads API use pagination with a page size of 100 records
* For faster extractions, select only the streams necessary for your analysis
* The `proprietarios` stream will only sync for properties that have a valid `prop_id` value
## Skills for agents
Arbo connector documentation as plain markdown, for use in AI agent contexts.
# Asaas as a data source
Source: https://docs.nekt.com/sources/asaas
Bring data from Asaas to Nekt.
Asaas is a payment processing platform focused on billing, recurring subscriptions, and financial operations. This connector extracts customer, payment, transfer, subscription, and financial transaction data from your Asaas account into your Nekt Catalog.
![]()
## Configuring Asaas as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button in the top-right corner, then select Asaas from the list of connectors.
Click **Next** and add your access details.
### 1. Add account access
The following configuration fields are available:
* **Access Token**: Token used to authenticate against the Asaas API. For details, check [Asaas authentication documentation](https://docs.asaas.com/docs/autentica%C3%A7%C3%A3o-1).
* **Start Date**: The earliest record date to sync.
* **Extract Full Payment Data**: Controls which payments endpoint is used:
* Enabled (`true`): extracts from `/payments` with the full payments schema.
* Disabled (`false`): extracts from `/lean/payments` with a reduced payments schema.
After configuring access, click **Next**.
### 2. Select streams
Choose which streams you want to sync. For faster extractions, select only the streams you need.
> Tip: You can search streams by name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how data should appear in your Catalog, including layer, table naming, and sync type.
* **Layer**: Select the destination layer where extracted tables will be created.
* **Table name**: Use the suggested name or customize it. You can also apply a prefix to all tables.
* **Sync Type**: Choose between `INCREMENTAL` and `FULL_TABLE`.
When you're done, click **Next**.
### 4. Configure data source
Add a clear description for this source so your team can identify it easily.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), choose how often data should be extracted (for example, daily, weekly, or at fixed times).
Optionally, configure additional settings:
* Delta Log Retention, which controls how long old table states are kept. Learn more [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync), to periodically complement incremental runs with a complete refresh.
When you are ready, click **Next** to finalize setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, trigger a run manually from the source actions.
To see data in the [Catalog](https://app.nekt.ai/catalog), at least one source run must complete successfully.
# Streams and Fields
Below are the currently available streams in the Asaas connector and their main fields.
Customer profile and billing identity information from the `/customers` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `object` | String | Type of object returned by the API. |
| `id` | String | Unique identifier of the record. |
| `date_created` | DateTime | Timestamp when the record was created. |
| `name` | String | Name of the record. |
| `email` | String | Email address of the customer. |
| `phone` | String | Landline phone number of the customer. |
| `mobile_phone` | String | Mobile phone number of the customer. |
| `address` | String | Street address of the customer. |
| `address_number` | String | Street number of the customer address. |
| `complement` | String | Additional address information. |
| `province` | String | District or neighborhood of the customer address. |
| `city` | Integer | Numeric identifier of the customer city. |
| `city_name` | String | Name of the customer city. |
| `state` | String | State code of the customer address. |
| `country` | String | Country of the customer address. |
| `postal_code` | String | Postal code of the customer address. |
| `cpf_cnpj` | String | Tax document number of the customer. |
| `person_type` | String | Type classification of the record. |
| `deleted` | Boolean | Indicates whether the record is deleted. |
| `additional_emails` | String | Additional email addresses for the customer. |
| `external_reference` | String | External reference identifier of the record. |
| `notification_disabled` | Boolean | Indicates whether customer notifications are disabled. |
| `observations` | String | Internal notes associated with the customer. |
| `foreign_customer` | Boolean | Indicates whether the customer is foreign. |
| `company` | String | Company name associated with the customer. |
| `municipal_inscription` | String | Municipal registration number of the customer. |
| `state_inscription` | String | State registration number of the customer. |
| `can_delete` | Boolean | Indicates whether the customer can be deleted. |
| `cannot_be_deleted_reason` | String | Reason why the customer cannot be deleted. |
| `can_edit` | Boolean | Indicates whether the customer can be edited. |
| `cannot_edit_reason` | String | Reason why the customer cannot be edited. |
Payment records. The stream endpoint and some fields depend on **Extract Full Payment Data**.
**Behavior by Configuration:**
* `true` - Uses `/payments` (full schema)
* `false` - Uses `/lean/payments` (short schema)
**Common Fields (both modes):**
| Field | Type | Description |
| :- | :- | :- |
| `object` | String | Type of object returned by the API. |
| `id` | String | Unique identifier of the record. |
| `date_created` | DateTime | Timestamp when the record was created. |
| `value` | Number | Monetary value associated with the record. |
| `net_value` | Number | Net monetary value associated with the payment. |
| `original_value` | Number | Original monetary value associated with the payment. |
| `interest_value` | Number | Interest monetary value associated with the payment. |
| `description` | String | Description of the payment. |
| `billing_type` | String | Billing method used for the payment. |
| `can_be_paid_after_due_date` | Boolean | Indicates whether the payment can be paid after the due date. |
| `status` | String | Current status of the record. |
| `due_date` | DateTime | Due date of the payment. |
| `original_due_date` | DateTime | Original due date of the payment. |
| `payment_date` | DateTime | Date when the payment was completed. |
| `installment_number` | Integer | Installment sequence number of the payment. |
| `external_reference` | String | External reference identifier of the record. |
| `deleted` | Boolean | Indicates whether the record is deleted. |
| `anticipated` | Boolean | Indicates whether the payment was anticipated. |
| `anticipable` | Boolean | Indicates whether the payment can be anticipated. |
| `transaction_receipt_url` | String | URL of the payment transaction receipt. |
| `discount` | Object | Discount details applied to the payment. |
| `fine` | Object | Fine details applied to the payment. |
| `interest` | Object | Interest details applied to the payment. |
| `postal_service` | Boolean | Indicates whether postal service is enabled for the payment. |
**Short Schema (`/lean/payments`) Additional Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `customer_id` | String | Identifier of the associated customer. |
| `subscription_id` | String | Identifier of the associated subscription. |
| `installment_id` | String | Identifier of the associated installment. |
| `payment_link_id` | String | Identifier of the associated payment link. |
| `pix_transaction_id` | String | Identifier of the associated PIX transaction. |
| `confirmed_date` | DateTime | Date when the payment was confirmed. |
| `customer_payment_date` | DateTime | Date when the customer paid the payment. |
| `credit_date` | String | Date when the payment amount is credited. |
| `duplicated_payment_id` | String | Identifier of the duplicated payment record. |
**Full Schema (`/payments`) Additional Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `customer` | String | Identifier of the associated customer. |
| `subscription` | String | Identifier of the associated subscription. |
| `installment` | String | Identifier of the associated installment. |
| `payment_link` | String | Identifier of the associated payment link. |
| `credit_card` | Object | Credit card details used for the payment. |
| `pix_transaction` | String | Identifier of the associated PIX transaction. |
| `pix_qr_code_id` | String | Identifier of the associated PIX QR code. |
| `confirmed_date` | DateTime | Date when the payment was confirmed. |
| `client_payment_date` | DateTime | Date when the client paid the payment. |
| `invoice_url` | String | URL of the payment invoice. |
| `invoice_number` | String | Invoice number associated with the payment. |
| `credit_date` | DateTime | Date when the payment amount is credited. |
| `estimated_credit_date` | DateTime | Estimated date when the payment amount is credited. |
| `last_invoice_viewed_date` | DateTime | Date when the invoice was last viewed. |
| `last_bank_slip_viewed_date` | DateTime | Date when the bank slip was last viewed. |
| `nosso_numero` | String | Bank slip identifier assigned by the bank. |
| `bank_slip_url` | String | URL of the payment bank slip. |
| `split` | Object | Split details for the payment distribution. |
| `days_after_due_date_to_registration_cancellation` | Integer | Number of days after due date to cancel registration. |
| `chargeback` | Object | Chargeback details associated with the payment. |
| `refunds` | Array | List of refunds associated with the payment. |
Recurring subscription contracts from the `/subscriptions` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `object` | String | Type of object returned by the API. |
| `id` | String | Unique identifier of the record. |
| `date_created` | DateTime | Timestamp when the record was created. |
| `customer` | String | Identifier of the associated customer. |
| `payment_link` | String | Identifier of the associated payment link. |
| `billing_type` | String | Billing method used for the subscription. |
| `cycle` | String | Recurrence cycle of the subscription. |
| `value` | Number | Monetary value associated with the record. |
| `gross_value` | Number | Gross monetary value associated with the subscription. |
| `next_due_date` | DateTime | Next due date for the subscription payment. |
| `end_date` | DateTime | Date when the subscription ends. |
| `description` | String | Description of the subscription. |
| `status` | String | Current status of the record. |
| `fine` | Object | Fine details applied to the subscription. |
| `interest` | Object | Interest details applied to the subscription. |
| `split` | Object | Split details for the subscription distribution. |
| `credit_card` | Object | Credit card details used for the subscription. |
| `discount` | Object | Discount details applied to the subscription. |
| `deleted` | Boolean | Indicates whether the record is deleted. |
| `max_payments` | Integer | Maximum number of payments for the subscription. |
| `external_reference` | String | External reference identifier of the record. |
| `send_payment_by_postal_service` | String | Indicates whether payment is sent by postal service. |
Financial account movement records from `/financialTransactions`.
| Field | Type | Description |
| :- | :- | :- |
| `object` | String | Type of object returned by the API. |
| `id` | String | Unique identifier of the record. |
| `value` | Number | Monetary value associated with the record. |
| `balance` | Number | Current account balance after the transaction. |
| `type` | String | Type classification of the record. |
| `date` | DateTime | Date of the financial transaction. |
| `description` | String | Description of the financial transaction. |
| `payment_id` | String | Identifier of the associated payment. |
| `split_id` | String | Identifier of the associated split. |
| `transfer_id` | String | Identifier of the associated transfer. |
| `anticipation_id` | String | Identifier of the associated anticipation. |
| `bill_id` | String | Identifier of the associated bill. |
| `invoice_id` | String | Identifier of the associated invoice. |
| `payment_dunning_id` | String | Identifier of the associated payment dunning process. |
| `pix_transaction_id` | String | Identifier of the associated PIX transaction. |
| `credit_bureau_report_id` | String | Identifier of the associated credit bureau report. |
Transfer operations and settlement details from the `/transfers` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique transfer identifier |
| `type` | String | Transfer type |
| `status` | String | Transfer status |
| `value` | Number | Transfer value |
| `net_value` | Number | Net transfer value |
| `transfer_fee` | Number | Transfer fee |
| `date_created` | DateTime | Timestamp when the record was created |
| `schedule_date` | DateTime | Scheduled date |
| `effective_date` | DateTime | Effective date |
| `authorized` | Boolean | Whether the transfer is authorized |
| `fail_reason` | String | Reason for failure |
| `external_reference` | String | External reference identifier |
| `transaction_receipt_url` | String | Transaction receipt URL |
| `operation_type` | String | Operation type |
| `description` | String | Description |
| `recurring` | Boolean | Whether the transfer is recurring |
| `end_to_end_identifier` | String | End to end identifier |
| `bank_account` | Object | Bank account information |
## Implementation Notes
### Data Freshness and Sync
* `transactions` uses `date` as replication key and is fetched in monthly windows. Missing starting timestamps gracefully fallback to the configured `start_date`.
* Other streams are available for standard source sync configuration and can be run as `INCREMENTAL` or `FULL_TABLE` in Nekt.
### Payments Schema Selection
* Enable **Extract Full Payment Data** for the complete payments payload (`/payments`).
* Disable it for a lighter and faster extraction using `/lean/payments`.
## Skills for agents
Asaas connector documentation as plain markdown, for use in AI agent contexts.
# Asana as a data source
Source: https://docs.nekt.com/sources/asana
Bring data from Asana to Nekt.
Asana is a powerful project management and team collaboration platform that helps organizations manage their work, track projects, and achieve their goals. It provides a comprehensive suite of tools for task management, project tracking, team coordination, and workflow optimization.
![]()
## Configuring Asana as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Asana option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need OAuth credentials to connect to Asana. You'll need to provide:
* **Authorization**: Log in with your Asana account and grant permissions for Nekt to access your workspace data.
* **Workspace ID**: The ID of your Asana workspace. You can get it from the Admin URL ([https://app.asana.com/admin/](https://app.asana.com/admin/)), it's the number that appears right next to `https://app.asana.com/admin/`. Alternatively, if you're logged in with your account, you can click [this link](https://app.asana.com/api/1.0/workspaces) and simply copy the `gid` value.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. The available streams include projects, tasks, teams, portfolios, and more.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Asana and their corresponding fields:
Stream for managing projects in your Asana workspace.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the project |
| `name` | String | Name of the project |
| `resource_type` | String | The project type |
Stream for managing tasks within projects.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the task |
| `name` | String | Name/title of the task |
| `resource_type` | String | The resource type (task) |
| `resource_subtype` | String | The resource subtype (e.g., default\_task) |
| `created_by` | Object | User who created the task (includes gid, resource\_type) |
| `approval_status` | String | The approval status of the task |
| `assignee_status` | String | Status of the task for the assignee |
| `completed` | Boolean | Whether the task is completed |
| `completed_at` | String | When the task was completed |
| `completed_by` | Object | User who completed the task (includes gid, resource\_type, name) |
| `created_at` | String | When the task was created |
| `dependencies` | Array | Array of tasks this task depends on |
| `dependents` | Array | Array of tasks that depend on this task |
| `due_at` | String | The due time for the task |
| `due_on` | String | The due date for the task |
| `html_notes` | String | Task notes in HTML format |
| `notes` | String | Task notes in plain text |
| `hearted` | Boolean | Whether the task is hearted/favorited |
| `hearts` | Array | Array of hearts on the task (includes user details) |
| `is_rendered_as_separator` | Boolean | Whether the task is rendered as a separator |
| `liked` | Boolean | Whether the task is liked |
| `likes` | Array | Array of likes on the task (includes user details) |
| `modified_at` | String | When the task was last modified |
| `num_hearts` | Integer | Number of hearts on the task |
| `num_likes` | Integer | Number of likes on the task |
| `num_subtasks` | Integer | Number of subtasks |
| `start_at` | String | The start time for the task |
| `start_on` | String | The start date for the task |
| `actual_time_minutes` | Number | The actual time spent on the task in minutes |
| `assignee` | Object | User assigned to the task (includes gid, resource\_type, name) |
| `assignee_section` | Object | Section this task is assigned to (includes gid, resource\_type, name) |
| `custom_fields` | Array | Array of custom fields with detailed information (configuration and values) |
| `custom_type` | Object | Custom type for this task (includes gid, resource\_type, name) |
| `custom_type_status_option` | Object | Custom type status option (includes gid, resource\_type, name) |
| `followers` | Array | Array of users following this task (includes gid, resource\_type, name) |
| `parent` | Object | Parent task details (includes gid, name, resource\_type, created\_by) |
| `projects` | Array | Array of projects this task belongs to (includes gid, resource\_type, name) |
| `tags` | Array | Array of tags applied to this task (includes gid, resource\_type, name) |
| `workspace` | Object | Workspace this task belongs to (includes gid, resource\_type, name) |
| `permalink_url` | String | Direct URL to the task in Asana |
Stream for managing subtasks within tasks.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the subtask |
| `task_id` | String | Parent task ID from context |
| `resource_type` | String | The resource type (task) |
| `name` | String | Name of the subtask |
| `resource_subtype` | String | The resource subtype (e.g., default\_task) |
| `created_by` | Object | User who created this subtask |
| `approval_status` | String | Approval status of the subtask |
| `assignee_status` | String | Assignee status of the subtask |
| `completed` | Boolean | Whether the subtask is completed |
| `completed_at` | String | When the subtask was completed |
| `completed_by` | Object | User who completed this subtask |
| `created_at` | String | When the subtask was created |
| `dependencies` | Array | Array of tasks this subtask depends on |
| `dependents` | Array | Array of tasks that depend on this subtask |
| `due_at` | String | Due time for the subtask |
| `due_on` | String | Due date for the subtask |
| `html_notes` | String | Subtask notes in HTML format |
| `notes` | String | Subtask notes in plain text |
| `hearted` | Boolean | Whether the current user hearted the subtask |
| `hearts` | Array | Array of hearts on the subtask |
| `is_rendered_as_separator` | Boolean | Whether rendered as a separator |
| `liked` | Boolean | Whether the current user liked the subtask |
| `likes` | Array | Array of likes on the subtask |
| `modified_at` | String | When the subtask was last modified |
| `num_hearts` | Integer | Number of hearts on the subtask |
| `num_likes` | Integer | Number of likes on the subtask |
| `num_subtasks` | Integer | Number of subtasks of this subtask |
| `start_at` | String | Start time for the subtask |
| `start_on` | String | Start date for the subtask |
| `actual_time_minutes` | Number | Actual time spent in minutes |
| `assignee` | Object | User assigned to this subtask |
| `assignee_section` | Object | Section this subtask is assigned to |
| `custom_fields` | Array | Array of custom fields with detailed information |
| `custom_type` | Object | Custom type for this subtask |
| `custom_type_status_option` | Object | Custom type status option |
| `followers` | Array | Array of users following this subtask |
| `parent` | Object | Parent task details |
| `projects` | Array | Array of projects this subtask belongs to |
| `tags` | Array | Array of tags applied to this subtask |
| `workspace` | Object | Workspace this subtask belongs to |
| `permalink_url` | String | Direct URL to the subtask in Asana |
Stream for managing teams in your workspace.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the team |
| `name` | String | Name of the team |
| `resource_type` | String | The resource type (team) |
| `description` | String | Team description |
| `html_description` | String | Team description in HTML format |
| `organization` | Object | Organization this team belongs to (includes gid, resource\_type, name) |
| `visibility` | String | Visibility of the team (public, private) |
| `created_at` | String | When this team was created |
| `updated_at` | String | When this team was last updated |
| `permalink_url` | String | Direct URL to the team in Asana |
Stream for managing users in your workspace.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the user |
| `name` | String | User's full name |
| `resource_type` | String | The resource type (user) |
| `email` | String | User's email address |
| `photo` | Object | User's profile photo URLs in various sizes |
| `workspaces` | Array | Array of workspaces the user belongs to (includes gid, resource\_type, name) |
Stream for managing tags used to categorize tasks.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the tag |
| `name` | String | Name of the tag |
| `resource_type` | String | The resource type (tag) |
| `color` | String | Color assigned to the tag |
| `created_at` | String | When the tag was created |
| `created_by` | Object | User who created this tag (includes gid, resource\_type, name) |
| `notes` | String | Tag description/notes |
| `followers` | Array | Array of users following this tag (includes gid, resource\_type, name) |
| `workspace` | Object | Workspace this tag belongs to (includes gid, resource\_type, name) |
| `permalink_url` | String | Direct URL to the tag in Asana |
Stream for tracking activity and comments on tasks.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the story |
| `task_id` | String | The parent task ID this story belongs to |
| `resource_type` | String | The resource type (story) |
| `created_at` | String | When the story was created |
| `created_by` | Object | User who created the story (includes gid, resource\_type, name) |
| `resource_subtype` | String | The resource subtype (e.g., comment\_added) |
| `text` | String | The plain text of the story |
| `html_text` | String | The HTML text of the story |
| `is_editable` | Boolean | Whether the story is editable |
| `target` | Object | Target object of this story (includes gid, resource\_type, name) |
| `action_taken` | String | The action taken |
| `old_resource_subtype` | String | The old resource subtype |
| `new_resource_subtype` | String | The new resource subtype |
| `old_text` | String | The old text |
| `new_text` | String | The new text |
| `old_name` | String | The old name |
| `new_name` | String | The new name |
| `old_dates` | Object | Previous dates (due\_on, due\_at, start\_on, start\_at) |
| `new_dates` | Object | Updated dates (due\_on, due\_at, start\_on, start\_at) |
| `old_number_value` | Number | The old number value |
| `new_number_value` | Number | The new number value |
| `old_enum_value` | Object | Previous enum value details |
| `new_enum_value` | Object | Updated enum value details |
| `old_multi_enum_values` | Array | Previous multi-enum values |
| `new_multi_enum_values` | Array | Updated multi-enum values |
| `old_custom_field_value` | Object | Previous custom field value details |
| `new_custom_field_value` | Object | Updated custom field value details |
| `previews` | Array | Array of previews for this story (includes text, links, headers) |
| `attachments` | Array | Array of attachments (includes metadata and URLs) |
Stream for managing custom fields in your workspace.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the custom field |
| `resource_type` | String | The resource type (custom\_field) |
| `resource_subtype` | String | The resource subtype (e.g., text, number, enum) |
| `name` | String | Name of the custom field |
| `description` | String | Description of the custom field |
| `type` | String | Type of the custom field (text, number, enum, multi\_enum, date, people) |
| `enum_options` | Array | Array of possible values for enum custom fields |
| `enabled` | Boolean | Whether the custom field is enabled |
| `representation_type` | String | The representation type of the custom field |
| `id_prefix` | String | The ID prefix for the custom field |
| `is_formula_field` | Boolean | Whether it's a formula field |
| `number_value` | Number | The number value for number custom fields |
| `text_value` | String | The text value for text custom fields |
| `display_value` | String | The display value for the custom field |
| `date_value` | Object | The date value for date custom fields (date and date\_time) |
| `enum_value` | Object | The enum value for enum custom fields |
| `multi_enum_values` | Array | Array of values for multi\_enum custom fields |
| `people_value` | Array | Array of users for people custom fields |
| `created_by` | Object | User who created the custom field |
| `precision` | Integer | Number of decimal places for number fields |
| `format` | String | Format of the custom field |
| `currency_code` | String | Currency code for currency fields |
| `custom_label` | String | Custom label for the field |
| `custom_label_position` | String | Position of custom label (prefix, suffix) |
| `is_global_to_workspace` | Boolean | Whether global to workspace |
| `has_notifications_enabled` | Boolean | Whether notifications are enabled |
| `asana_created_field` | String | The Asana created field type |
| `is_value_read_only` | Boolean | Whether the value is read-only |
| `privacy_setting` | String | Privacy setting for the custom field |
| `default_access_level` | String | Default access level for the custom field |
Stream for managing portfolios that group related projects.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the portfolio |
| `name` | String | Name of the portfolio |
| `resource_type` | String | The portfolio type |
| `archived` | Boolean | Whether the portfolio is archived |
| `color` | String | Color assigned to the portfolio |
| `created_at` | String | When the portfolio was created |
| `created_by` | Object | User who created the portfolio (includes gid, resource\_type, name) |
| `custom_field_settings` | Array | Array of custom field settings with detailed configuration |
| `current_status_update` | Object | Current status update details (includes title and type) |
| `due_on` | String | Due date for the portfolio |
| `custom_fields` | Array | Array of custom fields with values and configuration |
| `members` | Array | Array of users who are members of this portfolio |
| `owner` | Object | Owner of this portfolio (includes gid, resource\_type, name) |
| `start_on` | String | Start date for the portfolio |
| `workspace` | Object | Workspace this portfolio belongs to (includes gid, resource\_type, name) |
| `permalink_url` | String | Direct URL to the portfolio in Asana |
| `public` | Boolean | Whether the portfolio is public to its team |
| `default_access_level` | String | Default access level for the portfolio |
| `privacy_setting` | String | Privacy setting for the portfolio |
| `project_templates` | Array | Array of project templates associated with this portfolio |
Stream for managing items within portfolios.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the project/item |
| `resource_type` | String | The resource type (project) |
| `name` | String | The project/item name |
| `portfolio_id` | String | The portfolio ID this item belongs to |
Stream for managing custom fields specific to portfolios.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the custom field setting |
| `resource_type` | String | The resource type (custom\_field\_setting) |
| `project` | Object | Project this setting belongs to (if applicable) |
| `portfolio` | Object | Portfolio this setting belongs to |
| `custom_field` | Object | Detailed custom field information |
| `is_important` | Boolean | Whether this custom field is marked as important |
| `parent` | Object | Parent portfolio/project for this setting |
| `created_at` | String | When this setting was created |
| `created_by` | Object | User who created this setting |
| `portfolio_id` | String | Portfolio ID this setting belongs to |
Stream for managing custom fields specific to projects.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the custom field setting |
| `resource_type` | String | The resource type (custom\_field\_setting) |
| `project` | Object | Project this setting belongs to |
| `custom_field` | Object | Detailed custom field information |
| `is_important` | Boolean | Whether this custom field is marked as important |
| `parent` | Object | Parent project for this setting |
| `created_at` | String | When this setting was created |
| `created_by` | Object | User who created this setting |
| `project_id` | String | Project ID this setting belongs to |
Stream for tracking project status updates.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the status update |
| `project_id` | String | The parent project ID this status belongs to |
| `resource_type` | String | The resource type (project\_status) |
| `title` | String | Title of the status update |
| `text` | String | Text content of the status update |
| `html_text` | String | HTML content of the status update |
| `color` | String | Color of the status |
| `created_at` | String | When the status was created |
| `created_by` | Object | User who created the status (includes gid, resource\_type, name) |
| `author` | Object | Author of the status (includes gid, resource\_type, name) |
| `modified_at` | String | When the status was last modified |
| `status_type` | String | Type of status (on\_track, at\_risk, off\_track, on\_hold, complete) |
| `liked` | Boolean | Whether the current user has liked this status |
| `likes` | Array | Array of users who liked this status |
| `num_likes` | Integer | Number of likes on this status |
| `hearts` | Array | Array of users who hearted this status |
| `num_hearts` | Integer | Number of hearts on this status |
Stream for tracking time entries on tasks.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the time entry |
| `resource_type` | String | The resource type (time\_tracking\_entry) |
| `created_at` | String | When the time entry was created |
| `created_by` | Object | User who created the entry (includes gid, resource\_type, name) |
| `completed` | Boolean | Whether the entry is completed |
| `completed_at` | String | When the entry was completed |
| `duration_minutes` | Integer | Duration of the entry in minutes |
| `notes` | String | Notes for the time entry |
| `html_notes` | String | Notes in HTML format |
| `user` | Object | User who performed this time tracking |
| `task` | Object | Task this entry is associated with (includes gid, resource\_type, name) |
| `project` | Object | Project this entry is associated with (includes gid, resource\_type, name) |
| `workspace` | Object | Workspace this entry belongs to (includes gid, resource\_type, name) |
| `time_period` | Object | Time period details |
Stream for managing resource allocations.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the allocation |
| `project_id` | String | The project ID from context |
| `resource_type` | String | The resource type (allocation) |
| `resource_subtype` | String | The resource subtype (e.g., project\_allocation) |
| `start_date` | String | Start date of the allocation period |
| `end_date` | String | End date of the allocation period |
| `effort` | Object | Effort details (type and value) |
| `assignee` | Object | User assigned to this allocation (includes gid, name, resource\_type) |
| `created_by` | Object | User who created this allocation (includes gid, name, resource\_type) |
| `parent` | Object | Parent project this allocation belongs to (includes gid, name, resource\_type) |
Stream for managing organizational goals.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `gid` | String | Unique identifier for the goal |
| `resource_type` | String | The resource type (goal) |
| `name` | String | Name of the goal |
| `html_notes` | String | Goal notes in HTML format |
| `notes` | String | Goal notes in plain text |
| `due_on` | String | Due date for the goal |
| `start_on` | String | Start date for the goal |
| `is_workspace_level` | Boolean | Whether the goal is workspace-level |
| `liked` | Boolean | Whether the current user liked the goal |
| `likes` | Array | Array of likes on the goal |
| `num_likes` | Integer | Number of likes on the goal |
| `team` | Object | Team associated with this goal (includes gid, resource\_type, name) |
| `workspace` | Object | Workspace this goal belongs to (includes gid, resource\_type, name) |
| `followers` | Array | Array of users following this goal |
| `time_period` | Object | Time period details |
| `metric` | Object | Associated metric details (values and configuration) |
| `owner` | Object | Owner of this goal (includes gid, resource\_type, name) |
| `current_status_update` | Object | Current status update details |
| `status` | String | Status of the goal (green, yellow, red) |
# Data Model
The following diagram illustrates the relationships between the core data streams in Asana. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Organization"
Teams("Teams");
Users("Users");
CustomFields("Custom Fields");
end
subgraph "Project Management"
Portfolios("Portfolios");
PortfolioItems("Portfolio Items");
PortfolioCustomFields("Portfolio Custom Fields");
Projects("Projects");
ProjectCustomFields("Project Custom Fields");
ProjectStatuses("Project Statuses");
Allocations("Allocations");
end
subgraph "Task Management"
Tasks("Tasks");
Subtasks("Subtasks");
Stories("Stories");
Tags("Tags");
TimeTracking("Time Tracking Entries");
end
subgraph "Strategy"
Goals("Goals");
end
PortfolioItems -- "portfolio_id" --> Portfolios;
PortfolioCustomFields -- "portfolio_id" --> Portfolios;
ProjectCustomFields -- "project_id" --> Projects;
ProjectStatuses -- "project_id" --> Projects;
Allocations -- "project_id" --> Projects;
Tasks -- "project_id" --> Projects;
Tasks -- "assignee" --> Users;
Tasks -- "tags" --> Tags;
Subtasks -- "task_id" --> Tasks;
Stories -- "task_id" --> Tasks;
TimeTracking -- "task" --> Tasks;
Goals -- "team" --> Teams;
```
## Implementation Notes
### Rate Limiting and Plan Detection
The Asana API enforces request rate limits based on the workspace plan (150 requests per minute for free workspaces, and 1,500 for paid workspaces). The connector automatically detects the workspace's plan at the start of a run and strictly limits its request rate to 80% of the allowed capacity to prevent API blocks and ensure stability. In the event of a `429 Too Many Requests` response, the connector respects the `Retry-After` header and temporarily pauses operations before automatically resuming the sync.
### Performance Optimizations
To maximize throughput and significantly minimize network waiting time (especially on workspaces with a massive amount of nested tasks), the connector fetches per-task child streams (such as Subtasks and Stories) in parallel using a thread pool.
Additionally, it optimizes the subtask extraction process by intelligently skipping the API requests for tasks that report having exactly zero subtasks (`num_subtasks: 0`).
### Pagination and Sync Behavior
* **Project Stream Pagination**: The connector fetches all pages of the `Projects` stream upfront before syncing any nested child streams (such as Tasks, Subtasks, Stories, etc.). In Asana workspaces with large amounts of nested data, syncing children inline can cause the Asana pagination token to expire, failing the sync with a `400 "pagination token has expired"` error. Consuming the pagination fully in advance keeps API requests close together and prevents this expiration, ensuring data extraction reliability.
* **OAuth Token Refresh and Retries**: The connector proactively refreshes OAuth access tokens well ahead of their expiration (5 minutes prior) to avoid racing the expiration boundary. If an unexpected `401 Unauthorized` token expiration response occurs during a long sync, the connector automatically invalidates the expired token, fetches a fresh one, and seamlessly retries the request without interrupting the extraction.
## Skills for agents
Asana connector documentation as plain markdown, for use in AI agent contexts.
# AskSuite as a data source
Source: https://docs.nekt.com/sources/asksuite
Bring data from AskSuite to Nekt.
AskSuite is a conversational AI platform designed for the hospitality industry. It provides AI-powered chatbots and automation tools that help hotels manage guest inquiries, generate leads, and streamline communication across multiple channels including web chat, WhatsApp, and social media.
![]()
## Configuring AskSuite as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the AskSuite option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your AskSuite Integration User Token for this connection. This token authenticates your requests to the AskSuite API.
#### Obtaining Your Integration User Token
Log in to your AskSuite account at [control.asksuite.com](https://control.asksuite.com).
Go to **Menu** > **Integrações** (Integrations).
Locate or generate your Integration User Token. This token is used to authenticate API requests.
Store your token securely. Treat it like a password and do not share it publicly.
#### Configuration Fields
The following configurations are available:
* **Integration User Token**: The token to authenticate against the AskSuite API service. You can find your token at Menu > Integrações in AskSuite.
* **Company IDs**: The company IDs to filter conversation leads. You can specify multiple company IDs to extract data from multiple properties.
* **Start Date**: The earliest date from which records will be synced. This is used for incremental sync of conversation leads.
Company IDs allow you to filter conversations by specific properties or hotels in your AskSuite account. This is useful when managing multiple properties.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. AskSuite provides access to conversation data, tasks, tags, and messaging templates.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from AskSuite and their corresponding fields:
Stream containing conversation leads from your AskSuite chatbot interactions. This stream supports **incremental sync** based on the `updatedAt` field.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the conversation record |
| `conversationId` | String | Unique identifier for the conversation |
| `groupId` | String | Group identifier |
| `companyId` | String | Company identifier |
| `companyName` | String | Company name |
| `name` | String | Customer name |
| `email` | String | Customer email |
| `phone` | String | Customer phone |
| `city` | String | City |
| `state` | String | State or region |
| `country` | String | Country |
| `language` | String | Language code (e.g., pt-br) |
| `custom` | String | Custom field |
| `platform` | String | Platform where conversation originated |
| `source` | String | Source of the conversation (e.g., chat\_web) |
| `humanRequest` | Boolean | Whether human assistance was requested |
| `attendant` | String | Attendant name |
| `attendantEmail` | String | Attendant email |
| `linkToLiveChat` | String | Link to live chat |
| `page` | String | Page URL where conversation started |
| `arrivalDate` | String | Arrival date |
| `departureDate` | String | Departure date |
| `adults` | Integer | Number of adults |
| `children` | Integer | Number of children |
| `requestPrice` | Boolean | Whether price was requested |
| `linkQuotationOpened` | Boolean | Whether quotation link was opened |
| `hasReservation` | Boolean | Whether conversation has reservation |
| `groupQuote` | Boolean | Whether it's a group quote |
| `eventQuote` | Boolean | Whether it's an event quote |
| `totalReservation` | Number | Total reservation value |
| `lastReservationDate` | Integer | Last reservation date as timestamp |
| `lastReservationDateFormatted` | String | Last reservation date formatted |
| `lastReservationCurrency` | String | Last reservation currency |
| `lastReservationValue` | Number | Last reservation value |
| `lastReservationCode` | String | Last reservation code |
| `lastReservationSalesman` | String | Last reservation salesman |
| `minTotalValuePriceQuote` | Number | Minimum total value price quote |
| `maxTotalValuePriceQuote` | Number | Maximum total value price quote |
| `currencyPriceQuote` | String | Currency for price quote |
| `tags` | Array | List of tag names |
| `tagsString` | String | Tags as string |
| `events` | Array | List of event IDs |
| `utmSource` | String | UTM source parameter |
| `utmMedium` | String | UTM medium parameter |
| `utmCampaign` | String | UTM campaign parameter |
| `dateFormatted` | String | Formatted date and time |
| `lastMessageReceivedAt` | Integer | Timestamp of last message received in milliseconds |
| `finished` | Boolean | Whether conversation is finished |
| `unavailable` | Boolean | Whether unavailable |
| `updatedAt` | Integer | Last update timestamp in milliseconds |
| `createdAt` | Integer | Creation timestamp in milliseconds |
Stream containing tasks associated with conversations, such as follow-ups and reminders.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the conversation task |
| `conversationId` | String | Identifier of the conversation |
| `companyId` | String | Company identifier |
| `parentCompanyId` | String | Parent company identifier |
| `genericRuleEngineId` | Integer | Generic rule engine identifier |
| `conversationTaskMessage` | Object | Conversation task message with template details |
| `conversationTaskMessage.templateId` | String | Template ID |
| `conversationTaskMessage.templateCategory` | String | Template category |
| `conversationTaskMessage.companyId` | String | Company ID |
| `conversationTaskMessage.status` | String | Message status |
| `conversationTaskMessage.params` | Array | Array of parameters |
| `conversationTaskMessage.text` | String | Message text |
| `conversationTaskMessage.image` | String | Message image |
| `conversationTaskMessage.sentAt` | Integer | Sent timestamp |
| `conversationTaskMessage.deliveredAt` | Integer | Delivered timestamp |
| `conversationTaskMessage.answeredAt` | Integer | Answered timestamp |
| `conversationTaskMessage.failedAt` | Integer | Failed timestamp |
| `conversationTaskMessage.buttons` | Array | Message buttons |
| `conversationTaskMessage.quickReplies` | Array | Quick replies |
| `message` | Object | Message object with template ID and status |
| `message.text` | String | Message text |
| `message.image` | String | Message image |
| `message.params` | Array | Message parameters |
| `message.buttons` | Array | Message buttons |
| `message.quickReplies` | Array | Quick replies |
| `message.sentAt` | Integer | Sent timestamp |
| `message.failedAt` | Integer | Failed timestamp |
| `message.status` | String | Message status |
| `message.companyId` | String | Company ID |
| `message.answeredAt` | Integer | Answered timestamp |
| `message.templateId` | String | Template ID |
| `message.deliveredAt` | Integer | Delivered timestamp |
| `message.templateCategory` | String | Template category |
| `title` | String | Task title |
| `description` | String | Task description |
| `status` | String | Task status (e.g., TODO, DONE, EXPIRED, CANCELED) |
| `userCreatorId` | Integer | ID of the user who created the task |
| `userAssign` | Integer | ID of the user assigned to the task |
| `externalData` | Object | External data with lead and user information |
| `externalData.lead.name` | String | Lead name |
| `externalData.lead.email` | String | Lead email |
| `externalData.lead.phone` | String | Lead phone |
| `externalData.user.id` | Integer | User ID |
| `createdAt` | Integer | Creation timestamp in milliseconds |
| `updatedAt` | Integer | Last update timestamp in milliseconds |
| `expirationDate` | Integer | Expiration date as timestamp in milliseconds |
Stream containing all tags used to categorize and organize conversations.
| Field | Type | Description |
| :- | :- | :- |
| `tagId` | Integer | Unique identifier for the tag |
| `name` | String | Tag name |
| `color` | String | Tag color in hex format |
| `type` | String | Tag type (e.g., custom) |
| `status` | String | Tag status (e.g., active) |
| `active` | Boolean | Whether the tag is active |
| `standard` | Boolean | Whether the tag is standard |
| `visibleToClients` | Boolean | Whether the tag is visible to clients |
| `companyId` | String | Company identifier |
| `companiesIds` | Array | List of company IDs associated with the tag |
| `disabledCompaniesIds` | Array | List of disabled company IDs |
| `translations` | Object | Translations for the tag |
| `translations.en.TAG` | String | Tag translation in English |
| `translations.en.INFO` | String | Info translation in English |
| `translations.en.NOTIFICATION` | String | Notification translation in English |
| `translations.es.TAG` | String | Tag translation in Spanish |
| `translations.es.INFO` | String | Info translation in Spanish |
| `translations.es.NOTIFICATION` | String | Notification translation in Spanish |
| `translations.pt_br.TAG` | String | Tag translation in Brazilian Portuguese |
| `translations.pt_br.INFO` | String | Info translation in Brazilian Portuguese |
| `translations.pt_br.NOTIFICATION` | String | Notification translation in Brazilian Portuguese |
| `triggeredBy` | String | What triggered the tag |
| `createdAt` | Integer | Creation timestamp in milliseconds |
Stream containing broadcast/transmission lists for WhatsApp campaigns and bulk messaging.
| Field | Type | Description |
| :- | :- | :- |
| `transmissionListId` | String | Unique identifier for the transmission list |
| `name` | String | Name of the transmission list |
| `templateId` | String | Template identifier used for this transmission |
| `companyId` | String | Company identifier |
| `status` | String | Status of the transmission (e.g., SENT) |
| `messages` | Array | List of messages in the transmission |
| `messages.phone` | String | Recipient phone number |
| `messages.params` | Array | Array of parameters |
| `messages.channelRulesCheck` | Boolean | Channel rules check status |
| `failedMessages` | String | Failed messages, if any |
| `stopBot` | Boolean | Whether to stop the bot |
| `tags` | String | Tags associated with the transmission as key-value pairs |
| `trackData` | String | Tracking data |
| `actions` | String | Actions associated with the transmission |
| `createdAt` | Integer | Creation timestamp in milliseconds |
| `isCampaign` | Boolean | Whether this is a campaign |
| `expirationDate` | Integer | Expiration date as timestamp in milliseconds |
| `userCreatorId` | Integer | ID of the user who created the transmission |
Stream containing WhatsApp message templates used for automated and campaign messaging.
| Field | Type | Description |
| :- | :- | :- |
| `idMessageTemplate` | Integer | Unique identifier for the message template |
| `title` | String | Template title |
| `content` | String | Template content |
| `companyId` | String | Company identifier |
| `language` | String | Template language code |
| `category` | String | Template category (e.g., ACCOUNT\_UPDATE) |
| `templateType` | String | Template type (e.g., CAMPAIGN) |
| `status` | String | Template status (e.g., active) |
| `platformStatus` | String | Platform status (e.g., APPROVED) |
| `platform` | String | Platform name (e.g., GUPSHUP) |
| `platformId` | String | Platform identifier |
| `platformContent` | String | Platform-specific content |
| `reason` | String | Reason for template status |
| `buttons` | Array | List of buttons in the template |
| `buttons.id` | String | Button ID |
| `buttons.label` | String | Button label |
| `buttons.type` | String | Button type |
| `buttons.defaultValue` | String | Default value |
| `quickReplies` | Array | List of quick replies |
| `parameters` | Array | List of template parameters |
| `parameters.id` | String | Parameter ID |
| `parameters.name` | String | Parameter name |
| `parameters.type` | String | Parameter type |
| `parameters.visible` | Boolean | Whether parameter is visible |
| `parameters.contentReplace` | String | Content to replace |
| `parameters.dataSourceField` | String | Data source field |
| `image` | String | Template image URL or data |
| `userCreatorId` | Integer | ID of the user who created the template |
| `createdAt` | Integer | Creation timestamp in milliseconds |
| `lastCategoryUpdateAt` | Integer | Last category update timestamp in milliseconds |
| `categoryUpdates` | Integer | Number of category updates |
# Data Model
The following diagram illustrates the relationships between the core data streams in AskSuite. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
erDiagram
Conversations ||--o{ ConversationTasks : "conversationId"
Conversations ||--o{ Tags : "tagsIds"
TransmissionList ||--o| WhatsAppMessageTemplates : "templateId"
Conversations {
int id PK
string conversationId
string companyId
int updatedAt
string name
string email
string phone
}
ConversationTasks {
int id PK
string conversationId FK
string companyId
string status
int expirationDate
}
Tags {
int tagId PK
string companyId
string name
string color
}
TransmissionList {
string transmissionListId PK
string templateId FK
string companyId
string status
}
WhatsAppMessageTemplates {
int idMessageTemplate PK
string title
string content
string language
}
```
## Implementation Notes
### Data Sync Considerations
* **Incremental Sync**: The `Conversations` stream supports incremental sync using the `updatedAt` field. Other streams (Tags, Conversation Tasks, Transmission List, WhatsApp Message Templates) are extracted as full table syncs.
* **Timestamps**: All timestamps in AskSuite data are stored as milliseconds since epoch. You may need to convert these to standard datetime formats for analysis.
* **Company Filtering**: The `company_ids` configuration allows you to filter data by specific properties. Make sure to include all relevant company IDs to capture complete data.
### Multi-Property Support
If you manage multiple hotels or properties in AskSuite, you can specify multiple Company IDs in the configuration. This will extract conversation data from all specified properties in a single sync.
## Skills for agents
AskSuite connector documentation as plain markdown, for use in AI agent contexts.
# Atendare as a data source
Source: https://docs.nekt.com/sources/atendare
Bring data from Atendare to Nekt.
Atendare is a customer service and support management platform designed to help businesses handle customer inquiries, support tickets, and service requests efficiently. It provides tools for ticket management, customer communication, and service analytics to improve customer satisfaction.
## Configuring Atendare as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Atendare** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Atendare credentials to authorize Nekt to access your data.
The following configurations are available:
* **Access Token:** The token to authenticate against the API service. Get in touch with Atendare support team to get a token for your account. For detailed instructions, [check their documentation](https://atendare.readme.io/reference/authentication).
* **Start Date:** The date of the first record you want to sync with. We will extract every record created or updated after this. Format: `YYYY-MM-DD`.
Under **Advanced settings**, you can optionally turn on the file streams:
* **Enable contract files stream:** When enabled, the source adds a `contract_files` stream that downloads every contract's PDF document and uploads it to a Nekt volume. This makes extractions slower and heavier, since each PDF is downloaded from Atendare and re-uploaded to Nekt. Only enable this if you need the contract files stored in a Nekt volume.
* **Contract Files Volume:** The Nekt volume to upload contract PDFs to. Required when the contract files stream is enabled.
* **Enable proposal files stream:** When enabled, the source adds a `proposal_files` stream that downloads every proposal's PDF document and uploads it to a Nekt volume. Same performance caveat as above. Only enable this if you need the proposal files stored in a Nekt volume.
* **Proposal Files Volume:** The Nekt volume to upload proposal PDFs to. Required when the proposal files stream is enabled.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
When you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Skills for agents
Atendare connector documentation as plain markdown, for use in AI agent contexts.
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Authors | `authors` | Authors (content or record authors) registered in Atendare. |
| Companies | `companies` | Companies (accounts) managed in Atendare. |
| Contacts | `contacts` | Contacts (people) associated with companies and deals. |
| Contracts | `contracts` | Contracts (agreements) tied to each deal. |
| Contract Files | `contract_files` | PDF document of each contract, uploaded to a Nekt volume. Only available when **Enable contract files stream** is on. |
| Deals | `deals` | Sales deals (opportunities) tracked in Atendare. |
| Proposals | `proposals` | Proposals (budgets) generated for each deal. |
| Proposal Files | `proposal_files` | PDF document of each proposal, uploaded to a Nekt volume. Only available when **Enable proposal files stream** is on. |
| Qualifications | `qualifications` | Qualification records used to score and classify deals. |
| Users | `users` | Atendare users (team members) in the account. |
# Streams and Fields
Below you'll find the available documented data streams from Atendare and their key fields:
Stream containing deal records. This serves as the parent stream for proposals and contracts.
Stream for retrieving budgets/proposals associated with your Deals (`/deals/{deal_id}/budgets`).
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the proposal |
| `update_date` | Timestamp | Date and time the proposal was last updated. Used as the replication key for incremental merges. |
| `deal_id` | Integer | ID of the associated deal. |
| `model_id` | Integer | ID of the proposal model. |
| `model` | Object | Model object containing `id`, `name`, and `author_id`. |
| `name` | String | Name of the proposal. |
| `creation_date` | DateTime | Date and time the proposal was created. |
| `start_date` | DateTime | Start date of the proposal. |
| `end_date` | DateTime | End date of the proposal. |
| `signature_date` | DateTime | Date the proposal was signed. |
| `status` | Integer | Status code of the proposal. |
| `total` | Number | Total value of the proposal. |
| `total_discount` | Number | Total discount applied to the proposal. |
| `description` | String | Description of the proposal. |
| `version` | Integer | Version number of the proposal. |
| `link` | String | Link to the proposal. |
| `author` | Object | Author details including nested user information. |
Stream for retrieving agreements/contracts associated with your Deals (`/deals/{deal_id}/agreements`).
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the contract |
| `update_date` | Timestamp | Date and time the contract was last updated. Used as the replication key for incremental merges. |
| `deal_id` | Integer | ID of the associated deal. |
| `model_id` | Integer | ID of the contract model. |
| `model` | Object | Model object containing `id`, `name`, and `author_id`. |
| `name` | String | Name of the contract. |
| `creation_date` | DateTime | Date and time the contract was created. |
| `start_date` | DateTime | Start date of the contract. |
| `end_date` | DateTime | End date of the contract. |
| `signature_date` | DateTime | Date the contract was signed. |
| `status` | Integer | Status code of the contract. |
| `total` | Number | Total value of the contract. |
| `total_discount` | Number | Total discount applied to the contract. |
| `description` | String | Description of the contract. |
| `version` | Integer | Version number of the contract. |
| `link` | String | Link to the contract. |
| `author` | Object | Author details including nested user information. |
PDF documents of each deal's contracts, downloaded through `/deals/{deal_id}/agreements/{contract_id}/pdf` and uploaded to a Nekt volume. Child stream of `Contracts`, only registered when **Enable contract files stream** is on. It emits one metadata record per uploaded file so the original link to volume file mapping can be rebuilt downstream.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `contract_id` | Integer | ID of the contract the uploaded PDF belongs to. Primary key. |
| `deal_id` | Integer | ID of the deal the contract belongs to. |
| `update_date` | DateTime | When the contract was last updated in Atendare. Replication key. |
| `link` | String | Original Atendare-hosted public document link the PDF was generated for. Use it to map old links to the uploaded file. |
| `_nekt_file_*` | Metadata | Volume attachment metadata fields (such as path and filename) for the uploaded PDF. |
PDF documents of each deal's proposals, downloaded through `/deals/{deal_id}/budgets/{proposal_id}/pdf` and uploaded to a Nekt volume. Child stream of `Proposals`, only registered when **Enable proposal files stream** is on. It emits one metadata record per uploaded file so the original link to volume file mapping can be rebuilt downstream.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `proposal_id` | Integer | ID of the proposal the uploaded PDF belongs to. Primary key. |
| `deal_id` | Integer | ID of the deal the proposal belongs to. |
| `update_date` | DateTime | When the proposal was last updated in Atendare. Replication key. |
| `link` | String | Original Atendare-hosted public document link the PDF was generated for. Use it to map old links to the uploaded file. |
| `_nekt_file_*` | Metadata | Volume attachment metadata fields (such as path and filename) for the uploaded PDF. |
# Data Model
The following diagram illustrates the relationships between deals, their proposals and contracts, and the optional file streams:
```mermaid theme={null}
erDiagram
DEALS {
integer id PK
}
CONTRACTS {
integer id PK
integer deal_id FK
}
PROPOSALS {
integer id PK
integer deal_id FK
}
CONTRACT_FILES {
integer contract_id PK
integer deal_id FK
}
PROPOSAL_FILES {
integer proposal_id PK
integer deal_id FK
}
DEALS ||--o{ CONTRACTS : "has"
DEALS ||--o{ PROPOSALS : "has"
CONTRACTS ||--o| CONTRACT_FILES : "pdf"
PROPOSALS ||--o| PROPOSAL_FILES : "pdf"
```
# Implementation Notes
### Incremental Sync Behavior
* **Child Streams (Proposals & Contracts):** These streams act as child streams to the `Deals` stream. During incremental runs, the connector only traverses the subset of deals emitted by the parent.
* To prevent data loss from untouched deals, `proposals` and `contracts` are strictly configured as `INCREMENTAL` streams using `update_date` as the replication key (and `id` as the primary key). This setup guarantees the target destination handles the sync as a merge/upsert instead of a full table replacement.
* **API Limitations:** Note that the specific endpoints for proposals and contracts do not natively support query parameters for filtering by date or paging. Records arrive grouped per deal and are not globally sorted by `update_date`. The replication key is set purely to dictate the target's merge behavior.
### Sync Behaviors
* **Full Sync Integration:** When a stream's sync method is forced to `FULL_TABLE` (such as during a manual resync or an Additional Full Sync event), the connector automatically identifies this and bypasses any existing incremental date filters. This ensures all historical records are retrieved unfiltered without encountering API parameter errors.
### File Streams Behavior
* **Opt-in only:** The `contract_files` and `proposal_files` streams are not registered unless their corresponding **Enable ... files stream** flag is turned on in the source configuration, so accounts that don't need volume uploads are unaffected (no extra catalog entries, no per-record child syncs).
* **Why they exist:** Atendare hosts each generated document behind a public link (`https://web.atendare.com/public/docs/...`) that stops working if the account is decommissioned. The file streams preserve the documents in a Nekt volume and emit the original `link` alongside the `_nekt_file_*` columns so the old link to volume file mapping can be rebuilt downstream (for example, to regenerate links in a CRM).
* **Performance impact:** Enabling these streams makes extractions slower and heavier. The connector downloads each PDF (which can reach several MB) individually from the API's `/pdf` endpoints and uploads it to the designated Nekt volume.
* **Deduplication:** Uploaded files use a deterministic volume file name (`contract_{id}.pdf` or `proposal_{id}.pdf`), so subsequent runs overwrite the previous file in place instead of creating duplicates.
* **Replication strategy:** These are child streams of `contracts` and `proposals`, which are themselves incremental, so each run only downloads and uploads PDFs for the subset of contracts or proposals emitted by the parents. Like their parents, they use `update_date` as the replication key with `contract_id`/`proposal_id` as primary key so the destination merges records instead of replacing the table.
* **Skipped records:** Contracts or proposals without a generated document (no `link`) are skipped, and so is any record whose PDF endpoint answers with a JSON error body instead of a PDF. These skips are logged and do not fail the run.
# Attio as a data source
Source: https://docs.nekt.com/sources/attio
Bring data from Attio to Nekt.
Attio is a modern CRM (Customer Relationship Management) platform designed to help businesses manage their customer
relationships, sales pipelines, and contact data. It provides tools for tracking leads, managing deals, and organizing
customer information in a centralized system.
## 1. Add your Attio access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Attio option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* In order to generate the token you have to go into Settings -> Developers -> Access tokens. You have to give 'Read' access to all the options inside the 'Scopes' section in order for our connector to access the data. For more information, check the official [documentation](https://developers.attio.com/docs/integrations#access-tokens).
3. Click **Next**.
## 2. Select your Attio streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Attio data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Attio data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. In addition to the fixed streams, Attio discovers one stream per object and one per list in your workspace, so the exact set of slugs depends on your workspace configuration.
| Stream | Slug | Description |
| - | - | - |
| Workspace Members | `workspace_members` | Members belonging to the workspace. |
| Lists | `lists` | Lists defined in the workspace. |
| Objects | `objects` | Object definitions available in the workspace. |
| Tasks | `tasks` | Tasks created in the workspace. |
| Notes | `notes` | Notes attached to records. |
| Object Records | `records_{object}` | One stream per object (e.g. `records_people`, `records_companies`); holds the records of that object. |
| List Entries | `list_entries_{list}` | One stream per list (e.g. `list_entries_sales`); holds the entries of that list. |
The `records_{object}` and `list_entries_{list}` slugs are generated dynamically from the object and list `api_slug` values returned by the Attio API (lowercased and underscore-separated).
## Skills for agents
Attio connector documentation as plain markdown, for use in AI agent contexts.
# Autodoc as a data source
Source: https://docs.nekt.com/sources/autodoc
Bring data from Autodoc to your Lakehouse.
Autodoc is a Brazilian construction-industry platform for document management. Its GD4
module controls the legal and fiscal documentation of workers (colaboradores) and
suppliers (fornecedores) across construction sites (obras): required documents, analysis
status, expirations and site-access blocking. Nekt extracts this data through the official
GD4 API.
## Configuring Autodoc as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Autodoc option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are required:
* **API User**: The username of the API user of your GD4 account. An administrator can create it in the GD4 interface, under **Autodoc > Usuário API**. Module permissions for the API user are configured per construtora under **Autodoc > Construtora**; streams the user has no permission for are skipped with a warning.
* **API Password**: The password of that API user.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
The per-worker document streams (`colaboradores_historico` and the four `colaboradores_documentos_*` streams) and per-supplier document streams (`fornecedores_documentos*`) make one API call per worker/supplier, so syncs get slower as your workforce grows. Select only the ones you need.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: the Autodoc API does not expose update timestamps, so all streams are extracted as FULL\_TABLE: every run fetches the current state of the data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Autodoc and their corresponding fields.
Some detailed document payloads are not fully documented by the Autodoc API, so they are delivered as JSON strings (columns `historico`, `documentos` and `fornecedor` on the per-worker/per-supplier streams). You can parse them with SQL JSON functions in your Queries.
Workers registered in GD4, with their documentation status. Full table.
**Key Fields:**
* `id` - Unique identifier of the worker in GD4
* `nome` - Full name of the worker
* `cpf` - CPF (Brazilian individual taxpayer ID) of the worker
* `data_nascimento` - Birth date of the worker
* `data_admissao` - Date the worker was hired
* `data_cadastro` - Date the worker was registered in GD4
* `funcao` - Job function (role) of the worker
* `cargos_servicos` - Positions/services assigned to the worker
* `responsavel` - Name of the supplier (employer) responsible for the worker
* `relationships` - Details of the responsible supplier (JSON object)
* `obras` - Construction sites the worker is linked to
* `analise_documento` - Document analysis status for the worker
* `status_colaborador` - Current documentation status of the worker
* `estado_residencia` / `cidade_residencia` - Where the worker lives
* `facial_cadastrada` - Whether facial recognition is enrolled for turnstile access
Individual history entries of each worker (admissions, transfers, events). Full table. One API call per worker.
**Key Fields:**
* `colaborador_id` - Identifier of the worker this history entry belongs to
* `historico` - History entry of the worker as returned by GD4 (JSON object)
Documents required from each worker, split by requirement type: one-time (único), monthly (mensal), termination (encerramento) and transfer (transferência). Full table. One API call per worker per stream.
**Key Fields:**
* `colaborador_id` - Identifier of the worker the documents belong to
* `documentos` - Document entry with status, due dates and occurrences (JSON object)
Suppliers (contractors) monitored in GD4, with documentation status. Full table.
**Key Fields:**
* `id` - Unique identifier of the supplier in GD4
* `nome_fantasia` - Trade name of the supplier
* `razao_social` - Legal name of the supplier
* `cnpj` - CNPJ (Brazilian company taxpayer ID) of the supplier
* `categoria` - Category of the supplier
* `colaboradores` - Number of workers linked to the supplier
* `status_fornecedor` - Current documentation status of the supplier
* `data_uploads` - Date of the last document upload by the supplier
* `data_analise` - Date of the last document analysis for the supplier
Documents and documentation summary of each supplier, plus the per-type listings (único, mensal, encerramento). Full table. One API call per supplier per stream.
**Key Fields:**
* `fornecedor_id` - Identifier of the supplier
* `fornecedor` - Documentation summary and document list of the supplier (JSON object, summary stream)
* `documentos` - Document entry with status, due dates and occurrences (JSON object, per-type streams)
Administrative supplier registry with contacts, hiring relation and hierarchy. Full table.
**Key Fields:**
* `id` - Unique identifier of the supplier in GD4
* `cnpj` - CNPJ of the supplier
* `razao_social` / `nome_fantasia` - Legal and trade names of the supplier
* `data_vinculo` - Date the supplier was linked to the builder
* `codigo_fornecedor` - Internal code of the supplier
* `colaboradores_vinculados` - Number of workers linked to the supplier
* `contato` / `telefone` / `email` - Contact information of the supplier
* `relacao_contratacao` - Hiring relation of the supplier
* `fornecedor_pai` - Parent supplier when subcontracted
Construction sites managed in GD4. Full table.
**Key Fields:**
* `id` - Unique identifier of the construction site
* `nome` - Name of the construction site
* `endereco` - Address of the construction site
* `cei` - CEI/CNO registration number of the construction site
* `descricao` - Description of the construction site
* `bloqueado` - Whether the construction site is blocked
* `catracas` - Turnstiles installed at the site (JSON array of objects)
GD4 platform users of the builder account. Full table.
**Key Fields:**
* `id` - Unique identifier of the user
* `usuario` - Username (login) of the user
* `bloqueado` - Whether the user is blocked
* `usuario_colaborador` - Worker linked to the user account, when applicable
* `perfil` - Permission profile assigned to the user
Job functions (roles) configured in GD4. Full table.
**Key Fields:**
* `id` - Unique identifier of the job function
* `nome` - Name of the job function
* `codigo` - Numeric code of the job function
* `uuid` - UUID of the job function
* `documentos_vinculados` - Documents required for the job function (JSON array of objects)
* `funcao_vinculada` - Linked job function, when applicable
Positions/services configured in GD4. Full table.
**Key Fields:**
* `id` - Unique identifier of the position/service
* `nome` - Name of the position/service
* `bloqueado` - Whether the position/service is blocked
* `funcao` - Job functions linked to the position/service (JSON array of objects)
* `data_cadastro` - Date the position/service was registered
* `usuario` - User who registered the position/service (JSON object)
Document type definitions (requirement rules) configured in GD4 for workers and suppliers. Full table.
**Key Fields:**
* `id` - Unique identifier of the document type
* `nome` - Name of the document type
* `vencimento` - Expiration rule of the document type
* `tipo_exigencia` - Requirement type of the document (e.g. mensal, único)
* `peso` - Weight of the document in the supplier score (supplier documents)
* `criterio_exigencia` - Requirement criterion (supplier documents)
* `obras` - Construction sites the requirement applies to (supplier documents)
Permission profiles and document analysis profiles configured in GD4. Full table.
**Key Fields:**
* `id` - Unique identifier of the profile
* `nome` - Name of the profile
If you have any questions about the connector, feel free to reach out to us. We are happy to help you get your data flowing!
# Avendre as a data source
Source: https://docs.nekt.com/sources/avendre
Bring data from Avendre to Nekt.
Avendre is a Brazilian real estate platform that connects developers (incorporadoras), brokers (corretores), and real estate agencies (imobiliarias). It provides a comprehensive system for managing developments (empreendimentos), units, interactions, and sales coordination.
![]()
## Configuring Avendre as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Avendre option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a Bearer Token (JWT) to connect to the Avendre API.
The following configurations are available:
* **API Key**: Your JWT bearer token for authenticating with the Avendre API.
* **Start Date**: (Optional) The earliest date from which records will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer, a folder to organize it, a name for each table, and the type of sync.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization.
Define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers) based on how often you need data updated.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Avendre and their corresponding fields:
| Field | Type |
| - | - |
| id\_empreendimento | integer |
| id\_incorporadora | integer |
| incorporadora | string |
| logo\_incorporadora | string |
| nome | string |
| status\_obra | string |
| tipo | string |
| latitude | string |
| longitude | string |
| area\_de | string |
| area\_ate | string |
| cep | string |
| ano\_lancamento | integer |
| segmento | string |
| id\_cidade | integer |
| nome\_cidade | string |
| sigla | string |
| link\_site | string |
| mcmv | string |
| endereco | string |
| bairro | string |
| numero | string |
| complemento | string |
| quartos\_de | integer |
| quartos\_ate | integer |
| banheiros\_de | integer |
| banheiros\_ate | integer |
| vagas\_de\_garagem\_de | integer |
| vagas\_de\_garagem\_ate | integer |
| qtd\_unidades | integer |
| slug | string |
| oportunidades\_disponiveis | integer |
| unidades\_cadastradas | integer |
| total\_interesses | integer |
| total\_acessos | integer |
| caracteristicas | string (JSON) |
| comissao\_ate | integer |
| crm\_origem | string |
| galeria | string (JSON) |
| exibir\_valores\_unidades\_formas\_de\_pagamento | string |
| Field | Type |
| - | - |
| id\_unidade | integer |
| id\_empreendimento | integer |
| status | string |
| nome | string |
| ativo | string |
| data\_cad | string |
| area | number |
| posicao | string |
| andar | integer |
| quartos | integer |
| banheiros | integer |
| vagas\_garagem | integer |
| bloco | string |
| coluna | string |
| quadra | string |
| tipo | string |
| valor | string |
| codigo | string |
| qtd\_suites | integer |
| etapa | string |
| Field | Type |
| - | - |
| id\_corretor | integer |
| nome | string |
| tipo\_documento | string |
| documento | string |
| ativo | string |
| status | string |
| data\_cad | string |
| email | string |
| creci | string |
| telefone | string |
| descricao | string |
| endereco | string |
| tempo\_de\_mercado | integer |
| cep | string |
| bairro | string |
| complemento | string |
| numero | integer |
| facebook | string |
| twitter | string |
| instagram | string |
| linkedin | string |
| data\_ultimo\_acesso | string |
| avatar | string |
| slug | string |
| uf\_creci | string |
| tentativas\_erros | integer |
| id\_cidade | integer |
| id\_estado | integer |
| especialidades | string (JSON) |
| imobiliaria | string |
| possui\_imobiliaria | string |
| cadastro\_origem | string |
| email\_valido | string |
| termo\_aceite\_sso | string |
| tipo\_parceiro | string |
| Field | Type |
| - | - |
| id\_incorporadora | integer |
| nome | string |
| cnpj | string |
| ativo | string |
| data\_cad | string |
| email | string |
| telefone | string |
| descricao | string |
| endereco | string |
| tempo\_de\_mercado | integer |
| cep | string |
| facebook | string |
| twitter | string |
| instagram | string |
| linkedin | string |
| data\_ultimo\_acesso | string |
| avatar | string |
| slug | string |
| link\_site | string |
| tentativas\_erros | integer |
| plano | string |
| id\_cidade | integer |
| id\_estado | integer |
| unidades\_disponiveis | integer |
| crm\_origem | string |
| Field | Type |
| - | - |
| id | integer |
| titulo | string |
| tipo | string |
| imobiliaria | string |
| descricao | string |
| data\_interacao | string |
| status | string |
| data\_cad | string |
| id\_incorporadora | integer |
| nome\_incorporadora | string |
| id\_corretor | integer |
| nome\_corretor | string |
| email\_corretor | string |
| id\_coordenador | integer |
| nome\_coordenador | string |
| Field | Type |
| - | - |
| id | integer |
| nome | string |
| avatar | string |
| email | string |
| creci | string |
| uf\_creci | string |
| telefone | string |
| site | string |
| cep | string |
| endereco | string |
| bairro | string |
| numero | integer |
| aceitou\_termo | string |
| Field | Type |
| - | - |
| id | integer |
| nome | string |
| email | string |
| telefone | string |
| incorporadora | string |
| id\_incorporadora | integer |
| parceiros\_associados | integer |
| interesses\_recebidos | integer |
| empreendimentos\_associados | integer |
| principal | string |
| email\_valido | string |
| avatar | string |
| data\_cad | string |
| data\_ultimo\_acesso | string |
# AWS Cost Explorer as a data source
Source: https://docs.nekt.com/sources/aws-cost-explorer
Bring data from AWS Cost Explorer to Nekt.
AWS Cost Explorer is an AWS tool that provides detailed visibility into your cloud costs and usage. It allows you to analyze spending patterns, identify cost drivers, and track costs by service, linked account, or custom tags over time.
## Configuring AWS Cost Explorer as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the AWS Cost Explorer option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Setting up AWS permissions
You need to set up an IAM role or IAM user with permissions to access the Cost Explorer API. Choose one of the two authentication methods below.
**Create a role and add a custom trust policy**
* Open the AWS Console using the account that hosts the cost data you'd like to access.
* Enter the IAM (Identity and Access Management) service page.
* In the left panel under **Access management**, select **Roles**.
* Click **Create role**.
* In **Trusted entity type**, select **Custom trust policy**.
* Change the **Principal** value to:
```json theme={null}
{
"AWS": "arn:aws:iam::{AWS_ACCOUNT_ID}:role/nekt-ecs-task-role"
}
```
The variable must be replaced by the ID of the AWS account where the Nekt workspace is deployed. In order to find this information, click the dropdown in the top-right corner of the AWS console page after you log in with your account, then click on the "copy" icon to the right of your **Account ID**.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::{AWS_ACCOUNT_ID}:role/nekt-ecs-task-role"
},
"Action": "sts:AssumeRole"
}
]
}
```
* Click **Next**.
* Click **Next** again without adding any permission. We'll create an inline policy further on.
* Under **Role details**, you can type any **Role name** as you want. Example: `nekt-cost-explorer-source`.
* Optionally, type a **Description**. Example: `Used by Nekt to read AWS Cost Explorer data`.
* Click **Create role**.
You should get redirected back to the IAM roles page and a message of success should appear at the top.
**Add permissions to the role**
* Open the role you just created by clicking **View role** in the right end of the green banner that appeared at the top of the page. If you dismissed it already, you can type the given **Role name** in the search field and then select it when it appears in the list.
* Within **Permission policies (0)**, click **Add permissions**.
* Select **Create inline policy**.
* Within **Policy editor**, select `JSON`.
* Paste the following policy in the **Policy editor**:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "NektCostExplorerRead",
"Effect": "Allow",
"Action": [
"ce:GetCostAndUsage"
],
"Resource": "*"
}
]
}
```
* Click **Next**.
* Under **Policy details** type any Policy name as you want. Example: `nekt-cost-explorer-policy`.
* Click **Create policy**.
You should get redirected back to the role you just created and a message of success should appear at the top.
When setting up the connector at Nekt, you will use the ARN of the role you've just created.
If you prefer to use static credentials, create an IAM user with the `ce:GetCostAndUsage` permission.
* Open the AWS Console and go to the IAM service page.
* In the left panel, select **Users** and click **Create user**.
* Give the user a name (e.g., `nekt-cost-explorer`) and click **Next**.
* Attach a policy with the following permissions:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "NektCostExplorerRead",
"Effect": "Allow",
"Action": [
"ce:GetCostAndUsage"
],
"Resource": "*"
}
]
}
```
* Click **Next**, then **Create user**.
* Open the user you just created, go to the **Security credentials** tab, and click **Create access key**.
* Select **Third-party service**, confirm, and click **Create access key**.
* Copy the **Access Key ID** and **Secret Access Key**. You will need them in the next step.
Get in touch with us if you face any problem setting up permissions.
### 2. Add account access
The following configurations are available:
* **AWS Role ARN** *(recommended)*: The ARN of the IAM role you created in Step 1 (Option A). When provided, Nekt uses STS AssumeRole to obtain temporary credentials for cross-account access.
* **AWS Access Key ID**: Your AWS access key ID with Cost Explorer permissions. Required when using key/secret authentication (Option B).
* **AWS Secret Access Key**: Your AWS secret access key. Required when using key/secret authentication (Option B).
* **Start Date**: The earliest date from which cost records will be synced. AWS Cost Explorer only serves a moving window of history: 14 months by default, or 38 at monthly granularity if you enable multi-year data in your AWS Cost Management preferences. That window moves forward on the 1st of every month, so if this date ever falls outside it, the connector automatically starts from the oldest month your account still exposes and notes it in the run log. The source will not fail, and you never need to come back and update this date.
#### Advanced settings
* **Granularity**: Sets the aggregation granularity for cost data. Options are `DAILY` (default), `MONTHLY`, or `HOURLY`.
* **Cost Metric**: Which cost metric to retrieve. Available options:
* `UnblendedCost` *(default)*: the actual cost incurred by each account for each service, before any consolidation discounts are applied.
* `BlendedCost`: the average cost across all accounts in an AWS Organization for a given service, distributing bulk discounts evenly.
* `AmortizedCost`: spreads upfront reservation and Savings Plans payments across the months of the commitment term, giving a more even view of costs over time.
* `NetAmortizedCost`: same as AmortizedCost but after applicable discounts (e.g., private pricing, EDPs) are applied.
* `NetUnblendedCost`: same as UnblendedCost but after applicable discounts are applied.
* `NormalizedUsageAmount`: usage normalized to a common unit size, useful for comparing usage across different instance types.
* `UsageQuantity`: the raw quantity of usage (e.g., hours, GB) rather than a dollar amount.
* **Filter Tags**: A list of tag key/value pairs to filter costs by. Only resources matching **all** specified tags are included in the results. This also enables the `cost_by_tag` stream, which groups costs by the first tag key in the list.
You must first activate cost allocation tags in your AWS Billing console for them to appear in Cost Explorer data. See [AWS documentation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html) for details.
Once you're done, click **Next**.
### 3. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
Select the streams and click **Next**.
### 4. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, only new data since the last sync is fetched.
* Full table: every time the extraction happens, the current state of the data is fetched.
Once you are done configuring, click **Next**.
### 5. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. AWS cost data is typically updated once per day, so a daily trigger is recommended.
Once you are ready, click **Next** to finalize the setup.
### 6. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from AWS Cost Explorer and their corresponding fields:
Total aggregated cost and usage across all AWS services for each time period.
| Field | Description |
| - | - |
| `time_period_start` | Start date of the time period (e.g., 2024-01-01) |
| `time_period_end` | End date of the time period |
| `metric_name` | Name of the cost metric (e.g., UnblendedCost) |
| `amount` | The cost amount for the period |
| `unit` | The unit of the amount (e.g., USD) |
Cost and usage broken down by AWS service (e.g., Amazon EC2, Amazon S3, AWS Lambda) for each time period.
| Field | Description |
| - | - |
| `time_period_start` | Start date of the time period |
| `time_period_end` | End date of the time period |
| `group_by_key` | The grouping dimension (SERVICE) |
| `group_by_value` | The AWS service name (e.g., Amazon Elastic Compute Cloud - Compute) |
| `metric_name` | Name of the cost metric |
| `amount` | The cost amount for the period |
| `unit` | The unit of the amount |
Cost and usage broken down by linked AWS account for each time period. Useful for organizations with multiple AWS accounts under a consolidated billing setup.
| Field | Description |
| - | - |
| `time_period_start` | Start date of the time period |
| `time_period_end` | End date of the time period |
| `group_by_key` | The grouping dimension (LINKED\_ACCOUNT) |
| `group_by_value` | The AWS account ID |
| `metric_name` | Name of the cost metric |
| `amount` | The cost amount for the period |
| `unit` | The unit of the amount |
Cost and usage broken down by the first tag key specified in **Filter Tags**. This stream is only available when **Filter Tags** is configured in the source settings.
| Field | Description |
| - | - |
| `time_period_start` | Start date of the time period |
| `time_period_end` | End date of the time period |
| `group_by_key` | The tag key prefixed with "tag:" (e.g., tag:Environment) |
| `group_by_value` | The tag value (e.g., production, staging) |
| `metric_name` | Name of the cost metric |
| `amount` | The cost amount for the period |
| `unit` | The unit of the amount |
You must first activate cost allocation tags in your AWS Billing console for them to appear in Cost Explorer data. See [AWS documentation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html) for details.
# AWS S3 (CSV) as a data source
Source: https://docs.nekt.com/sources/aws-s3-csv
Bring data from AWS S3 CSV files to Nekt.
Amazon Simple Storage Service (S3) is a scalable cloud storage service that can store various types of files, including CSV (Comma-Separated Values) files. This connector allows you to extract data from CSV files stored in your S3 buckets, making it easy to analyze and process structured data stored in the cloud.
## Configuring AWS S3 (CSV) as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the AWS S3 (CSV) option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide the following credentials to connect to AWS S3:
* **Bucket Name**: The name of your AWS S3 bucket where the CSV files are stored
Additionally, you'll need to configure your CSV streams. You can add multiple streams, each representing a different set of CSV files, although the schema must be consistent between files for each stream.
For each stream, configure:
* **Stream Name**: A unique identifier for the stream (e.g., `sales_data`). Must start with a letter and contain only lowercase letters, numbers, and underscores.
* **Delimiter**: The character that separates fields in your CSV files (default: `,`). Common options:
* Comma (`,`) for standard CSV files
* Semicolon (`;`) for some European formats
* Tab (`\t`) for TSV files
Note: The delimiter must be a single character.
* **S3 Folder Path** (optional): The path within your bucket where the files are located (e.g., `raw_data/sales/`). Leave empty if files are in the bucket's root.
* **File Search Pattern**: A regex pattern to match the files you want to process (default: `.*`). Examples:
* `.*` - matches all files
* `.*\.csv$` - matches only files ending in .csv
* `sales_.*\.csv$` - matches CSV files starting with "sales\_"
* `2024-.*\.csv$` - matches CSV files from 2024
* **Unique Key Columns**: The column(s) that uniquely identify each row in your CSV files (e.g., `id` or `order_number`). This helps with deduplication and incremental syncs.
#### Required bucket permissions
The credentials used by the source need to list the bucket and read its objects. Attach a policy like the following, replacing `your-bucket-name`:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::your-bucket-name"
},
{
"Sid": "ReadObjects",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::your-bucket-name/*"
}
]
}
```
If the bucket is encrypted with SSE-KMS, the same principal also needs `kms:Decrypt` on the key.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only files that are added or modified since the last extraction.
* Full table: every time the extraction happens, we'll process all matching files from the bucket.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Additional Information
## Best Practices
### File Organization
* Keep related CSV files in dedicated folders
* Use consistent naming patterns for your files
* Consider using date-based prefixes for time-series data (e.g., `YYYY-MM-DD-sales.csv`)
### CSV Structure
* Ensure consistent column names across files
* Use appropriate data types for each column
* Avoid special characters in column names
* Include a header row in all CSV files
### Performance Optimization
* Use compression (e.g., .gz) for large files
* Set appropriate file search patterns to limit unnecessary file scanning
* Choose unique key columns that have good cardinality
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `403 Forbidden` when reading an object | The bucket policy does not grant the required permissions. | Grant `s3:ListBucket` on the bucket and `s3:GetObject` on its contents. If the bucket uses SSE-KMS, also grant `kms:Decrypt` on the key. See [Required bucket permissions](#required-bucket-permissions). |
| Bucket not found | The bucket name is wrong in the source configuration. | Confirm the **Bucket Name** exactly as it appears in AWS. |
| Numeric or date columns arrive as text | In CSV files every column is read as text. | Cast the types in a Query or Notebook. |
| Column names with accents appear altered | Names are normalized for the destination warehouse. | Use the normalized names in your queries, or standardize the headers at the source. |
### File Not Found
* Verify the bucket name is correct
* Check the file search pattern
* Ensure the S3 folder path is correct
* Confirm file permissions
### Parsing Errors
* Verify the correct delimiter is specified
* Check for special characters or quotes in the CSV
* Ensure consistent column counts across rows
* Look for hidden characters (BOM, etc.)
### Authentication Issues
* Verify AWS credentials are correct
* Check bucket permissions
* Ensure the credentials have not expired
* Confirm the bucket region
### Performance Issues
* Review file search patterns for efficiency
* Check file sizes and consider splitting large files
* Monitor AWS request quotas
* Consider using compression for large files
## Skills for agents
AWS S3 (CSV) connector documentation as plain markdown, for use in AI agent contexts.
# AWS S3 (JSON) as a data source
Source: https://docs.nekt.com/sources/aws-s3-json
Bring data from AWS S3 JSON files to Nekt.
AWS S3 JSON refers to JSON (JavaScript Object Notation) files stored in Amazon Simple Storage Service (S3), which is a
scalable cloud storage service. This allows you to access and process structured data files stored in the cloud,
providing flexible data storage for applications and analytics.
![]()
## 1. Add your AWS S3 JSON access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the AWS S3 (JSON) option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access. You will need to provide some details to locate your file on your AWS S3 bucket.
* **Bucket name:** The AWS bucket where files are stored.
* **S3 folder name:** This is a prefix to apply after the bucket name, but before the file search pattern, to allow you to find files in 'directories' inside the bucket. Leave empty if the files aren in the bucket's root level.
* **File search pattern:** This is a regex the tap will use to find files in the specified bucket. Ex: .\*.json\$ will look for all .json files inside the bucket.
* **Primary key:** The unique identifier of the objects extracted from the files to be used for deduplication in the generated table.
* **Start date:** Documents added from this date onwards will be considered for extraction.
* \*\*Sample file for schema: Optionally upload a sample file to generate a fixed schema if schema consistency is needed between extractions.
* **Max. number of files to sample for generating the schema:** The total number of files to use for generating the schema. If you have a very strict schema in the target files, this number can be low. The higher this number is, the longer it's going to take to generate the sampled schema, however the more consistent it is going to be.
3. Click **Next**.
## 2. Select your AWS S3 (JSON) streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your AWS S3 (JSON) data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your AWS S3 (JSON) data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `403 Forbidden` when reading an object | The bucket policy does not grant the required permissions. | Grant `s3:ListBucket` on the bucket and `s3:GetObject` on its contents. If the bucket uses SSE-KMS, also grant `kms:Decrypt` on the key. The full policy is on the [AWS S3 (CSV)](/sources/aws-s3-csv#required-bucket-permissions) page. |
| Bucket not found | The bucket name is wrong in the source configuration. | Confirm the **Bucket Name** exactly as it appears in AWS. |
| Column names with accents appear altered | Names are normalized for the destination warehouse. | Use the normalized names in your queries, or standardize the field names at the source. |
## Skills for agents
AWS S3 (JSON) connector documentation as plain markdown, for use in AI agent contexts.
# AWS S3 Parquet as a data source
Source: https://docs.nekt.com/sources/aws-s3-parquet
Bring data from AWS S3 Parquet files to Nekt.
AWS S3 Parquet refers to Parquet files stored in Amazon Simple Storage Service (S3), which is a scalable cloud storage service. Parquet is a columnar storage format optimized for big data processing, providing efficient compression and fast query performance for analytical workloads in the cloud.
![]()
## 1. Add your AWS S3 Parquet access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located at the top right of your screen. Then, select the AWS S3 Parquet option from the list of connectors.
2. Click **Next** and you'll be prompted to add the connector configuration. The configuration is split into global settings that apply to all data streams, and a list of stream-specific settings.
### Global configuration
* **Bucket name**: the AWS bucket name where files are stored.
* **Assume role ARN**: the ARN for a role that has access to the S3 bucket you want to extract files from and allows Nekt to assume the role via trust policy.
* **Start date**: Documents added or modified from this date onwards will be considered for extraction.
### Stream configuration
This connector supports extracting data from multiple locations within the same S3 bucket, with each location being treated as a separate data stream. You need to define at least one stream. For each stream, you can specify:
* **Stream name**: A unique name for the data stream.
* **Folder path**: this is a prefix to apply after the bucket name, but before the file search pattern, to allow you to find files in 'directories' inside the bucket. Leave empty if the files are in the bucket's root directory.
* **File search pattern**: This is a regex the connector will use to find files in the specified bucket. Ex: `\\.parquet$` will look for all files containing the extension `.parquet` inside the bucket. For detailed instructions in constructing the regex, [click here](https://docs.python.org/3.5/library/re.html#regular-expression-syntax).
* **Primary keys**: A list of column names to be used as primary keys for the stream.
* **Max parallelism**: The maximum number of files to process in parallel. Defaults to 5. Increasing this value can improve performance but will also increase memory and CPU usage.
* **Sample file for schema**: by default, the schema is generated from sampling a file from the bucket. Assuming all files have a similar structure, this should be enough to generate a consistent schema. However, if there's high variability on your Parquet files, consider uploading a sample file that fully represents the structure that might be found in the different documents.
Consider you have your Parquet files in the following bucket structure:
```
| bucket
| -- stream_A_files
| ---- file_1.parquet
| ---- file_2.parquet
| -- stream_B_files
| ---- file_7.parquet
| ---- file_8.parquet
```
If you want to extract files from both `stream_A_files` and `stream_B_files` as two separate streams, your streams configuration should look like this:
```json theme={null}
[
{
"name": "stream_a",
"folder_path": "stream_A_files/",
"file_pattern": "\\.parquet$",
"primary_keys": ["id"]
},
{
"name": "stream_b",
"folder_path": "stream_B_files/",
"file_pattern": "\\.parquet$",
"primary_keys": ["uuid"],
"max_parallelism": 10
}
]
```
1. Create a new role in the same account where the bucket is located and add the following policy:
```json theme={null}
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::/*",
"arn:aws:s3:::"
]
}
```
2. Add the following trust policy to allow Nekt to assume the recently created role.
```json theme={null}
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:::role/nekt-ecs-task-role"
},
"Action": "sts:AssumeRole"
}
```
Here it should be the same AWS account ID where the Nekt workspace is configured.
3. Add the following policy to your S3 bucket:
```json theme={null}
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:::role/"
},
"Action": [
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::/*",
"arn:aws:s3:::"
]
}
```
4. Finally, copy the ARN of your role and use it in the connector configuration.
3. Click **Next**.
## 2. Select your AWS S3 Parquet streams
1. The next step is selecting the data streams. You will see a list of all the streams you configured in the previous step. You can select which streams you want to enable.
Besides the schema inferred from the Parquet file, Nekt automatically injects
three more fields to the schema: `_nekt_unique_id`: a concatenation of `folder_path`,
`filename` and `row` position at which the record was extracted; `_file_last_modified`:
the date the object was last modified in the bucket; `_file_origin`: the full
s3 path of the file.
2. Click **Next**.
## 3. Configure your AWS S3 Parquet data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: this connector allows INCREMENTAL sync based on the date the documents were last modified. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your AWS S3 Parquet data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via AWS S3 Parquet, and we'll gladly assist you!
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `403 Forbidden` when reading an object | The bucket policy does not grant the required permissions. | Grant `s3:ListBucket` on the bucket and `s3:GetObject` on its contents. If the bucket uses SSE-KMS, also grant `kms:Decrypt` on the key. The full policy is on the [AWS S3 (CSV)](/sources/aws-s3-csv#required-bucket-permissions) page. |
| Bucket not found | The bucket name or region is wrong in the source configuration. | Confirm the **Bucket Name** and set **AWS Region** to the bucket's region. |
| Column names with accents appear altered | Names are normalized for the destination warehouse. | Use the normalized names in your queries, or standardize the column names at the source. |
## Skills for agents
AWS S3 (Parquet) connector documentation as plain markdown, for use in AI agent contexts.
# Axon as a data source
Source: https://docs.nekt.com/sources/axon
Bring data from Axon (AppLovin) to Nekt.
Axon is AppLovin's growth and monetization platform for mobile apps. It provides reporting APIs for advertiser campaign performance and publisher monetization data. With this connector, you can sync campaign metrics such as impressions, clicks, cost, and conversions, as well as publisher revenue data into Nekt.
## Configuration
To connect Axon to Nekt, you need a **Reporting API Key**.
You can find it in the Axon dashboard under **Account > Keys**.
| Field | Description |
| - | - |
| **Reporting API Key** | Your Axon reporting API key for authentication. |
| **Start Date** | The earliest date to sync data from (YYYY-MM-DD). Maximum 45 days back. |
| **End Date** | Optional. The latest date to sync data to. Defaults to today. |
The Axon Reporting API only supports data from the last 45 days. If you set a start date older than 45 days, the API will return an error.
Choose which streams you want to sync:
* **advertiser\_report** - Campaign performance data including impressions, clicks, cost, and conversions.
* **publisher\_report** - Monetization data including impressions, revenue, and eCPM.
For each selected stream, configure the following:
| Setting | Description |
| - | - |
| **Layer** | The data layer to write to (Bronze, Silver, or Gold). |
| **Folder** | The folder path within the selected layer. |
| **Table name** | The name of the destination table. |
| **Sync Type** | Incremental (recommended) or Full. |
Set general source settings:
| Setting | Description |
| - | - |
| **Description** | A description for this data source. |
| **Triggers** | Schedule or event-based triggers. |
| **Delta Log Retention** | How long to keep change history. |
Review your configuration and save the new source. Nekt will start syncing data on the configured schedule.
## Streams and fields
Campaign performance data from the Axon advertiser reporting API.
**Sync mode:** Incremental (by day)
| Field | Type | Description |
| - | - | - |
| day | date | Report date |
| campaign | string | Campaign name |
| campaign\_id\_external | string | External campaign ID |
| campaign\_type | string | Campaign type |
| campaign\_ad\_type | string | Campaign ad type |
| ad | string | Ad name |
| ad\_type | string | Ad type |
| ad\_creative\_type | string | Ad creative type |
| application | string | Application name |
| app\_id\_external | string | External app ID |
| platform | string | Platform (iOS, Android) |
| country | string | Country code |
| device\_type | string | Device type |
| placement\_type | string | Placement type |
| traffic\_source | string | Traffic source |
| target\_event | string | Target event name |
| creative\_set | string | Creative set name |
| creative\_set\_id | string | Creative set ID |
| size | string | Ad size |
| impressions | integer | Number of impressions |
| clicks | integer | Number of clicks |
| ctr | number | Click-through rate |
| conversions | integer | Number of conversions |
| conversion\_rate | number | Conversion rate |
| cost | number | Total cost |
| average\_cpa | number | Average cost per acquisition |
| average\_cpc | number | Average cost per click |
| sales | number | Total sales revenue |
| first\_purchase | integer | Number of first purchases |
| target\_event\_count | integer | Target event count |
Monetization data from the Axon publisher reporting API.
**Sync mode:** Incremental (by day)
| Field | Type | Description |
| - | - | - |
| day | date | Report date |
| application | string | Application name |
| package\_name | string | App package name |
| store\_id | string | Store ID |
| platform | string | Platform (iOS, Android) |
| country | string | Country code |
| ad\_type | string | Ad type |
| device\_type | string | Device type |
| placement\_type | string | Placement type |
| zone | string | Ad zone name |
| zone\_id | string | Ad zone ID |
| size | string | Ad size |
| impressions | integer | Number of impressions |
| clicks | integer | Number of clicks |
| ctr | number | Click-through rate |
| ecpm | number | Effective cost per mille |
| revenue | number | Total revenue |
# Azure Cosmos DB as a data source
Source: https://docs.nekt.com/sources/azure-cosmos-db
Bring data from Azure Cosmos DB for NoSQL to Nekt.
Azure Cosmos DB is Microsoft's fully managed NoSQL database. This connector reads accounts that use the **NoSQL API** (also called the Core or SQL API), Cosmos DB's native API, which stores schemaless JSON documents and queries them with a SQL-like dialect.
Cosmos DB also offers APIs that emulate other databases. If your account uses the MongoDB API, connect it with the [MongoDB](/sources/mongo-db) source instead; if it uses the PostgreSQL API, use the [PostgreSQL](/sources/postgres) source. This connector is only for accounts on the NoSQL API.
An account organizes data as `databases → containers → items`. A container holds JSON documents; an item is one document. This connector discovers your databases and containers at run time and creates one stream per container.
## Configuring Azure Cosmos DB as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Azure Cosmos DB option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Check the instructions next to each configuration option to discover where you can find the required parameters for the connection. These are the available configurations for this source:
* **Account endpoint:** The URI of your Cosmos DB account. In the [Azure portal](https://portal.azure.com), open the account and copy the value of **URI** from the Overview page. It looks like `https://my-account.documents.azure.com:443/`.
* **Authentication type:** How Nekt authenticates against the account. Choose **Account key** unless your organization has disabled key-based access on the account, in which case choose **Microsoft Entra ID**. Depending on your choice, different credential fields will be requested.
* **Account key:** An access key for the account. In the Azure portal, open the account, go to **Keys**, and copy a key from the **Read-only Keys** tab: Nekt only reads data, so a read-only key is all it needs. Only required and visible if you chose the **Account key** authentication type.
* **Directory (tenant) ID:** The Microsoft Entra ID tenant that owns the app registration. Only required and visible for **Microsoft Entra ID** authentication.
* **Application (client) ID:** The application ID of the Microsoft Entra ID app registration. Only required and visible for **Microsoft Entra ID** authentication.
* **Client secret:** A client secret of the Microsoft Entra ID app registration. The service principal also needs the **Cosmos DB Built-in Data Reader** role on this account: the control-plane **Reader** role does not grant access to the data. Only required and visible for **Microsoft Entra ID** authentication.
* **Databases:** Extract only these databases. Leave empty to discover every database the credentials can read.
* **Containers:** Extract only containers with these names, across the selected databases. Leave empty to discover every container.
* **Start date:** The earliest document modification date to extract on a stream's first incremental sync.
* **Max item count:** How many documents Cosmos DB returns per page. Lower it if extraction times out on containers with large documents.
* **Max throttling wait (seconds)** and **Max throttling retries:** How long, and how many times, to keep retrying while Cosmos DB is throttling the extraction because a container has run out of provisioned throughput.
* **Connection timeout (seconds):** How long to wait for Cosmos DB to answer a single request.
* **Preferred regions:** For accounts replicated across several Azure regions, the regions to read from in order of preference. Leave empty to read from the account's write region.
For Microsoft Entra ID authentication, the service principal needs the **Cosmos DB Built-in Data Reader** role on the account. This is a *data-plane* role and has to be assigned separately: the control-plane **Reader** role lets a principal see the account but not read any documents, which is the most common cause of a permission error here.
Once you're done, click **Next**.
### 2. Select streams
The next step is letting us know which streams you want to bring. Each stream in that list corresponds to one container in your account, named `{database}_{container}`. You can select entire groups of streams or only a subset of them.
Containers that hold no documents are not listed, since a stream with no documents would have nothing to write.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest the same name as the container, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* **Full table**: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog. However, keep in mind this increases resource usage such as computing time and storage.
For incremental syncs, the connector defaults to `_ts`, a timestamp Cosmos DB maintains on every document. See [Choosing a replication key](#choosing-a-replication-key) before selecting a different field.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Streams correspond dynamically to the containers present in your account. Two documents in the same container can share nothing but their `id`, so a container has no fixed schema to map into columns. Every stream therefore has the same shape: the fields Cosmos DB maintains itself become columns, and the document travels whole in a JSON string.
All extracted streams have the following standard properties:
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The document identifier set by your application. Unique within a logical partition, but not necessarily within the container (see the note below). |
| `_rid` | String | The resource identifier assigned by Cosmos DB. Unique across the container and stable for the document's lifetime. This is the table's primary key. |
| `_etag` | String | A version tag Cosmos DB changes on every write, which can be used to detect updates. |
| `_ts` | Integer | The time of the document's last write, in seconds since 1970-01-01 UTC. Maintained by Cosmos DB and used as the default incremental replication key. |
| `partition_key` | String | The document's value for the container's partition key. Containers with a hierarchical partition key report the levels as a JSON array, in order. |
| `document` | String | A stringified JSON version of the document payload. |
| `replication_key` | String / Integer | When you choose a document field as the incremental key, that field is also extracted into its own column under its own name. |
# Implementation Notes
## Why `_rid` is the primary key
Cosmos DB guarantees `id` to be unique **within a logical partition**, not within a container. A container partitioned by `/tenant_id` can legitimately hold a document with `id` `order-1` for one tenant and another with the same `id` for a different tenant. Keying the table on `id` would treat those as the same record and keep only one of them, so the connector keys on `_rid`, which Cosmos DB guarantees to be unique across the container.
`_rid` is stable for as long as a document exists. It does change if a document is deleted and recreated, or if a container is migrated or restored from a backup; after any of those, run a full sync so the table matches the source again.
## Choosing a replication key
`_ts` is the default and the recommended choice. It exists on every document, and Cosmos DB updates it itself on every write, so no application behavior can leave it stale.
Any other top-level document field can be used instead, but two things are worth knowing before you do:
**Documents missing the field are never extracted.** An incremental query filters on the replication key, so a document that does not have that field never matches and never reaches your catalog. Cosmos DB also compares values by type: if some documents store a date as a string and others as a number, one incremental query only returns the ones matching the type of the stored bookmark, and the rest are silently skipped. The connector samples the field when discovering streams and warns you when it is missing or inconsistently typed.
The connector also re-reads the boundary of each incremental window. `_ts` has one-second resolution, so many documents can share a value; the query uses "greater than or equal to" the last value seen, which means the final second of the previous run is read again. Those records are simply overwritten in your catalog. The alternative, a strict comparison, would permanently drop every document written in that second.
## Full sync reads the whole container
A full sync ignores the **Start date** setting and always reads every document. A full sync replaces the destination table, so filtering it would not make the read cheaper: it would delete every older document from your catalog. **Start date** applies only to a stream's first incremental run.
## Request units and throttling
Cosmos DB bills by **request unit** (RU) rather than by request, and reading a whole container costs RUs in proportion to the data read. When a container runs out of provisioned throughput, Cosmos DB rejects requests until throughput frees up, and the extraction waits and retries.
If runs fail with a throughput error, you have three options: raise the container's provisioned throughput, switch the container to autoscale, or schedule the source outside your application's peak hours. Raising **Max throttling wait (seconds)** makes the extraction more patient, which helps when throttling is brief but does not help when a container is permanently short of throughput.
### Best practices
* **Filter to the containers you need.** Every container you extract costs request units on your Azure bill. Use the **Databases** and **Containers** settings to skip the ones you do not analyze; you can always add more streams later.
* **Prefer a read-only key.** Cosmos DB accounts have separate read-only keys. The connector never writes, so giving it a read-write key grants access it does not use.
* **Use `_ts` for incremental syncs** unless you have a specific reason to use an application field, and make sure that field exists with a consistent type on every document if you do.
## Skills for agents
Azure Cosmos DB connector documentation as plain markdown, for use in AI agent contexts.
# Barte as a data source
Source: https://docs.nekt.com/sources/barte
Bring data from Barte to Nekt.
Barte is a Brazilian payments platform for online billing, in-person card terminals, and recurring subscriptions. This connector extracts orders, charges, subscriptions, buyers, plans, payment links, and the account's financial statement from your Barte account into your Nekt Catalog.
## Configuring Barte as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button in the top-right corner, then select Barte from the list of connectors.
Click **Next** and add your access details.
### 1. Add account access
The following configuration fields are available:
* **API key**: Token used to authenticate against the Barte API, sent as the `X-Token-Api` header.
* If your account is a **seller** (vendedor), find it in the Barte seller portal under **Configurações » Integração**.
* If your account is a **payment intermediary** (intermediador), find each seller's key in the intermediary portal under **Vendedores » select the seller » Integração**.
* For details, check the [Barte authentication documentation](https://docs.barte.com/guias/seguranca/autenticacao).
* **Environment**: Which Barte environment to read from, `production` or `sandbox`. Keys are issued per environment and are not interchangeable.
* **Start date** (advanced): The earliest record date to sync. It bounds the first run of Charges and the date windows of the two Statement streams. Orders, Buyers, Plans, and Payment links have no date filter in the Barte API and are always read in full.
* **Requests per minute** (advanced): Maximum number of requests per minute the connector sends. Barte publishes no numeric quota but answers HTTP 429 when an account goes too fast, so this acts as a safety belt. Lower it if your account gets rate limited.
* **Window size (days)** (advanced): Number of days requested per call on the Charges stream and the two Statement streams. Lower it if Barte rejects large ranges; raise it to reduce the number of requests on the first sync. Default: 30.
After configuring access, click **Next**.
### 2. Select streams
Choose which streams you want to sync. For faster extractions, select only the streams you need.
> Tip: You can search streams by name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how data should appear in your Catalog, including layer, table naming, and sync type.
* **Layer**: Select the destination layer where extracted tables will be created.
* **Table name**: Use the suggested name or customize it. You can also apply a prefix to all tables.
* **Sync Type**: Choose between `INCREMENTAL` and `FULL_TABLE`.
Only the **Charges** stream supports `INCREMENTAL` sync. Barte's other list endpoints have no creation-date filter, so every other stream reads in full on each run.
When you're done, click **Next**.
### 4. Configure data source
Add a clear description for this source so your team can identify it easily.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), choose how often data should be extracted (for example, daily, weekly, or at fixed times).
Optionally, configure additional settings:
* Delta Log Retention, which controls how long old table states are kept. Learn more [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync), to periodically complement incremental runs with a complete refresh.
When you are ready, click **Next** to finalize setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, trigger a run manually from the source actions.
To see data in the [Catalog](https://app.nekt.ai/catalog), at least one source run must complete successfully.
## Good to know
* **Orders vs. Charges.** An order (pedido) is the commercial intent: what was sold, to whom, for how much. A charge (cobrança) is an individual payment attempt against it. A one-off sale has a single charge; an instalment plan or a subscription cycle has several. Join them on `order_uuid`.
* **Incremental sync filters on creation, not update.** A charge whose status changes long after it was created (a late refund or a chargeback) is not picked up again by an incremental run. Schedule an [Additional Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync) if you need those corrections reflected. The two Statement streams re-read their whole window on every run, so they already repair themselves.
* **Charges are requested in time windows.** The Charges stream always carries both a start and end creation date on every request, walking the sync range in chunks defined by the **Window size (days)** setting. This satisfies Barte's API requirements (which otherwise returns a `BAR-3011` error) and keeps pages stable while paginating.
* **Subscriptions come from charges.** Barte publishes no endpoint that lists subscriptions, so this connector reaches them through the charges they generated. A subscription that has never produced a charge is not available through the API, and on an incremental run you see the subscriptions billed inside the window.
* **Statement entries may not be available.** The `/report/statement` endpoint is documented under Barte's payment-intermediary section. If your key is a plain seller key and Barte refuses it, the connector skips that stream with a warning in the run log and continues; use **Statement with balance** instead.
* **Nested data is stored as JSON strings.** Fields such as `charges` and `refunds` are text columns containing JSON, so they can be parsed in a Query or Notebook without the table schema changing when Barte adds a field.
# Streams and Fields
Below are the currently available streams in the Barte connector and their main fields.
Orders (pedidos) created on the account, from the `/v2/orders` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier of the order, used as the primary key. |
| `status` | String | Current status of the order: SENT, ABANDONED, PAID, CANCELED, LATE or PARTIALLY\_PAID. |
| `title` | String | Title of the order, shown to the buyer on the checkout and receipts. |
| `description` | String | Free-text description of what is being charged. |
| `value` | Number | Total amount of the order, in the account's currency. |
| `installments` | Integer | Number of instalments the order was split into (1 for a single payment). |
| `start_date` | Date | Date the order's billing starts. |
| `payment` | String | Payment method chosen for the order: CREDIT\_CARD, DEBIT\_CARD, PIX, BANK\_SLIP, CREDIT\_CARD\_EARLY\_SELLER or CREDIT\_CARD\_EARLY\_BUYER. |
| `customer_uuid` | String | Unique identifier of the buyer the order belongs to. |
| `customer_document` | String | Tax document of the buyer (CPF or CNPJ in Brazil). |
| `customer_type` | String | Type of the buyer's tax document: CPF, CNPJ or SSN. |
| `customer_document_country` | String | Country that issued the buyer's tax document. |
| `customer_name` | String | Full name of the buyer. |
| `customer_email` | String | Primary e-mail address of the buyer. |
| `customer_phone` | String | Phone number of the buyer. |
| `customer_alternative_email` | String | Secondary e-mail address of the buyer, when one was provided. |
| `idempotency_key` | String | Idempotency key sent when the order was created, used to avoid duplicate orders. |
| `charges` | String | Charges generated by this order (JSON array of objects). Each charge is also published as its own row in the charges stream. |
| `serial_number` | String | Serial number of the card terminal that processed the transaction, for in-person (POS) orders. |
| `terminal_id` | String | Logical identifier of the card terminal that processed the transaction, for in-person (POS) orders. |
| `origin_request` | String | Channel the order came from: API, PAYMENT\_LINK, POS, TAP\_TO\_PAY, ECOMMERCE or TEF. |
Individual payment attempts, from the `/v2/charges` endpoint. This is the only stream that supports `INCREMENTAL` sync.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier of the charge, used as the primary key. |
| `title` | String | Title of the charge, shown to the buyer on the checkout and receipts. |
| `expiration_date` | Date | Date the charge expires and can no longer be paid. |
| `value` | Number | Amount actually charged, after any discount or interest. |
| `original_value` | Number | Amount of the charge before discounts, interest or fees were applied. |
| `payment_method` | String | Payment method used: CREDIT\_CARD, DEBIT\_CARD, PIX, BANK\_SLIP, CREDIT\_CARD\_EARLY\_SELLER or CREDIT\_CARD\_EARLY\_BUYER. |
| `status` | String | Current status of the charge: SENT, PENDING, AUTHORIZED, CAPTURED, PAID or CANCELED. |
| `installments` | Integer | Number of instalments this charge was split into. |
| `brand` | String | Card brand used on the transaction (null for Pix and bank slips). |
| `customer_document` | String | Tax document of the buyer (CPF or CNPJ in Brazil). |
| `customer_type` | String | Type of the buyer's tax document: CPF, CNPJ or SSN. |
| `customer_name` | String | Full name of the buyer. |
| `customer_email` | String | Primary e-mail address of the buyer. |
| `customer_phone` | String | Phone number of the buyer. |
| `authorization_code` | String | Authorisation code returned by the card issuer when the transaction was approved. |
| `authorization_nsu` | String | NSU (sequential transaction number) assigned by the acquirer. |
| `refunds` | String | Refunds issued against this charge (JSON array of objects with id, createdAt, amount and requestStatus). |
| `error_reason` | String | Reason the payment was declined, when it failed. |
| `created_at` | DateTime | Timestamp the charge was created, normalised to UTC and used as the incremental replication key. |
| `paid_date` | Date | Date the charge was actually paid. |
| `order_uuid` | String | Unique identifier of the order this charge belongs to. |
| `subscription_uuid` | String | Unique identifier of the subscription that generated this charge, when it came from recurring billing. |
Recurring billing agreements, from the the subscription detail endpoint endpoint. Barte has no endpoint that lists subscriptions, so they are reached through the charges they generated.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier of the subscription, used as the primary key. |
| `status` | String | Current status of the subscription: PENDING, ACTIVE, DEFAULTER or INACTIVE. |
| `start_date` | Date | Date the subscription's billing cycle starts. |
| `payment_method` | String | Payment method used for the recurring charges: CREDIT\_CARD, PIX or BANK\_SLIP. |
| `customer_uuid` | String | Unique identifier of the subscriber. |
| `customer_name` | String | Full name of the subscriber. |
| `customer_email` | String | Primary e-mail address of the subscriber. |
| `customer_document` | String | Tax document of the subscriber (CPF or CNPJ in Brazil). |
| `value` | String | Recurring amount and billing frequency of the subscription (JSON object with type \[MONTHLY, SEMESTER or YEARLY] and valuePerMonth). |
| `additional_value` | String | One-off amount billed on top of the recurring value (JSON object with installments and value). |
| `charges` | String | Charges generated by this subscription (JSON array of objects). Each charge is also published as its own row in the charges stream. |
Customers registered on the account, from the `/v2/buyers` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier of the buyer, used as the primary key. |
| `document` | String | Tax document of the buyer (CPF or CNPJ in Brazil). |
| `name` | String | Full name of the buyer. |
| `country_code` | String | Country code of the buyer's phone number. |
| `phone` | String | Phone number of the buyer. |
| `email` | String | Primary e-mail address of the buyer. |
| `alternative_email` | String | Secondary e-mail address of the buyer, when one was provided. |
Subscription plan catalogue, from the `/v2/plans` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier of the plan, used as the primary key. |
| `title` | String | Title of the plan, shown to subscribers. |
| `description` | String | Description of what the plan includes. |
| `active` | Boolean | Whether the plan is currently active and can be subscribed to. |
| `bullets` | String | Benefits listed on the plan (JSON array of objects). |
| `values` | String | Price of the plan per billing frequency (JSON array of objects with the charge type and amount). |
| `accept_payment_methods` | Array | Payment methods the plan accepts. |
Shareable checkout links, from the `/v2/payment-links` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier of the payment link, used as the primary key. |
| `link_type` | String | Kind of link: ORDER for a one-off charge, SUBSCRIPTION for recurring billing, FAST\_ORDER for quick checkout. |
| `created_at` | DateTime | Timestamp the link was created, normalised to UTC. |
| `title` | String | Title of the link, shown to the payer on the checkout page. |
| `value` | Number | Amount the link charges, in the account's currency. |
| `url` | String | Public checkout URL that is shared with the payer. |
| `processed` | Integer | Number of payments already collected through this link. |
The account's financial ledger, from the `/report/statement` endpoint. Only available to payment intermediary accounts; the stream is skipped when the key is not allowed to read it.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique and stable identifier of the statement entry, used as the primary key. |
| `execution_date` | Date | Date the movement was posted to the account. |
| `charge_paid_date` | DateTime | Timestamp the underlying charge was paid, normalised to UTC. |
| `uuid_charge` | String | Unique identifier of the charge that produced this movement, linking to the charges stream. |
| `operacao` | String | Type of movement: CRÉDITO, ESTORNO, CHARGEBACK, TRANSFERÊNCIA, CORREÇÃO DÉBITO, CORREÇÃO CRÉDITO, CRÉDITO: DESCONTO DE TAXA, CRÉDITO: ANTECIPAÇÃO DE RECEITA or DÉBITO: ANTECIPAÇÃO DE RECEITA. |
| `payment_method` | String | Payment method of the underlying charge. |
| `entrada_bruta` | Number | Gross amount credited to the account by this movement, before fees. |
| `saida_bruta` | Number | Gross amount debited from the account by this movement, before fees. |
| `entrada_liquida` | Number | Net amount credited to the account by this movement, after fees. |
| `saida_liquida` | Number | Net amount debited from the account by this movement, after fees. |
| `taxa_operacao` | String | Fee applied to this movement. |
| `bandeira` | String | Card brand of the underlying charge (null for Pix and bank slips). |
The same movements plus the running account balance, from the `/report/statement_with_balance` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `pedido` | String | Unique identifier of the order the entry refers to, linking to the orders stream. |
| `data_lancamento` | Date | Date the movement was posted to the account. |
| `metodo_pagamento` | String | Payment method used on the underlying transaction. |
| `bandeira` | String | Card brand of the underlying transaction (null for Pix and bank slips). |
| `parcelas_total` | Integer | Total number of instalments the order was split into. |
| `parcelas_lancamento` | String | Which instalment of the order this entry settles. |
| `descricao` | String | Description of the movement. |
| `valor_lancamento` | Number | Net amount of the entry, after fees. |
| `valor_bruto_lancamento` | Number | Gross amount of the entry, before fees. |
| `taxa_lancamento` | Number | Fee charged on this entry. |
| `saldo_conta` | Number | Balance of the account immediately after this entry was posted. |
# Better Stack as a data source
Source: https://docs.nekt.com/sources/betterstack
Bring your uptime monitoring, incident and on-call data from Better Stack to Nekt.
Better Stack is an uptime monitoring, incident management and log management platform. This
connector extracts the operational record behind it: the monitors and heartbeats that watch your
services, every incident they raised and how long it took to acknowledge and resolve it, the
availability figures Better Stack computes, the on-call schedules and escalation policies that
decide who gets paged, the public status pages you publish, and the configuration of your telemetry
sources and collectors.
## 1. Add your Better Stack access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Better Stack option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API token**: required. In Better Stack, go to **API tokens** and copy a token from the **Global API tokens** section. A global token is the right choice for a pipeline: it covers every team and both the Uptime and Telemetry products, so no table is left out.
* **Start date** (advanced): earliest date to read incidents and on-call shifts from. Leave empty to start from the oldest data Better Stack still serves. Every other endpoint returns its full contents on each sync, so this setting does not affect the other tables.
* **Incidents lookback (days)** (advanced): how far before the last synced incident to re-read on each run. 30 by default.
* **Page size** (advanced): records requested per page. 250 by default, which is Better Stack's maximum.
* **Requests per minute** (advanced): how fast the connector is allowed to call the API. 300 by default.
A team-scoped **Uptime API token** also works, but it only reaches one team's Uptime resources. The
four `telemetry_*` tables cannot be read with it and will stay empty. Use a global token unless you
deliberately want to limit the source to a single team.
3. Click **Next**.
## 2. Select your Better Stack streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Four streams cost one request per parent record: `monitor_availability` and
`monitor_response_times` per monitor, `heartbeat_availability` per heartbeat,
`telemetry_source_metrics` per telemetry source. On an account with hundreds of monitors they make
the sync noticeably longer, so select them only if you need them.
2. Click **Next**.
## 3. Configure your Better Stack data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Better Stack data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Before you rely on the data
**`monitor_response_times` cannot be backfilled.** Better Stack keeps individual check timings for
the last 24 hours only and offers no date filter on them, so this table builds up history from your
first sync onwards. Schedule the pipeline to run at least once a day, or the series will have gaps
that nothing can recover.
A few behaviours are worth knowing before you build on these tables:
* **Only `incidents` can sync incrementally.** It is the one Better Stack endpoint that accepts a
date range. Every other table is read in full on each run, which is why almost all of them are
FULL\_TABLE.
* **Incidents keep changing after they start.** An incident's `started_at` never moves, but
`acknowledged_at` and `resolved_at` are filled in later. The connector re-reads the last 30 days
before the bookmark on every run so those updates land; raise **Incidents lookback (days)** if
incidents in your account often stay open for longer than that.
* **The availability tables are rollups, not history.** `monitor_availability` and
`heartbeat_availability` hold lifetime figures that Better Stack recomputes on every call, so each
sync replaces the previous answer. To keep a time series of your SLA, snapshot the table with a
[History](https://docs.nekt.com/histories/overview).
* **Join incidents to monitors through `monitor_id`.** The connector lifts it out of the API's
relationships object into its own column. It is empty for incidents raised by an integration or an
incoming webhook rather than by a monitor.
* **Severities are called urgencies in the API.** Better Stack renamed the concept in the product but
kept the old name on the wire, so `severities` and `severity_groups` come from the `urgencies`
endpoints. The `type` column still reads `urgency`.
* **Nested data arrives as JSON text.** Fields with sub-structure (an incident's `metadata`, a
policy's `steps`, a status page resource's `status_history`, a catalog record's `attributes`) are
stored as JSON strings. Read them with `JSON_VALUE(column, '$.field')` in your queries.
* **`team_members` mixes people and invitations.** A row is an accepted member when `type` is
`team_member` and a pending invitation when it is `team_member_invitation`; invitations carry
`invited_at` instead of `created_at`.
* **Credentials are not extracted.** The heartbeat ping URL, a telemetry source's ingest token, a
collector's secret, an escalation policy's incident token and a monitor's basic-auth password,
environment variables and Playwright script are all write credentials, which have no place in a
warehouse table.
* **Log lines and metric points are out of scope.** Better Stack serves them through a separate
ClickHouse-compatible query API that runs SQL over your retention window. The telemetry tables here
describe where your telemetry comes from and how it is shaped, not the events themselves.
## Available streams
| Stream | Sync | Description |
| - | - | - |
| `monitors` | FULL\_TABLE | Every check configured in the account: the URL or host, the check type (status code, keyword, ping, TCP, SMTP, DNS, Playwright and others), current status, check frequency and timeouts, the regions it runs from, the maintenance window, expiration alerting for SSL certificates and domains, and which channels an incident may use. |
| `monitor_groups` | FULL\_TABLE | Groups monitors are organized into, with their sort order and whether the whole group is paused. |
| `monitor_availability` | FULL\_TABLE | Lifetime availability summary per monitor: uptime percentage, total downtime, number of incidents, and the longest and average incident duration, all in seconds. One request per monitor. |
| `monitor_response_times` | INCREMENTAL | Individual check timings, one row per measurement per region, broken down into DNS lookup, connection, TLS handshake and data transfer time. Covers the last 24 hours only. One request per monitor. |
| `heartbeats` | FULL\_TABLE | Checks that expect a ping from your own job instead of calling out: the expected period and grace, the group they belong to, the maintenance window, and their current status. |
| `heartbeat_groups` | FULL\_TABLE | Groups heartbeats are organized into, with their sort order and whether the whole group is paused. |
| `heartbeat_availability` | FULL\_TABLE | Lifetime availability summary per heartbeat, in the same shape as `monitor_availability`. One request per heartbeat. |
| Stream | Sync | Description |
| - | - | - |
| `incidents` | INCREMENTAL | Every incident raised, with the cause, the monitor behind it, when it started, when it was acknowledged and by whom, when it was resolved and by whom, the regions that observed the failure, the response captured at the time, the channels that were alerted, the Slack channels opened for it and any custom metadata attached. This is the table time-to-acknowledge and time-to-resolve are computed from. |
| `incident_comments` | FULL\_TABLE | Comments written on an incident, with the author and when they were posted. One request per incident in the sync window. |
| `on_call_calendars` | FULL\_TABLE | On-call schedules, whether each is its team's default, and who is on call right now. |
| `on_call_calendar_events` | FULL\_TABLE | Individual on-call shifts generated by each schedule, past and upcoming, with the people covering them and whether the shift is a manual override. |
| `escalation_policies` | FULL\_TABLE | The ordered steps taken when an incident is not handled (who is alerted, after how long, at which severity, and which policy it falls back to), plus how many times the whole policy repeats. |
| `escalation_policy_groups` | FULL\_TABLE | Groups escalation policies are organized into. |
| `severities` | FULL\_TABLE | Severity levels and which alert channels each is allowed to use: SMS, phone call, e-mail and push. |
| `severity_groups` | FULL\_TABLE | Groups severities are organized into. |
| Stream | Sync | Description |
| - | - | - |
| `status_pages` | FULL\_TABLE | Public status pages, with their branding, subdomain or custom domain, the history window they display, whether visitors can subscribe, whether incidents are published automatically, and the state currently shown. |
| `status_page_groups` | FULL\_TABLE | Groups status pages are organized into. |
| `status_page_sections` | FULL\_TABLE | Sections a status page groups its resources into. |
| `status_page_resources` | FULL\_TABLE | Monitors and heartbeats published on a status page, with the public name shown to visitors, the availability over the displayed window, the rules deciding when the resource is marked down or degraded, and the daily status history. |
| `status_page_reports` | FULL\_TABLE | Incident and maintenance reports published on a page, with the title, when the event started and ended, and which resources it affected. |
| `status_page_report_updates` | FULL\_TABLE | Individual updates posted while a report was open, with the message, when it was published, whether subscribers were notified, and the state each affected resource was put in. |
| `status_page_subscribers` | FULL\_TABLE | People subscribed to a page's updates, with the resources their subscription is limited to and when they confirmed their address. |
| Stream | Sync | Description |
| - | - | - |
| `metadata` | FULL\_TABLE | Custom key/value metadata attached to a monitor, heartbeat, incident or integration: the values incident routing rules match on. |
| `catalog_relations` | FULL\_TABLE | The tables your service catalog is made of, with what each one catalogs. |
| `catalog_attributes` | FULL\_TABLE | Attributes (columns) defined on each catalog relation, and which one identifies the record. |
| `catalog_records` | FULL\_TABLE | Rows held by each catalog relation, with the value of every attribute. |
| `team_members` | FULL\_TABLE | Everyone in the organization plus pending invitations, with the role each holds and the mobile platforms they can receive push alerts on. |
| `organization_roles` | FULL\_TABLE | Roles available in the organization, including custom ones, and the built-in role each maps to. |
| Stream | Sync | Description |
| - | - | - |
| `telemetry_sources` | FULL\_TABLE | Sources your logs and metrics are ingested into, with the platform they come from, the table they are queryable under, the retention configured for logs and metrics, the data region, and the transformation applied on ingestion. |
| `telemetry_source_groups` | FULL\_TABLE | Groups telemetry sources are organized into. |
| `telemetry_source_metrics` | FULL\_TABLE | Metric definitions extracted from each source's log lines: the SQL expression evaluated, its storage type and the aggregations it can be charted with. One request per source. |
| `telemetry_collectors` | FULL\_TABLE | Better Stack collectors running on your own infrastructure, with their platform, online status, sample rates and batch settings, the source they write into and when they last reported in. |
# BigQuery as a data source
Source: https://docs.nekt.com/sources/bigquery
Bring data from BigQuery to Nekt.
BigQuery is Google Cloud's fully managed, serverless data warehouse that enables super-fast SQL queries using the processing power of Google's infrastructure. It's designed for large-scale data analytics and provides tools for data analysis, machine learning, and business intelligence.
![]()
## Configuring BigQuery as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the BigQuery option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a Google Cloud service account with the appropriate permissions to connect to BigQuery.
**Required IAM Roles**: Your service account **must** have the following roles assigned at the project level:
* **BigQuery Data Viewer** (`roles/bigquery.dataViewer`): Allows reading data from tables and views
* **BigQuery Job User** (`roles/bigquery.jobUser`): Required to run queries and extract data
Without the **BigQuery Job User** role, the connector will fail to execute queries even if data viewer permissions are granted.
1. Go to the [Google Cloud Console](https://console.cloud.google.com/)
2. Navigate to **IAM & Admin** → **Service Accounts**
3. Click **Create Service Account**
4. Enter a name and description for the service account
5. Click **Create and Continue**
1. In the "Grant this service account access to project" section
2. Add the role **BigQuery Data Viewer** (`roles/bigquery.dataViewer`)
3. Click **Add Another Role**
4. Add the role **BigQuery Job User** (`roles/bigquery.jobUser`)
5. Click **Continue** and then **Done**
1. Click on the newly created service account
2. Go to the **Keys** tab
3. Click **Add Key** → **Create new key**
4. Select **JSON** format
5. Click **Create**: the JSON file will be downloaded automatically
6. Keep this file secure: you'll upload it to Nekt
### 2. Configuration Options
* **GCP Project ID**: The Google Cloud Platform project ID where your BigQuery instance is located. [Find your Project ID](https://support.google.com/googleapi/answer/7014113?hl=en)
* **Service Account Credentials (JSON)**: The JSON credentials file for the service account. Upload the file downloaded in the previous step.
* **Filter Datasets** (Advanced): Filter to extract only specific datasets. If left blank, all available datasets will be discovered.
* **Filter Tables** (Advanced): Filter to extract only specific tables. Shell patterns are supported (e.g., `orders_*`, `*_backup`). If left blank, all available tables will be discovered.
* **Initial Sync Date** (Advanced): Starting point for incremental extraction. When a stream has a replication key but no bookmark yet (first run), only records with replication key `>=` this date are extracted. Required for BigQuery tables with partition filters that mandate a `WHERE` clause. ISO 8601 format (e.g., `2024-01-01T00:00:00Z`).
* **Google Storage Bucket** (Advanced): Required for large datasets. When supplied, data is extracted as files and temporarily saved in the bucket.
**Performance Tip**: It's recommended to use the dataset and table filters to speed up the discovery and extraction process, especially in projects with many datasets.
### 3. Google Storage Bucket (for Large Datasets)
For extracting large datasets efficiently, you can optionally configure a Google Cloud Storage bucket. When configured, the connector will:
1. Export query results to temporary files in the bucket
2. Stream the data from those files
3. Automatically delete the temporary files after extraction
**Additional Permissions Required for GCS Bucket**: If you use this option, your service account needs these additional permissions on the bucket:
* `storage.objects.create`
* `storage.objects.delete`
* `storage.objects.get`
* `storage.objects.list`
You can grant these by assigning the **Storage Object Admin** (`roles/storage.objectAdmin`) role on the specific bucket.
### 4. Select streams
The connector automatically discovers all tables and views in your BigQuery project (filtered by your configuration). Choose which streams you want to sync.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your service account has access to the dataset containing it.
Some complex nested types (deeply nested STRUCTs or ARRAYs) may not be fully supported. If you encounter issues during discovery, please contact Nekt support.
Select the streams and click **Next**.
### 5. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table and the type of sync.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once.
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: Every time the extraction happens, we'll get only the new data based on the replication key you configure. Good for append-only tables or when you want to keep historical records.
* **Full Table**: Every time the extraction happens, we'll get the current state of the data. Good when you want the latest snapshot without deleted records.
For incremental sync to work, you must specify a **replication key** column (e.g., `updated_at`, `created_at`, or an auto-incrementing ID).
Once you are done configuring, click **Next**.
### 6. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define additional settings:
* Configure **Delta Log Retention** to determine how long old states of this table should be stored. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement incremental data extractions, ensuring your data is completely synchronized periodically.
Once you are ready, click **Next** to finalize the setup.
### 7. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Streams and Fields
BigQuery is a generic database connector: streams are dynamically discovered based on your project's datasets and tables. Each table or view in your BigQuery project becomes a stream with fields matching the table's schema.
### Supported Data Types
| BigQuery Type | Mapped Type |
| - | - |
| STRING | String |
| INT64, INTEGER | Integer |
| FLOAT, FLOAT64, NUMERIC | Number |
| BOOL | Boolean |
| DATE | Date |
| DATETIME, TIMESTAMP | DateTime |
| ARRAY | Array |
| STRUCT | Object |
### Stream Naming
Streams are named using the pattern `{dataset}-{table}`, for example:
* `analytics-events` (from dataset `analytics`, table `events`)
* `sales-orders` (from dataset `sales`, table `orders`)
## Implementation Notes
### Required Permissions Summary
| Permission/Role | Purpose | Required |
| - | - | - |
| BigQuery Data Viewer | Read access to tables and views | Yes |
| BigQuery Job User | Execute queries for data extraction | Yes |
| Storage Object Admin (on bucket) | Temporary file storage for large extracts | Only if using GCS bucket |
### Performance Considerations
* **Filter your datasets**: Use the `Filter Datasets` option to limit discovery to only the datasets you need
* **Filter your tables**: Use shell patterns in `Filter Tables` to include only relevant tables
* **Large datasets**: Configure a Google Storage bucket for efficient extraction of large tables
* **Incremental sync**: When possible, use incremental sync with a proper replication key to minimize data transfer
### Data Quality
* **Primary Keys**: The connector attempts to detect primary keys from table constraints and unique indexes
* **Nested Data**: STRUCT and ARRAY types are supported and mapped to nested JSON objects/arrays
* **Views**: Both tables and views are discovered and can be synced
### Troubleshooting
| Issue | Possible Cause | Solution |
| - | - | - |
| No streams discovered | Missing permissions | Ensure service account has `BigQuery Data Viewer` role |
| Query execution failed | Missing job permissions | Ensure service account has `BigQuery Job User` role |
| Discovery is slow | Too many datasets/tables | Use filter options to limit scope |
| Large table extraction fails | Memory/timeout issues | Configure a Google Storage bucket |
| Permission denied on bucket | Missing storage permissions | Grant `Storage Object Admin` on the specific bucket |
## Skills for agents
BigQuery connector documentation as plain markdown, for use in AI agent contexts.
# Bing Webmaster Tools as a data source
Source: https://docs.nekt.com/sources/bing-webmasters
Bring data from Bing Webmaster Tools to Nekt.
Bing Webmaster Tools is a free service from Microsoft that helps you monitor and maintain your site's presence in Bing search results. It provides search performance data (clicks, impressions, top queries and pages) as well as crawl statistics and crawl issues for every site you have verified.
## Configuring Bing Webmaster Tools as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Bing Webmaster Tools option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The connector authenticates with a single **API key**. To generate it:
1. Sign in to [Bing Webmaster Tools](https://www.bing.com/webmasters).
2. Open **Settings → API access → API Key**.
3. Generate the key and copy it.
The following configurations are available:
* **API Key**: Your Bing Webmaster Tools API key. The key inherits access to every site verified by the associated account.
* **Start date**: (Optional) The earliest date to keep for date-partitioned streams (rank and traffic, queries, pages, and crawl stats). Records before this date are dropped.
The Bing API exposes no server-side date filter, so the start date and the incremental bookmark are applied by the connector after fetching each response.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select all streams or only specific ones.
> Tip: You can quickly find streams by typing their names.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data.
* Full table: every time the extraction happens, we'll get the current state of the data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
To see data in the [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Bing Webmaster Tools and their corresponding fields:
The sites verified in the account, used as the parent for every site-scoped stream.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `url` | String | The site URL, starting with http and ending with a /. |
| `is_verified` | Boolean | Whether the site has been verified in Bing Webmaster Tools. |
| `authentication_code` | String | Verification code issued by Bing for the site. |
| `dns_verification_code` | String | DNS verification code issued by Bing for the site. |
Daily clicks and impressions per site. Incremental by `date`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `site_url` | String | The site URL these stats belong to. |
| `date` | Date | The day the metrics were measured (Pacific time). |
| `clicks` | Integer | Number of clicks from Bing search results that day. |
| `impressions` | Integer | Number of times the site appeared in Bing results that day. |
Weekly clicks and impressions for the site's top search queries. Incremental by `date`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `site_url` | String | The site URL. |
| `query` | String | The search query term. |
| `date` | Date | End date of the weekly bucket the metrics cover. |
| `clicks` | Integer | Clicks for the query. |
| `impressions` | Integer | Impressions for the query. |
| `avg_click_position` | Number | Average position of the site in results that were clicked. |
| `avg_impression_position` | Number | Average position of the site across impressions. |
Weekly clicks and impressions for the site's top pages. The page URL is returned in the `query` field. Incremental by `date`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `site_url` | String | The site URL. |
| `query` | String | The page URL the statistics apply to. |
| `date` | Date | End date of the weekly bucket the metrics cover. |
| `clicks` | Integer | Clicks for the page. |
| `impressions` | Integer | Impressions for the page. |
| `avg_click_position` | Number | Average position of the page in results that were clicked. |
| `avg_impression_position` | Number | Average position of the page across impressions. |
Daily crawl statistics per site. Incremental by `date`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `site_url` | String | The site URL. |
| `date` | Date | The day the crawl stats apply to. |
| `crawled_pages` | Integer | Pages crawled by Bing. |
| `in_index` | Integer | Pages currently in the Bing index. |
| `in_links` | Integer | Inbound links discovered. |
| `crawl_errors` | Integer | Total crawl errors. |
| `code2xx` | Integer | URLs that returned a 2xx status. |
| `code301` | Integer | URLs that returned a 301 redirect. |
| `code302` | Integer | URLs that returned a 302 redirect. |
| `code4xx` | Integer | URLs that returned a 4xx status. |
| `code5xx` | Integer | URLs that returned a 5xx status. |
| `all_other_codes` | Integer | URLs that returned any other status code. |
| `blocked_by_robots_txt` | Integer | URLs blocked by robots.txt. |
| `contains_malware` | Integer | URLs flagged as containing malware. |
URLs with crawl issues per site.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `site_url` | String | The site URL. |
| `url` | String | The URL the issue applies to. |
| `http_code` | Integer | HTTP status code Bing received for the URL. |
| `in_links` | Integer | Number of inbound links to the URL. |
| `issues` | Integer | Bitmask flag describing the crawl issue type(s) for the URL. |
## Implementation Notes
* The connector authenticates with an API key passed as the `apikey` query parameter against the Bing Webmaster Tools JSON API (`https://ssl.bing.com/webmaster/api.svc/json`).
* The `sites` stream (`GetUserSites`) is the parent stream; each site URL is passed to the site-scoped streams as the `siteUrl` parameter, so a single connection covers every verified site.
* Microsoft JSON dates (`/Date(…)/`) are converted to ISO-8601 dates.
* Date-partitioned streams (rank and traffic, queries, pages, crawl stats) replicate incrementally by `date`. Because the API has no server-side date filter, the start date and incremental bookmark are applied client-side.
* Every site-scoped record is stamped with the `site_url` it belongs to.
## Skills for agents
Bing Webmaster Tools connector documentation as plain markdown, for use in AI agent contexts.
# Bitrix24 as a data source
Source: https://docs.nekt.com/sources/bitrix
Bring data from Bitrix24 to Nekt.
Bitrix24 is a comprehensive business management platform that combines CRM, project management, communication tools, and business automation features. It provides solutions for sales, marketing, customer service, and team collaboration in a single integrated platform.
![]()
## Configuring Bitrix24 as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Bitrix24** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The connector supports two authentication modes:
* **OAuth** (default): Install the Nekt app on your Bitrix24 instance using the install button, provide your **Domain URL** (e.g. `https://yourworkspace.bitrix24.com/`), and sign in with Bitrix24 to authorize Nekt.
* **Webhook**: Provide an **Incoming webhook URL** for your Bitrix24 portal. Check the [Bitrix documentation](https://apidocs.bitrix24.com/first-steps/first-rest-api-call.html) for how to create an incoming webhook and copy its URL.
The following configurations are available:
* **Authentication mode**: `oauth` (default) or `webhook`.
* **Domain URL**: The domain URL of your Bitrix24 workspace. Required in OAuth mode; in webhook mode the domain is read from the webhook URL.
* **Incoming webhook URL**: The Bitrix24 incoming webhook URL used for API calls. Required in webhook mode.
* **Extract deal contacts**: (Default: `false`) When enabled, performs an additional API call per deal to retrieve related contact IDs. Enable only if your deals have multiple clients and you need that list.
* **Enable deal comments stream**: (Default: `false`) When enabled, the `deal_comments` stream becomes available. Selecting it triggers one extra API call per deal to fetch comments. Enable only if needed, as it increases extraction time.
* **Enable deal products stream**: (Default: `false`) When enabled, the `deal_products` stream becomes available. Selecting it triggers one extra API call per deal to fetch products. Enable only if needed, as it increases extraction time.
* **Enable deal IDs stream**: (Default: `false`) When enabled, the `deal_ids` stream becomes available: a full snapshot, on every run, of the IDs of all deals that currently exist in Bitrix24. Use it to detect deals deleted in Bitrix24 (IDs present in the `deals` table but absent from `deal_ids`). Only the `ID` column is requested, so it is much lighter than a full extraction of deals.
* **Enable quote products stream**: (Default: `false`) When enabled, the `quote_products` stream becomes available. Selecting it triggers one extra API call per quote to fetch its products. Enable only if needed, as it increases extraction time.
* **Enable Open Channels sessions stream**: (Default: `false`) When enabled, the `imopenlines_sessions` stream becomes available: one row per Open Channels (Canais Abertos) conversation session, with the channel it came in on, the operator, the response-time metrics and the CRM record the conversation was attached to.
* **Enable Open Channels messages stream**: (Default: `false`) When enabled, the `imopenlines_messages` stream becomes available: the full text of every conversation, one row per message. It lists every session in the replication window before reading its messages, so it makes the extraction considerably longer. Enable only if you need the conversation content.
* **Operating time threshold**: (Default: 200) If Bitrix returns an "operating time" greater than this value (in seconds), the tap will pause for that duration before continuing. Use this to respect rate limits.
* **Filter deals by funnel ID**: (Optional) List of Bitrix deal category IDs (funnels). If set, only deals (and deal IDs) from these categories are extracted. The ID of a funnel can be found in the Bitrix URL when opening the funnel's kanban page (`crm/deal/kanban/category/{ID}`). Leave empty to sync all deals.
* **SPA Column Name Overrides**: (Advanced, optional) Map specific Bitrix SPA field codes to exact column names, to control how case-colliding custom fields are separated. Example: `{"ufCrm10Fotoprincipal": "ufCrm10Fotoprincipal_2"}`. Leave empty to use automatic collision resolution.
* **Enable BI reports**: (Default: `false`) Enables the Bitrix24 BI Connector report streams (such as Open Channels session statistics). The BI Connector is a separate integration from the webhook and requires its own key.
* **BI connector key**: Bitrix24 BI connector key, generated under **CRM** -> **Analytics** -> **BI analytics** -> **Manage keys**. Required when 'Enable BI reports' is on.
* **BI reports start date**: First day to pull BI report data from on the initial sync.
* **BI reports window size (days)**: (Default: 30) Size in days of each BI Connector request window. Smaller windows reduce per-request row counts to stay under plan limits.
* **BI reports lookback (days)**: (Default: 30) Number of days re-pulled before the saved bookmark on incremental BI report syncs, to capture updates to recently created sessions.
* **Open Channels start date**: (Advanced, default: `2024-01-01`) First day to pull Open Channels sessions and messages from on the initial sync, before any state exists.
* **Open Channels sessions lookback (days)**: (Advanced, default: 30) Number of days re-walked before the saved bookmark on incremental syncs of the `imopenlines_sessions` stream, to capture sessions answered, rated or closed after they were created.
* **Open Channels messages lookback (days)**: (Advanced, default: 30) Number of days re-walked before the saved bookmark on incremental syncs of the `imopenlines_messages` stream. Conversations that keep receiving messages for longer than this stop being refreshed, so raise it if your sessions stay open for weeks.
Scopes that must be included in your webhook (or granted to the Nekt app in OAuth mode) so Nekt can retrieve data:
* **CRM** (`crm`): contacts, companies, deals, deal categories, leads, activities, products, status list, SPA types (and optionally deal comments, deal products, deal IDs, quote products).
* **Users** (`user`, `user_brief`, `user_basic`): user list and profile data.
* **Company structure** (`departments`): departments.
* **Telephony** (`telephony`): calls (Voximplant statistics).
* **Open lines** (`imopenlines`): open line configurations, and optionally Open Channels sessions and the conversation messages themselves. The `im` scope is not required.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. Visibility depends on your webhook scopes and on the optional settings above (deal comments, deal products, deal IDs, quote products). You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has the required scope. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Pipeline fails with `Duplicate custom fields detected in Bitrix` error | Custom fields in Bitrix share codes that differ only by letter case or symbols (e.g., `ufCrm10FotoPrincipal` vs `ufCrm10Fotoprincipal`). Because column names are sanitized before reaching the destination, these fields collapse to the same column name, which causes a failure. | Remove or rename one field of each colliding pair directly in your Bitrix CRM settings, then re-run the pipeline. |
| The load stops after a custom field starts accepting multiple values | The field changed from single value to list at the source, and the destination column type does not follow automatically. | Run a full sync of the affected stream to recreate the table with the correct type. |
| Records deleted at the source keep showing up | Bitrix does not expose logical deletion in a way the API can replay. | For deals, enable the `deal_ids` stream, which re-extracts a full snapshot of current deal IDs on every run, and use it to detect deletions. For the other entities, schedule an additional full sync, preferably on the weekend. |
| A stream does not appear in the list | The matching scope was not granted on the authorization. | Review the scopes listed on this page and re-authorize. |
| SPA streams are missing or `Smart Process (SPA) streams were skipped` warning in logs | The connected Bitrix24 user lacks permission to list Smart Processes (`crm.type.list` access denied). | Ensure the user has read access to Smart Processes in Bitrix24 and the `crm` scope is granted, then re-authorize or re-run the sync. |
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked *optional* are only discovered when the corresponding setting is enabled.
| Stream | Slug | Description |
| - | - | - |
| Activities | `activities` | CRM activities (calls, meetings, tasks, emails). |
| Activity fields | `activity_fields` | Field definitions/metadata for activities. |
| Calls | `calls` | Telephony (Voximplant) call statistics. Requires the `telephony` scope. |
| Companies | `companies` | CRM companies. |
| Company fields | `company_fields` | Field definitions/metadata for companies. |
| Contact fields | `contact_fields` | Field definitions/metadata for contacts. |
| Contacts | `contacts` | CRM contacts. |
| Deal categories | `deal_categories` | Deal categories (pipelines/funnels). |
| Deal comments | `deal_comments` | Comments attached to deals. *Optional* (one API call per deal; enabled via "Enable deal comments stream"). |
| Deal fields | `deal_fields` | Field definitions/metadata for deals. |
| Deal IDs | `deal_ids` | Full snapshot of the IDs of every deal currently in Bitrix24, re-extracted on every run; use it to detect deleted deals. *Optional* (enabled via "Enable deal IDs stream"). |
| Deal products | `deal_products` | Products (line items) linked to deals. Incremental via the parent deal's `DATE_MODIFY`. *Optional* (one API call per deal; enabled via "Enable deal products stream"). |
| Deal stage history | `deal_stage_history` | History of deal stage (pipeline step) changes. |
| Deals | `deals` | CRM deals. |
| Departments | `departments` | Company structure / departments. Requires the `department` scope. |
| Lead fields | `lead_fields` | Field definitions/metadata for leads. |
| Lead stage history | `lead_stage_history` | History of lead stage changes. |
| Leads | `leads` | CRM leads. |
| Open Channels messages | `imopenlines_messages` | Every message exchanged in Open Channels conversations, one row per message, with the author classified as client, operator or system. Requires the `imopenlines` scope. *Optional* (enabled via "Enable Open Channels messages stream"). |
| Open Channels sessions | `imopenlines_sessions` | One row per Open Channels conversation session, with the channel, the operator, the response-time metrics and the linked CRM record. Requires the `imopenlines` scope. *Optional* (enabled via "Enable Open Channels sessions stream"). |
| Open lines | `open_lines` | Open Lines (communication channels) configuration. Requires the `imopenlines` scope. |
| Product | `product` | CRM product catalog. |
| Product fields | `product_fields` | Field definitions/metadata for products. |
| Quotes | `quotes` | CRM quotes. Incremental sync via `DATE_MODIFY`. |
| Quote fields | `quote_fields` | Field definitions/metadata for quotes. |
| Quote products | `quote_products` | Products (line items) linked to quotes. Incremental via the parent quote's `DATE_MODIFY`. *Optional* (one API call per quote; enabled via "Enable quote products stream"). |
| SPA types | `spa_types` | Smart Process Automation (SPA) entity types, used to discover the dynamic SPA streams. |
| SPA stage history | `spa_stage_history` | Stage (pipeline step) change history for SPA items. Incremental sync via `CREATED_TIME`. |
| Status list | `status_list` | CRM status list (deal/lead status options, etc.). |
| Users | `users` | Bitrix24 users. Requires the `user`, `user_brief` and `user_basic` scopes. |
| SPA data (dynamic) | `spa_{entityTypeId}` | *Dynamic.* One data stream per SPA entity type in your portal (e.g. `spa_128`). |
| SPA fields (dynamic) | `spa_fields_{entityTypeId}` | *Dynamic.* Field definitions/metadata for each SPA entity type (e.g. `spa_fields_128`). |
## Fields by stream
Below you'll find all available data streams from Bitrix24 and their main fields. CRM entity streams (contacts, companies, deals, leads, activities, product) have schemas built dynamically from the Bitrix API, so custom fields (e.g. `UF_*`) may vary per portal.
Stream for CRM activities (calls, meetings, tasks, etc.). Schema is built from the Bitrix API.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `CREATED` | String | Replication key (incremental sync) |
| Custom fields | Varies | Additional fields depend on your Bitrix CRM activity configuration |
Metadata stream for activity field definitions (types, labels, options).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `isRequired`, `isReadOnly`, `items` (for list fields), etc. |
Stream for telephony (Voximplant) call statistics. Requires scope `telephony`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | String | Unique identifier |
| `CALL_START_DATE` | String | Replication key |
| `PORTAL_USER_ID` | Integer | ID of the portal user |
| `PHONE_NUMBER` | String | Phone number |
| `CALL_DURATION` | Integer | Call duration |
| `CALL_RECORD_URL` | String | URL to the call recording |
| `CRM_ENTITY_TYPE` | String | Related CRM entity type |
| `CRM_ENTITY_ID` | Integer | Related CRM entity ID |
| `COST` | Number | Cost of the call |
| `COST_CURRENCY` | String | Currency of the cost |
Stream for CRM companies. Schema is built from the Bitrix API.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `DATE_MODIFY` | String | Replication key |
| Custom fields | Varies | Standard and custom (`UF_*`) company fields from your portal |
Metadata stream for company field definitions.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `statusType`, `isRequired`, `items`, `listLabel`, `formLabel`, etc. |
Stream for CRM contacts. Schema is built from the Bitrix API.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `DATE_MODIFY` | String | Replication key |
| Custom fields | Varies | Standard and custom (`UF_*`) contact fields from your portal |
Metadata stream for contact field definitions.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `statusType`, `isRequired`, `items`, `listLabel`, `formLabel`, etc. |
Stream for deal categories (pipelines/funnels). Schema is built from the Bitrix API.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the category. |
| `name` | String | Name of the pipeline/funnel. |
| `sort` | Integer | Sort order of the category. |
| `entityTypeId` | Integer | Identifier of the entity type this category belongs to. |
| `isDefault` | Boolean | Whether this is the default category. |
| `originId` | String | Origin identifier for external integrations. |
| `originatorId` | String | Originator identifier for external integrations. |
Stream for comments attached to deals. Available only when "Enable deal comments stream" is turned on. One API call per deal.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| Content data | Varies | Comment content and metadata; structure depends on Bitrix API response. |
Metadata stream for deal field definitions.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `statusType`, `isRequired`, `items`, `listLabel`, `formLabel`, etc. |
Full snapshot of the IDs of every deal that currently exists in Bitrix24. Available only when "Enable deal IDs stream" is turned on. Bitrix24 has no deletion flag (a deleted deal goes to the recycle bin and simply disappears from the API), so this stream is re-extracted in full on every run, requesting only the `ID` column, and can be used to detect deals deleted in Bitrix24 (IDs present in the `deals` table but absent from `deal_ids`). It honours the "Filter deals by funnel ID" setting.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | String | Identifier of a deal that currently exists in Bitrix24. Same type as `deals.ID`, so the tables join without casting. |
Stream for products (line items) linked to deals. Available only when "Enable deal products stream" is turned on. One API call per deal. Supports incremental sync based on the parent deal's modification date (`DEAL_DATE_MODIFY`).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | String | Unique identifier of the record. |
| `OWNER_ID` | String | Identifier of the associated deal. |
| `OWNER_TYPE` | String | Type of the owner entity. |
| `PRODUCT_ID` | Integer | Identifier of the associated product. |
| `PRODUCT_NAME` | String | Name of the product. |
| `ORIGINAL_PRODUCT_NAME` | String | Original product name from the catalog. |
| `PRODUCT_DESCRIPTION` | String | Description of the product. |
| `PRICE` | Number | Monetary value associated with the record. |
| `PRICE_EXCLUSIVE` | Number | Price excluding tax. |
| `PRICE_NETTO` | Number | Net price of the product. |
| `PRICE_BRUTTO` | Number | Gross price of the product. |
| `PRICE_ACCOUNT` | String | Price in account currency. |
| `QUANTITY` | Integer | Quantity of the product. |
| `DISCOUNT_TYPE_ID` | Integer | Identifier of the discount type. |
| `DISCOUNT_RATE` | Number | Discount rate applied. |
| `DISCOUNT_SUM` | Number | Discount amount. |
| `TAX_RATE` | Number | Tax rate applied to the product. |
| `TAX_INCLUDED` | String | Whether tax is included in the price. |
| `CUSTOMIZED` | String | Whether the row was customized. |
| `MEASURE_CODE` | Integer | Code of the measure unit. |
| `MEASURE_NAME` | String | Name of the measure unit. |
| `SORT` | Integer | Sort order of the row. |
| `XML_ID` | String | External identifier for integration. |
| `TYPE` | Integer | Type classification of the record. |
| `STORE_ID` | Integer | Identifier of the associated store. |
| `RESERVE_ID` | Integer | Identifier of the reserve. |
| `DATE_RESERVE_END` | String | End date of the reservation. |
| `RESERVE_QUANTITY` | Integer | Reserved quantity. |
| `DEAL_DATE_MODIFY` | DateTime | Timestamp when the parent deal was last updated; inherited from the parent stream and used as the incremental bookmark. |
Stream for CRM deals. Schema is built from the Bitrix API. Optionally can include `CONTACT_IDS` per deal when "Extract deal contacts" is enabled.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `DATE_MODIFY` | String | Replication key |
| `CATEGORY_ID` | Integer | Funnel/pipeline category ID |
| Custom fields | Varies | Standard and custom (`UF_*`) deal fields |
Stream for deal stage (pipeline step) change history.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| History data | Varies | Identifiers and timestamps for stage transitions; structure depends on Bitrix API. |
Stream for company structure (departments). Requires scope `departments`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `NAME` | String | Department name |
| `SORT` | Integer | Sort order |
| `PARENT` | Integer | Parent department ID |
| `UF_HEAD` | Integer | Department head user ID |
Metadata stream for lead field definitions.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `statusType`, `isRequired`, `items`, `listLabel`, `formLabel`, etc. |
Stream for CRM leads. Schema is built from the Bitrix API.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `DATE_MODIFY` | String | Replication key |
| Custom fields | Varies | Standard and custom lead fields from your portal |
Stream for lead stage change history.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| History data | Varies | Identifiers and timestamps for stage transitions; structure depends on Bitrix API. |
Stream for Open Channels (Canais Abertos) conversation sessions. One row per session, which is a single conversation segment inside a chat. Requires scope `imopenlines`, and the "Enable Open Channels sessions stream" setting. Incremental via `dateCreate`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the session |
| `configId` | Integer | Identifier of the open line the session belongs to |
| `source` | String | Connector the conversation came in through, e.g. `whatsapp`, `facebook`, `livechat` |
| `operatorId` | Integer | Identifier of the assigned operator; `0` when unassigned |
| `userId` | Integer | Identifier of the client user record created by the connector |
| `userCode` | String | Connector-side client identifier, stable across sessions with the same client |
| `chatId` | Integer | Identifier of the chat that holds the session's messages |
| `dateCreate` | DateTime | Replication key. Timestamp when the session was opened |
| `dateClose` | DateTime | Timestamp when the session was closed; null while still open |
| `dateFirstAnswer` | DateTime | Timestamp of the first answer sent in the session |
| `dateOperatorAnswer` | DateTime | Timestamp of the first answer sent by a human operator |
| `status` | String | Session status, e.g. `new`, `answered`, `closed`, `spam`, `paused` |
| `closeReason` | String | Reason the session was closed, when Bitrix recorded one |
| `vote` | String | Client satisfaction vote, e.g. `none`, `like`, `dislike` |
| `voteHead` | Integer | Supervisor rating given to the session |
| `commentHead` | String | Supervisor comment left on the session |
| `crmEntityType` | String | Type of CRM record the conversation was attached to: `lead`, `deal`, `contact` or `company` |
| `crmEntityId` | Integer | Identifier of the attached CRM record |
| `queueTransfers` | Integer | Number of times the session was transferred between operators or queues |
| `waitAnswer` | Integer | Seconds the client waited for the first answer |
| `waitClose` | Integer | Seconds between the last message and the session being closed |
| `kpiFirstAnswer` | Integer | First-answer KPI outcome; null when no KPI is configured on the line |
| `messageCount` | Integer | Client and operator messages counted by Bitrix. Excludes system messages, so it reads lower than the row count in `imopenlines_messages` |
Stream for the messages exchanged in Open Channels conversations. One row per message, carrying the session context alongside the message itself. Requires scope `imopenlines`, and the "Enable Open Channels messages stream" setting. Incremental via `session_date_create`.
Message text is landed exactly as Bitrix stores it, BBCode markup included (`[b]`, `[USER=23]Name[/USER]`, `[URL=...]`). Clean it up in a transformation if you need plain text.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `message_id` | String | Unique identifier of the message |
| `session_id` | Integer | Session the message belongs to. Joins to `imopenlines_sessions.id` |
| `chat_id` | Integer | Chat holding the session |
| `message_chat_id` | Integer | Chat identifier as reported on the message itself |
| `config_id` | Integer | Open line the conversation came in on |
| `session_source` | String | Connector the conversation came in through |
| `session_status` | String | Status of the session at extraction time |
| `session_date_create` | DateTime | Replication key. Creation timestamp of the session |
| `session_date_close` | DateTime | Timestamp when the session was closed; null while still open |
| `session_operator_id` | Integer | Operator assigned to the session |
| `session_user_code` | String | Connector-side client identifier for the conversation |
| `crm_entity_type` | String | Type of CRM record the conversation was attached to |
| `crm_entity_id` | Integer | Identifier of the attached CRM record |
| `date` | DateTime | Timestamp when the message was posted |
| `text` | String | Message body as Bitrix stores it, BBCode markup included |
| `text_legacy` | String | Legacy rendering of the same body, where some markup is HTML rather than BBCode |
| `author_id` | Integer | Author of the message; `0` for messages Bitrix posts itself |
| `author_name` | String | Display name of the author |
| `author_first_name` | String | First name of the author |
| `author_last_name` | String | Last name of the author |
| `author_work_position` | String | Job title of the author, for staff users who have one |
| `author_external_auth_id` | String | Authentication source of the author; `imconnector` marks the external client |
| `author_is_bot` | Boolean | Whether the author is a chatbot |
| `is_system` | Boolean | Message posted by Bitrix itself, e.g. "conversation started", "operator accepted the chat" |
| `is_client` | Boolean | Message sent by the external client |
| `is_operator` | Boolean | Message sent by a member of staff or a bot |
| `connector_message_ids` | Array of String | Message identifiers assigned by the external channel, e.g. WhatsApp message ids. Use these to reconcile with the messaging provider |
| `is_deleted` | Boolean | Whether the message was deleted in Bitrix |
| `is_edited` | Boolean | Whether the message was edited after being sent |
| `component_id` | String | Interface component that rendered the message, set on system messages |
| `system_event_class` | String | CSS class Bitrix tags system messages with, e.g. `bx-messenger-content-item-ol-end` for a session being closed. Useful to classify system events |
| `files` | Array of Object | Attachments, with `id`, `name`, `type`, `extension`, `size`, `date`, `author_id`, `is_voice_note` and `is_video_note` |
| `attach` | String (JSON) | Rich attachment blocks rendered under the message, such as the CRM record card Bitrix posts when a deal is linked |
| `params` | String (JSON) | Remaining message parameters not promoted to their own columns. The set of keys varies widely by message type |
Stream for Open Lines (communication channels) configuration. Requires scope `imopenlines`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `ACTIVE` | Boolean | Whether the line is active |
| `LINE_NAME` | String | Communication channel name |
| `CRM` | Boolean | CRM routing flag |
| `DATE_CREATE` | String | Creation date |
| `DATE_MODIFY` | String | Modification date |
| Settings | Varies | Queue, welcome, worktime, KPI and other channel settings |
Stream for CRM product catalog. Schema is built from the Bitrix API.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `TIMESTAMP_X` | String | Replication key |
| Custom fields | Varies | Standard and custom product fields from your portal |
Metadata stream for product field definitions.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `statusType`, `isRequired`, `items`, `listLabel`, `formLabel`, etc. |
Stream for CRM quotes. Schema is built from the Bitrix API. Supports incremental sync using the `DATE_MODIFY` replication key.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `DATE_MODIFY` | String | Replication key |
| Custom fields | Varies | Standard and custom (`UF_*`) quote fields from your portal |
Metadata stream for quote field definitions.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field identifier |
| `type` | String | Field type |
| `title` | String | Display name |
| Metadata | Varies | `statusType`, `isRequired`, `isReadOnly`, `isImmutable`, `isMultiple`, `isDynamic`, `isDeprecated`, `items`, `listLabel`, `formLabel`, `filterLabel`, `settings`, `upperName`, etc. |
Stream for products (line items) linked to quotes. Available only when "Enable quote products stream" is turned on. One API call per quote. Supports incremental sync based on the parent quote's modification date (`QUOTE_DATE_MODIFY`).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | String | Unique identifier of the record. |
| `OWNER_ID` | String | Identifier of the associated quote. |
| `OWNER_TYPE` | String | Type of the owner entity. |
| `PRODUCT_ID` | Integer | Identifier of the associated product. |
| `PRODUCT_NAME` | String | Name of the product. |
| `ORIGINAL_PRODUCT_NAME` | String | Original product name from the catalog. |
| `PRODUCT_DESCRIPTION` | String | Description of the product. |
| `PRICE` | Number | Monetary value associated with the record. |
| `PRICE_EXCLUSIVE` | Number | Price excluding tax. |
| `PRICE_NETTO` | Number | Net price of the product. |
| `PRICE_BRUTTO` | Number | Gross price of the product. |
| `PRICE_ACCOUNT` | String | Price in account currency. |
| `QUANTITY` | Integer | Quantity of the product. |
| `DISCOUNT_TYPE_ID` | Integer | Identifier of the discount type. |
| `DISCOUNT_RATE` | Number | Discount rate applied. |
| `DISCOUNT_SUM` | Number | Discount amount. |
| `TAX_RATE` | Number | Tax rate applied to the product. |
| `TAX_INCLUDED` | String | Whether tax is included in the price. |
| `CUSTOMIZED` | String | Whether the row was customized. |
| `MEASURE_CODE` | Integer | Code of the measure unit. |
| `MEASURE_NAME` | String | Name of the measure unit. |
| `SORT` | Integer | Sort order of the row. |
| `XML_ID` | String | External identifier for integration. |
| `TYPE` | Integer | Type classification of the record. |
| `STORE_ID` | Integer | Identifier of the associated store. |
| `RESERVE_ID` | Integer | Identifier of the reserve. |
| `DATE_RESERVE_END` | String | End date of the reservation. |
| `RESERVE_QUANTITY` | Integer | Reserved quantity. |
| `QUOTE_DATE_MODIFY` | DateTime | Timestamp when the parent quote was last updated; inherited from the parent stream and used as the incremental bookmark. |
Stream for SPA (Smart Process Automation) entity types. Used to discover dynamic SPA streams.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| Identifiers | Varies | Entity type identifiers used to build `spa_{entityTypeId}` and `spa_fields_{entityTypeId}` streams. |
Dynamic streams created per SPA entity type in your Bitrix portal (e.g. `spa_128`, `spa_fields_128`). Data and field metadata for each SPA entity type; structure depends on your Bitrix configuration.
Stream for CRM status list (e.g. deal/lead status options).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| Status data | Varies | Status identifiers, labels and configuration; structure depends on Bitrix API. |
Stream for Bitrix24 users. Requires scopes `user`, `user_brief`, `user_basic`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier |
| `NAME` | String | First name |
| `LAST_NAME` | String | Last name |
| `EMAIL` | String | Email address |
| `ACTIVE` | Boolean | Is user active |
| `LAST_LOGIN` | String | Last login date |
| `DATE_REGISTER` | String | Registration date |
| `WORK_POSITION` | String | Job title |
| `UF_DEPARTMENT` | Array | Department IDs |
| Profile fields | Varies | `PERSONAL_*`, `WORK_*` and other profile fields |
Stream containing the stage history records for Smart Process Automations (SPAs). Incremental sync is supported using `CREATED_TIME`.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | String | Unique identifier of the record. |
| `TYPE_ID` | String | Type classification of the record. |
| `OWNER_ID` | String | Identifier of the associated SPA item. |
| `CREATED_TIME` | DateTime | Timestamp when the record was created. |
| `CATEGORY_ID` | String | Identifier of the associated category (funnel). |
| `STAGE_SEMANTIC_ID` | String | Semantic identifier of the stage. |
| `STAGE_ID` | String | Current stage of the record. |
# Data Model
The following diagram illustrates the relationships between the core data streams in Bitrix24. CRM entities relate as follows: **Contacts** and **Companies** are standalone; **Deals** and **Leads** can be linked to contacts and companies. **Deals** are organized by **Deal Categories** (funnels/pipelines). **Activities** can be attached to deals, contacts, and other entities. **product** is the catalog; **deal\_products** links products to deals. **Users** and **departments** describe your organization; **calls** and **open\_lines** are telephony and communication data. Field streams (`*_fields`) describe the schema of their entity (contact, company, deal, lead, activity, product, SPA).
```mermaid theme={null}
graph TD;
subgraph "CRM Entities"
Contacts("Contacts")
Companies("Companies")
Leads("Leads")
Deals("Deals")
DealCategories("Deal Categories")
end
subgraph "Activities & Communications"
Activities("Activities")
Calls("Calls")
OpenLines("Open Lines")
end
subgraph "Products"
Product("Product Catalog")
DealProducts("Deal Products")
end
subgraph "Organization"
Users("Users")
Departments("Departments")
end
Deals -- "CATEGORY_ID" --> DealCategories
Deals -- "linked to" --> Contacts
Deals -- "linked to" --> Companies
Leads -- "linked to" --> Contacts
Leads -- "linked to" --> Companies
DealProducts -- "linked to" --> Deals
DealProducts -- "linked to" --> Product
Activities -- "attached to" --> Deals
Activities -- "attached to" --> Contacts
Activities -- "attached to" --> Leads
Users -- "UF_DEPARTMENT" --> Departments
```
## BI Connector streams
Below you'll find the specific data streams available when using the **BI Connector** in Bitrix24 (ensure `Enable BI reports` is turned on). The connector also fetches standard CRM and organizational streams via the outbound webhook (documented above).
Open Channels dialog statistics from the Bitrix24 BI Connector. It exposes one row per Open Channels session (dialog) with pre-computed SLA metrics: response times, dialog duration, timestamps, operator, channel and status.
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier of the Open Channels session (dialog). |
| `MODE` | String | Session direction, e.g. 'Entrada' for inbound. |
| `STATUS` | String | Human-readable session status, e.g. 'Fechado', 'Spam'. |
| `SPAM` | String | Spam flag, 'Y' if the session was marked as spam, otherwise 'N'. |
| `SOURCE` | String | Source channel of the session, e.g. 'whatsapp', 'facebook', 'fbinstagramdirect', 'facebookcomments'. |
| `CONFIG_ID` | Integer | Identifier of the open line (channel configuration) the session belongs to. |
| `CONFIG_NAME` | String | Name of the open line the session belongs to. |
| `CONFIG` | String | Open line label prefixed with its id, e.g. '\[4] Instagram'. |
| `CLIENT_ID` | Integer | Identifier of the client (contact) in the conversation. |
| `CLIENT_NAME` | String | Display name of the client in the conversation. |
| `CLIENT` | String | Client label prefixed with its id, e.g. '\[26] Guest'. |
| `OPERATOR_ID` | Integer | Identifier of the assigned operator; 0 when no operator was assigned. |
# Implementation Notes
* **Optional streams**: `deal_comments` and `deal_products` appear only when enabled in the source configuration and add one API call per deal; `quote_products` appears only when enabled and adds one API call per quote; `deal_ids` appears only when enabled and is a lightweight ID-only snapshot; `imopenlines_sessions` and `imopenlines_messages` appear only when enabled and read the Open Channels conversation history. Use only if needed.
* **Product streams are incremental through their parent**: `deal_products` and `quote_products` have no modification date of their own, so they inherit the parent's `DATE_MODIFY` as `DEAL_DATE_MODIFY` / `QUOTE_DATE_MODIFY` and use it as the replication key. Only products of deals/quotes modified since the last run are re-fetched.
* **Detecting deleted deals**: Bitrix24 does not expose a deletion flag, so the incremental `deals` stream never learns about deals sent to the recycle bin. Enable the `deal_ids` stream and compare it with your `deals` table: any `ID` present in `deals` but absent from the latest `deal_ids` snapshot was deleted in Bitrix24.
* **Deal filtering**: Use "Filter deals by funnel ID" to sync only specific funnels and reduce volume. The filter applies to both `deals` and `deal_ids`.
* **Rate limits**: If Bitrix returns high "operating time" values, the tap pauses automatically; you can adjust "Operating time threshold" if needed.
* **SPA streams**: `spa_types` is synced first to discover entity types, followed by one `spa_{id}` and one `spa_fields_{id}` stream per type. If the connected user lacks permission to list Smart Processes (e.g., access denied to `crm.type.list`), these streams are safely skipped without failing the rest of the extraction.
* **Connection resilience**: Schema discovery includes connection pooling and an automatic retry mechanism with exponential backoff (up to 5 attempts). This improves stability when dealing with temporary network issues or API timeouts during the initialization phase.
* **SPA custom field collisions**: Bitrix24 Smart Process Automations can return custom fields whose codes differ only by letter case (e.g., `ufCrm10FotoPrincipal` vs `ufCrm10Fotoprincipal`). To avoid data collapsing into a single column, the connector preserves the original column names exactly as returned by Bitrix and automatically resolves collisions by appending a numeric suffix (e.g., `_2`, `_3`) to the overlapping fields. If you need to enforce a specific mapping without changing your Bitrix24 setup, pin the names using the **SPA Column Name Overrides** setting.
### Open Channels conversations
The `imopenlines_sessions` and `imopenlines_messages` streams need only the `imopenlines` scope that the connector already asks for. The `im` scope is **not** involved.
* **How the messages are fetched**: Bitrix returns a session's entire message history in one call, but takes one session at a time. Rather than one request per session, the connector lists the sessions in the replication window and then packs 50 history calls into each batched request, which keeps the extraction fast and well inside Bitrix's operating-time budget.
* **Both streams replicate on the session's creation timestamp**: Messages cannot bookmark on their own timestamp, because a session is fetched whole or not at all. That is why `imopenlines_messages` uses `session_date_create` as its replication key.
* **Long-running conversations**: A conversation that keeps receiving messages for longer than the lookback stops being refreshed. Raise **Open Channels messages lookback (days)** on portals where sessions stay open for weeks.
* **Row counts do not match `messageCount`**: The `messageCount` on a session counts only client and operator messages, so it reads lower than the number of `imopenlines_messages` rows for that session, which also include Bitrix's own system messages ("conversation started", "operator accepted the chat", "deal attached to the conversation").
* **Telling clients from operators**: Use the `is_client`, `is_operator` and `is_system` flags rather than trying to infer it from the author id. They are derived at extraction time from information Bitrix only returns alongside the messages.
* **Attachments**: File names, types and sizes are landed, but download URLs are not. Bitrix signs them with a short-lived token that would already be expired by the time the table is read.
* **Message text is raw**: Bodies keep their BBCode markup. Strip it in a transformation if you need plain text.
* **Relationship with `imopenlines_session_stats`**: Both describe Open Channels sessions. `imopenlines_sessions` comes from the REST API, needs no BI connector key, and carries the chat and CRM record identifiers; `imopenlines_session_stats` comes from the BI Connector and carries Bitrix's pre-computed SLA metrics. Use the first when you need to join conversations to the CRM, the second when you need the ready-made report figures.
### BI Connector Replication & Data Quirks
When utilizing the **BI Connector** to extract reports like `imopenlines_session_stats`, be aware of the following behaviors and quirks:
* **Incremental Syncs on `DATE_CREATE`**: The BI dataset has no `DATE_MODIFY` column, so `DATE_CREATE` is used as the replication key.
* **Mutable Sessions**: A session created today might be answered or closed days/weeks later, which backfills its stats (e.g. `TIME_*`, `DATE_*`). The primary key is `ID`, and downstream merges should perform an **upsert**. The `bi_reports_lookback_days` configuration handles re-pulling recent sessions to capture late updates.
* **Timestamps**: All timestamps are portal-local (e.g. `YYYY-MM-DD HH:MM:SS` in the account's configured timezone) and are landed **as-is** without timezone conversions.
* **String replication key**: Because BI datasets expose naive, portal-local timestamp strings, the replication key is landed unchanged as a string type rather than a timestamp type. The connector safely parses the raw bookmark value internally to calculate the proper lookback windows without causing timestamp validation errors, ensuring reliable incremental syncs for BI reports.
* **Inflated Response Times**: Unanswered sessions will show constantly accruing `TIME_*` metrics while a client waits. Filter on answered status states before computing SLA aggregates.
* **No Operator on Certain Channels**: Sessions coming from Facebook or Instagram comments might register an `OPERATOR_ID` of `0` with a null operator name.
## Skills for agents
Bitrix24 connector documentation as plain markdown, for use in AI agent contexts.
# BIU as a data source
Source: https://docs.nekt.com/sources/biu
Bring data from BIU to Nekt.
BIU is a Brazilian financial management platform focused on automating invoice issuance (NFe and NFs) for digital product sales. It integrates with major sales platforms like Hotmart, Eduzz, and others, consolidating sales data, fiscal notes, and financial dashboards in one place.
![]()
## Configuring BIU as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the BIU option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your BIU API token to authenticate. You can generate a token at [https://app.somosbiu.com/api-token](https://app.somosbiu.com/api-token).
The following configurations are available:
* **API Token**: Your BIU API bearer token for authentication.
* **Start Date**: (Optional) Earliest date to sync data from. If not set, all available data will be extracted.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: you can customize the name or add a **prefix** to all tables at once.
* **Sync Type**: choose between INCREMENTAL and FULL\_TABLE.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from BIU and their corresponding fields:
Consolidated financial performance summary including sales totals, fiscal notes (NFe/NFs), and operational alerts for a given competence period.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `competence` | String | Monthly period in YYYY-MM format (primary key) |
| `start_date` | String | Start date of the period |
| `end_date` | String | End date of the period |
| `sales_date_field` | String | Date field used for filtering |
| `sales_warning` | Number | Sales amount flagged as warning |
| `sales_warning_qtt` | Integer | Number of warning sales |
| `sales_pending` | Number | Pending sales amount |
| `sales_pending_qtt` | Integer | Number of pending sales |
| `sales_canceled` | Number | Canceled sales amount |
| `sales_canceled_qtt` | Integer | Number of canceled sales |
| `sales_coproducer` | Number | Coproducer sales amount |
| `sales_coproducer_qtt` | Integer | Number of coproducer sales |
| `sales_processed` | Number | Processed sales amount |
| `sales_processed_qtt` | Integer | Number of processed sales |
| `sales_dont_issue` | Number | Sales marked as "don't issue" amount |
| `sales_dont_issue_qtt` | Integer | Number of "don't issue" sales |
| `sales_total_qtt` | Integer | Total number of sales |
| `sales_processed_pct` | Number | Percentage of processed sales |
| `fnp_warning` | Number | NFe warnings amount |
| `fnp_warning_qtt` | Integer | Number of NFe warnings |
| `fnp_pending` | Number | Pending NFe amount |
| `fnp_pending_qtt` | Integer | Number of pending NFe |
| `fnp_canceled` | Number | Canceled NFe amount |
| `fnp_canceled_qtt` | Integer | Number of canceled NFe |
| `fnp_coproducer` | Number | Coproducer NFe amount |
| `fnp_coproducer_qtt` | Integer | Number of coproducer NFe |
| `fnp_processed` | Number | Processed NFe amount |
| `fnp_processed_qtt` | Integer | Number of processed NFe |
| `fnp_dont_issue` | Number | "Don't issue" NFe amount |
| `fnp_dont_issue_qtt` | Integer | Number of "don't issue" NFe |
| `fnp_total_qtt` | Integer | Total NFe count |
| `fnp_processed_pct` | Number | Percentage of processed NFe |
| `fns_warning` | Number | NFs warnings amount |
| `fns_warning_qtt` | Integer | Number of NFs warnings |
| `fns_pending` | Number | Pending NFs amount |
| `fns_pending_qtt` | Integer | Number of pending NFs |
| `fns_canceled` | Number | Canceled NFs amount |
| `fns_canceled_qtt` | Integer | Number of canceled NFs |
| `fns_coproducer` | Number | Coproducer NFs amount |
| `fns_coproducer_qtt` | Integer | Number of coproducer NFs |
| `fns_processed` | Number | Processed NFs amount |
| `fns_processed_qtt` | Integer | Number of processed NFs |
| `fns_dont_issue` | Number | "Don't issue" NFs amount |
| `fns_dont_issue_qtt` | Integer | Number of "don't issue" NFs |
| `fns_total_qtt` | Integer | Total NFs count |
| `fns_processed_pct` | Number | Percentage of processed NFs |
| `alerts_pending_products_qtt` | Integer | Number of pending products |
| `alerts_expired_certificates_qtt` | Integer | Number of expired certificates |
| `alerts_is_single_company` | Boolean | Whether the account is a single company |
| `alerts_cert_expires` | String | Certificate expiration date |
| `alerts_cert_days_left` | Integer | Days until certificate expires |
| `alerts_cert_expired` | Boolean | Whether the certificate is expired |
**Notes:**
* **Replication**: The stream uses `competence` as the replication key. Incremental syncs request data starting from the last replicated month.
* **Pagination**: The connector iterates one month at a time from the configured `start_date` (or the last saved state) up to the current month to fetch all records seamlessly.
## API Limits
The BIU API enforces a rate limit of 20 requests per minute per company. Exceeding it blocks the company for 1 hour. The connector automatically handles this by restricting calls to 15 requests per 60 seconds to stay safely under the limit.
## Skills for agents
BIU connector documentation as plain markdown, for use in AI agent contexts.
# Bling as a data source
Source: https://docs.nekt.com/sources/bling
Bring data from Bling to Nekt.
Bling is a comprehensive business management platform that provides tools for invoicing (NFe, NFCe, NFSe), inventory control, sales management, financial operations (accounts payable/receivable), and CRM. It helps businesses streamline their operations with integrated solutions for e-commerce, accounting, and logistics.
![]()
## Configuring Bling as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Bling option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need OAuth credentials to connect to Bling. Log in with your Bling account to generate the access token. Make sure you use an account with sufficient privileges to access your Bling data.
The following configurations are available:
* **Start Date**: Records created or updated after this date will be extracted from the source.
* **Lookback Window**: Optional. The number of days to look back when re-fetching records based on the replication key. Leave it empty to use the connector's per-stream defaults (a few hours for streams filtered by modification date, up to 90 days for contracts); when set, it overrides those defaults for **all** streams, so higher values increase run time.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. The available streams include invoices (NFe, NFSe), products, sales orders, accounts payable/receivable, cash registers, stock balances, contacts, and more.
> Tip: The stream can be found more easily by typing its name.
If the connected Bling user lacks permission to a specific module, the connector skips the corresponding stream instead of failing the run. If a stream's table stays empty or is not updated, verify that the user has the necessary module permissions in Bling, reconnect the account and run a new sync.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Bling API only allows fetching historical data from 2025-01-01.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## API Limitations & Performance
**Important**: Bling has strict API rate limits that affect extraction performance:
* **\~3 requests per second** (the connector spaces requests at least 0.4 seconds apart to prevent burst rejections; the quota is per Bling account, so other sources of the same account share it)
* **120,000 requests per day** (daily quota). If this limit is reached during a run, the extraction will stop gracefully, save its state, and resume from where it left off in the next scheduled run.
### Understanding Sync Times
Bling uses a **parent-child stream architecture** for detailed data. For each parent record (e.g., a Product or Sales Order), an additional API request is made to fetch the detailed information. This means:
* **Products + Product Details**: 1 request for the list + 1 request per product
* **Product Stores + Product Store Details**: 1 request for the list + 1 request per product-store link
* **Sales Orders + Sales Order Details**: 1 request for the list + 1 request per order
* **Purchase Orders + Purchase Order Details**: 1 request for the list + 1 request per order
* **Contracts + Contract Details**: 1 request for the list + 1 request per contract
* **Commercial Proposals + Commercial Proposal Details**: 1 request for the list + 1 request per proposal
* **NFe + NFe Details + NFe Document**: 1 request for the list + 1 request per invoice for each child stream
* **NFe Purchase + NFe Purchase Details**: 1 request for the list + 1 request per invoice
* **NFCe + NFCe Details**: 1 request for the list + 1 request per invoice
* **Contacts + Contact Details**: 1 request for the list + 1 request per contact
* **Cash Registers + Cash Register Details**: 1 request for the list + 1 request per entry
* **Stock Balances**: Batch requests for multiple products to optimize API usage
**Tip for faster syncs**: If you don't need the detailed information, consider syncing only the parent streams (e.g., `Products` without `Product Details`). This significantly reduces the number of API calls and sync time.
For example, if you have 10,000 products and select both `Products` and `Product Details` streams:
* At 3 requests/second, fetching all product details alone would take approximately **55 minutes**
* With 120,000 daily requests, you're limited to syncing approximately **120,000 detailed records per day**
* If you have more than 120,000 detailed records in your account, an initial sync might hit the daily limit. The connector handles this by gracefully pausing the extraction when the limit is reached and automatically resuming during the next run until all historical data is fetched.
**Recommendations:**
1. Start with a minimal set of streams and add more as needed
2. Use incremental sync when available to reduce the number of records fetched
3. Schedule full syncs during off-peak hours if you have large datasets
4. Consider syncing only parent streams if detailed data isn't required
## Stream to Endpoint Mapping
| Stream | API Endpoint | Sync Type | Notes |
| - | - | - | - |
| NFe | `/nfe` | Incremental | Uses `dataEmissao` as replication key |
| NFe Details | `/nfe/{id}` | Incremental | Child stream, 1 request per NFe |
| NFe Document | `/nfe/documento/{chaveAcesso}` | Incremental | Child stream, 1 request per NFe; fetches the XML using the parent's access key |
| NFe Purchase | `/nfe` (`tipo=0`) | Incremental | Purchase (inbound) NFe; uses `dataEmissao` as replication key |
| NFe Purchase Details | `/nfe/{id}` | Incremental | Child stream, 1 request per purchase NFe |
| NFCe | `/nfce` | Incremental | Uses `dataEmissao` as replication key |
| NFCe Details | `/nfce/{id}` | Incremental | Child stream, 1 request per NFCe |
| NFSe | `/nfse` | Incremental | Uses `dataEmissao` as replication key |
| NFe Situation | Static | Full Table | Reference data |
| NFSe Situation | Static | Full Table | Reference data |
| Operation Types | `/naturezas-operacoes` | Full Table | |
| Products | `/produtos` | Incremental | Uses `dataFiltroAlteracao` as replication key |
| Product Details | `/produtos/{id}` | Incremental | Child stream, 1 request per product |
| Product Categories | `/categorias/produtos` | Full Table | |
| Product Stores | `/produtos/lojas` | Full Table | |
| Product Store Details | `/produtos/lojas/{id}` | Full Table | Child stream, 1 request per product-store link |
| Product Suppliers | `/produtos/fornecedores` | Full Table | |
| Sales Orders | `/pedidos/vendas` | Incremental | Uses `dataFiltroAlteracao` as replication key |
| Sales Order Details | `/pedidos/vendas/{id}` | Incremental | Child stream, 1 request per order |
| Purchase Orders | `/pedidos/compras` | Incremental | Uses `data` as replication key |
| Purchase Order Details | `/pedidos/compras/{id}` | Incremental | Child stream, 1 request per order |
| Commercial Proposals | `/propostas-comerciais` | Incremental | Uses `data` as replication key |
| Commercial Proposal Details | `/propostas-comerciais/{id}` | Incremental | Child stream, 1 request per proposal |
| Contracts | `/contratos` | Incremental | Uses `data` as replication key |
| Contract Details | `/contratos/{id}` | Incremental | Child stream, 1 request per contract |
| Sales Channels | `/canais-venda` | Full Table | |
| Sellers | `/vendedores` | Full Table | |
| Logistics | `/logisticas` | Full Table | |
| Accounts Receivable | `/contas/receber` | Incremental | Uses `dataEmissao` as replication key |
| Accounts Receivable Details | `/contas/receber/{id}` | Incremental | Child stream, 1 request per receivable |
| Borderos | `/borderos/{bordero_id}` | Incremental | Child stream, 1 request per bordero |
| Accounts Payable | `/contas/pagar` | Incremental | Uses `dataFiltro` as replication key |
| Accounts Payable Details | `/contas/pagar/{id}` | Incremental | Child stream, 1 request per payable |
| Boleto | `/contas/receber/boletos` | Incremental | Child stream of Accounts Receivable, 1 request per receivable that has a boleto link and an origin; uses `dataEmissaoNfe` as replication key |
| Cash Registers | `/caixas` | Incremental | Uses `data` as replication key |
| Cash Register Details | `/caixas/{id}` | Incremental | Child stream, 1 request per entry |
| Accounting Accounts | `/contas-contabeis` | Full Table | |
| Accounting Account Details | `/contas-contabeis/{id}` | Full Table | Child stream, 1 request per account |
| Deposits | `/depositos` | Full Table | |
| Revenue Expense Categories | `/categorias/receitas-despesas` | Full Table | |
| Revenue Expense Category Details | `/categorias/receitas-despesas/{id}` | Full Table | Child stream |
| Store Categories | `/categorias/lojas` | Full Table | |
| Store Category Details | `/categorias/lojas/{id}` | Full Table | Child stream |
| Situation Modules | `/situacoes/modulos` | Full Table | |
| Situations | `/situacoes/modulos/{module_id}` | Full Table | Child stream, 1 request per module |
| Stock Balances | `/estoques/saldos` | Incremental | Child stream, batch requests for multiple products |
| Contacts | `/contatos` | Incremental | Uses `dataFiltroAlteracao` as replication key |
| Contact Details | `/contatos/{id}` | Incremental | Child stream, 1 request per contact |
# Streams and Fields
Below you'll find all available data streams from Bling and their corresponding fields:
### Invoices & Fiscal Documents
## Contacts & CRM
Stream for managing contacts (customers, suppliers, partners). Supports incremental sync using modification date filter.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The contact ID |
| `nome` | String | The contact name |
| `codigo` | String | The contact code |
| `situacao` | String | The contact status (A = Active, I = Inactive, S = Without movement) |
| `numeroDocumento` | String | The contact document number (CPF/CNPJ) |
| `telefone` | String | The contact phone number |
| `celular` | String | The contact mobile number |
| `dataFiltroAlteracao` | DateTime | The filter date used for incremental sync (YYYY-MM-DD HH:mm:ss format) |
Stream for detailed contact information (child of Contacts).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The contact ID |
| `pessoasContato` | Array | Contact persons associated with this contact |
| `data` | String | The contact creation/update date |
| `dataFiltroAlteracao` | DateTime | The filter date (YYYY-MM-DD HH:mm:ss format) |
| *(Additional fields)* | *Various* | *Inherits comprehensive structure (address, financial details, documentation) from Bling's detailed contact API* |
## Invoices & Fiscal Documents
Stream for managing electronic invoices (NFe) for products.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The NFe ID |
| `tipo` | Integer | The NFe type |
| `situacao` | Integer | The NFe status |
| `numero` | String | The NFe number |
| `dataEmissao` | DateTime | The emission date (YYYY-MM-DD HH:MM:SS format) |
| `dataOperacao` | String | The operation date (YYYY-MM-DD HH:MM:SS format) |
| `chaveAcesso` | String | The access key |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The document number |
| `contato.ie` | String | State registration |
| `contato.rg` | String | RG number |
| `contato.telefone` | String | Phone number |
| `contato.email` | String | Email address |
| `contato.endereco.endereco` | String | Street address |
| `contato.endereco.numero` | String | Address number |
| `contato.endereco.complemento` | String | Address complement |
| `contato.endereco.bairro` | String | Neighborhood |
| `contato.endereco.cep` | String | ZIP code |
| `contato.endereco.municipio` | String | City |
| `contato.endereco.uf` | String | State |
| `contato.endereco.pais` | String | Country |
| `naturezaOperacao.id` | Integer | The nature of operation ID |
| `loja.id` | Integer | The store ID |
Stream for detailed NFe information (child of NFe stream).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The NFe ID |
| `tipo` | Integer | The NFe type |
| `situacao` | Integer | The NFe status |
| `numero` | String | The NFe number |
| `dataEmissao` | String | The emission date |
| `dataOperacao` | String | The operation date |
| `chaveAcesso` | String | The access key |
| `serie` | Integer | The NFe series |
| `valorNota` | Number | The invoice value |
| `valorFrete` | Number | The freight value |
| `xml` | String | The XML content |
| `linkDanfe` | String | The DANFE link |
| `linkPDF` | String | The PDF link |
| `optanteSimplesNacional` | Boolean | Simple National taxpayer flag |
| `numeroPedidoLoja` | String | Store order number |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The document number |
| `contato.ie` | String | State registration |
| `contato.rg` | String | RG number |
| `contato.telefone` | String | Phone number |
| `contato.email` | String | Email address |
| `contato.endereco.endereco` | String | Street address |
| `contato.endereco.numero` | String | Address number |
| `contato.endereco.complemento` | String | Address complement |
| `contato.endereco.bairro` | String | Neighborhood |
| `contato.endereco.cep` | String | ZIP code |
| `contato.endereco.municipio` | String | City |
| `contato.endereco.uf` | String | State |
| `contato.endereco.pais` | String | Country |
| `naturezaOperacao.id` | Integer | The nature of operation ID |
| `loja.id` | Integer | The store ID |
| `transporte.fretePorConta` | Integer | Freight responsibility |
| `transporte.transportador.nome` | String | Transporter name |
| `transporte.transportador.numeroDocumento` | String | Transporter document number |
| `transporte.volumes[].id` | Integer | Volume ID |
| `transporte.etiqueta.nome` | String | Label name |
| `transporte.etiqueta.endereco` | String | Label street address |
| `transporte.etiqueta.numero` | String | Label address number |
| `transporte.etiqueta.complemento` | String | Label address complement |
| `transporte.etiqueta.municipio` | String | Label city |
| `transporte.etiqueta.uf` | String | Label state |
| `transporte.etiqueta.cep` | String | Label ZIP code |
| `transporte.etiqueta.bairro` | String | Label neighborhood |
| `vendedor.id` | Integer | The seller ID |
| `itens[].codigo` | String | Item code |
| `itens[].descricao` | String | Item description |
| `itens[].unidade` | String | Item unit |
| `itens[].quantidade` | Number | Item quantity |
| `itens[].valor` | Number | Item value |
| `itens[].valorTotal` | Number | Item total value |
| `itens[].tipo` | String | Item type |
| `itens[].pesoBruto` | Number | Item gross weight |
| `itens[].pesoLiquido` | Number | Item net weight |
| `itens[].numeroPedidoCompra` | String | Purchase order number |
| `itens[].classificacaoFiscal` | String | Fiscal classification |
| `itens[].cest` | String | CEST code |
| `itens[].codigoServico` | String | Service code |
| `itens[].origem` | Integer | Origin |
| `itens[].informacoesAdicionais` | String | Additional information |
| `itens[].gtin` | String | GTIN code |
| `itens[].cfop` | String | CFOP code |
| `itens[].documentoReferenciado.chaveAcesso` | String | Referenced document access key |
| `itens[].documentoReferenciado.numero` | String | Referenced document number |
| `itens[].documentoReferenciado.serie` | String | Referenced document series |
| `itens[].documentoReferenciado.numeroItem` | String | Referenced item number |
| `itens[].impostos.valorAproximadoTotalTributos` | Number | Approximate total tax value |
| `itens[].impostos.icms.st` | Number | ICMS ST |
| `itens[].impostos.icms.origem` | Integer | ICMS origin |
| `itens[].impostos.icms.modalidade` | Integer | ICMS origin |
| `itens[].impostos.icms.aliquota` | Number | ICMS rate |
| `itens[].impostos.icms.valor` | Number | ICMS value |
| `parcelas[].data` | String | Installment date |
| `parcelas[].valor` | Number | Installment value |
| `parcelas[].observacoes` | String | Installment observations |
| `parcelas[].caut` | String | CAUT code |
| `parcelas[].formaPagamento.id` | Integer | Payment method ID |
Stream for retrieving the XML document content of an NFe (child of NFe stream). Supports incremental sync.
NFe records without an access key (`chaveAcesso`) are skipped, as their XML document is not available in Bling.
| Field | Type | Description |
| - | - | - |
| `chaveAcesso` | String | The 44-digit NFe access key (from the parent NFe) |
| `nome` | String | The name of the NFe document |
| `conteudo` | String | The content of the NFe document (XML) |
| `dataEmissaoNfe` | DateTime | The emission date of the NFe (from the parent NFe) |
Stream for managing electronic service invoices (NFSe). Unlike NFe which is for products, NFSe is specifically for service-based transactions.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | Unique identifier for the NFSe |
| `numero` | String | The NFSe number |
| `numeroRPS` | String | The RPS (Recibo Provisório de Serviços) number |
| `serie` | String | The NFSe series |
| `situacao` | Integer | The NFSe status |
| `dataEmissao` | String | The emission date (YYYY-MM-DD format) |
| `valor` | Number | The service value |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The document number |
| `contato.email` | String | Email address |
Static stream providing a mapping of NFe situation IDs to their descriptions.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The NFe situation ID |
| `situacao` | String | The NFe situation status description (e.g., "Pendente", "Autorizada", "Cancelada") |
Static stream providing a mapping of NFSe situation IDs to their descriptions.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The NFSe situation ID |
| `situacao` | String | The NFSe situation status description (e.g., "Pendente", "Emitida", "Cancelada") |
Stream for managing different types of business operations (nature of operations for invoices).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The operation type ID |
| `descricao` | String | The operation type description |
| `situacao` | Integer | The operation type status (1 = Active, 0 = Inactive) |
| `padrao` | Boolean | Indicates if this is the default operation type |
### Sales & Orders
Stream for managing sales orders. Supports incremental sync using modification date filter.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The sales order ID |
| `numero` | Integer | The sales order number |
| `numeroLoja` | String | The store order number |
| `data` | Date | The order date (YYYY-MM-DD format) |
| `dataSaida` | Date | The departure date (YYYY-MM-DD format) |
| `dataPrevista` | Date | The expected delivery date (YYYY-MM-DD format) |
| `totalProdutos` | Number | The total products value |
| `total` | Number | The total order value |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.tipoPessoa` | String | Person type (F = Individual, J = Legal Entity) |
| `contato.numeroDocumento` | String | The document number |
| `situacao.id` | Integer | The status ID |
| `situacao.valor` | Integer | The status value |
| `loja.id` | Integer | The store ID |
| `dataFiltroAlteracao` | DateTime | The filter date used for incremental sync (replication key) |
Stream for detailed sales order information (child of Sales Orders).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The sales order ID |
| `numero` | Integer | The sales order number |
| `numeroLoja` | String | The store order number |
| `data` | Date | The order date (YYYY-MM-DD format) |
| `dataSaida` | Date | The departure date (YYYY-MM-DD format) |
| `dataPrevista` | Date | The expected delivery date (YYYY-MM-DD format) |
| `totalProdutos` | Number | The total products value |
| `total` | Number | The total order value |
| `numeroPedidoCompra` | String | Purchase order number |
| `outrasDespesas` | Number | Other expenses |
| `observacoes` | String | Order observations |
| `observacoesInternas` | String | Internal observations |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.tipoPessoa` | String | Person type (F = Individual, J = Legal Entity) |
| `contato.numeroDocumento` | String | The document number |
| `situacao.id` | Integer | The status ID |
| `situacao.valor` | Integer | The status value |
| `loja.id` | Integer | The store ID |
| `desconto.valor` | Number | Discount value |
| `desconto.unidade` | String | Discount unit (REAL, PERCENTUAL) |
| `categoria.id` | Integer | The category ID |
| `notaFiscal.id` | Integer | The invoice ID |
| `tributacao.totalICMS` | Number | Total ICMS tax |
| `tributacao.totalIPI` | Number | Total IPI tax |
| `itens[].id` | Integer | Item ID |
| `itens[].codigo` | String | Item code |
| `itens[].unidade` | String | Item unit |
| `itens[].quantidade` | Number | Item quantity |
| `itens[].desconto` | Number | Item discount |
| `itens[].valor` | Number | Item value |
| `itens[].aliquotaIPI` | Number | IPI rate |
| `itens[].descricao` | String | Item description |
| `itens[].descricaoDetalhada` | String | Detailed description |
| `itens[].produto.id` | Integer | Product ID |
| `itens[].comissao.base` | Number | Commission base |
| `itens[].comissao.aliquota` | Number | Commission rate |
| `itens[].comissao.valor` | Number | Commission value |
| `itens[].naturezaOperacao.id` | Integer | Natureza Operação ID |
| `parcelas[].id` | Integer | Installment ID |
| `parcelas[].dataVencimento` | Date | Due date |
| `parcelas[].valor` | Number | Installment value |
| `parcelas[].observacoes` | String | Installment observations |
| `parcelas[].caut` | String | CAUT code |
| `parcelas[].formaPagamento.id` | Integer | Payment method ID |
| `transporte.fretePorConta` | Integer | Freight responsibility |
| `transporte.frete` | Number | Freight value |
| `transporte.quantidadeVolumes` | Integer | Number of volumes |
| `transporte.pesoBruto` | Number | Gross weight |
| `transporte.prazoEntrega` | Integer | Delivery term |
| `transporte.contato.id` | Integer | Transporter contact ID |
| `transporte.contato.nome` | String | Transporter name |
| `transporte.etiqueta.nome` | String | Label name |
| `transporte.etiqueta.endereco` | String | Label address |
| `transporte.etiqueta.numero` | String | Label address number |
| `transporte.etiqueta.complemento` | String | Label address complement |
| `transporte.etiqueta.municipio` | String | Label city |
| `transporte.etiqueta.uf` | String | Label state |
| `transporte.etiqueta.cep` | String | Label ZIP code |
| `transporte.etiqueta.bairro` | String | Label neighborhood |
| `transporte.etiqueta.nomePais` | String | Label country name |
| `transporte.volumes[].id` | Integer | Volume ID |
| `transporte.volumes[].servico` | String | Service |
| `transporte.volumes[].codigoRastreamento` | String | Tracking code |
| `vendedor.id` | Integer | The seller ID |
| `intermediador.cnpj` | String | Intermediator CNPJ |
| `intermediador.nomeUsuario` | String | Username |
| `taxas.taxaComissao` | Number | Commission rate |
| `taxas.custoFrete` | Number | Freight cost |
| `taxas.valorBase` | Number | Base value |
### Products & Inventory
Stream for managing product catalog. Supports incremental sync using modification date filter.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The product ID |
| `idProdutoPai` | Integer | The parent product ID (for variations) |
| `nome` | String | The product name |
| `codigo` | String | The product code/SKU |
| `preco` | Number | The product price |
| `precoCusto` | Number | The product cost price |
| `estoque.saldoVirtualTotal` | Number | Total virtual stock balance |
| `tipo` | String | The product type (P = Product, S = Service) |
| `situacao` | String | The product status (A = Active, I = Inactive) |
| `formato` | String | The product format (S = Simple, V = Variable) |
| `descricaoCurta` | String | Short product description |
| `imagemURL` | String | Product image URL |
| `dataFiltroAlteracao` | Date | The filter date used for incremental sync (replication key) |
Stream for detailed product information (child of Products).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The product ID |
| `nome` | String | The product name |
| `codigo` | String | The product code |
| `preco` | Number | The product price |
| `tipo` | String | The product type (P = Product, S = Service) |
| `situacao` | String | The product status (A = Active, I = Inactive) |
| `formato` | String | The product format (S = Simple, V = Variable) |
| `descricaoCurta` | String | Short product description |
| `imagemURL` | String | Product image URL |
| `dataValidade` | String | Product validity date |
| `unidade` | String | Product unit |
| `pesoLiquido` | Number | Net weight |
| `pesoBruto` | Number | Gross weight |
| `volumes` | Number | Number of volumes |
| `itensPorCaixa` | Number | Items per box |
| `gtin` | String | GTIN code |
| `gtinEmbalagem` | String | Package GTIN code |
| `tipoProducao` | String | Production type |
| `condicao` | Integer | Product condition |
| `freteGratis` | Boolean | Free shipping flag |
| `marca` | String | Product brand |
| `descricaoComplementar` | String | Complementary description |
| `linkExterno` | String | External link |
| `observacoes` | String | Product observations |
| `descricaoEmbalagemDiscreta` | String | Discrete packaging description |
| `actionEstoque` | String | Stock action |
| `categoria.id` | Integer | The category ID |
| `estoque.minimo` | Number | Minimum stock |
| `estoque.maximo` | Number | Maximum stock |
| `estoque.crossdocking` | Number | Crossdocking quantity |
| `estoque.localizacao` | String | Stock location |
| `estoque.saldoVirtualTotal` | Number | Total virtual stock balance |
| `fornecedor.id` | Integer | The supplier ID |
| `fornecedor.contato.id` | Integer | Supplier contact ID |
| `fornecedor.contato.nome` | String | Supplier contact name |
| `fornecedor.codigo` | String | Supplier code |
| `fornecedor.precoCusto` | Number | Cost price |
| `fornecedor.precoCompra` | Number | Purchase price |
| `dimensoes.largura` | Number | Width |
| `dimensoes.altura` | Number | Height |
| `dimensoes.profundidade` | Number | Depth |
| `dimensoes.unidadeMedida` | Integer | Measurement unit |
| `tributacao.origem` | Integer | Origin |
| `tributacao.nFCI` | String | NFCI code |
| `tributacao.ncm` | String | NCM code |
| `tributacao.cest` | String | CEST code |
| `tributacao.codigoListaServicos` | String | Service list code |
| `tributacao.spedTipoItem` | String | SPED item type |
| `tributacao.codigoItem` | String | Item code |
| `tributacao.percentualTributos` | Number | Tax percentage |
| `tributacao.valorBaseStRetencao` | Number | ST retention base value |
| `tributacao.valorStRetencao` | Number | ST retention value |
| `tributacao.valorICMSSubstituto` | Number | ICMS substitute value |
| `tributacao.codigoExcecaoTipi` | String | TIPI exception code |
| `tributacao.classeEnquadramentoIpi` | String | IPI classification class |
| `tributacao.valorIpiFixo` | Number | Fixed IPI value |
| `tributacao.codigoSeloIpi` | String | IPI seal code |
| `tributacao.valorPisFixo` | Number | Fixed PIS value |
| `tributacao.valorCofinsFixo` | Number | Fixed COFINS value |
| `tributacao.codigoANP` | String | ANP code |
| `tributacao.descricaoANP` | String | ANP description |
| `tributacao.percentualGLP` | Number | GLP percentage |
| `tributacao.percentualGasNacional` | Number | National gas percentage |
| `tributacao.percentualGasImportado` | Number | Imported gas percentage |
| `tributacao.valorPartida` | Number | Starting value |
| `tributacao.tipoArmamento` | Integer | Armament type |
| `tributacao.descricaoCompletaArmamento` | String | Complete armament description |
| `tributacao.dadosAdicionais` | String | Additional data |
| `tributacao.grupoProduto.id` | Integer | The product group ID |
| `midia.video.url` | String | Video URL |
| `midia.imagens.imagensURL` | Array | Images URL |
| `midia.imagens.externas[].link` | String | External image link |
| `midia.imagens.internas[].link` | String | Internal image link |
| `midia.imagens.internas[].linkMiniatura` | String | Thumbnail link |
| `midia.imagens.internas[].validade` | String | Image validity |
| `midia.imagens.internas[].ordem` | Integer | Image order |
| `midia.imagens.internas[].anexo.id` | Integer | Attachment ID |
| `midia.imagens.internas[].anexoVinculo.id` | Integer | Attachment link ID |
| `linhaProduto.id` | Integer | The product line ID |
| `estrutura.tipoEstoque` | String | Stock type |
| `estrutura.lancamentoEstoque` | String | Stock release type |
| `estrutura.componentes[].produto.id` | Integer | Component product ID |
| `estrutura.componentes[].quantidade` | Number | Component quantity |
| `camposCustomizados[].idCampoCustomizado` | Integer | Custom field ID |
| `camposCustomizados[].idVinculo` | String | Link ID for updating |
| `camposCustomizados[].valor` | String | Custom field value |
| `camposCustomizados[].item` | String | Custom field item |
| `variacoes[]` | Array | Product variations (contains same fields as parent product) |
| `product_id` | Integer | Parent product ID from parent stream |
Stream for managing product categories.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The category ID |
| `descricao` | String | The category description |
| `categoriaPai.id` | Integer | Parent category ID |
Stream for managing product-store links (produtos lojas).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The product-store link ID |
| `codigo` | String | The product code in this store |
| `preco` | Number | The product price in this store |
| `precoPromocional` | Number | The promotional price in this store |
| `produto.id` | Integer | The product ID |
| `loja.id` | Integer | The store ID |
| `fornecedorLoja.id` | Integer | The store supplier ID |
| `marcaLoja.id` | Integer | The store brand ID |
| `categoriasProdutos[].id` | Integer | The product category ID |
Stream for detailed product-store link information (child of Product Stores).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The product-store link ID |
| `codigo` | String | The product code in this store |
| `preco` | Number | The product price in this store |
| `precoPromocional` | Number | The promotional price in this store |
| `produto.id` | Integer | The product ID |
| `loja.id` | Integer | The store ID |
| `fornecedorLoja.id` | Integer | The store supplier ID |
| `marcaLoja.id` | Integer | The store brand ID |
| `categoriasProdutos[].id` | Integer | The product category ID |
Stream for retrieving product suppliers (produtos fornecedores). Extracts all suppliers linked to a product, and is extracted in full as no date filter is available.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The product-supplier link ID |
| `descricao` | String | The product description used by this supplier |
| `codigo` | String | The product code used by this supplier |
| `precoCusto` | Number | The product cost price from this supplier |
| `precoCompra` | Number | The product purchase price from this supplier |
| `padrao` | Boolean | Whether this is the product's default supplier |
| `produto.id` | Integer | The product ID |
| `fornecedor.id` | Integer | The supplier contact ID (joins with contact\_details.id) |
Stream for retrieving current stock balances by product and deposit. This stream provides real-time inventory information across different warehouses and locations.
| Field | Type | Description |
| - | - | - |
| `idProduto` | Integer | The product ID |
| `produto.id` | Integer | The product ID (nested object) |
| `produto.codigo` | String | The product code/SKU |
| `saldoFisicoTotal` | Number | Total physical stock balance |
| `saldoVirtualTotal` | Number | Total virtual stock balance |
| `depositos[].id` | Integer | The deposit/warehouse ID |
| `depositos[].saldoFisico` | Number | Physical stock balance in this deposit |
| `depositos[].saldoVirtual` | Number | Virtual stock balance in this deposit (physical minus reserved) |
| `dataFiltroAlteracao` | Date | The filter date inherited from the parent product stream |
### Financial
Stream for managing accounts receivable (amounts to be received).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The account receivable ID |
| `situacao` | Integer | The account receivable status |
| `vencimento` | Date | The due date (YYYY-MM-DD format) |
| `valor` | Number | The amount to be received |
| `idTransacao` | String | The transaction ID |
| `linkQRCodePix` | String | The PIX QR code link |
| `linkBoleto` | String | The boleto (payment slip) link |
| `dataEmissao` | Date | The emission date (YYYY-MM-DD format) |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The contact document number |
| `contato.tipo` | String | The contact type (F = Person, J = Company) |
| `formaPagamento.id` | Integer | The payment method ID |
| `formaPagamento.codigoFiscal` | Integer | The fiscal code |
| `contaContabil.id` | Integer | The accounting account ID |
| `contaContabil.descricao` | String | The accounting account description |
| `origem.id` | Integer | The origin ID |
| `origem.tipoOrigem` | String | The origin type (e.g., 'venda') |
| `origem.numero` | String | The origin number |
| `origem.dataEmissao` | Date | The origin emission date |
| `origem.valor` | Number | The origin value |
| `origem.situacao` | Integer | The origin status |
| `origem.url` | String | The origin URL |
Stream for detailed accounts receivable information (child of Accounts Receivable).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The account receivable ID |
| `situacao` | Integer | The account receivable status |
| `vencimento` | Date | The due date (YYYY-MM-DD format) |
| `valor` | Number | The amount to be received |
| `idTransacao` | String | The transaction ID |
| `linkQRCodePix` | String | The PIX QR code link |
| `linkBoleto` | String | The boleto link |
| `dataEmissao` | Date | The emission date (YYYY-MM-DD format) |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The contact document number |
| `contato.tipo` | String | The contact type (F = Person, J = Company) |
| `formaPagamento.id` | Integer | The payment method ID |
| `formaPagamento.codigoFiscal` | Integer | The fiscal code |
| `contaContabil.id` | Integer | The accounting account ID |
| `contaContabil.descricao` | String | The accounting account description |
| `origem.id` | Integer | The origin ID |
| `origem.tipoOrigem` | String | The origin type |
| `origem.numero` | String | The origin number |
| `origem.dataEmissao` | String | The origin emission date |
| `origem.valor` | Number | The origin value |
| `origem.situacao` | Integer | The origin status |
| `origem.url` | String | The origin URL |
| `saldo` | Number | The balance for the account receivable |
| `vencimentoOriginal` | String | The original due date (YYYY-MM-DD format) |
| `numeroDocumento` | String | The document number |
| `competencia` | String | The competence date (YYYY-MM-DD format) |
| `historico` | String | The history of the receivable |
| `numeroBanco` | String | The bank number |
| `portador.id` | Integer | The portador ID |
| `categoria.id` | Integer | The category ID |
| `vendedor.id` | Integer | The salesperson ID |
| `borderos` | Array | List of borderos IDs |
| `ocorrencia.tipo` | Integer | The occurrence type |
Stream for managing borderos (settlement batches with the actual receipt/payment date). This is a child stream of Accounts Receivable Details.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The bordero ID |
| `data` | Date | The settlement date (YYYY-MM-DD format) |
| `historico` | String | The bordero description |
| `portador.id` | Integer | The portador ID |
| `categoria.id` | Integer | The category ID |
| `pagamentos[].contato.id` | Integer | The contact ID |
| `pagamentos[].numeroDocumento` | String | The document number |
| `pagamentos[].valorPago` | Number | The paid amount |
| `pagamentos[].juros` | Number | Interest amount |
| `pagamentos[].desconto` | Number | Discount amount |
| `pagamentos[].acrescimo` | Number | Surcharge amount |
| `pagamentos[].tarifa` | Number | Payment method fee |
Stream for managing payment slips (boletos) associated with accounts receivable (child of Accounts Receivable). Only receivables that have a boleto link and an origin are queried.
| Field | Type | Description |
| - | - | - |
| `origemId` | Integer | The origin ID (links to sale/invoice) |
| `dataEmissaoNfe` | String | The NFe emission date |
| `venda.numero` | String | Sale number |
| `notaFiscal.numero` | String | Invoice number |
| `valorTotal` | Number | Total value |
| `contas[].id` | Integer | Account ID |
| `contas[].numeroExterno` | String | External number |
| `contas[].vencimento` | String | Due date |
| `contas[].valor` | Number | Value |
| `contas[].situacao` | Integer | Status |
Stream for managing accounts payable (amounts to be paid).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The account payable ID |
| `situacao` | Integer | The account payable status |
| `vencimento` | Date | The due date (YYYY-MM-DD format) |
| `valor` | Number | The amount to be paid |
| `dataEmissao` | String | The emission date (YYYY-MM-DD format) |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The contact document number |
| `contato.tipo` | String | The contact type (F = Person, J = Company) |
| `formaPagamento.id` | Integer | The payment method ID |
| `formaPagamento.codigoFiscal` | Integer | The fiscal code |
| `dataFiltro` | Date | The filter date (YYYY-MM-DD format) |
Stream for detailed accounts payable information (child of Accounts Payable).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The account payable ID |
| `situacao` | Integer | The account payable status |
| `vencimento` | Date | The due date (YYYY-MM-DD format) |
| `valor` | Number | The amount to be paid |
| `dataEmissao` | Date | The emission date (YYYY-MM-DD format) |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.numeroDocumento` | String | The contact document number |
| `contato.tipo` | String | The contact type |
| `formaPagamento.id` | Integer | The payment method ID |
| `formaPagamento.codigoFiscal` | Integer | The fiscal code |
| `contaContabil.id` | Integer | The accounting account ID |
| `contaContabil.descricao` | String | The accounting account description |
| `saldo` | Number | The balance |
| `vencimentoOriginal` | Date | The original due date |
| `numeroDocumento` | String | The document number |
| `competencia` | Date | The competence date |
| `historico` | String | The history |
| `numeroBanco` | String | The bank number |
| `portador.id` | Integer | The portador ID |
| `categoria.id` | Integer | The category ID |
| `ocorrencia.tipo` | Integer | The occurrence type |
Stream for managing cash register entries. Supports incremental sync using date filter.
| Field | Type | Description |
| - | - | - |
| `id` | String | The cash register entry ID |
| `debCred` | String | Debit/Credit indicator (D = Debit, C = Credit) |
| `situacao` | String | The entry status (R = Reconciled) |
| `valor` | Number | The entry value (negative for debits) |
| `data` | Date | The entry date (YYYY-MM-DD format, used as replication key) |
| `observacoes` | String | The entry observations/notes |
| `descricao` | String | The entry description |
| `origem.id` | Integer | The origin ID |
| `contato.id` | Integer | The contact ID |
| `contato.nome` | String | The contact name |
| `contato.cnpj` | String | The contact CNPJ |
| `contaFinanceira.id` | Integer | The financial account ID |
| `contaFinanceira.descricao` | String | The financial account description |
Stream for managing accounting accounts (bank accounts, payment integrations).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The accounting account ID |
| `descricao` | String | The account description |
| `tipo` | String | The account type (e.g., 'integracao-pagamento') |
| `aliasIntegracao` | String | The integration alias (e.g., 'BlingPagamentos') |
| `saldoInicial` | Number | The initial balance |
| `dataInicioTransacoes` | String | The start date for transactions (YYYY-MM-DD format) |
Stream for managing revenue and expense categories.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The category ID |
| `idCategoriaPai` | Integer | The parent category ID (0 if no parent) |
| `descricao` | String | The category description |
| `tipo` | Integer | The category type (1 = Revenue, 2 = Expense) |
Stream for detailed revenue/expense category information (child of Revenue Expense Categories).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The category ID |
| `idCategoriaPai` | Integer | The parent category ID |
| `descricao` | String | The category description |
| `tipo` | Integer | The category type (1 = Revenue, 2 = Expense) |
| `situacao` | Integer | The category status |
### Configuration
Stream for managing store categories (for e-commerce integrations).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The store category ID |
| `descricao` | String | The category description |
| `codigo` | String | The category code |
| `loja.id` | Integer | Store ID |
| `categoriaProduto.id` | Integer | Product category mapping ID |
Stream for detailed store category information (child of Store Categories).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The store category ID |
| `descricao` | String | The category description |
| `codigo` | String | The category code |
| `loja.id` | Integer | Store ID |
| `categoriaProduto.id` | Integer | Product category mapping ID |
Stream for retrieving situation modules (situações módulos).
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The module ID |
| `nome` | String | The module name |
| `descricao` | String | The module description |
| `criarSituacoes` | Boolean | Whether the module allows creating situations |
Stream for retrieving situations per module (situações por módulo). Child stream of Situation Modules.
| Field | Type | Description |
| - | - | - |
| `id` | Integer | The situation ID |
| `nome` | String | The situation name |
| `idHerdado` | Integer | The inherited situation ID |
| `cor` | String | The situation color |
# Data Model
# Blip as a data source
Source: https://docs.nekt.com/sources/blip
Bring data from Blip to Nekt.
Blip is a messaging and chatbot platform that helps businesses create conversational experiences for customer
communication. It provides tools for building chatbots, managing messaging channels, and automating customer interactions
to improve engagement and support.
![]()
## Configuring Blip as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Blip** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the following credentials from your Blip account:
* **Authorization token**: The token to authenticate against the API service. Note that it must be a HTTP token. It should be generated using the `Connect using HTTP` option. For more information on how to generate the token, please check the [Blip documentation](https://docs.blip.ai/#authentication).
* **Company identifier (contract ID)**: The company identifier (also known as contract ID) used to send commands through the API. Its value can be identified as being part of your URL, in the following format: `https://{contract_id}.http.msging.net/commands`.
* **Start date**: Records created or updated after the start date will be extracted from the source. Format: `YYYY-MM-DD`.
Once you have all the required credentials, add the account access and click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Blip and their corresponding fields:
Stream for tracking daily message activity metrics.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the report |
| `interval_start` | Timestamp | Start time of the reporting interval |
| `interval_end` | Timestamp | End time of the reporting interval |
| `count` | Integer | Number of active messages in the interval |
Stream for real-time agent performance metrics.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `sync_date` | Timestamp | When the metrics were collected |
| `identity` | String | Agent identifier |
| `status` | String | Current agent status |
| `is_enabled` | Boolean | Whether the agent is enabled |
| `opened_tickets` | Integer | Number of open tickets |
| `agent_name` | String | Name of the agent |
| `break_duration_in_seconds` | Integer | Duration of break time |
| `current_status_date_time` | Timestamp | When the current status was set |
| `closed_tickets` | Integer | Number of closed tickets |
| `average_attendance_time` | String | Average time spent on tickets |
| `average_response_time` | String | Average time to respond |
| `tickets_count` | Integer | Total number of tickets handled |
Stream for daily agent performance metrics.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the report |
| `sync_date` | Timestamp | Date of the report |
| `identity` | String | Agent identifier |
| `status` | String | Agent status |
| `is_enabled` | Boolean | Whether the agent is enabled |
| `opened_tickets` | Integer | Number of open tickets |
| `agent_name` | String | Name of the agent |
| `break_duration_in_seconds` | Integer | Duration of break time |
| `current_status_date_time` | Timestamp | When the current status was set |
| `closed_tickets` | Integer | Number of closed tickets |
| `average_attendance_time` | String | Average time spent on tickets |
| `average_response_time` | String | Average time to respond |
| `average_first_response_time` | String | Average time for first response |
| `average_wait_time` | String | Average wait time for customers |
| `tickets_count` | Integer | Total number of tickets handled |
Stream for overall ticket performance metrics.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `sync_date` | Timestamp | When the metrics were collected |
| `max_queue_time` | String | Maximum time in queue |
| `max_first_response_time` | String | Maximum time for first response |
| `max_without_first_response_time` | String | Maximum time without first response |
| `avg_queue_time` | String | Average time in queue |
| `avg_first_response_time` | String | Average time for first response |
| `avg_wait_time` | String | Average wait time |
| `avg_response_time` | String | Average response time |
| `avg_attendance_time` | String | Average attendance time |
| `tickets_per_attendant` | String | Number of tickets per agent |
Stream for detailed ticket information.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the ticket |
| `sequential_id` | Integer | Sequential ticket number |
| `owner_identity` | String | Identity of the ticket owner |
| `customer_identity` | String | Identity of the customer |
| `customer_domain` | String | Domain of the customer |
| `agent_identity` | String | Identity of the assigned agent |
| `provider` | String | Service provider |
| `status` | String | Current ticket status |
| `storage_date` | Timestamp | When the ticket was stored |
| `open_date` | Timestamp | When the ticket was opened |
| `close_date` | Timestamp | When the ticket was closed |
| `status_date` | Timestamp | When the status was last updated |
| `rating` | Integer | Customer rating |
| `team` | String | Assigned team |
| `unread_messages` | Integer | Number of unread messages |
| `closed` | Boolean | Whether the ticket is closed |
| `closed_by` | String | Who closed the ticket |
| `tags` | Array | Tags associated with the ticket |
| `first_response_date` | Timestamp | When the first response was sent |
| `priority` | Integer | Ticket priority level |
| `is_automatic_distribution` | Boolean | Whether automatically distributed |
| `distribution_type` | String | Type of distribution |
Stream for daily ticket statistics.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the report |
| `date` | Timestamp | Date of the report |
| `waiting` | Integer | Number of tickets waiting |
| `open` | Integer | Number of open tickets |
| `closed` | Integer | Number of closed tickets |
| `closed_attendant` | Integer | Tickets closed by attendants |
| `closed_client` | Integer | Tickets closed by clients |
| `transferred` | Integer | Number of transferred tickets |
| `missed` | Integer | Number of missed tickets |
| `in_attendance` | Integer | Number of tickets in attendance |
# Data Model
The following diagram illustrates the relationships between the core data streams in Blip. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Tickets("Tickets");
AgentMetrics("Agent Metrics");
end
subgraph "Performance Reports"
ActiveMessages("Active Messages Daily Report");
AgentsDaily("Agents Daily Report");
TicketMetrics("Ticket Metrics");
TicketsDaily("Tickets Daily Report");
end
Tickets -- "agent_identity -> identity" --> AgentMetrics;
Tickets -- "agent_identity -> identity" --> AgentsDaily;
```
## Use Cases for Data Analysis
Here are some valuable business intelligence use cases when consolidating Blip data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Agent Performance Analysis
Track agent productivity and response times.
**Business Value:**
* Monitor individual agent performance
* Identify training needs
* Optimize workload distribution
* Improve customer response times
```sql theme={null}
WITH
agent_daily_stats AS (
SELECT
"identity",
DATE_TRUNC ('day', "sync_date") AS "report_date",
"opened_tickets",
"closed_tickets",
"tickets_count",
CAST(
SPLIT_PART ("average_response_time", ':', 1) AS DOUBLE
) * 3600 + CAST(
SPLIT_PART ("average_response_time", ':', 2) AS DOUBLE
) * 60 + CAST(
SPLIT_PART ("average_response_time", ':', 3) AS DOUBLE
) AS "avg_response_seconds",
CAST(
SPLIT_PART ("average_first_response_time", ':', 1) AS DOUBLE
) * 3600 + CAST(
SPLIT_PART ("average_first_response_time", ':', 2) AS DOUBLE
) * 60 + CAST(
SPLIT_PART ("average_first_response_time", ':', 3) AS DOUBLE
) AS "avg_first_response_seconds",
CAST(
SPLIT_PART ("average_attendance_time", ':', 1) AS DOUBLE
) * 3600 + CAST(
SPLIT_PART ("average_attendance_time", ':', 2) AS DOUBLE
) * 60 + CAST(
SPLIT_PART ("average_attendance_time", ':', 3) AS DOUBLE
) AS "avg_attendance_seconds",
"break_duration_in_seconds"
FROM
"nekt_raw"."blip_agents_daily_report"
WHERE
"sync_date" >= CURRENT_DATE - INTERVAL '30' DAY
),
agent_summary AS (
SELECT
"identity",
COUNT(DISTINCT "report_date") AS "active_days",
SUM("closed_tickets") AS "total_closed_tickets",
SUM("tickets_count") AS "total_tickets_handled",
AVG("avg_response_seconds") AS "avg_response_time_seconds",
AVG("avg_first_response_seconds") AS "avg_first_response_time_seconds",
AVG("avg_attendance_seconds") AS "avg_attendance_time_seconds",
SUM("break_duration_in_seconds") / 3600.0 AS "total_break_hours"
FROM
agent_daily_stats
GROUP BY
"identity"
)
SELECT
"identity",
"active_days",
"total_closed_tickets",
"total_tickets_handled",
ROUND(
CAST("total_closed_tickets" AS DOUBLE) / NULLIF("total_tickets_handled", 0) * 100,
2
) AS "resolution_rate",
ROUND(
"total_tickets_handled" / NULLIF("active_days", 0),
1
) AS "avg_daily_tickets",
ROUND("avg_response_time_seconds" / 60.0, 2) AS "avg_response_time_minutes",
ROUND("avg_first_response_time_seconds" / 60.0, 2) AS "avg_first_response_time_minutes",
ROUND("avg_attendance_time_seconds" / 60.0, 2) AS "avg_attendance_time_minutes",
ROUND("total_break_hours", 2) AS "total_break_hours"
FROM
agent_summary
ORDER BY
"total_tickets_handled" DESC
```
### 2. Ticket Resolution Analysis
Analyze ticket resolution patterns and identify bottlenecks.
**Business Value:**
* Improve ticket resolution efficiency
* Reduce customer wait times
* Identify common issues
* Optimize team allocation
```sql theme={null}
WITH daily_metrics AS (
SELECT
DATE_TRUNC('day', date) AS report_date,
waiting,
open,
closed,
closed_attendant,
closed_client,
transferred,
missed,
in_attendance,
closed + waiting + open + in_attendance AS total_tickets,
ROUND(
CAST(closed AS DOUBLE) / NULLIF(closed + waiting + open + in_attendance, 0) * 100,
2
) AS resolution_rate
FROM nekt_raw.blip_tickets_daily_report
WHERE date >= CURRENT_DATE - INTERVAL '30' DAY
),
ticket_timing AS (
SELECT
DATE_TRUNC('day', open_date) AS created_date,
status,
priority,
team,
CASE
WHEN close_date IS NOT NULL
THEN DATE_DIFF('minute', open_date, close_date)
END AS resolution_time_minutes
FROM nekt_raw.blip_tickets
WHERE open_date >= CURRENT_DATE - INTERVAL '30' DAY
)
SELECT
dm.report_date,
dm.total_tickets,
dm.waiting AS tickets_waiting,
dm.open AS tickets_open,
dm.closed AS tickets_closed,
dm.transferred AS tickets_transferred,
dm.missed AS tickets_missed,
dm.resolution_rate AS daily_resolution_rate,
ROUND(AVG(tt.resolution_time_minutes), 2) AS avg_resolution_time_minutes,
COUNT(DISTINCT CASE WHEN tt.priority >= 2 THEN tt.id END) AS high_priority_tickets,
COUNT(DISTINCT CASE WHEN tt.resolution_time_minutes > 120 THEN tt.id END) AS long_resolution_tickets
FROM daily_metrics dm
LEFT JOIN ticket_timing tt ON dm.report_date = tt.created_date
GROUP BY
dm.report_date,
dm.total_tickets,
dm.waiting,
dm.open,
dm.closed,
dm.transferred,
dm.missed,
dm.resolution_rate
ORDER BY dm.report_date DESC
```
## Implementation Notes
### Data Quality Considerations
* Monitor agent status changes for accurate reporting
* Validate ticket resolution times for outliers
* Ensure consistent rating data collection
* Track message delivery status
### Automation Opportunities
* Schedule daily agent performance reports
* Set up alerts for long wait times
* Automate customer satisfaction reporting
* Generate team workload distribution reports
These SQL transformations provide a foundation for comprehensive Blip analytics. Customize the date ranges, filters, and metrics based on your specific business requirements and reporting needs.
## Skills for agents
Blip connector documentation as plain markdown, for use in AI agent contexts.
# BotConversa as a data source
Source: https://docs.nekt.com/sources/botconversa
Bring data from BotConversa to Nekt.
BotConversa is a chatbot platform that helps businesses automate customer interactions and support. It provides tools for building, deploying, and managing chatbots across various messaging channels, improving customer engagement and operational efficiency.
## Configuring BotConversa as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **BotConversa** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your BotConversa data.
The following configurations are available:
* **API Key**: The token to authenticate against the API service. This field is required.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings (if available):
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### 5. Check your new source
Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
Below you'll find all available data streams from BotConversa:
Stream containing campaign data from BotConversa.
Stream containing message flows configured in BotConversa.
Stream containing data about your BotConversa subscribers and contacts.
Stream containing tags used to organize and segment your subscribers.
## Skills for agents
BotConversa connector documentation as plain markdown, for use in AI agent contexts.
# Botmaker as a data source
Source: https://docs.nekt.com/sources/botmaker
Bring Botmaker conversations, contacts, and agent activity to Nekt.
Botmaker is a conversational platform for WhatsApp, Instagram, Messenger, and other channels, with bot automation and human-agent handoff. This connector extracts your chats, messages, sessions, calls, contacts, and agent data from the Botmaker API v2.0 and lands them in your Nekt Catalog as queryable tables.
![]()
## Configuring Botmaker as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Botmaker option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a Botmaker API access token. Generate it in Botmaker under **Configuration → Integrations → API** (the Credentials tab). Once you have it, add the account access, then click **Next**.
The following configurations are available:
* **Access Token**: Botmaker API access token, sent in the `access-token` request header.
* **Start Date**: The earliest date you want to extract data from.
* **Long-term Search**: Enable historical backfill for chats, messages, sessions, and calls.
By default, the chats, messages, sessions, and calls streams only return Botmaker's recent activity window. To backfill older history from your Start Date, enable **Long-term Search**. Botmaker bills long-term searches as BI data-source consumption, so enable it only when you need the full history.
### 2. Select streams
Choose which data streams you want to sync. The available streams are described in the [Streams and Fields](#streams-and-fields) section below. Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You can add a **prefix** to all tables at once.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: only new or updated records since the last successful run are extracted (available for chats, messages, sessions, calls, and tickets).
* Full table: every run fetches the full current state of the selected stream.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated.
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Botmaker and their corresponding fields:
Chats with their current state, filtered by last-activity window. Incremental on `lastUserMessageDatetime`.
| Field | Type | Description |
| :- | :- | :- |
| `chat_id` | String | Botmaker chat id. |
| `channel_id` | String | Channel id of the chat. |
| `contact_id` | String | Contact id (for example, the user's phone number on WhatsApp). |
| `creationTime` | DateTime | When the chat was created. |
| `lastSessionCreationTime` | String | Start time of the last session. |
| `externalId` | String | External identifier for the chat user. |
| `firstName` | String | User's first name. |
| `lastName` | String | User's last name. |
| `country` | String | User's country. |
| `email` | String | User's email. |
| `whatsAppWindowCloseDatetime` | DateTime | When the WhatsApp 24-hour window closes. |
| `variables` | String | JSON-encoded chat variables. |
| `tags` | Array | Tags applied to the chat. |
| `queueId` | String | Queue the chat is assigned to. |
| `agentId` | String | Agent currently assigned. |
| `onHoldAgentId` | String | Agent the chat is on hold for. |
| `lastUserMessageDatetime` | DateTime | Timestamp of the last user message. |
| `listMessagesURL` | String | API URL to list this chat's messages. |
| `isBanned` | Boolean | Whether the user is banned. |
| `isTester` | Boolean | Whether the user is a tester. |
| `isBotMuted` | Boolean | Whether the bot is muted for this chat. |
Messages exchanged across chats, filtered by a datetime window. Incremental on `creationTime`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Message id. |
| `creationTime` | DateTime | When the message was created. |
| `from` | String | Sender of the message. |
| `agentId` | String | Agent who sent the message, if any. |
| `queueId` | String | Queue associated with the message. |
| `content` | Object | Message content (type, text, buttons, media, location, call, template). |
| `encryptionParams` | Object | Encryption metadata, when applicable. |
| `sessionCreationTime` | DateTime | Start time of the session the message belongs to. |
| `sessionId` | String | Session id. |
| `chat_id` | String | Botmaker chat id. |
| `channel_id` | String | Channel id of the chat. |
| `contact_id` | String | Contact id of the chat. |
Conversation sessions, filtered by a datetime window. Incremental on `creationTime`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Session id. |
| `creationTime` | DateTime | When the session started. |
| `startingCause` | String | What triggered the session. |
| `chat` | Object | Chat the session belongs to. |
| `messages` | Array | Session messages (only when included by the API). |
| `events` | Array | Session events (only when included by the API). |
| `aiAnalysis` | Object | AI analysis of the session (summary, quality score, etc.). |
Voice and video calls, filtered by a datetime window. Incremental on `creationDatetime`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Call id. |
| `chat_id` | String | Botmaker chat id. |
| `channel_id` | String | Channel id of the chat. |
| `contact_id` | String | Contact id of the chat. |
| `agentId` | String | Agent involved in the call. |
| `creationDatetime` | DateTime | When the call record was created. |
| `startDatetime` | DateTime | When the call started. |
| `endDatetime` | DateTime | When the call ended. |
| `recording` | Object | Call recording (id and url). |
| `type` | String | Call type. |
| `participants` | Array | Call participants and their join/leave times. |
Contacts and their channels of communication.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Contact id. |
| `firstName` | String | Contact's first name. |
| `lastName` | String | Contact's last name. |
| `birthday` | String | Contact's birthday. |
| `pictureUrl` | String | Profile picture URL. |
| `language` | String | Preferred language. |
| `country` | String | Country. |
| `companyId` | String | Associated company id. |
| `jobTitle` | String | Job title. |
| `phoneNumbers` | Array | Phone numbers (value and label). |
| `emails` | Array | Email addresses (value and label). |
| `addresses` | Array | Addresses (value and label). |
| `websites` | Array | Websites (value and label). |
| `instagramIds` | Array | Linked Instagram ids. |
| `facebookIds` | Array | Linked Facebook ids. |
| `twitterIds` | Array | Linked Twitter ids. |
| `notes` | Array | Free-text notes. |
| `chats` | Array | Chats linked to this contact. |
| `whatsappBsuids` | Array | Linked WhatsApp business user ids. |
Agents configured in your Botmaker account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Agent id. |
| `email` | String | Agent's email. |
| `name` | String | Agent's name. |
| `alias` | String | Display alias. |
| `isOnline` | Boolean | Whether the agent is online. |
| `status` | String | Agent status. |
| `role` | String | Agent role. |
| `queues` | Array | Queues the agent belongs to. |
| `slots` | Integer | Concurrent chat slots. |
| `priority` | String | Assignment priority. |
| `groups` | Array | Groups the agent belongs to. |
| `additionalInfo` | String | JSON-encoded additional info. |
| `creationTime` | DateTime | When the agent was created. |
Communication channels connected to your account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Channel id. |
| `platform` | String | Channel platform (for example, whatsapp, instagram\_chat). |
| `active` | Boolean | Whether the channel is active. |
| `name` | String | Channel name. |
| `webhookId` | String | Associated webhook id. |
Bot intents defined in your account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Intent id. |
| `name` | String | Intent name. |
| `bot` | Object | Bot the intent belongs to (id and name). |
| `topics` | Array | Topics associated with the intent. |
| `triggers` | Array | Triggers that activate the intent. |
| `active` | Boolean | Whether the intent is active. |
Support tickets. Incremental on `creationTime`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Ticket id. |
| `businessId` | String | Business id. |
| `ticketCode` | String | Human-readable ticket code. |
| `state` | String | Ticket state. |
| `lastStateModification` | String | When the state last changed. |
| `creationTime` | DateTime | When the ticket was created. |
| `lastModificationBy` | String | JSON-encoded last modifier. |
| `subject` | String | Ticket subject. |
| `content` | String | Ticket content. |
| `contentFormat` | String | Format of the content. |
| `searchableContent` | String | Searchable content. |
| `companyId` | Integer | Company id. |
| `authorId` | Integer | Author id. |
| `assignedTo` | String | JSON-encoded assignee. |
| `followBy` | Array | User ids following the ticket. |
| `waitingQueueIds` | Array | Queues waiting on the ticket. |
| `waitingAssignationSince` | String | Since when the ticket is awaiting assignment. |
| `assignedQueueId` | String | Queue the ticket is assigned to. |
| `firstAnswer` | String | First answer timestamp. |
| `resolveDate` | String | Resolution date. |
| `finishedSlasMetrics` | String | JSON-encoded SLA metrics. |
| `attachments` | Array | Ticket attachments. |
| `chatIds` | String | JSON-encoded related chat ids. |
| `fieldValues` | Array | Custom field values. |
## Implementation Notes
The connector authenticates against the Botmaker API v2.0 using your access token as the `access-token` request header.
Most streams paginate by following the absolute `nextPage` URL returned in each response. The tickets stream uses `offset`/`limit` pagination based on the response's `total` count.
Some Botmaker endpoints (like messages) reply with a silent `204 No Content` empty body instead of an empty list or an error when a parameter is out of bounds (such as a page size outside the 250-1500 range, or datetimes with microsecond precision). The connector handles this by requesting a fixed page size of 1000, enforcing millisecond precision on datetimes, and gracefully treating any 204 response as a final, empty page.
The chats, messages, sessions, and calls endpoints only return Botmaker's recent activity window unless **Long-term Search** is enabled, which sends `long-term-search=true`. Botmaker bills long-term searches as BI data-source consumption, so enable it only when you need to backfill history from your Start Date.
## Skills for agents
Botmaker connector documentation as plain markdown, for use in AI agent contexts.
# Brevo as a data source
Source: https://docs.nekt.com/sources/brevo
Bring data from Brevo to Nekt.
Connect Brevo with Nekt to seamlessly sync and manage your customer relationship data. Import email, SMS, and chat interactions into Nekt's data hub to visualize engagement metrics, create reports, and enhance customer communication strategies.
## Configuring Brevo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Brevo** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Brevo data. Check [Brevo docs](https://developers.brevo.com/docs/getting-started) to discover where you can find the Brevo API key.
The following configuration is available:
* **Brevo API key**: The token to authenticate against the API service. This field is required and stored securely.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings (if available):
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### 5. Check your new source
Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
Below you'll find all available data streams from Brevo:
Stream containing contact information and details from your Brevo account.
Stream containing email marketing campaigns, including their settings, status, and basic information.
Stream containing detailed statistics of email campaigns broken down by individual contact engagement (e.g., opens, clicks, bounces).
## Skills for agents
Brevo connector documentation as plain markdown, for use in AI agent contexts.
# BSoft TMS as a data source
Source: https://docs.nekt.com/sources/bsoft-tms
Bring data from BSoft TMS to Nekt.
BSoft TMS is a Transport Management System that provides financial and operational data for logistics and fleet operations. The connector extracts data from the BSoft TMS API (financeiro module), including companies, receivables, payables, payments, receipts, transfers, cost centers, chart of accounts, and related master data.
![]()
## Configuring BSoft TMS as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the BSoft TMS option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide credentials for the BSoft TMS API. The integration uses HTTP Basic Authentication with a user that has the integration profile in BSoft TMS.
The following configurations are available:
* **Domain**: Your BSoft TMS domain name (e.g. `yourdomain.bsoft.app`). The connector will request data from `https://{domain}/services/index.php`.
* **Username**: Username with integration profile in BSoft TMS.
* **Password**: Password for the BSoft TMS integration user.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, used for streams that have a replication key (e.g. titulos\_receber, titulos\_pagar, pagamentos, recebimentos, transferencias).
* Full table: every time the extraction happens, we'll get the current state of the data, used for master/reference streams (empresas, centros de custo, formas de pagamento, etc.).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from BSoft TMS and their corresponding fields:
Stream containing companies (empresas) registered in BSoft TMS.
**Key Fields:**
* `id` - Company identifier (primary key)
* `descricao` - Company name
* `cnpj` - Brazilian company tax registration number (CNPJ)
Stream for cost centers (centros de custo) used for allocation and reporting.
**Key Fields:**
* `cod_cc` - Cost center identifier code (primary key)
* `cod_estendido` - Extended hierarchical code of the cost center
* `descricao` - Cost center name or description
* `status` - Status of the cost center (e.g. Aberto, Fechado)
Stream for payment conditions (e.g. 15 dias, 30 dias).
**Key Fields:**
* `id` - Payment condition identifier (primary key)
* `descricao` - Payment condition description
* `ativo` - Indicates whether the payment condition is active
Stream for financial accounts (bank accounts and similar).
**Key Fields:**
* `id` - Financial account identifier (primary key)
* `descricao` - Financial account description (including bank name and number)
* `codBanco` - Bank code
Stream for product and service families used to group services/products.
**Key Fields:**
* `id` - Product/service family identifier (primary key)
* `descricao` - Product/service family description
* `cod_estendido` - Extended hierarchical code of the family
* `nome_interno` - Internal name used for system identification
* `familiaServicos` - Indicates if this family is a service family (S/N)
* `familiaProdutos` - Indicates if this family is a product family (S/N)
* `irrestrito` - Indicates if access is unrestricted (S/N)
Stream for payment methods (e.g. boleto, PIX).
**Key Fields:**
* `id` - Payment method identifier (primary key)
* `descricao` - Payment method description
* `tipoCobranca` - Billing type code (e.g. BLO for boleto)
Stream for invoice models (modelos de notas fiscais).
**Key Fields:**
* `id` - Invoice model identifier (primary key)
* `descricao` - Invoice model description
* `serie` - Invoice series number
* `empresa` - Company name associated with the invoice model
Stream for document type codes and their descriptions (e.g. NF, BOL, CTR). The API returns a key-value map; each entry is exposed as one record.
**Key Fields:**
* `codigo` - Document type code (primary key)
* `descricao` - Full document type description
Stream for payment types and their settings.
**Key Fields:**
* `id` - Payment type identifier (primary key)
* `descricao` - Payment type description
* `leContaDeposito` - Indicates if this payment type reads a deposit account (S/N)
* `obrigaContaDeposito` - Indicates if a deposit account is required (S/N)
* `contas_financeiras_id` - Associated financial account identifier
* `nome_interno` - Internal name used for system identification
Stream for transaction types (e.g. TED, DOC, Cheque).
**Key Fields:**
* `id` - Transaction type identifier (primary key)
* `descricao` - Transaction type description
* `visivel` - Contexts where this transaction type is available (e.g. Pagamento, Recebimento)
Stream for billing suspension types (contract/item suspension, upgrade/downgrade, etc.).
**Key Fields:**
* `id` - Suspension type identifier (primary key)
* `ativo` - Indicates whether the suspension type is active (S/N)
* `descricao` - Suspension type description
* `periodoIndeterminado` - Indicates if the suspension has an indefinite period (S/N)
* `validoParaContrato` - Indicates if applicable to contracts (S/N)
* `validoParaItensContrato` - Indicates if applicable to contract items (S/N)
* `upgradeDowngrade` - Indicates if this type represents an upgrade or downgrade (S/N)
* `disponivelParaRevisaoFaturamento` - Indicates if available for billing review (S/N)
Stream for the management chart of accounts (plano de contas gerencial). Some records are category headers without a management code.
**Key Fields:**
* `descricao` - Account description including hierarchical code and name (primary key)
* `codGerencial` - Management account code (absent for category headers)
Stream for the fiscal chart of accounts (plano de contas fiscal).
**Key Fields:**
* `cod_fiscal` - Fiscal account code (primary key)
* `cod_fiscal_ext` - Extended hierarchical fiscal account code
* `nome` - Fiscal account name
* `ativo` - Indicates whether the account is active (S/N)
Stream for cost apportionment codes (rateios) used in allocations.
**Key Fields:**
* `codRateio` - Apportionment code identifier (primary key)
* `codEstendido` - Extended hierarchical code of the apportionment
* `descricao` - Apportionment description
Stream for services and products (serviços) with pricing and family.
**Key Fields:**
* `id` - Service/product identifier (primary key)
* `familia` - Product/service family name
* `descricao` - Service or product description
* `valorCompra` - Purchase price
* `valorVenda` - Sale price
* `codigo` - Service or product code
* `valorComissao` - Commission value for the service or product
Stream for receivables (titulos a receber). Records are built from the API response: base data is flattened from `dados_basicos`, and nested arrays (gerencias, rateios, duplicatas, etc.) are kept. Supports incremental sync by `data_ultimaModificacao`.
**Key Fields (dados\_basicos):**
* `codFatura` - Invoice (fatura) identifier (primary key)
* `cod_cliente` - Customer identifier code
* `cliente` - Customer name
* `tipo` - Document type code (e.g. ADT, AMV, BOL)
* `numero` - Document number
* `valor` - Total invoice amount
* `data_emissao` - Invoice issue date
* `data_entrada` - Invoice entry date in the system
* `forma_pagamento` - Payment method description
* `nro_parcelas` - Number of installments
* `historico` - Invoice description or payment history
* `previsao` - Indicates if this is a forecast entry (S/N)
* `data_ultimaModificacao` - Last modification timestamp (replication key)
* `data_criacao` - Record creation timestamp
* `empresaId`, `empresa` - Company identifier and name
* `acrescimos`, `abatimentos` - Additional charges and discounts
* `status` - Invoice status (e.g. Aberto, Quitação total, Quitação parcial)
* `referencia` - Reference period (e.g. 05/2022)
* `observacao` - Additional notes
**Nested arrays:**
* `gerencias` - Management account allocations (grupo, cod\_gerencial, descricao, percentual, valor)
* `rateios` - Cost apportionment allocations (grupo, cod\_rateio, descricao, percentual, valor)
* `gerencias_acrescimos_abatimentos` - Allocations for surcharges and discounts
* `duplicatas` - Installment schedule (parcela, loteId, data\_vencimento, data\_previsao\_pagamento, valor, saldo)
Stream for payables (titulos a pagar). Same structure as Títulos a Receber: base data from `dados_basicos` plus nested arrays. Supports incremental sync by `data_ultimaModificacao`.
**Key Fields (dados\_basicos):**
* `codFatura` - Invoice (fatura) identifier (primary key)
* `cod_fornecedor` - Supplier identifier code
* `fornecedor` - Supplier name
* `tipo` - Document type code (e.g. NF, FAT, CF)
* `numero` - Document number
* `valor` - Total invoice amount
* `data_emissao` - Invoice issue date
* `data_entrada` - Invoice entry date in the system
* `nro_parcelas` - Number of installments
* `historico` - Invoice description or payment history
* `previsao` - Indicates if this is a forecast entry (S/N)
* `data_ultimaModificacao` - Last modification timestamp (replication key)
* `data_criacao` - Record creation timestamp
* `empresaId`, `empresa` - Company identifier and name
* `acrescimos`, `abatimentos` - Additional charges and discounts
* `status` - Invoice status (e.g. Aberto, Quitação total)
* `observacao` - Additional notes
**Nested arrays:**
* `gerencias` - Management account allocations
* `rateios` - Cost apportionment allocations
* `duplicatas` - Installment schedule (parcela, loteId, data\_vencimento, valor, saldo, nroDoc)
Stream for payments (pagamentos) registered in the system. Supports incremental sync by `modificado`.
**Key Fields:**
* `id` - Payment identifier (primary key)
* `codigo` - Payment code reference
* `lancamento` - Payment posting date
* `conciliado` - Bank reconciliation date
* `conta` - Account name associated with the payment
* `valor` - Payment amount
* `numeroTransacao` - Transaction number
* `usuario` - User who registered the payment
* `modificado` - Last modification timestamp (replication key)
* `historico` - Payment history or description
Stream for receipts (recebimentos) registered in the system. Supports incremental sync by `modificado`.
**Key Fields:**
* `id` - Receipt identifier (primary key)
* `codigo` - Receipt code reference
* `lancamento` - Receipt posting date
* `conciliado` - Bank reconciliation date
* `conta` - Account name associated with the receipt
* `valor` - Receipt amount
* `numeroTransacao` - Transaction number
* `usuario` - User who registered the receipt
* `modificado` - Last modification timestamp (replication key)
* `historico` - Receipt history or description
Stream for transfers (transferências) between accounts. Supports incremental sync by `modificado`.
**Key Fields:**
* `id` - Transfer identifier (primary key)
* `lancamento` - Transfer posting date
* `conciliado` - Bank reconciliation date
* `conta` - Account name associated with the transfer
* `valor` - Transfer amount
* `numeroTransacao` - Transaction number
* `usuario` - User who registered the transfer
* `modificado` - Last modification timestamp (replication key)
* `historico` - Transfer history or description
# Data Model
The following diagram illustrates the relationships between the main data streams in BSoft TMS. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Master & Reference"
Empresas("Empresas");
CentrosCusto("Centros de Custo");
ContasFinanceiras("Contas Financeiras");
FormasPagamento("Formas de Pagamento");
PlanoContasGerencial("Plano Contas Gerencial");
PlanoContasFiscal("Plano Contas Fiscal");
Rateios("Rateios");
Servicos("Serviços");
FamiliaProdutos("Família Produtos e Serviços");
end
subgraph "Receivables & Payables"
TitulosReceber("Títulos a Receber");
TitulosPagar("Títulos a Pagar");
end
subgraph "Movements"
Pagamentos("Pagamentos");
Recebimentos("Recebimentos");
Transferencias("Transferências");
end
TitulosReceber -- "empresaId" --> Empresas;
TitulosPagar -- "empresaId" --> Empresas;
Servicos -- "familia" --> FamiliaProdutos;
TitulosReceber -- "gerencias.cod_gerencial / rateios.cod_rateio" --> PlanoContasGerencial;
TitulosReceber -- "gerencias.cod_gerencial / rateios.cod_rateio" --> Rateios;
TitulosPagar -- "gerencias / rateios" --> PlanoContasGerencial;
TitulosPagar -- "gerencias / rateios" --> Rateios;
```
# Use Cases for Data Analysis
This section outlines useful analyses when consolidating BSoft TMS data, with example SQL you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Receivables by status and company
Summarize receivables (títulos a receber) by status and company to monitor open and paid amounts.
**Business value:**
* Track open receivables per company
* Compare totals by status (Aberto, Quitação total, Quitação parcial)
* Support cash flow and aging analysis
```sql theme={null}
SELECT
empresa,
empresaId,
status,
COUNT(*) AS qtd_titulos,
SUM(TRY_CAST(valor AS DECIMAL(20, 2))) AS valor_total
FROM
nekt_raw.bsoft_tms_titulos_receber
GROUP BY
empresa,
empresaId,
status
ORDER BY
empresa,
status
```
```sql theme={null}
SELECT
empresa,
empresaId,
status,
COUNT(*) AS qtd_titulos,
SUM(SAFE_CAST(valor AS FLOAT64)) AS valor_total
FROM
`nekt_raw.bsoft_tms_titulos_receber`
GROUP BY
empresa,
empresaId,
status
ORDER BY
empresa,
status
```
### 2. Payments and receipts summary
Aggregate payments and receipts by posting date for a cash flow view.
**Business value:**
* Daily cash in and out
* Compare payments vs receipts over time
* Support reconciliation and reporting
```sql theme={null}
SELECT
lancamento AS data_lancamento,
SUM(TRY_CAST(valor AS DECIMAL(20, 2))) AS total_pagamentos
FROM
nekt_raw.bsoft_tms_pagamentos
GROUP BY
lancamento
ORDER BY
lancamento DESC
LIMIT 30
```
```sql theme={null}
SELECT
lancamento AS data_lancamento,
SUM(SAFE_CAST(valor AS FLOAT64)) AS total_pagamentos
FROM
`nekt_raw.bsoft_tms_pagamentos`
GROUP BY
lancamento
ORDER BY
lancamento DESC
LIMIT 30
```
## Implementation Notes
### Authentication
* The connector uses **HTTP Basic Authentication** (username and password). Ensure the user has the integration profile in BSoft TMS and that the domain is reachable from the Nekt environment.
### Pagination
* The BSoft TMS API uses limit-based pagination in the form `limit=offset,count`. The tap requests pages of a fixed size and stops when a partial page is returned.
### Incremental sync
* Streams with a replication key support incremental sync: **titulos\_receber** (`data_ultimaModificacao`), **titulos\_pagar** (`data_ultimaModificacao`), **pagamentos** (`modificado`), **recebimentos** (`modificado`), **transferencias** (`modificado`). The tap uses bookmarking only (no date filter is sent to the API). On the first run, all records are synced; on subsequent runs, only new or updated records are emitted based on the replication key.
### Data types
* Monetary and numeric values are often returned as strings by the API. Use casting in SQL (e.g. `TRY_CAST(valor AS DECIMAL(20,2))` or `SAFE_CAST(valor AS FLOAT64)`) for aggregations and calculations.
## Skills for agents
BSoft TMS connector documentation as plain markdown, for use in AI agent contexts.
# BTG Parceiros as a data source
Source: https://docs.nekt.com/sources/btg-parceiros
Bring data from BTG Parceiros to Nekt.
BTG Parceiros is a financial services platform designed for BTG Partners, which are offices or consultancy firms that
offer and manage BTG products for their clients. It provides tools for financial data management, reporting, and client
portfolio tracking for investment and wealth management services.
This source is available for BTG Partners only. Partners are offices or consultancy
firms that offer and manage BTG products for their clients. This connector is not
meant to be used for `PJ` or `PF` BTG accounts.
## Configuring BTG Parceiros as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the BTG Parceiros option from the list of connectors.
Click **Next** and you'll be prompted to add your access and configurations.
### 1. Add account access
You will need to provide the following credentials:
* **Client ID**: the client ID of the BTG Parceiros developer account.
* **Client secret**: the client secret of the BTG Parceiros developer account.
These credentials can be generated from the [Developer Partner](https://developer-partner.btgpactual.com/) portal.
* Go to `Autorização`, then click on `Gerar novas chaves`
* Store the new keys safely and use them to fill the required fields at Nekt
Besides generating the credentials, you'll have to configure a webhook URL on the `Webhooks -> Manutenção` section. Please get in touch with Nekt support to get the webhook URL that should be used there.
Once you're done, click **Next**.
### 2. Select streams
The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
The streams available bring D-1 reports.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings (if available):
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
When you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
## Available streams
The table below details the available streams you can sync from BTG Parceiros.
| Stream | Slug | Description |
| - | - | - |
| Asset Allocation | `reports_asset_allocation` | Asset allocation report (Relatório de Alocação de Ativos): one row per account. |
| Closed Fund Reserve | `reports_closed_fund_reserve` | Closed fund reserve report (Reservas de Fundos Fechados): one row per reservation in a fund offering. |
| Pension Campaign | `reports_pension_campaign` | Pension campaign report (Campanha de Previdência): one row per pension operation in the campaign. |
| Subscription | `reports_subscription` | Subscription report (Relatório de Subscrição): one row per account and subscription right. |
| Trade Idea | `reports_trade_idea` | Trade idea report (Recomendação de Operação): fixed income positions with premium/discount and a recommended operation, one row per position and recommendation. |
Depending on your BTG Parceiros access, you may also see other report streams available (such as Account Base, Banking, Commission, Operations, etc.).
## Fields by stream
Asset allocation report (Relatório de Alocação de Ativos): one row per account.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `nr_conta` | String | BTG account number |
| `total_fin` | Number | Total financial amount (BRL) of the account |
| `asset_total` | Number | Total number of assets held in the account |
| `dt_referencia` | Date | Reference date of the positions in the report |
| `profitability_mtd` | Number | Account profitability month to date |
| `profitability_last_12_months_accumulated` | Number | Account profitability accumulated over the last 12 months |
| `profitability_last_6_months_accumulated` | Number | Account profitability accumulated over the last 6 months |
| `profitability_last_3_months_accumulated` | Number | Account profitability accumulated over the last 3 months |
| `profitability_ytd` | Number | Account profitability year to date |
| `score_desenquadramento` | Number | Score of how far the portfolio deviates from the account's suitability profile |
| `modelo_rentabilidade` | String | Profitability classification of the account, e.g. 'Rentabilidade Acima do CDI' |
| `score_exposicao_risco` | Number | Risk exposure score of the portfolio |
| `liquidez` | Number | Liquidity indicator of the portfolio |
**Asset Classes Breakdown:**
The report breaks the portfolio into multiple asset classes: `pos_fixado` (post-fixed income), `inflacao` (inflation-linked), `pre_fixado` (pre-fixed income), `retorno_absoluto` (absolute return), `renda_variavel` (equities), `internacional` (international), `alternativo` (alternative investments), and `nao_classificado` (unclassified).
For each class, the following fields are returned:
| Field Pattern | Type | Description |
| :- | :- | :- |
| `_fin` | Number | Financial amount (BRL) allocated to the asset class |
| `_allocation` | Number | Share of the account's total financial amount allocated to the asset class (0 to 1) |
| `_asset_total` | Number | Number of assets held in the asset class |
| `_margem` | Number | Gap (BRL) between the amount allocated to the asset class and the account's target allocation (Not available for `nao_classificado`) |
*(Standard common report properties are also included in this stream).*
Closed fund reserve report (Reservas de Fundos Fechados): one row per reservation in a fund offering.
| Field | Type | Description |
| :- | :- | :- |
| `nr_conta` | String | BTG account number |
| `cod_oferta` | String | Code of the fund offering |
| `vl_oferta` | Number | Amount (BRL) reserved in the offering |
| `dt_solicitacao` | DateTime | When the reservation was requested |
| `flag_investflex` | String | Whether the reservation uses InvestFlex ('Sim'/'Não') |
| `flag_vinculo_pessoa` | String | Whether the investor is a related party of the offering ('Sim'/'Não') |
| `condicao_oferta` | String | Condition the investor set for the reservation, e.g. placement of the total offering amount |
| `vl_condicao` | Number | Amount (BRL) tied to the reservation condition |
| `status_oferta` | String | Status of the offering request, e.g. 'Solicitada' |
| `flag_operacao` | String | Whether the reservation generated an operation, as sent by BTG |
| `tx_motivo` | String | Reason attached to the reservation status, as sent by BTG |
| `tipo_oferta` | String | Type of the offering, as sent by BTG |
| `cge_fundo` | String | CGE code of the fund, as sent by BTG |
| `dt_liquidacao` | DateTime | Settlement date of the offering |
| `nm_fundo` | String | Name of the fund being offered |
| `status_reserva` | String | Status of the reservation, e.g. 'Reserva Aprovada' |
| `boletim` | String | Status of the subscription form sent to the investor, e.g. 'Enviado com sucesso' |
| `id_control` | String | Control identifier of the reservation; its meaning is given by `id_control_desc` |
| `id_control_desc` | String | What `id_control` identifies, e.g. 'Id Operação' |
| `dt_aprovacao` | DateTime | When the reservation was approved |
*(Standard common report properties are also included in this stream).*
Pension campaign report (Campanha de Previdência): one row per pension operation in the campaign.
| Field | Type | Description |
| :- | :- | :- |
| `hash_id` | String | Identifier of the operation in the report |
| `nr_conta` | String | BTG account number |
| `certificado` | String | Pension plan certificate at BTG |
| `certificado_extrno` | String | Pension plan certificate at the external insurer |
| `operacao` | String | Type of operation, e.g. 'Portabilidade de Entrada' |
| `cnpj_fundo` | String | CNPJ of the pension fund |
| `fundo` | String | Name of the pension fund |
| `vlr_solicitado` | Number | Amount (BRL) requested in the operation |
| `vlr_liquidado_total` | Number | Total amount (BRL) settled in the operation |
| `status` | String | Status of the operation, e.g. 'Finalizada' |
| `pontos_btg` | Number | Total BTG Pontos granted by the campaign |
| `pontos_primeira_parcela` | Number | BTG Pontos of the first installment |
| `pontos_segunda_parcela` | Number | BTG Pontos of the second installment |
| `beneficio` | String | Type of benefit granted, e.g. 'BTG\_PONTOS' |
| `seguradora_externa` | String | External insurer the plan was transferred from |
| `dt_solicitacao` | Date | Date the operation was requested |
| `dt_liquidacao` | Date | Date the operation was settled |
| `elegibilidade` | String | Eligibility classification of the operation, e.g. 'SEM\_ENTRADA\_ELEGIVEL' |
| `motivo_perda_segunda_parcela` | String | Why the second installment was lost, e.g. 'MOVIMENTOU\_APOS\_AGOSTO' |
| `flag_conta_ativa` | String | Whether the account is active ('Sim'/'Não') |
| `flag_tem_entrada_elegivel` | String | Whether the account has an eligible inflow ('Sim'/'Não') |
| `flag_valor_minimo_portabilizado` | String | Whether the transferred amount reaches the campaign minimum ('Sim'/'Não') |
| `flag_saldo_elegivel` | String | Whether the balance is eligible ('Sim'/'Não') |
| `flag_btg_pontos` | String | Whether the benefit is paid in BTG Pontos ('Sim'/'Não') |
| `flag_programa_no_prazo` | String | Whether the account joined the program within the deadline ('Sim'/'Não') |
| `flag_solicitacao_no_prazo` | String | Whether the request was made within the deadline ('Sim'/'Não') |
| `flag_conclusao_no_prazo` | String | Whether the operation was completed within the deadline ('Sim'/'Não') |
| `flag_sem_movimentacao_apos_agosto` | String | Whether there was no withdrawal or transfer after August ('Sim'/'Não') |
| `flag_elegivel_primeira_parcela` | String | Whether the account is eligible for the first installment of the benefit ('Sim'/'Não') |
| `flag_elegivel_segunda_parcela` | String | Whether the account is eligible for the second installment of the benefit ('Sim'/'Não') |
| `dt_extracao` | Date | Date BTG extracted the data for the report |
*(Standard common report properties are also included in this stream).*
Subscription report (Relatório de Subscrição): one row per account and subscription right.
| Field | Type | Description |
| :- | :- | :- |
| `nr_conta` | String | BTG account number |
| `nome_empresa` | String | Name of the company or fund offering the subscription |
| `titulo_direito` | String | Ticker of the subscription right held by the account, e.g. 'XPLG12' |
| `titulo_recebido` | String | Ticker of the security received when the right is exercised, e.g. 'XPLG13' |
| `quantidade_atual` | Number | Quantity of the underlying security currently held by the account |
| `disponivel_a_vista` | Number | Quantity of rights available to be exercised on the spot market |
| `perc_direito` | Number | Subscription ratio: rights granted per security held |
| `disponivel_btc` | Number | Quantity available under securities lending (BTC) |
| `solicitado_a_vista` | Number | Quantity already requested for subscription on the spot market |
| `preco_emissao` | Number | Issue price (BRL) of each subscribed security |
| `saldo_atual` | Number | Current cash balance (BRL) of the account |
| `valor_direito` | Number | Amount (BRL) needed to exercise the available rights |
| `saldo_final` | Number | Cash balance (BRL) of the account after exercising the rights |
| `dt_limite_para_subscricao` | Date | Last day to exercise the subscription right |
*(Standard common report properties are also included in this stream).*
Trade idea report (Recomendação de Operação): fixed income positions with premium/discount and a recommended operation, one row per position and recommendation.
| Field | Type | Description |
| :- | :- | :- |
| `nr_conta` | String | BTG account number |
| `tipo` | String | Type of the fixed income asset held, e.g. 'CRA, CRI' |
| `emissor` | String | Issuer of the asset held |
| `ativo` | String | Code of the asset held |
| `indexador` | String | Index of the asset held, e.g. '100% CDI+' |
| `taxa_compra` | Number | Rate at which the asset was bought |
| `dt_emissao` | Date | Issue date of the asset |
| `dt_aquisicao` | Date | Date the account acquired the asset |
| `dt_vencimento` | Date | Maturity date of the asset |
| `quantidade` | Number | Quantity of the asset held |
| `vl_pu_custo` | Number | Unit cost (BRL) of the asset |
| `vl_total_custo` | Number | Total cost (BRL) of the position |
| `vl_pu_curva` | Number | Unit value (BRL) of the asset on the yield curve |
| `vl_total_curva` | Number | Total value (BRL) of the position on the yield curve |
| `vl_pu_mercado` | Number | Unit market value (BRL) of the asset |
| `vl_total_mercado` | Number | Total market value (BRL) of the position |
| `tipo_cotacao` | String | Source of the market quote, e.g. 'Cotação Anbima' |
| `taxa_mercado` | Number | Current market rate of the asset |
| `duration` | Number | Duration of the asset |
| `agio_desagio` | String | Whether the position trades at a premium ('Ágio') or a discount ('Deságio') |
| `vl_agio_desagio` | Number | Premium (positive) or discount (negative) amount (BRL) of the position |
| `percent_agio_desagio` | Number | Premium or discount as a percentage of the position |
| `vl_acumulado_proventos` | Number | Accumulated income (BRL) paid by the asset |
| `vl_carrego_cdi` | Number | Amount (BRL) the position would earn if held to maturity at CDI |
| `rentabilidade` | Number | Profitability (BRL) of the position |
| `percent_rentabilidade` | Number | Profitability of the position as a percentage |
| `percent_carrego_cdi` | Number | Carry of the position as a percentage of CDI |
| `carrego_acima_cdi` | String | Whether the carry of the position is above CDI ('Sim'/'Não') |
| `id` | String | Identifier of the position in the report |
| `write_timestamp` | DateTime | When BTG wrote the row |
| `tipo_recomendacao` | String | Type of recommended operation, e.g. 'Troca' |
| `oferta_recomendacao` | String | Type of asset recommended, e.g. 'CRI' |
| `emissor_recomendacao` | String | Issuer of the recommended asset |
| `indexador_recomendacao` | String | Index of the recommended asset, e.g. 'CDI' |
| `taxa_indexador_recomendacao` | Number | Percentage of the index paid by the recommended asset |
| `taxa_adicional_recomendacao` | Number | Additional rate paid by the recommended asset on top of the index |
| `vencimento_recomendacao` | String | Maturity of the recommended asset, as sent by BTG (the example has a day count, e.g. '1825') |
| `roa_indicativa` | Number | Indicative return on assets of the recommended operation |
| `receita_indicativa` | Number | Indicative revenue (BRL) of the recommended operation |
| `last_updated` | DateTime | When the recommendation was last updated |
| `estimated_recovery` | Number | Estimated recovery (BRL) of the recommended operation |
| `carrego_proventos_cdi` | Number | Carry of the accumulated income relative to CDI |
| `percentual_carrego_sem_carrego_proventos` | Number | Carry of the position as a percentage, excluding the carry of the accumulated income |
| `percentual_sem_carrego_proventos` | Number | Profitability of the position as a percentage, excluding the carry of the accumulated income |
| `rentabilidade_sem_carrego_proventos` | Number | Profitability (BRL) of the position, excluding the carry of the accumulated income |
| `is_stvm` | String | Whether the asset is held in custody at BTG's broker-dealer (STVM), as sent by BTG |
| `vl_carrego_ipca` | Number | Amount (BRL) the position would earn if held to maturity at IPCA |
| `percent_carrego_ipca` | Number | Carry of the position as a percentage of IPCA |
| `carrego_acima_ipca` | String | Whether the carry of the position is above IPCA ('Sim'/'Não') |
*(Standard common report properties are also included in this stream).*
## Implementation Notes
* **Primary Keys:** The connector safely handles records with missing composite primary key fields by generating a reliable hash ID (`record_id`). This ensures data integrity and prevents pipeline failures when the source API omits certain fields for specific reports.
* **Report Parsing and Delimiter Detection:** To prevent row loss caused by HTTP streaming chunk issues, report files are fully downloaded to a temporary location before being parsed. The connector dynamically detects the appropriate CSV delimiter (`,` or `;`) based on the report header to safely adapt to varying BTG formatting standards. Additionally, the connector safely folds surplus column values back into expected free-text columns when BTG's source contains unquoted delimiters that split them unexpectedly, ensuring columns stay perfectly aligned.
## Skills for agents
BTG Parceiros connector documentation as plain markdown, for use in AI agent contexts.
# Bubble as a data source
Source: https://docs.nekt.com/sources/bubble
Bring data from Bubble to Nekt.
Bubble is a no-code platform for building web applications. It allows users to create custom databases, workflows, and user interfaces without writing code. The Bubble Data API provides access to all application data types and their records.
![]()
## Configuring Bubble as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Bubble option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Bubble Data API key and app details. The following configurations are available:
* **API Key**: Your Bubble Data API key (bearer token). You can find it in your Bubble app under **Settings > API > Data API**. Make sure the Data API is enabled and the key has read access to the data types you want to sync.
* **App Name**: Your Bubble app subdomain. For example, if your app URL is `myapp.bubbleapps.io`, enter `myapp`.
* **Data Types**: A list of Bubble data type names you want to sync. Enter the exact names as they appear in your Bubble database (e.g., `user`, `order`, `product`). Each data type will become a separate stream.
* **Environment**: Choose between `Live` (production data) or `Test` (development data). Defaults to `live`.
* **Custom Domain** *(optional)*: If your Bubble app uses a custom domain instead of the default `bubbleapps.io`, enter it here (e.g., `app.example.com`).
* **Start Date** *(optional)*: The earliest date from which records will be synced. Only applicable for incremental sync.
* **Discovery Record Sample** *(optional)*: Number of records to sample for schema discovery. Increase if some fields are missing from the schema. Defaults to `1000`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. Each configured data type appears as a separate stream. For faster extractions, select only the streams that are relevant to your analysis.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, only new or modified records are fetched, based on the `modified_date` field.
* Full table: every time the extraction happens, the current state of all records is fetched.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Streams are dynamically created based on the data types you configure. During extraction, the connector samples records (up to the configured `Discovery Record Sample`) to automatically discover your schema and infer field types.
All field names are sanitized during extraction: they are lowercased, spaces and special characters are replaced with underscores, and accents are removed (e.g., `My Field!` becomes `my_field`).
Every Bubble data type includes the following base fields:
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique record identifier |
| `created_date` | Datetime | When the record was created |
| `modified_date` | Datetime | When the record was last modified |
| `created_by` | String | User who created the record |
| `slug` | String | URL-friendly identifier |
Additional fields from your Bubble data types are dynamically discovered and included in the sync. The data types mapped include strings, numbers, booleans, and datetimes. Complex types (objects, lists) are serialized as JSON strings.
Since Bubble data types are user-defined, the specific fields available depend on your application's database schema. If you notice missing fields, try increasing the **Discovery Record Sample** configuration.
## Implementation Notes
### API Limits & Pagination
* **50k Record Limit**: The Bubble Data API has a limit of 50,000 results per query. This connector automatically bypasses this limit by using date-windowed pagination based on the `created_date` field, splitting queries into smaller time windows when approaching the limit.
* **Schema Discovery**: Since Bubble lacks a descriptive metadata endpoint, the connector samples records to discover available fields. This involves an initial API scan before extraction begins.
## Skills for agents
Bubble connector documentation as plain markdown, for use in AI agent contexts.
# Buzz Monitor as a data source
Source: https://docs.nekt.com/sources/buzz-monitor
Bring social listening data from Buzzmonitor to Nekt.
Buzzmonitor (by E.life) is a social listening and brand-monitoring platform. It collects posts, comments, mentions, and direct messages about your brands across social networks (X/Twitter, Instagram, Facebook, YouTube, Bluesky, TikTok, and more), classifies sentiment, and organizes everything into reports.
This connector uses the Buzzmonitor API with your account email and API key, both found in your Buzzmonitor **Settings > Developer** (Username and Access Token). If you cannot find your Access Token there, request it from the Buzzmonitor support team.
## How this connector works
Buzzmonitor's API does not let you list your reports programmatically, so you tell the connector which reports to bring by **copying each report's JSON payload from the Buzzmonitor Dashboard and pasting it into Nekt**. You add one entry per report, give it a name, and paste its payload. Each report becomes its own table. You decide how many reports to bring, one, two, or several (for example, one for *listening* and one for *responding/monitoring*).
### Copy a report's payload
Buzzmonitor shows each report's payload directly in the Dashboard. For every report you want to sync:
1. Log into the [Buzzmonitor Dashboard](https://app.buzzmonitor.com.br) and open the report (for example, under **Listening** or **Customer Care**).
2. In the toolbar below the report, click **JSON**. The report's request payload appears in a code panel.
3. Click the **copy** icon at the top-right of that panel to copy the entire JSON.
4. Paste it as the report's **payload** in Nekt.
You don't need to edit the copied payload, Nekt keeps its scope and filters and automatically overrides the authentication and the date window with your own settings.
## 1. Add your Buzz Monitor access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Buzz Monitor option from the list of connectors.
2. Click **Next** and you'll be prompted to fill the configuration.
* **Account Email**: The email of the Buzzmonitor account used to authenticate API requests. Found in Settings > Developer > Username.
* **API Key**: The Buzzmonitor API key of that account. Found in Settings > Developer > Access Token.
* **Reports**: One entry per report to extract. For each entry, provide a **name** (which becomes the table name) and paste the **payload** copied from the report's **JSON** view in the Buzzmonitor Dashboard. Add as many reports as you need.
* **Start Date** *(advanced)*: The earliest record date to sync. Defaults to full history.
3. Click **Next**.
## 2. Select your Buzz Monitor streams
1. The next step is letting us know which streams you want to bring. Each report you configured appears as its own stream, named after the report. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Buzz Monitor data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Buzz Monitor data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source.
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. If needed, manually trigger the pipeline by clicking on the refresh icon.
> If you encounter any issues, reach out to us, and we'll gladly assist you!
# Streams and Fields
Buzz Monitor streams are defined by the **Reports** you configure: each report becomes one stream, named after the report. Every stream shares the same fields and is synced **incrementally on `date`** (the publish time), with `elasticsearch_id` as the primary key.
`elasticsearch_id` is used as the primary key because it is the only identifier present and unique across every record kind (posts, comments, direct messages, story mentions). It is stable across syncs, so incremental loads update records in place rather than duplicating them.
Every post, comment, mention, and message returned by the report, one row per record. The same fields back listening records (public mentions) and responding records (DMs, private messages, story mentions); they differ by the `type`/`interaction_type` of each row.
**Fields:**
* `elasticsearch_id`: `String` - Buzzmonitor's unique row identifier (primary key), present and unique across every record kind
* `external_id`: `String` - Buzzmonitor's own identifier of the record, when provided
* `post_id`: `String` - Identifier of the post within its source network (may be null for some private messages, and shared across a post's comments)
* `service`: `String` - Source network of the record (twitter, facebook, instagram, youtube, bluesky, tiktok, news, blogs, and others)
* `type`: `String` - Kind of content within the network (post, comment, reply, video, direct\_message, story\_mention, and others)
* `interaction_type`: `String` - Source block that matched the record (monitoring\_brands, twitter\_dms, and others)
* `date`: `DateTime` - Timestamp when the record was published on the source network, used as the incremental replication key
* `collected_time`: `DateTime` - Timestamp when Buzzmonitor collected the record
* `content`: `String` - Text content of the post, comment, or message
* `title`: `String` - Title of the content, when available
* `caption`: `String` - Media caption, when available
* `description`: `String` - Longer description of the content, when available
* `url`: `String` - Public URL of the post, when provided
* `link`: `String` - Alternative link to the content, when provided
* `sentiment`: `String` - Sentiment classified by Buzzmonitor (positive, neutral, negative)
* `language`: `String` - Language detected for the content
* `author`: `String` - Author of the record (id, name, username, avatar, bio, and follower count), serialized as a JSON string
* `gender`: `String` - Gender inferred for the author (male, female, unknown)
* `brand`: `String` - Monitoring brand (project) in Buzzmonitor that matched the record
* `engagement`: `Number` - Engagement metric computed by Buzzmonitor for the record
* `archived`: `Boolean` - Whether the record is archived in Buzzmonitor
* `thread_id`: `String` - Identifier of the conversation thread (private messages and DMs)
* `solved`: `Boolean` - Whether the interaction is marked as solved (responding/inbox)
* `answer_ids`: `Array of Strings` - Identifiers of replies/answers to the interaction
* `elapsed_time`: `Integer` - Elapsed time metric reported for the interaction
* `likes`: `Integer` - Number of likes/reactions on the record
* `shares`: `Integer` - Number of shares/reposts of the record
* `replies`: `Integer` - Number of replies to the record
* `comment_replies`: `Integer` - Number of replies on comments of the record
* `hashtags`: `Array of Strings` - Hashtags present in the content
* `fb_reaction_love`: `Integer` - Facebook-specific count of Love reactions
* `fb_reaction_wow`: `Integer` - Facebook-specific count of Wow reactions
* `fb_reaction_haha`: `Integer` - Facebook-specific count of Haha reactions
* `fb_reaction_sorry`: `Integer` - Facebook-specific count of Sorry reactions
* `fb_reaction_anger`: `Integer` - Facebook-specific count of Anger reactions
* `fb_reaction_thankful`: `Integer` - Facebook-specific count of Thankful reactions
* `fb_reaction_pride`: `Integer` - Facebook-specific count of Pride reactions
* `view_count`: `Integer` - Number of views (video content)
* `dislikes_count`: `Integer` - Number of dislikes (YouTube content)
* `favorite_count`: `Integer` - Number of favorites (YouTube content)
* `video_id`: `String` - Identifier of the parent video, for video comments and replies
* `channel_id`: `String` - Identifier of the YouTube channel that published the content
* `channel_name`: `String` - Name of the YouTube channel that published the content
* `page_id`: `String` - Identifier of the Facebook page related to the record
* `reply_id`: `String` - Identifier of the reply, for reply records
* `content_type`: `String` - Content type flag reported by Buzzmonitor
* `collected_from`: `String` - Collection source that captured the record
* `origin`: `String` - Origin of the record within the network, serialized as JSON when structured
* `application`: `String` - Application used to publish the record, when reported
* `source`: `String` - Original source attribution of the content
* `location_name`: `String` - Name of the location attached to the record, when available
* `location_id`: `String` - Identifier of the location attached to the record, when available
* `location_geo`: `String` - Geographic coordinates of the record (latitude and longitude), serialized as a JSON string
* `icon`: `String` - URL of the author's avatar or icon, when provided
# Buzzlead
Source: https://docs.nekt.com/sources/buzzlead
Connect your Buzzlead referral marketing data to Nekt.
![Buzzlead Logo]()
Buzzlead is a referral marketing platform that helps businesses grow through ambassador programs, referral tracking, and reward management. This connector extracts data about ambassadors, bonuses, referrals, rescues, and NPS evaluations from your Buzzlead account.
## Configuration
To connect Buzzlead, you need:
* **Account Email**: The email registered as the account owner in Buzzlead.
* **API Key** and **API Token**: Both available in your Buzzlead dashboard under **Integracoes > API**.
The API has a rate limit of 600 requests per 5 minutes. The connector handles this automatically with backoff.
Choose which data streams to sync:
* **Ambassadors**: All ambassadors with personal referral links.
* **Bonuses**: Bonus records across all campaigns.
* **Referrals**: Referral records per campaign (requires Campaign IDs).
* **Rescues**: Reward rescue/redemption records.
* **NPS**: NPS evaluations per campaign (requires Campaign IDs).
Fill in the required configuration:
* **Account Email** (required): Your Buzzlead registration email.
* **API Key** (required): Your API key from the Buzzlead dashboard.
* **API Token** (required): Your API token from the Buzzlead dashboard.
* **Campaign IDs** (optional): A list of campaign IDs to fetch referrals and NPS data for. If not provided, the referrals and NPS streams are skipped.
* **Start Date** (optional): The earliest date to fetch data from (format: YYYY-MM-DD). Used for filtering bonuses and referrals.
After saving the configuration, run a sync to verify that data is flowing correctly. Check that all selected streams produce records.
## Streams and Fields
List of ambassadors registered in your Buzzlead account with their personal referral links.
| Field | Type | Description |
| - | - | - |
| `nome` | string | Ambassador name |
| `email` | string | Ambassador email (primary key) |
| `telefone` | string | Phone number |
| `empresa` | string | Company name |
| `documento` | string | Document number (CPF/CNPJ) |
| `cnpj` | string | CNPJ number |
| `hashid` | string | Unique hash identifier |
| `link` | string | Personal referral link |
| `campanha` | string | Campaign name |
| `campanha_id` | string | Campaign identifier |
| `created` | datetime | Creation date |
Bonus records generated from referral conversions across all campaigns.
| Field | Type | Description |
| - | - | - |
| `origem_nome` | string | Ambassador name |
| `origem_email` | string | Ambassador email |
| `origem_telefone` | string | Ambassador phone |
| `origem_empresa` | string | Ambassador company |
| `origem_cnpj` | string | Ambassador CNPJ |
| `data` | datetime | Bonus creation date |
| `numeroVenda` | string | Sale number (primary key) |
| `status` | string | Bonus status (pendente/confirmado) |
| `valorVenda` | number | Sale value |
| `recompensa` | number | Reward amount |
| `cliente_nome` | string | Converted customer name |
| `cliente_email` | string | Converted customer email |
| `cliente_documento` | string | Customer document |
| `campanha_id` | string | Campaign identifier |
| `campanha_nome` | string | Campaign name |
Referral records per campaign. Requires Campaign IDs to be configured.
| Field | Type | Description |
| - | - | - |
| `campaign_id` | string | Campaign identifier (partition key) |
| `codigo` | string | Referral code (primary key) |
| `numero` | string | Numeric referral code |
| `data` | datetime | Referral date |
| `canal` | string | Referral channel (whatsapp/email/link/facebook) |
| `origem_nome` | string | Referrer name |
| `origem_email` | string | Referrer email |
| `origem_telefone` | string | Referrer phone |
| `origem_empresa` | string | Referrer company |
| `origem_documento` | string | Referrer document |
| `destino_nome` | string | Referred person name |
| `destino_email` | string | Referred person email |
| `destino_telefone` | string | Referred person phone |
| `destino_empresa` | string | Referred person company |
| `destino_documento` | string | Referred person document |
| `campanha_id` | string | Campaign identifier |
| `campanha_nome` | string | Campaign name |
| `situacao` | string | Referral status (Valida/Invalida) |
| `situacao_motivo` | string | Status reason |
| `divulgador_nome` | string | Promoter name |
| `divulgador_email` | string | Promoter email |
| `conversao` | string | Conversion details (JSON) |
Reward rescue and redemption records.
| Field | Type | Description |
| - | - | - |
| `hashid` | string | Unique hash identifier (primary key) |
| `nome` | string | Ambassador name |
| `email` | string | Ambassador email |
| `telefone` | string | Phone number |
| `status` | string | Rescue status |
| `valor` | number | Rescue value |
| `campanha_id` | string | Campaign identifier |
| `campanha_nome` | string | Campaign name |
| `data` | datetime | Rescue date |
| `tipo` | string | Rescue type |
NPS evaluation records per campaign. Requires Campaign IDs to be configured.
| Field | Type | Description |
| - | - | - |
| `campaign_id` | string | Campaign identifier (partition key) |
| `nome` | string | Respondent name |
| `email` | string | Respondent email |
| `nota` | integer | NPS score |
| `comentario` | string | Comment |
| `data` | datetime | Evaluation date |
# C2S as a data source
Source: https://docs.nekt.com/sources/c2s
Bring data from C2S (Contact2Sale) to Nekt.
C2S (Contact2Sale) is a lead management CRM widely used by real estate and automotive companies in Brazil. It centralizes leads coming from portals, social media and WhatsApp, and distributes them to sellers with configurable rules and queues. This connector extracts your leads, including customer, product, seller, status, messages and activity history, along with sellers, companies, tags and distribution settings.
## 1. Add your C2S access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the C2S option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Token**: an integration token generated inside C2S. In the C2S web app, go to **Integrações -> Informações de integração por e-mail e API** and click **Gerar Token**. The token is shown only once, so copy it right away and store it securely. For more information, check the official [C2S API guide](https://sites.google.com/view/guiadeajudac2s/menus/integra%C3%A7%C3%B5es/integra%C3%A7%C3%A3o-via-api-c2s/informa%C3%A7%C3%B5es-da-api-c2s).
* **Start Date** (advanced): earliest record date to sync for the incremental leads stream. When empty, all history is synced.
3. Click **Next**.
## 2. Select your C2S streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, feel free to get in touch and we'll help you out!
2. Click **Next**.
## 3. Configure your C2S data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your C2S data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
## Available streams
Leads of your company, synced incrementally by last update. Nested structures (customer, product, seller, messages, etc.) are delivered as JSON strings.
**Key Fields:**
* `id` - Unique identifier of the lead
* `internal_id` - Internal numeric identifier of the lead in C2S
* `description` - Free-text description of the lead's interest (e.g. the property or product sought)
* `observation` - Latest observation or reminder note registered on the lead
* `is_favorite` - Whether the lead is marked as favorite by the seller
**Related Entities (JSON):**
* `customer` - Customer contact details: name, email and phone numbers
* `product` - Product or property the lead is interested in, with price, reference and location
* `seller` - Seller (broker) currently responsible for the lead
* `collaborators` - Additional users collaborating on the lead
* `created_by` - User who created the lead
* `company` - Company (branch) that owns the lead
* `lead_source` - Origin channel that generated the lead, e.g. a portal or campaign
* `channel` - Communication channel of the lead, e.g. Showroom or WhatsApp
* `lead_status` - Current status of the lead with id, alias and display name
* `funnel_status` - Position of the lead in the sales funnel
* `lost_reasons` - Reason registered when the lead was lost
* `done_details` - Closed-deal details: done flag, notes and final price
* `archive_details` - Archiving details: archived flag and notes
* `tags` - Tags applied to the lead
* `log` - Chronological activity log entries of the lead
* `messages` - Messages exchanged on the lead, with sender, recipient and timestamps
* `schedulated_actions` - Scheduled follow-up actions for the lead, with type, date and status
* `facebook_attributes` - Attributes captured from Facebook lead ads, when applicable
* `custom_attributes` - Account-specific custom attributes of the lead
**Timestamps:**
* `created_at` - When the lead was created
* `updated_at` - Last update, used as the incremental replication key
* `external_created_at` - Creation timestamp in the external system that originated the lead
* `last_activity_date` - Last activity registered on the lead
* `read_at` - When the lead was first read by the seller
* `replied_at` - When the lead was first replied to
* `done_deal_at` - When the deal was closed, when applicable
**Other Fields:**
* `from_hierarchy_company` - Name of the hierarchy (company) that generated the lead
* `url` - URL associated with the lead, when provided by the source
* `demo_company` - Whether the lead belongs to a demo company
Sellers (brokers) of your company and its group companies.
**Key Fields:**
* `id` - Unique identifier of the seller
* `name` - Full name of the seller
* `email` - Email address of the seller
* `phone` - Phone number of the seller
* `company` - Name of the company (branch) the seller belongs to
* `company_id` - Identifier of the company (branch) the seller belongs to
* `is_recipient` - Lead distribution recipient flag/level for the seller
* `can_view_bucket` - Whether the seller can view the lead pool (bucket)
* `external_id` - Identifier of the seller in an external system, when mapped
* `external_name` - Name of the seller in an external system, when mapped
* `is_master` - Whether the seller is a master (admin) user
Companies (branches) of your company's group.
**Key Fields:**
* `id` - Unique identifier of the company branch
* `name` - Display name of the company branch
Tags configured in your company.
**Key Fields:**
* `id` - Unique identifier of the tag
* `name` - Display name of the tag
* `enabled` - Whether the tag is active
* `autofill` - Whether the tag is applied automatically by C2S
* `instructions` - Usage instructions shown to sellers for when to apply the tag
Lead distribution rules (regions) of your company and its branches.
**Key Fields:**
* `cod_1` - First matching code of the rule (e.g. development/property reference)
* `cod_2` - Second matching code of the rule (e.g. development name)
* `cod_3` - Third matching code of the rule (e.g. negotiation type)
* `priority` - Priority of the rule; lower numbers are evaluated first
* `type_rule` - Type of the rule: distribution (direct assignment) or rotation (round-robin)
* `seller_id` - Identifier of the seller the rule assigns leads to
* `company_id` - Identifier of the company (branch) the rule belongs to
Lead distribution queues of your parent company.
**Key Fields:**
* `id` - Unique numeric identifier of the distribution queue
* `name` - Display name of the queue
* `priority` - Priority of the queue; higher-priority queues receive leads first
* `status` - Whether the queue is enabled or disabled
* `queue_type` - Type of the queue (e.g. custom\_queue)
* `leads_count` - Number of leads currently held by the queue
* `redistributable_leads_count` - Number of leads eligible for redistribution in the queue
* `redistributable` - Whether leads in the queue can be redistributed
* `check_in` - Whether sellers must check in to receive leads from the queue
* `check_in_qr_code` - Whether check-in is done via QR code
* `group_memory_lead` - Whether returning leads are routed to the seller who previously handled them
* `receptionist_enabled` - Whether the receptionist flow is enabled for the queue
* `today_working_time` - Working hours of the queue for the current day (JSON string)
* `sales_stand_form_id` - Identifier of the sales-stand form linked to the queue, when applicable
* `advanced_queue_edit` - Whether advanced queue editing is enabled
* `rules` - Matching rules that route leads into the queue (JSON string)
* `next_seller_index` - Index of the next seller in the rotation
* `next_seller` - Seller next in line to receive a lead from the queue (JSON string)
* `distribution_sellers_quantity` - Number of sellers participating in the queue
Information about the authenticated company and its sub-companies.
**Key Fields:**
* `company_id` - Unique identifier of the authenticated company
* `company_name` - Display name of the authenticated company
* `sub_companies` - Sub-companies (branches) linked to the authenticated company (JSON string)
## Good to know
* **Rate limit**: the C2S API tolerates roughly 10 requests per minute. The connector throttles itself accordingly, so large first syncs of the leads stream may take a while; subsequent incremental runs only fetch what changed.
* **Encrypted IDs**: all identifiers returned by C2S are encrypted strings and should be used as-is.
* **Incremental sync**: the leads stream uses the `updated_at` timestamp, so leads that change status, receive messages or are updated in any way are re-synced automatically.
# Cakto as a data source
Source: https://docs.nekt.com/sources/cakto
Bring data from Cakto to your Lakehouse.
Cakto is a Brazilian checkout and payments platform for digital products. It handles product pages, checkouts, order bumps, upsells, affiliate commissions and recurring subscriptions, and this source brings that sales data (orders, customers, products, offers and subscriptions) into your Lakehouse.
## 1. Add your Cakto access
In the [Cakto panel](https://app.cakto.com.br/dashboard/cakto-api), go to **Integracoes** and then **Cakto API**, and click **Criar Chave de API**.
Give the key a descriptive name and select the scopes it needs. Nekt only reads data, so the key needs the `read` scope plus the scope of each resource you want to sync:
| Scope | Enables |
| - | - |
| `read` | Required. Allows the key to read data at all. |
| `orders` | Orders and customers |
| `products` | Products, checkouts and order bumps |
| `offers` | Offers |
| `subscriptions` | Subscriptions, billing cycles and charge attempts |
| `webhooks` | Webhooks and the webhook event history |
Cakto shows the Client Secret only once, at the moment the key is created. Copy it before closing the dialog: if it is lost, you have to create a new key.
In the [Sources](https://app.nekt.ai/sources) tab, click **Add source**, select **Cakto** from the list of connectors and click **Next**. You'll be prompted to add your access:
* **Client ID**: the Client ID of the API key you created.
* **Client Secret**: the Client Secret of the same key.
* **Initial sync date** (optional): the earliest date to read for the incremental streams, orders and the webhook event history. Defaults to the beginning of the year two years ago.
* **Order update lookback (days)** (optional): how far before the last synced order the update filter reaches back. Defaults to 30 days. See the note below.
Click **Next**.
**Why the order lookback exists.** Cakto's order endpoint lets you filter by last update but does not return an update timestamp on the record itself, so Nekt bookmarks the creation date and reaches the update filter back by the lookback window. That window is what makes a refund or a chargeback recorded against an older order show up in a later run. The default of 30 days covers the period in which most refunds happen; raise it if you issue refunds later than that.
## 2. Select your Cakto streams
Choose which streams you want to sync. For faster extractions, select only the streams relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
`subscription_billing_cycles` is read one subscription at a time, so it costs one request per subscription on every run. Cakto allows 5,000 requests per hour, so select it only if you need the recurring-billing history. `subscription_billing_cycle_attempts` comes from the same responses and adds no requests of its own.
Click **Next**.
## 3. Configure your Cakto data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Click **Next**.
## 4. Configure your Cakto data source
1. Describe your data source for easy identification within your organization.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source.
3. Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source
Click **Next** to finalize the setup. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Cakto and their corresponding fields. Fields described as JSON hold a serialized object or array in a text column: query them with your warehouse's JSON functions.
Every purchase made through your checkouts, with the product, the buyer, the traffic source and the commission split of each sale.
Replicated incrementally on `created_at`.
**Fields:**
* `id` - Unique identifier of the order in Cakto.
* `ref_id` - Short reference code of the order, the one shown to the buyer.
* `status` - Current status of the order: processing, authorized, paid, refused, refunded, chargedback, waiting\_payment, canceled or in\_protest.
* `type` - Kind of product bought: 'unique' for one-off sales, 'subscription'.
* `offer_type` - Where in the funnel the offer was bought: main, upsell, downsell or orderbump.
* `base_amount` - Value of the order before discounts and instalment fees.
* `discount` - Discount applied to the order, when a coupon was used.
* `amount` - Total value of the order after discounts and fees, as charged.
* `fees` - Total fees Cakto charged on the order.
* `coupon_code` - Code of the discount coupon used in the purchase, if any.
* `coupon` - Full coupon applied to the purchase: code, discount and validity window (JSON object).
* `reason` - Reason the payment was refused, as reported by the acquirer.
* `refund_reason` - Reason given when the order was refunded.
* `product_id` - Identifier of the product bought in this order.
* `product_name` - Name of the product bought.
* `product_type` - Sale model of the product: 'unique' or 'subscription'.
* `product_status` - Status of the product at the time of reading: active, blocked or deleted.
* `product_price` - List price of the product, which may differ from the amount paid.
* `product_category_id` - Identifier of the category the product belongs to.
* `product_category_name` - Name of the category the product belongs to.
* `checkout` - Identifier of the checkout page where the purchase was made.
* `checkout_url` - URL of the checkout page where the purchase was made.
* `subscription` - Identifier of the subscription this order belongs to, when recurring.
* `subscription_period` - Which billing period of the subscription this order paid for.
* `installments` - Number of instalments the buyer split the payment into.
* `payment_method` - How the order was paid: credit\_card, pix, boleto, picpay and so on.
* `created_at` - When the order was created. Used as the incremental replication key; note that the records themselves are filtered by their last update, so an older order can reappear after a refund.
* `due_date` - When the payment is scheduled to be processed (subscriptions).
* `paid_at` - When the payment was confirmed.
* `release_date` - Estimated date the commissions of this order become available.
* `refunded_at` - When the order was refunded.
* `chargedback_at` - When a chargeback was opened against the order.
* `canceled_at` - When the order was canceled, usually for subscriptions.
* `customer_name` - Name of the buyer.
* `customer_email` - Email of the buyer. Only returned when the product shares buyer contact details with this account.
* `customer_phone` - Phone number of the buyer, when contact details are shared.
* `customer_doc_type` - Kind of Brazilian tax document of the buyer: cpf or cnpj.
* `customer_doc_number` - Tax document number of the buyer, when contact details are shared.
* `customer_birth_date` - Date of birth of the buyer, when informed at checkout.
* `address_country` - Country of the buyer's address, as an ISO 3166-1 alpha-2 code.
* `address_state` - State of the buyer's address, as an ISO 3166-2 code (for example SP).
* `address_city` - City of the buyer's address.
* `address_zipcode` - Postal code (CEP) of the buyer's address.
* `address_street` - Street of the buyer's address.
* `address_neighborhood` - Neighbourhood of the buyer's address.
* `address_complement` - Additional address information, such as apartment or reference point.
* `address_number` - Street number of the buyer's address.
* `shipping` - Shipping details of the order, for products that are delivered physically (JSON object).
* `commissioned_users` - Producers, co-producers and affiliates who earned from this order: id and email of each (JSON array).
* `commissions` - Commission split of the order: user, role, percentage and value for each participant (JSON array).
* `utm_source` - Traffic source recorded at checkout, such as google or instagram.
* `utm_medium` - Traffic medium recorded at checkout, such as cpc, email or social.
* `utm_campaign` - Campaign name recorded at checkout.
* `utm_term` - Campaign term recorded at checkout.
* `utm_content` - Campaign content recorded at checkout, such as the creative used.
* `sck` - Free-form tracking parameter the seller can attach to a checkout link.
Buyers who have completed at least one paid order. Cakto masks the phone number and the tax document on this endpoint; the unmasked values, when the product shares buyer contact details with your account, are on the order record.
**Fields:**
* `id` - Unique identifier of the customer.
* `name` - Full name of the customer.
* `email` - Email address of the customer.
* `birth_date` - Date of birth of the customer, when informed at checkout.
* `phone` - Phone number of the customer, partially masked by Cakto.
* `doc_type` - Kind of Brazilian tax document of the customer: cpf or cnpj.
* `doc_number` - Tax document number of the customer, partially masked by Cakto.
The products you sell. Summary view: the per-product settings (affiliation rules, upsell pages, tracking pixels) are not synced, because reading them would cost one request per product on every run.
**Fields:**
* `id` - Unique identifier of the product.
* `name` - Name of the product.
* `description` - Description of the product shown to buyers.
* `price` - List price of the product, in Brazilian reais.
* `type` - Sale model of the product: 'unique' for one-off, 'subscription'.
* `status` - Current status of the product: active, blocked or deleted.
* `image` - URL of the product image.
* `content_deliveries` - How the product content is delivered to the buyer, such as the Cakto members area, email access, Telegram or Discord.
* `email_access_link` - Link emailed to the buyer to access the content after purchase.
* `sales_page` - URL of the product's sales page.
* `payment_methods` - Payment methods the product accepts at checkout.
* `category_id` - Identifier of the category the product belongs to.
* `category_name` - Name of the category the product belongs to.
The price points configured for each product: the default offer plus any promotional price, annual plan or upsell.
**Fields:**
* `id` - Unique identifier of the offer.
* `name` - Name of the offer, shown at checkout and in the Cakto panel.
* `image` - URL of the image used for the offer.
* `price` - Price charged by this offer.
* `units` - How many units of the product the buyer gets with this offer.
* `default` - Whether this is the product's default offer.
* `product` - Identifier of the product this offer belongs to.
* `status` - Current status of the offer: active, disabled or deleted.
* `type` - Payment model of the offer: 'unique' or 'subscription'.
* `interval_type` - Unit of the access period granted by a one-off offer: day, week, month or year.
* `interval` - How many interval units of access the offer grants.
* `recurrence_period` - Days between charges, for subscription offers.
* `quantity_recurrences` - How many charges the subscription makes; -1 means indefinitely.
* `trial_days` - Free trial days granted before the first charge.
* `max_retries` - How many times a failed subscription charge is retried.
* `retry_interval` - Days between retries of a failed subscription charge.
The checkout pages of each product, with their visit counter. Read one product at a time.
**Fields:**
* `product_id` - Identifier of the product this checkout page belongs to.
* `id` - Unique identifier of the checkout page.
* `name` - Name given to the checkout page.
* `default` - Whether this is the product's default checkout page.
* `visits` - Number of visits the checkout page has received.
* `created_at` - When the checkout was created.
* `updated_at` - When the checkout was last changed.
The complementary offers displayed on a product's checkout. Read one product at a time.
**Fields:**
* `owner_product_id` - Identifier of the product whose checkout displays this order bump.
* `id` - Unique identifier of the order bump.
* `product` - Identifier of the product offered as the order bump.
* `installments` - Number of instalments allowed for the offered product.
* `reference_price` - Crossed-out reference price displayed next to the order bump.
* `offer_id` - Identifier of the offer sold through this order bump.
* `offer_name` - Name of the offer sold.
* `offer_price` - Price charged by the offer sold.
* `offer_type` - Payment model of the offer sold: 'unique' or 'subscription'.
* `offer_status` - Status of the offer sold: active, disabled or deleted.
* `cta` - Text of the call-to-action button of the order bump.
* `title` - Title displayed on the order bump.
* `description` - Descriptive copy displayed on the order bump.
* `position` - Position of the order bump in the checkout, lowest shown first.
* `image` - URL of the order bump image.
* `show_image` - Whether the image is displayed alongside the order bump.
Your recurring subscriptions in every state: active, paused, canceled, expired and trial.
**Fields:**
* `id` - Unique identifier of the subscription.
* `status` - Current state of the subscription: active, inactive, canceled, expired, paused or trial.
* `current_period` - Billing period the subscription is currently in.
* `recurrence_period` - Days between charges of this subscription.
* `quantity_recurrences` - How many charges are configured in total; -1 means indefinitely.
* `trial_days` - Free trial days granted before the first charge.
* `max_retries` - How many times a failed charge is retried before giving up.
* `retry_interval` - Days between retries of a failed charge.
* `amount` - Amount charged on each recurrence.
* `paid_payments_quantity` - How many charges of this subscription have been paid so far.
* `retention` - How long the subscription has stayed active, as reported by Cakto.
* `product_id` - Identifier of the product the subscription gives access to.
* `product_name` - Name of the subscribed product.
* `offer_id` - Identifier of the offer (price point) the subscription was sold on.
* `offer_name` - Name of the offer subscribed to.
* `offer_price` - Price configured on the offer the subscription was sold on.
* `customer_id` - Identifier of the subscriber.
* `customer_name` - Name of the subscriber.
* `customer_email` - Email of the subscriber.
* `customer_phone` - Phone number of the subscriber, when shared with this account.
* `customer_doc_number` - Tax document of the subscriber, when shared with this account.
* `parent_order_id` - Identifier of the order that originated the subscription.
* `parent_order_ref_id` - Short reference code of the order that originated the subscription.
* `payment_method` - Payment method used to charge the subscription. A plain method name such as credit\_card, or a JSON object with the card details when Cakto returns them.
* `orders` - Orders generated by this subscription so far (JSON array).
* `next_payment_date` - Estimated date of the next charge.
* `created_at` - When the subscription was created.
* `updated_at` - When the subscription was last changed.
* `canceled_at` - When the subscription was canceled, if it was.
The billing cycles of each subscription: what was charged, when it was due and how it ended. Read one subscription at a time, so this stream is the most request-hungry of the connector.
**Fields:**
* `subscription_id` - Identifier of the subscription this billing cycle belongs to.
* `id` - Unique identifier of the billing cycle.
* `cycle_number` - Sequential number of the cycle within the subscription.
* `due_date` - When the cycle was due to be paid.
* `amount` - Amount charged in this cycle.
* `status` - Outcome of the cycle, such as paid or pending.
* `total_attempts` - How many charge attempts were made for this cycle.
* `completed_at` - When the cycle was settled.
* `created_at` - When the cycle was created.
The individual charge attempts of each billing cycle, with the reason a charge failed. Extracted from the billing cycle response, so selecting it costs no extra requests.
**Fields:**
* `subscription_id` - Identifier of the subscription this charge attempt belongs to.
* `billing_cycle_id` - Identifier of the billing cycle this charge attempt belongs to.
* `id` - Unique identifier of the charge attempt.
* `attempt_number` - Sequential number of the attempt within its billing cycle.
* `amount` - Amount the attempt tried to charge.
* `result` - Outcome of the attempt: success or failure.
* `failure_reason` - Why the charge failed, as reported by the payment provider.
* `scheduled_for` - When the attempt was scheduled to run.
* `started_at` - When the attempt started.
* `completed_at` - When the attempt finished.
* `created_at` - When the attempt was recorded.
The webhooks configured on your account to receive Cakto events. The signing secret of each webhook is never synced.
**Fields:**
* `id` - Unique identifier of the webhook.
* `name` - Name given to the webhook.
* `status` - State of the webhook: active, disabled or waiting\_config.
* `url` - Destination URL events are delivered to.
* `products` - Products whose events trigger this webhook (JSON array).
* `events` - Events subscribed by this webhook (JSON array).
* `created_at` - When the webhook was created.
* `updated_at` - When the webhook configuration was last changed.
The delivery history of your webhooks: which event was sent, to which URL, what the destination answered and how long it took. The signing secret is stripped from the payload before it reaches your Lakehouse.
Replicated incrementally on `dispatched_at`.
**Fields:**
* `id` - Unique identifier of this delivery in the webhook history.
* `app_id` - Identifier of the webhook that produced this delivery.
* `app_name` - Name of the webhook.
* `app_status` - State of the webhook: active, disabled or waiting\_config.
* `url` - Destination URL this delivery was sent to.
* `event_id` - Event that triggered the delivery, such as purchase\_approved, refund, chargeback or subscription\_canceled.
* `event_name` - Human-readable name of the event that triggered the delivery.
* `event_status` - HTTP status code returned by the destination. 2xx means the delivery was accepted; anything else means the receiver rejected or failed to handle it.
* `response_time` - How long the destination took to answer, in milliseconds.
* `payload` - Body Cakto sent to the destination, with the webhook signing secret removed (JSON object).
* `response` - Body the destination answered with (JSON object).
* `steps` - Steps Cakto went through while delivering the event (JSON).
* `scheduled_at` - When the delivery was queued.
* `dispatched_at` - When the HTTP request was actually sent. Used as the incremental replication key.
* `sent_at` - Deprecated by Cakto, kept for backwards compatibility. Use scheduled\_at and dispatched\_at instead.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Calendly as a data source
Source: https://docs.nekt.com/sources/calendly
Bring data from Calendly to Nekt.
Calendly is a scheduling automation platform that helps individuals and businesses manage meetings and appointments. It provides tools for setting up event types, sharing booking links, and tracking scheduled events and invitees.
![]()
## Configuring Calendly as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Calendly option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a Calendly Personal Access Token to connect. To generate one:
1. Log in to your [Calendly account](https://calendly.com)
2. Go to **Integrations & Apps** > **API & Webhooks**
3. Click **Generate New Token** and copy the token
The following configurations are available:
* **Personal Access Token**: Your Calendly Personal Access Token for authentication.
* **Start Date**: (Optional) The earliest date from which events will be synced. If not provided, all events will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables.
* **Table name**: we suggest a name, but feel free to customize it. You can add a **prefix** to all tables at once.
* **Sync Type**: choose between INCREMENTAL and FULL\_TABLE.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Calendly and their corresponding fields:
All scheduled meetings and events in your Calendly organization.
**Key Fields:**
* `uri` - Unique identifier for the event
* `name` - Name of the event
* `status` - Status of the event (active, canceled)
* `start_time` - Start time of the event
* `end_time` - End time of the event
* `event_type` - URI of the event type
* `location` - Location details (JSON)
* `created_at` - When the event was created
* `updated_at` - When the event was last updated
**Invitee Counters:**
* `invitees_counter_total` - Total number of invitees
* `invitees_counter_active` - Number of active invitees
* `invitees_counter_limit` - Maximum number of invitees
**Additional Fields:**
* `event_memberships` - Host memberships (JSON)
* `event_guests` - Guest list (JSON)
* `calendar_event` - Calendar event details (JSON)
* `meeting_notes_plain` - Meeting notes in plain text
* `meeting_notes_html` - Meeting notes in HTML
* `cancellation_reason` - Reason for cancellation
* `cancellation_canceled_by` - Who canceled the event
Invitees for each scheduled event (child stream of Scheduled Events).
**Key Fields:**
* `uri` - Unique identifier for the invitee
* `email` - Invitee's email address
* `name` - Invitee's full name
* `first_name` - Invitee's first name
* `last_name` - Invitee's last name
* `status` - Invitee status (active, canceled)
* `timezone` - Invitee's timezone
* `created_at` - When the invitee was created
* `updated_at` - When the invitee was last updated
**Scheduling Details:**
* `questions_and_answers` - Responses to scheduling questions (JSON)
* `tracking` - UTM and tracking parameters (JSON)
* `rescheduled` - Whether the invitee rescheduled
* `cancel_url` - URL to cancel the event
* `reschedule_url` - URL to reschedule the event
* `scheduled_event_uri` - URI of the parent scheduled event
**Additional Fields:**
* `text_reminder_number` - Phone number for text reminders
* `old_invitee` - Previous invitee URI (if rescheduled)
* `new_invitee` - New invitee URI (if rescheduled)
* `routing_form_submission` - Routing form submission details (JSON)
* `payment` - Payment information (JSON)
* `no_show` - No-show status details (JSON)
Event types configured in your Calendly organization.
**Key Fields:**
* `uri` - Unique identifier for the event type
* `name` - Name of the event type
* `active` - Whether the event type is active
* `slug` - URL-friendly slug
* `scheduling_url` - Public scheduling URL
* `duration` - Duration in minutes
* `kind` - Event kind (solo, group)
* `type` - Event type (StandardEventType, AdhocEventType)
* `created_at` - When the event type was created
* `updated_at` - When the event type was last updated
**Appearance & Settings:**
* `color` - Display color
* `description_plain` - Description in plain text
* `description_html` - Description in HTML
* `internal_note` - Internal notes
* `secret` - Whether the event type is secret
* `admin_managed` - Whether the event type is admin-managed
* `pooling_type` - Pooling type for round-robin events
* `custom_questions` - Custom scheduling questions (JSON)
* `profile` - Profile information (JSON)
Members of your Calendly organization.
**Key Fields:**
* `uri` - Unique identifier for the membership
* `role` - Role in the organization (admin, user, owner)
* `organization` - Organization URI
* `created_at` - When the membership was created
* `updated_at` - When the membership was last updated
**User Details:**
* `user_uri` - User's unique identifier
* `user_name` - User's display name
* `user_slug` - User's URL slug
* `user_email` - User's email address
* `user_timezone` - User's timezone
* `user_avatar_url` - User's avatar URL
* `user_created_at` - When the user was created
* `user_updated_at` - When the user was last updated
# Cardápio Web as a data source
Source: https://docs.nekt.com/sources/cardapioweb
Bring data from Cardápio Web to Nekt.
Cardápio Web is a Brazilian digital-menu and delivery management platform for restaurants. It handles the online menu, orders placed through the digital menu or through marketplaces such as iFood, table and counter service, customers, loyalty and cashback, coupons and reviews. This connector brings the establishment's orders, menu, customers, coupons and reviews into your catalog.
## Configuring Cardápio Web as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Cardápio Web option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the establishment's API token. In the Cardápio Web Portal, go to **Configurações → Integrações → API de integração** and generate the token.
The following configurations are available:
* **API Key**: The establishment's API token, sent to Cardápio Web on every request. One token covers one establishment, so create a separate source for each establishment you want to extract.
* **Partner Key**: (Advanced) The integrator token. Only a few endpoints require it, and it has to be requested from Cardápio Web at `integracao@cardapioweb.com`. Leave it empty unless you have one; see the note below.
* **Environment**: (Advanced) `production` or `sandbox`. Leave it as `production` unless you are extracting from a Cardápio Web test establishment.
* **Initial sync date**: (Advanced) The earliest date to extract orders and reviews from. Cardápio Web does not accept dates older than 3 years, so earlier values are automatically raised to that limit.
* **Lookback Days**: (Advanced) How many days before the last synced date to re-read on each incremental run. Defaults to 7 days; see the note in [Order history](#order-history).
The **Payment Methods** stream is the only one Cardápio Web requires the integrator token for. Without a Partner Key it is skipped with a message in the run log, and the rest of the extraction runs normally.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
**Order Details** makes one request per order, so a first extraction covering years of history takes considerably longer than the other streams. Select it only if you need item, discount or payment level data.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Only **Orders** and **Reviews** can be filtered by date on the Cardápio Web side, so they are the streams that support INCREMENTAL. Every other stream is a full snapshot of the current state: the API exposes no date filter for customers, coupons or the menu.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
### 5. Verify data extraction
Once the source is created, the first extraction will start according to the trigger you defined. You can follow the progress in the [Runs](https://app.nekt.ai/runs) tab and find your tables in the [Catalog](https://app.nekt.ai/catalog) as soon as the run finishes.
## How Cardápio Web exposes orders
Cardápio Web has two separate order endpoints, and the connector uses both because neither covers the whole picture on its own.
### Order history
The **Orders** stream reads the order history, which only returns orders that reached a terminal status: completed (`closed`) or cancelled (`canceled`). This is the stream to use for sales analysis.
Because an order only appears here once it is finished, and that can happen days after it was created, each incremental run re-reads the last few days of orders (the **Lookback Days** setting, 7 days by default). An order created on Monday and closed on Thursday is therefore still picked up.
The history goes back at most 3 years, and Cardápio Web accepts a window of at most 6 months per request, so the connector walks the history in successive windows.
### Recent orders
The **Recent Orders** stream reads the orders modified in the last 8 hours, in **any** status. It is the only way to see orders that are still in progress, or stuck in an intermediate status. The 8-hour window is fixed by Cardápio Web, so this stream is only complete for sources that run at least three times a day.
## Rate limits
Cardápio Web applies its limits per establishment, and the connector paces its requests to stay inside them:
| Endpoints | Limit |
| - | - |
| Store, full catalog and order history | 5 requests per minute |
| Everything else | 300 requests per 3 minutes |
Because of the 5-per-minute limit, the first extraction of a long order history is slow by design. The run log reports when the connector is pacing itself.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Merchant | `merchant` | Full table | Profile, address, opening hours and settings of the establishment. |
| Payment Methods | `payment_methods` | Full table | Payment methods enabled for the establishment. Requires the Partner Key. |
| Customers | `customers` | Full table | Customers registered at the establishment, with loyalty and cashback balances. |
| Coupons | `coupons` | Full table | Discount coupons and their rules. |
| Reviews | `reviews` | Incremental | Reviews submitted by customers after an order. |
| Orders | `orders` | Incremental | Completed and cancelled orders, with status and dates. |
| Order Details | `order_details` | Full table | Full detail of each order: items, options, discounts and payments. |
| Recent Orders | `recent_orders` | Full table | Orders modified in the last 8 hours, in any status. |
| Catalog Categories | `catalog_categories` | Full table | Menu categories. |
| Catalog Items | `catalog_items` | Full table | Menu items, with prices, stock and availability. |
| Catalog Option Groups | `catalog_option_groups` | Full table | Add-on groups attached to each item. |
| Catalog Options | `catalog_options` | Full table | Options available within each add-on group. |
The four `catalog_*` streams are all built from a single request to Cardápio Web, so selecting all of them costs no more than selecting one.
## Fields by stream
Below you'll find all available data streams from Cardápio Web and their corresponding fields:
Profile, address, opening hours and settings of the establishment. Always a single record.
**Key Fields:**
* `id` - Unique identifier of the establishment
* `uuid` - Universally unique identifier of the establishment
* `name` - Name of the establishment
* `slug` - Identifier used in the digital menu URL (`app.cardapioweb.com/{slug}`)
**Profile:**
* `description` - Free-text description of the establishment
* `main_color` - Hexadecimal code of the primary brand color
* `instagram` - Instagram handle
* `whatsapp` - WhatsApp number, with country and area code, unformatted
* `custom_domain` - Custom domain configured for the digital menu
* `status` - Whether the establishment is ACTIVE or INACTIVE
* `created_at` - When the establishment was created
* `updated_at` - When the establishment was last modified
**Address:**
* `address_street` - Street
* `address_number` - Street number
* `address_address_block` - Block, in cities that use block and lot instead of a number
* `address_address_lot` - Lot, in cities that use block and lot instead of a number
* `address_neighborhood` - Neighborhood
* `address_complement` - Unit, floor and similar details
* `address_reference` - Landmark that helps locate the address
* `address_postal_code` - Postal code (CEP), unformatted
* `address_city` - City
* `address_state` - Two-letter state code
* `address_latitude` - Latitude
* `address_longitude` - Longitude
**Nested data (JSON strings):**
* `logo_image` - Logo image, with the standard and thumbnail URLs
* `banner_image` - Banner image, with the standard and thumbnail URLs
* `operation_modes` - Which modes the establishment works with: delivery, takeout, onsite, immediate and scheduled
* `opening_hours` - Opening hours per weekday, custom dates, forced open/closed state and the establishment's timezone
* `payment_methods` - Payment methods configured, with their fees and availability
* `schedule_setting` - Scheduling rules: lead times, slot interval, orders per slot and available hours
Payment methods enabled for the establishment. Requires the Partner Key.
**Key Fields:**
* `id` - Unique identifier of the payment method
* `name` - Display name of the payment method, such as `Pix`
* `kind` - Type of the payment method: `money`, `credit_card`, `debit_card`, `food_voucher`, `meal_voucher` or `pix`
**Nested data (JSON strings):**
* `brands` - Card brands accepted for this payment method, each with an id and a name
Customers registered at the establishment. Cardápio Web exposes no date filter for customers, so every run reads the whole customer base.
**Key Fields:**
* `id` - Unique identifier of the customer
* `name` - Name of the customer
* `email` - Email address
* `phone_number` - Phone number, without the country code
* `ddi` - Country calling code of the phone number
**Profile:**
* `birth_date` - Date of birth
* `gender` - Gender: `female`, `male`, `non-binary` or `other`
* `created_at` - When the customer was registered
* `notifications_enabled` - Whether the customer allows WhatsApp messages from the establishment
**Loyalty and cashback:**
* `loyalty_points` - Total loyalty points currently held
* `loyalty_points_expires_at` - When the loyalty point balance expires
* `cashback_balance` - Cashback balance available
* `cashback_expires_at` - When the cashback balance expires
Discount coupons configured by the establishment.
**Key Fields:**
* `id` - Unique identifier of the coupon
* `uid` - Identifier used to apply the coupon automatically through the digital menu URL
* `code` - Code the customer types at checkout; coupons without a code are available to everyone
* `name` - Name of the coupon, shown to the customer
**Discount:**
* `type` - `free_shipping`, `percent_discount` or `flat_discount`
* `value` - Discount amount: a percentage for `percent_discount`, an absolute amount for `flat_discount`, empty for `free_shipping`
* `minimum_order_value` - Minimum order value required to use the coupon
**Availability:**
* `active` - Whether the coupon is active
* `status` - `not_active`, `not_yet_available`, `expired`, `usage_limit_reached`, `not_available` or `valid`
* `use_limit` - Maximum number of times the coupon can be used in total
* `new_customers_only` - Whether the coupon is restricted to first-time customers
* `customer_multi_use` - Whether a single customer may use the coupon more than once
* `availability_start_time` - Time of day from which the coupon can be used
* `availability_end_time` - Time of day until which the coupon can be used
* `available_days` - Weekdays the coupon can be used on; empty means every day
* `available_order_types` - Order types the coupon applies to; empty means every type
* `available_from` - Date from which the coupon becomes available
* `expires_at` - Date the coupon expires, valid until 23:59 that day
**Nested data (JSON strings):**
* `items` - Menu items the discount applies to, each with an id and a name
Reviews submitted by customers after an order. Incremental on `created_at`.
**Key Fields:**
* `id` - Unique identifier of the review
* `order_id` - Order the review refers to
* `customer_id` - Customer who submitted the review
* `created_at` - When the review was recorded, used as the incremental replication key
**Review:**
* `rating` - Average of the scores given by the customer
* `comment` - Free-text comment left by the customer
**Nested data (JSON strings):**
* `answers` - Answer to each survey question, with the question id, the question text and the answer
Completed and cancelled orders, from the order history. Incremental on `created_at`.
**Key Fields:**
* `id` - Unique identifier of the order
* `created_at` - When the order was created, used as the incremental replication key
* `updated_at` - When the order was last modified
**Classification:**
* `status` - `closed` or `canceled` in this stream
* `order_type` - `delivery`, `takeout`, `onsite` or `closed_table` (table or tab)
* `order_timing` - `immediate` or `scheduled`
* `sales_channel` - `catalog`, `store_front_catalog`, `portal`, `whatsapp_extension` or `ifood`
Full detail of each order. One request per order, so this is the slowest stream on a first extraction.
**Key Fields:**
* `id` - Unique identifier of the order
* `display_id` - Friendlier order number shown to the establishment; not unique
* `merchant_id` - Establishment the order belongs to
* `created_at` - When the order was created
* `updated_at` - When the order was last modified
**Classification:**
* `status` - Status of the order, from `waiting_confirmation` through to `closed` or `canceled`
* `order_type` - `delivery`, `takeout`, `onsite` or `closed_table`
* `order_timing` - `immediate` or `scheduled`
* `sales_channel` - Channel the order came through
* `customer_origin` - Free-text tag identifying which variant of the digital-menu link the order came from
* `delivered_by` - Who delivers the order: `merchant`, `ifood`, `ifood_shipping`, `foody_delivery`, `food99`, `keeta` or `aiqfome`
* `table_number` - Table or tab number, for table orders
* `estimated_time` - Estimated delivery time in minutes from creation
* `cancellation_reason` - Why the order was cancelled
**Amounts:**
* `total` - Total amount of the order
* `delivery_fee` - Delivery fee charged, for delivery orders
* `service_fee` - Service fee charged, for table and tab orders
* `additional_fee` - Extra fee freely charged by the establishment
**Marketplace:**
* `external_display_id` - Order number in the marketplace; not unique
* `external_order_id` - Unique identifier of the order in the marketplace
* `external_merchant_id` - Unique identifier of the store in the marketplace
* `external_merchant_name` - Name of the store in the marketplace
**Customer:**
* `customer_id` - Customer who placed the order; joins to the Customers stream
* `customer_name` - Name of the customer
* `customer_phone` - Phone number of the customer
* `customer_ddi` - Country calling code of the customer's phone number
**Delivery address:**
* `delivery_address_street` - Street
* `delivery_address_number` - Street number
* `delivery_address_address_block` - Block, in cities that use block and lot instead of a number
* `delivery_address_address_lot` - Lot, in cities that use block and lot instead of a number
* `delivery_address_neighborhood` - Neighborhood
* `delivery_address_complement` - Unit, floor and similar details
* `delivery_address_reference` - Landmark that helps locate the address
* `delivery_address_postal_code` - Postal code (CEP), unformatted
* `delivery_address_city` - City
* `delivery_address_state` - Two-letter state code
* `delivery_address_latitude` - Latitude
* `delivery_address_longitude` - Longitude
**Scheduling and staff:**
* `scheduled_date_time_start` - Start of the window the order was scheduled for
* `scheduled_date_time_end` - End of the window the order was scheduled for
* `user_id` - Establishment user responsible for the order, for table and tab orders
* `user_name` - Name of that user
**Notes:**
* `fiscal_document` - Customer tax id (CPF or CNPJ) for the receipt, unformatted
* `observation` - General notes about the order
* `internal_observation` - Notes kept for the establishment's own use
**Nested data (JSON strings):**
* `items` - Items of the order, each with the catalog item id, quantity, unit and total price, production status and the selected options
* `discounts` - Discounts applied, with their kind, category (coupon, loyalty or other), amount and the coupon or reward involved
* `payments` - Payments of the order, with the amount, method, online/offline type, status, payment fee and card details
Orders modified in the last 8 hours, in any status. Same fields as the Orders stream.
**Key Fields:**
* `id` - Unique identifier of the order
* `created_at` - When the order was created
* `updated_at` - When the order was last modified
**Classification:**
* `status` - Any status, including orders still in progress
* `order_type` - `delivery`, `takeout`, `onsite` or `closed_table`
* `order_timing` - `immediate` or `scheduled`
* `sales_channel` - Channel the order came through
Menu categories of the establishment.
**Key Fields:**
* `id` - Unique identifier of the category
* `name` - Name of the category
**Category:**
* `description` - Description of the category
* `index` - Display position of the category in the menu
* `status` - `ACTIVE` (visible), `INACTIVE` (hidden) or `MISSING` (out of stock)
**Nested data (JSON strings):**
* `image` - Image of the category, with the standard and thumbnail URLs
* `allowed_times` - Time ranges the category is available; an empty array means always available
Menu items of the establishment.
**Key Fields:**
* `id` - Unique identifier of the item
* `category_id` - Category the item belongs to
* `name` - Name of the item
* `external_code` - Code identifying the item in the establishment's own systems
**Item:**
* `description` - Description of the item
* `kind` - `regular_item` or `combo`
* `unit_type` - Unit of measure: `UN`, `KG` or `L`
* `status` - `ACTIVE`, `INACTIVE` or `MISSING`
* `index` - Display position of the item within its category
* `highlighted` - Whether the item is featured on the digital menu
* `badge` - Highlight badge: `best_seller`, `new_item`, `recommended` or `limited_edition`
* `adults_only` - Whether the item is restricted to customers over 18
* `hide_observation_field` - Whether the customer notes field is hidden for this item
* `available_for` - Where the item is available: `delivery`, `table`, `internal_table`, `service_desk` or `view_only`
**Pricing and stock:**
* `price` - Base price of the item
* `cost_price` - Cost price of the item
* `auto_calculate_cost_price` - Whether the cost price is calculated from the item's supplies
* `promotional_price_active` - Whether the promotional price is currently active
* `promotional_price` - Promotional price of the item
* `stock` - Quantity available in stock
* `active_stock_control` - Whether stock control is enabled for the item
**Nested data (JSON strings):**
* `image` - Image of the item, with the standard and thumbnail URLs
* `extra_images` - Additional images of the item
* `allowed_times` - Time ranges the item is available; an empty array means always available
* `promotional_price_schedules` - Days and times the promotional price applies
* `combo_steps` - Steps of the combo, each with its base price and the items available in it
Add-on groups attached to each menu item. The same group can be attached to several items, so the primary key is `item_id` plus `id`.
**Key Fields:**
* `id` - Unique identifier of the option group
* `item_id` - Item the group is attached to
* `category_id` - Category the item belongs to
* `name` - Name of the option group
**Rules:**
* `status` - `ACTIVE`, `INACTIVE` or `MISSING`
* `choice_type` - `SINGLE` (only one option), `MULTIPLE` (several, no repetition) or `SUMMABLE` (several, with repetition)
* `price_calculation_type` - How the price is calculated with several options selected: `SUM`, `MEAN`, `MAX` or `MIN`
* `minimum_quantity` - Minimum number of options the customer must choose
* `maximum_quantity` - Maximum number of options the customer may choose
* `index` - Display position of the group within the item
Options available within each add-on group. An option can belong to more than one group and be priced differently in each, so the primary key is `item_id` plus `option_group_id` plus `id`.
**Key Fields:**
* `id` - Unique identifier of the option
* `option_group_id` - Option group the option was read under
* `item_id` - Item the option group is attached to
* `category_id` - Category the item belongs to
* `name` - Name of the option
* `external_code` - Code identifying the option in the establishment's own systems
**Option:**
* `description` - Description of the option
* `status` - `ACTIVE`, `INACTIVE` or `MISSING`
* `index` - Display position of the option within the group
* `badge` - Highlight badge: `best_seller`, `new_item`, `recommended` or `limited_edition`
* `max_quantity` - Maximum quantity of this option that can be selected, when the group is `SUMMABLE`
**Pricing and stock:**
* `price` - Price of the option inside this group
* `cost_price` - Cost price of the option
* `stock` - Quantity available in stock
* `active_stock_control` - Whether stock control is enabled for the option
## Known limitations
* **Inventory and aggregated summaries are not available.** Cardápio Web restricts the supplies (`insumos`) endpoints and the aggregated order summaries to OAuth applications published on the CW App Store; they cannot be read with an establishment API token. Raw orders are available, so the same aggregates can be computed in your catalog.
* **The order history only covers terminal orders.** Orders still in progress appear only in the Recent Orders stream, within its 8-hour window.
* **The order history reaches back at most 3 years** and is read in windows of at most 6 months.
* **One source per establishment.** The API token identifies a single establishment, so multi-store groups need one source per store.
# Carta as a data source
Source: https://docs.nekt.com/sources/carta
Bring data from Carta to Nekt.
Carta is a platform that helps companies and investors manage their cap tables, valuations, investments, and equity plans. It provides a centralized system for tracking and managing equity, ensuring compliance and simplifying complex financial processes.
![]()
## Configuring Carta as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Carta option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Carta API credentials for this connection. In order to get access to the API, you need to ask for access directly to Carta ([read this guide](https://docs.carta.com/carta/docs/introduction) if you want to know more). Once you have API access, add your access details:
* **Client ID**: Your Client ID for the Carta API.
* **Client Secret**: Your Client Secret for the Carta API.
When this is done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Carta and their corresponding fields:
Stream for managing firms in your Carta account.
**Key Fields:**
* `id`: Unique identifier for the firm.
* `name`: Name of the firm.
Stream for managing funds within a firm.
**Key Fields:**
* `id`: Unique identifier for the fund.
* `firmId`: Identifier for the firm associated with the fund.
* `name`: Name of the fund.
Stream containing information about investments made by funds.
**Key Fields:**
* `id`: Unique identifier for the investment.
* `firmId`: Identifier for the firm associated with the investment.
* `fundId`: Identifier for the fund associated with the investment.
* `companyId`: Identifier for the company associated with the investment.
* `legalName`: Legal name of the company.
* `doingBusinessAsName`: Doing business as name of the company.
* `website`: Website of the company.
* `country`: Country of the company.
* `stateOfIncorporation`: State of incorporation of the company.
* `cartaCompany`: Whether the company is a Carta company.
* `reference_cap_table_id`: Reference capitalization table ID.
Stream for managing capitalization tables for investments.
**Key Fields:**
* `id`: Unique identifier for the capitalization table.
* `firmId`: Identifier for the firm associated with the capitalization table.
* `fundId`: Identifier for the fund associated with the capitalization table.
* `companyId`: Identifier for the company associated with the capitalization table.
* `name`: Name of the capitalization table.
* `type`: Type of the capitalization table.
* `asOfDate.value`: The date as of which the capitalization table is valid.
Stream for detailed information within a capitalization table.
**Key Fields:**
* `id`: Unique identifier for the capitalization table detail.
* `firmId`: Identifier for the firm associated with the capitalization table.
* `fundId`: Identifier for the fund associated with the capitalization table.
* `companyId`: Identifier for the company associated with the capitalization table.
* `capitalizationTableId`: Identifier for the capitalization table.
* `asOfDate.value`: The date as of which the capitalization table details are valid.
* `summary`: An object containing summary information for the capitalization table (e.g., `fullyDilutedShares`, `outstandingShares`, `cashRaised`).
* `shareClassDetails`: A list of objects with details for each share class.
* `optionPoolSummaries`: A list of objects with summaries for each option pool.
* `warrantBlockDetails`: A list of objects with details for each warrant block.
Stream for managing fund investment documents.
**Key Fields:**
* `documentId`: Unique identifier for the document.
* `fundId`: Identifier for the fund associated with the document.
* `firmId`: Identifier for the firm associated with the document.
* `documentName`: Name of the document.
* `fundName`: Name of the fund.
* `firmName`: Name of the firm.
* `effectiveDate.value`: Effective date of the document.
* `sharedDate.value`: Date when the document was shared.
* `portfolioId`: Identifier for the portfolio this document belongs to.
Stream for downloading fund investment documents.
**Key Fields:**
* `portfolioId`: Identifier for the portfolio.
* `documentId`: Unique identifier for the document.
* `documentName`: Name of the document.
* `downloadUrl`: URL to download the document.
* `downloadUrlExpirationTime`: Expiration time for the download URL.
Stream for managing issuers in your Carta account.
**Key Fields:**
* `id`: Unique identifier for the issuer.
* `legalName`: Legal name of the issuer.
* `doingBusinessAsName`: Doing business as name of the issuer.
* `website`: Website URL of the issuer.
* `status`: Status of the issuer.
* `createdAt`: Timestamp when the issuer was created.
* `updatedAt`: Timestamp when the issuer was last updated.
Stream for the capitalization table of a specific issuer.
**Key Fields:**
* `issuerId`: Unique identifier for the issuer.
* `asOfDate.value`: The date as of which the capitalization table is valid.
* `summary`: An object containing summary information for the capitalization table.
* `shareClassDetails`: A list of objects with details for each share class.
* `shareClassSummaries`: A list of objects with summaries for each share class.
* `optionPoolSummaries`: A list of objects with summaries for each option pool.
* `warrantBlockDetails`: A list of objects with details for each warrant block.
* `warrantBlockSummaries`: A list of objects with summaries for each warrant block.
* `noteBlockSummaries`: A list of objects with summaries for each note block.
Stream for the stakeholder capitalization table of a specific issuer.
**Key Fields:**
* `issuerId`: Unique identifier for the issuer.
* `asOfDate`: Date as of which the capitalization table is valid.
* `stakeholderGroups`: A list of objects, each representing a stakeholder group with detailed information.
Stream for the stakeholder capitalization table of a specific investment.
**Key Fields:**
* `id`: Unique identifier for the stakeholder capitalization table.
* `companyId`: Identifier for the company.
* `asOfDate`: Date as of which the capitalization table is valid.
* `firmId`: Identifier for the firm.
* `fundId`: Identifier for the fund.
* `capitalizationTableId`: Identifier for the capitalization table.
* `stakeholderGroups`: A list of objects, each representing a stakeholder group with detailed information.
Stream for managing shareholder portfolios.
**Key Fields:**
* `portfolioId`: Unique identifier for the portfolio.
* `legalName`: Legal name of the portfolio.
* `createTime`: Timestamp when the portfolio was created.
Stream for issuers within a specific portfolio.
**Key Fields:**
* `id`: Unique identifier for the issuer.
* `portfolioId`: Identifier for the portfolio associated with the issuer.
* `legalName`: Legal name of the issuer.
* `doingBusinessAsName`: Doing business as name of the issuer.
* `website`: Website URL of the issuer.
## Skills for agents
Carta connector documentation as plain markdown, for use in AI agent contexts.
# CasaSoft as a data source
Source: https://docs.nekt.com/sources/casasoft
Bring data from CasaSoft to Nekt.
CasaSoft is a Brazilian property-management ERP for real-estate agencies, delivered in the cloud as cloudSLIM. It runs the whole rental operation: the property portfolio and its owners, the tenant contracts and the guarantees backing them, the charges billed every month, the municipal property tax, the brokerage deals, and the agency's own service tickets and filed documents. This connector brings that operation into your catalog.
![]()
## Configuring CasaSoft as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the CasaSoft option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a cloudSLIM user and the company code of your account. Ask CasaSoft support for a user dedicated to the integration rather than reusing a person's login, so the extraction keeps working when people join or leave and its access can be reviewed on its own.
The following configurations are available:
* **User**: The *Identificação* used to sign in to cloudSLIM, usually an email address.
* **Password**: The password of that user. It is stored encrypted.
* **Company code**: The *Empresa* code of your account, a number such as `1930`. It is the same code you type when signing in, and it selects which company's data is read.
* **Start date**: (Advanced) The earliest date to read, for the four streams CasaSoft lets us filter. See the note below on which ones. Leave it empty to load the whole history.
The **Start date** applies to four streams only: **Tickets**, **Non-conformities**, **Property Tax (IPTU)** and **Charges**. Everything else is always read in full, for two different reasons. Properties, tenants, owners and their insurance can only be filtered by a status or signature date, so filtering on one would leave out records registered years ago that are still active today. **Documents** and **Brokerage Deals** cannot be filtered at all. CasaSoft's own date filter on those two endpoints returns a server error instead of data (see *Known CasaSoft limitations* below).
**Charges** is the one stream where the start date really decides how much data you get. CasaSoft serves that ledger one month at a time and refuses an open-ended request, so the connector walks month by month from your start date to the current month. With the field left empty it loads the **last 12 months** rather than the whole history. Set a start date to go further back.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
CasaSoft exposes the same entity through more than one endpoint, each carrying a different set of columns: Properties, Properties (base view) and Properties (detailed view) describe the same properties, and so do the two Owners streams and the two Surety Bonds streams. Pick the view that carries the columns you need rather than all of them; selecting several means reading the same records more than once.
Selecting a stream your cloudSLIM plan does not include will fail the extraction rather than silently produce an empty table. If a run fails saying CasaSoft refused access to a stream, unselect that stream or ask CasaSoft to release the module for the integration user.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Every CasaSoft stream is a **full table** sync. None of the CasaSoft endpoints exposes a "last modified" column, so there is nothing an incremental sync could reliably use: a contract edited months after it was signed would never come back. Reading each collection in full on every run is what keeps the tables faithful to what is in cloudSLIM right now.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Properties | `imoveis` | Full table | Properties with their commercial terms |
| Properties (base view) | `imoveis_base` | Full table | Properties with utility accounts and current tenant |
| Properties (detailed view) | `imoveis_detalhes` | Full table | Properties with their building and assigned staff |
| Property Fire Insurance | `imoveis_seguro_incendio` | Full table | Fire insurance policies for the managed properties |
| Tenant Contracts | `inquilinos` | Full table | Rental contracts and the tenants holding them |
| Tenant Contracts (detailed view) | `inquilinos_detalhes` | Full table | Contracts with their guarantee and payment agreements |
| Terminated Contracts | `inquilinos_contratos_baixados` | Full table | Contracts that were closed, with the reason |
| Surety Bonds (by contract) | `inquilinos_fiancas_seguro` | Full table | Surety bonds reached through the tenant endpoint |
| Owners | `proprietarios` | Full table | Owners with their payout and tax settings |
| Owners (base view) | `proprietarios_base` | Full table | Owners with nationality and occupation |
| Surety Bonds | `fianca_seguros` | Full table | Surety bonds guaranteeing rental contracts |
| Property Tax (IPTU) | `iptus` | Full table | Municipal property tax records |
| Brokerage Deals | `intermediacoes` | Full table | Deals closed by the agency and their commission |
| Charges | `lancamentos` | Full table | Individual charges and credits posted to a contract, read one month at a time |
| Recurring Charges | `taxas_fixas` | Full table | Charges configured to repeat every month |
| Tickets | `tickets` | Full table | Support tickets opened with the agency |
| Non-conformities | `rncs` | Full table | Non-conformity records from the quality process |
| Documents | `geds` | Full table | Index of the documents filed in cloudSLIM |
## Fields by stream
Below you'll find all available data streams from CasaSoft and their corresponding fields:
Every property the agency manages, with the commercial terms agreed with its owner.
**Endpoint:** `/imoveis` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the property record in cloudSLIM. |
| `referencia` | String | Business reference of the property in the format 00000.000, used across cloudSLIM to link the property to its owner, tenants and charges. |
| `ref_proprietario` | String | Reference of the owner this property belongs to, in the format 00000.00. |
| `situacao_locacao` | String | Rental status of the property: Locado (rented), Divulgação (advertised), Disponível (available), Baixado (deactivated) or Próprio (owned by the agency). |
| `data_cadastro` | String | Date the property was registered in cloudSLIM. |
| `endereco` | String | Street name of the property address. |
| `tipo_endereco` | String | Type of thoroughfare of the address, such as Rua, Avenida or Travessa. |
| `numero` | String | Street number of the property. |
| `complemento` | String | Address complement, such as the apartment, block or unit. |
| `cep` | String | Brazilian postal code (CEP) of the property. |
| `bairro` | String | Neighbourhood of the property. |
| `cidade` | String | City of the property. |
| `estado` | String | Brazilian state (UF) of the property. |
| `finalidade` | String | Intended use of the property, such as residential or commercial. |
| `area_total` | Number | Total area of the property in square metres. |
| `area_util` | Number | Usable (private) area of the property in square metres. |
| `data_desde_locacao` | String | Date the property became available for rent. |
| `data_desde_venda` | String | Date the property became available for sale. |
| `tipo_imovel` | String | Category of the property, such as apartment, house, shop or land. |
| `tipo_construcao` | String | Construction type of the property, such as masonry or wood. |
| `data_baixa` | String | Date the property was deactivated in cloudSLIM, when it no longer is managed. |
| `locacao` | Boolean | Whether the property is managed for rental. |
| `matricula_agua` | String | Water utility account number registered for the property. |
| `matricula_energia` | String | Electricity utility account number registered for the property. |
| `matricula_gas` | String | Gas utility account number registered for the property. |
| `data_angariacao` | String | Date the agency took the property into its portfolio. |
| `comissao` | Number | Management commission charged on the rent, as a percentage. |
| `comissao_sobre_taxas` | String | Whether and how the commission also applies to the charges billed with the rent. |
| `indice_reajuste` | String | Inflation index used to adjust the rent, such as IGPM or IPCA. |
| `forma_reajuste` | String | How the rent adjustment is applied, for example annually on the contract date. |
| `valor_aluguel` | Number | Rent amount currently registered for the property. |
| `multa_desconto` | String | Whether the amount in valor\_multa\_desconto is a penalty or a discount. |
| `valor_multa_desconto` | Number | Penalty or discount amount applied to the rent. |
| `ultimo_inquilino` | String | Name of the most recent tenant of the property. |
| `dia_extrato` | Integer | Day of the month on which the owner statement is issued for this property. |
| `comissao_minima` | Boolean | Whether a minimum commission applies to this property. |
| `motivo_baixa_complemento` | String | Free-text reason recorded when the property was deactivated. |
| `divulgar_internet` | Boolean | Whether the property may be advertised on the agency's website and portals. |
| `numero_mes_garantido` | Integer | Number of months of rent guaranteed to the owner regardless of tenant payment. |
| `grupo` | String | Grouping the agency uses to classify the property. |
| `administra_condominio` | Boolean | Whether the agency also administers the condominium fee of this property. |
The same properties, seen through the endpoint that carries the utility accounts and the current tenant instead of the commercial terms.
**Endpoint:** `/imoveis/base` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the property record in cloudSLIM. |
| `referencia` | String | Business reference of the property in the format 00000.000, used across cloudSLIM to link the property to its owner, tenants and charges. |
| `ref_proprietario` | String | Reference of the owner this property belongs to, in the format 00000.00. |
| `situacao_locacao` | String | Rental status of the property. |
| `data_cadastro` | String | Date the property was registered in cloudSLIM. |
| `endereco` | String | Street name of the property address. |
| `tipo_endereco` | String | Type of thoroughfare of the address, such as Rua, Avenida or Travessa. |
| `numero` | String | Street number of the property. |
| `complemento` | String | Address complement, such as the apartment, block or unit. |
| `cep` | String | Brazilian postal code (CEP) of the property. |
| `bairro` | String | Neighbourhood of the property. |
| `cidade` | String | City of the property. |
| `estado` | String | Brazilian state (UF) of the property. |
| `finalidade` | String | Intended use of the property, such as residential or commercial. |
| `area_total` | Number | Total area of the property in square metres. |
| `area_util` | Number | Usable (private) area of the property in square metres. |
| `data_desde_locacao` | String | Date the property became available for rent. |
| `data_desde_venda` | String | Date the property became available for sale. |
| `tipo_imovel` | String | Category of the property, such as apartment, house, shop or land. |
| `tipo_construcao` | String | Construction type of the property, such as masonry or wood. |
| `data_baixa` | String | Date the property was deactivated in cloudSLIM. |
| `locacao` | Boolean | Whether the property is managed for rental. |
| `matricula_agua` | String | Water utility account number registered for the property. |
| `matricula_energia` | String | Electricity utility account number registered for the property. |
| `matricula_gas` | String | Gas utility account number registered for the property. |
| `data_agua` | String | Date the water utility account was registered or last updated. |
| `situacao_agua` | String | Status of the water utility account for the property. |
| `data_energia` | String | Date the electricity utility account was registered or last updated. |
| `situacao_energia` | String | Status of the electricity utility account for the property. |
| `data_gas` | String | Date the gas utility account was registered or last updated. |
| `situacao_gas` | String | Status of the gas utility account for the property. |
| `indicacoes_fiscais` | JSON String | Municipal fiscal registrations (indicações fiscais) linked to the property, as a JSON array of objects. |
| `inquilino` | JSON String | Summary of the tenant currently occupying the property, as a JSON object. |
The same properties, enriched with the building the unit belongs to and the staff assigned to it.
**Endpoint:** `/imoveis/detalhes` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the property record in cloudSLIM. |
| `referencia` | String | Business reference of the property in the format 00000.000. |
| `ref_proprietario` | String | Reference of the owner this property belongs to, in the format 00000.00. |
| `situacao_locacao` | String | Rental status of the property. |
| `data_cadastro` | String | Date the property was registered in cloudSLIM. |
| `endereco` | String | Street name of the property address. |
| `tipo_endereco` | String | Type of thoroughfare of the address. |
| `numero` | String | Street number of the property. |
| `complemento` | String | Address complement. |
| `cep` | String | Brazilian postal code (CEP) of the property. |
| `bairro` | String | Neighbourhood of the property. |
| `cidade` | String | City of the property. |
| `estado` | String | Brazilian state (UF) of the property. |
| `finalidade` | String | Intended use of the property. |
| `area_total` | Number | Total area of the property in square metres. |
| `area_util` | Number | Usable (private) area of the property in square metres. |
| `data_desde_locacao` | String | Date the property became available for rent. |
| `data_desde_venda` | String | Date the property became available for sale. |
| `tipo_imovel` | String | Category of the property. |
| `tipo_construcao` | String | Construction type of the property. |
| `data_baixa` | String | Date the property was deactivated in cloudSLIM. |
| `locacao` | Boolean | Whether the property is managed for rental. |
| `matricula_agua` | String | Water utility account number. |
| `matricula_energia` | String | Electricity utility account number. |
| `matricula_gas` | String | Gas utility account number. |
| `data_angariacao` | String | Date the agency took the property into its portfolio. |
| `comissao` | Number | Management commission charged on the rent, as a percentage. |
| `comissao_sobre_taxas` | String | Whether and how the commission also applies to the charges billed. |
| `indice_reajuste` | String | Inflation index used to adjust the rent. |
| `forma_reajuste` | String | How the rent adjustment is applied. |
| `valor_aluguel` | Number | Rent amount currently registered for the property. |
| `multa_desconto` | String | Whether the amount in valor\_multa\_desconto is a penalty or a discount. |
| `valor_multa_desconto` | Number | Penalty or discount amount applied to the rent. |
| `ultimo_inquilino` | String | Name of the most recent tenant of the property. |
| `dia_extrato` | Integer | Day of the month on which the owner statement is issued. |
| `comissao_minima` | Boolean | Whether a minimum commission applies to this property. |
| `motivo_baixa_complemento` | String | Free-text reason recorded when the property was deactivated. |
| `divulgar_internet` | Boolean | Whether the property may be advertised on the agency's website and portals. |
| `numero_mes_garantido` | Integer | Number of months of rent guaranteed to the owner. |
| `grupo` | String | Grouping the agency uses to classify the property. |
| `administra_condominio` | Boolean | Whether the agency also administers the condominium fee of this property. |
| `edificio` | JSON String | Building the unit belongs to, as a JSON object. |
| `atendente` | JSON String | Agency staff assigned to the property, as a JSON array of objects. |
Fire insurance policies contracted for the managed properties.
**Endpoint:** `/imoveis/seguroincendio` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the fire insurance record in cloudSLIM. |
| `referencia` | String | Reference of the insured property, in the format 00000.000. |
| `seguradora_id` | String | Internal identifier of the insurance company that issued the policy. |
| `seguradora` | String | Name of the insurance company that issued the policy. |
| `apolice` | String | Policy number of the fire insurance. |
| `data_apolice` | String | Date the policy was issued. |
| `data_renovacao_seguro` | String | Date the policy is due for renewal. |
| `situacao` | String | Status of the policy: Normal, Cancelado (cancelled), Baixado (closed) or Simulação (quote only). |
| `data_situacao` | String | Date the policy reached its current status. |
| `data_inicio` | String | Date the coverage starts. |
| `data_final` | String | Date the coverage ends. |
| `valor_seguro` | Number | Insured value of the property. |
| `valo_segurado` | Number | Amount effectively insured. The field name carries the typo cloudSLIM uses in its own payload. |
| `valor_predial` | Number | Coverage amount for the building structure. |
| `valor_moveis_utencilio` | Number | Coverage amount for furniture and fittings. |
| `valor_vendaval` | Number | Coverage amount for windstorm damage to the building. |
| `valor_vendaval_conteudo` | Number | Coverage amount for windstorm damage to the contents. |
| `valor_danos_eletricos` | Number | Coverage amount for electrical damage. |
| `valor_perda_aluguel` | Number | Coverage amount for loss of rent while the property is uninhabitable. |
| `valor_primeira_parcela` | Number | Amount of the first instalment billed for the policy. |
| `valor_parcela` | Number | Amount of each remaining instalment billed for the policy. |
| `mes_ano_inicial` | String | First month billed for the policy, as MM/YYYY. |
| `mes_ano_final` | String | Last month billed for the policy, as MM/YYYY. |
| `codigo_taxa` | String | Code of the charge type used to bill the policy, from 000 to 999. |
| `nao_debitar_proprietario` | Boolean | Whether the instalments must not be debited from the owner. |
| `nao_debitar_inquilino` | Boolean | Whether the instalments must not be debited from the tenant. |
| `observacao` | String | Free-text notes recorded about the policy. |
One row per rental contract: who rents, which property, on what terms.
**Endpoint:** `/inquilinos` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the tenant contract in cloudSLIM. |
| `referencia` | String | Business reference of the tenant contract in the format 00000.000.00. The first two blocks match the reference of the rented property. |
| `ref_proprietario` | String | Reference of the owner of the rented property, in the format 00000.00. |
| `ref_imovel` | String | Reference of the rented property, in the format 00000.000. |
| `imovel_id` | String | Internal identifier of the rented property, matching id on the property streams. |
| `nome` | String | Full name or company name of the tenant. |
| `cpf_cnpj` | String | Brazilian taxpayer number of the tenant: CPF for individuals, CNPJ for companies. |
| `tipo` | String | Whether the tenant is an individual or a company. |
| `situacao` | String | Status of the tenant contract: Normal, Baixado (terminated), Bloqueado (blocked) or Desocupação (moving out). |
| `email` | String | Email address of the tenant. |
| `telefone` | String | Primary phone number of the tenant. |
| `telefone_secundario` | String | Secondary phone number of the tenant. |
| `whats_app` | String | WhatsApp number of the tenant. |
| `data_cadastro` | String | Date the tenant was registered in cloudSLIM. |
| `data_locacao` | String | Date the rental contract was signed. |
| `data_ocupacao` | String | Date the tenant took occupancy of the property. |
| `data_vigencia` | String | Date the current contract term ends. |
| `data_baixa` | String | Date the contract was terminated, when it no longer is active. |
| `motivo_baixa` | String | Reason recorded for terminating the contract. |
| `data_ultimo_reajuste` | String | Date of the most recent rent adjustment on this contract. |
| `data_proxio_reajuste` | String | Date of the next scheduled rent adjustment. The field name carries the typo cloudSLIM uses in its own payload. |
| `data_revisional` | String | Date the contract is due for a rent review. |
| `indice_reajuste` | String | Inflation index used to adjust this contract's rent, such as IGPM or IPCA. |
| `prazo_locacao` | Integer | Contract term, usually expressed in months. |
| `valor_aluguel` | Number | Rent amount agreed in this contract. |
| `multa_desconto` | String | Whether the amount in valor\_multa\_desconto is a penalty or a discount. |
| `valor_multa_desconto` | Number | Penalty or discount amount applied to this contract's rent. |
| `fianca` | String | Type of guarantee backing the contract, such as a surety bond, a guarantor or a security deposit. |
| `utilizacao_imovel` | String | Declared use of the property under this contract, such as residential or commercial. |
| `grupo` | String | Grouping the agency uses to classify the tenant. |
| `data_nascimento` | String | Date of birth of the tenant, for individuals. |
| `estado_civil` | String | Marital status of the tenant. |
| `nacionalidade` | String | Nationality of the tenant. |
| `profissao` | String | Occupation of the tenant. |
| `sexo` | String | Gender recorded for the tenant. |
| `atendente` | JSON String | Agency staff assigned to this contract, as a JSON array of objects. Serialized as text so the column stays stable as cloudSLIM adds attributes to it. |
The same contracts, plus the surety bond backing them and any payment agreement negotiated with the tenant.
**Endpoint:** `/inquilinos/detalhes` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the tenant contract in cloudSLIM. |
| `referencia` | String | Business reference of the tenant contract in the format 00000.000.00. |
| `ref_proprietario` | String | Reference of the owner of the rented property, in the format 00000.00. |
| `ref_imovel` | String | Reference of the rented property, in the format 00000.000. |
| `imovel_id` | String | Internal identifier of the rented property, matching id on the property streams. |
| `nome` | String | Full name or company name of the tenant. |
| `cpf_cnpj` | String | Brazilian taxpayer number of the tenant. |
| `tipo` | String | Whether the tenant is an individual or a company. |
| `situacao` | String | Status of the tenant contract. |
| `email` | String | Email address of the tenant. |
| `telefone` | String | Primary phone number of the tenant. |
| `telefone_secundario` | String | Secondary phone number of the tenant. |
| `whats_app` | String | WhatsApp number of the tenant. |
| `data_cadastro` | String | Date the tenant was registered in cloudSLIM. |
| `data_locacao` | String | Date the rental contract was signed. |
| `data_ocupacao` | String | Date the tenant took occupancy of the property. |
| `data_vigencia` | String | Date the current contract term ends. |
| `data_baixa` | String | Date the contract was terminated, when it no longer is active. |
| `motivo_baixa` | String | Reason recorded for terminating the contract. |
| `data_ultimo_reajuste` | String | Date of the most recent rent adjustment on this contract. |
| `data_proxio_reajuste` | String | Date of the next scheduled rent adjustment. |
| `data_revisional` | String | Date the contract is due for a rent review. |
| `indice_reajuste` | String | Inflation index used to adjust this contract's rent. |
| `prazo_locacao` | Integer | Contract term, usually expressed in months. |
| `valor_aluguel` | Number | Rent amount agreed in this contract. |
| `multa_desconto` | String | Whether the amount in valor\_multa\_desconto is a penalty or a discount. |
| `valor_multa_desconto` | Number | Penalty or discount amount applied to this contract's rent. |
| `fianca` | String | Type of guarantee backing the contract. |
| `utilizacao_imovel` | String | Declared use of the property under this contract. |
| `grupo` | String | Grouping the agency uses to classify the tenant. |
| `data_nascimento` | String | Date of birth of the tenant, for individuals. |
| `estado_civil` | String | Marital status of the tenant. |
| `nacionalidade` | String | Nationality of the tenant. |
| `profissao` | String | Occupation of the tenant. |
| `sexo` | String | Gender recorded for the tenant. |
| `atendente` | JSON String | Agency staff assigned to this contract, as a JSON array of objects. |
| `fianca_seguro` | JSON String | Surety bond backing this contract, as a JSON object. The same records are available as columns in the inquilinos\_fiancas\_seguro stream. |
| `acordo` | JSON String | Payment agreement negotiated with the tenant over unpaid charges, as a JSON object: the instalments, amounts and status of the agreement. |
Contracts that were closed, with the reason and who handled it.
**Endpoint:** `/inquilinos/contratos/baixados` · **Primary key:** `referencia`, `data_baixa`
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference of the terminated tenant contract, in the format 00000.000.00. |
| `data_baixa` | String | Date the contract was terminated. |
| `motivo` | String | Reason recorded for terminating the contract. |
| `tipo_fianca` | String | Type of guarantee that backed the contract. |
| `cia_seguro` | String | Insurance company of the surety bond, when the guarantee was a bond. |
| `atendente_nome` | String | Name of the agency staff who handled the termination. |
The same bonds as the Surety Bonds stream, reached through the tenant endpoint.
**Endpoint:** `/inquilinos/fiancas/seguro` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the surety bond record in cloudSLIM. |
| `referencia` | String | Reference of the tenant contract the bond guarantees, in the format 00000.000.00. |
| `cia_seguro` | String | Insurance company that issued the surety bond. |
| `numero_apolice` | String | Policy number of the surety bond. |
| `tipo` | String | Type of surety bond contracted. |
| `situacao` | String | Status of the bond, such as Normal or Cancelado. |
| `data_situacao` | String | Date the bond reached its current status. |
| `data_inicial` | String | Date the coverage starts. |
| `data_final` | String | Date the coverage ends. |
| `primeira_parcela` | String | Month and year of the first instalment charged for the bond, as MM/YYYY. |
| `ultima_parcela` | String | Month and year of the last instalment charged for the bond, as MM/YYYY. |
| `valor` | Number | Total amount of the surety bond. |
| `premio_liquido` | Number | Net premium of the policy, before charges and taxes. |
| `encargos` | Number | Charges and taxes added to the net premium. |
| `valor_inquilino` | Number | Share of the bond amount charged to the tenant. |
| `valor_proprietario` | Number | Share of the bond amount charged to the owner. |
| `debito_seguro` | String | Who the bond instalment is debited from. |
| `codigo_taxa` | String | Code of the charge type used to bill the bond, from 000 to 999. |
| `sobre` | String | Base the bond amount is calculated on, such as the rent or the total charges. |
| `observacao` | String | Free-text notes recorded about the bond. |
Property owners, with their contact details, payout terms and tax settings.
**Endpoint:** `/proprietarios` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the owner record in cloudSLIM. |
| `referencia` | String | Business reference of the owner in the format 00000.00, used to link the owner to their properties. |
| `nome` | String | Full name or company name of the owner. |
| `cpf_cnpj` | String | Brazilian taxpayer number of the owner: CPF for individuals, CNPJ for companies. |
| `tipo` | String | Whether the owner is an individual or a company. |
| `email` | String | Email address of the owner. |
| `telefone` | String | Primary phone number of the owner. |
| `telefone_secundario` | String | Secondary phone number of the owner. |
| `telefone_whats_app` | String | WhatsApp number of the owner. |
| `endereco` | String | Street name of the owner's mailing address. |
| `numero` | String | Street number of the owner's mailing address. |
| `complemento` | String | Address complement of the owner's mailing address. |
| `cep` | String | Brazilian postal code (CEP) of the owner's mailing address. |
| `bairro` | String | Neighbourhood of the owner's mailing address. |
| `cidade` | String | City of the owner's mailing address. |
| `estado` | String | Brazilian state (UF) of the owner's mailing address. |
| `est_civil` | String | Marital status of the owner. |
| `grupo` | String | Grouping the agency uses to classify the owner. |
| `sub_grupo` | String | Sub-grouping the agency uses to classify the owner. |
| `data_cadastro` | String | Date the owner was registered in cloudSLIM. |
| `sexo` | String | Gender recorded for the owner. |
| `data_nascimento` | String | Date of birth of the owner, for individuals. |
| `data_baixa` | String | Date the owner was deactivated in cloudSLIM. |
| `participacao` | Number | Share of the property this owner holds, as a percentage, when a property has more than one owner. |
| `dia_extrato` | Integer | Day of the month on which this owner's statement is issued. |
| `calcular_irrf` | Boolean | Whether Brazilian withholding income tax (IRRF) is calculated on this owner's rental income. |
| `irrf_desconto_simplificado` | Boolean | Whether the simplified IRRF deduction is applied to this owner. |
The same owners, seen through the endpoint that carries nationality and occupation instead of the payout terms.
**Endpoint:** `/proprietarios/base` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the owner record in cloudSLIM. |
| `referencia` | String | Business reference of the owner in the format 00000.00, used to link the owner to their properties. |
| `nome` | String | Full name or company name of the owner. |
| `cpf_cnpj` | String | Brazilian taxpayer number of the owner. |
| `tipo` | String | Whether the owner is an individual or a company. |
| `email` | String | Email address of the owner. |
| `telefone` | String | Primary phone number of the owner. |
| `telefone_secundario` | String | Secondary phone number of the owner. |
| `telefone_whats_app` | String | WhatsApp number of the owner. |
| `endereco` | String | Street name of the owner's mailing address. |
| `numero` | String | Street number of the owner's mailing address. |
| `complemento` | String | Address complement of the owner's mailing address. |
| `cep` | String | Brazilian postal code (CEP) of the owner's mailing address. |
| `bairro` | String | Neighbourhood of the owner's mailing address. |
| `cidade` | String | City of the owner's mailing address. |
| `estado` | String | Brazilian state (UF) of the owner's mailing address. |
| `est_civil` | String | Marital status of the owner. |
| `grupo` | String | Grouping the agency uses to classify the owner. |
| `sub_grupo` | String | Sub-grouping the agency uses to classify the owner. |
| `data_cadastro` | String | Date the owner was registered in cloudSLIM. |
| `nacionalidade` | String | Nationality of the owner. |
| `profissao` | String | Occupation of the owner. |
Surety bonds (seguro fianca) contracted to guarantee rental contracts.
**Endpoint:** `/fiancaSeguros` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the surety bond record in cloudSLIM. |
| `referencia` | String | Reference of the tenant contract the bond guarantees, in the format 00000.000.00. |
| `cia_seguro` | String | Insurance company that issued the surety bond. |
| `numero_apolice` | String | Policy number of the surety bond. |
| `tipo` | String | Type of surety bond contracted. |
| `situacao` | String | Status of the bond, such as Normal or Cancelado. |
| `data_situacao` | String | Date the bond reached its current status. |
| `data_inicial` | String | Date the coverage starts. |
| `data_final` | String | Date the coverage ends. |
| `primeira_parcela` | String | Month and year of the first instalment charged for the bond, as MM/YYYY. |
| `ultima_parcela` | String | Month and year of the last instalment charged for the bond, as MM/YYYY. |
| `valor` | Number | Total amount of the surety bond. |
| `premio_liquido` | Number | Net premium of the policy, before charges and taxes. |
| `encargos` | Number | Charges and taxes added to the net premium. |
| `valor_inquilino` | Number | Share of the bond amount charged to the tenant. |
| `valor_proprietario` | Number | Share of the bond amount charged to the owner. |
| `debito_seguro` | String | Who the bond instalment is debited from. |
| `codigo_taxa` | String | Code of the charge type used to bill the bond, from 000 to 999. |
| `sobre` | String | Base the bond amount is calculated on, such as the rent or the total charges. |
| `observacao` | String | Free-text notes recorded about the bond. |
Municipal property tax records tied to the managed properties. Honours the source's start date.
**Endpoint:** `/iptus` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the property tax record in cloudSLIM. |
| `referencia` | String | Reference of the property the tax record belongs to, in the format 00000.000. |
| `codigo_iptu` | String | Municipal fiscal registration code (indicação fiscal) the tax is assessed on. |
| `codigo_imovel_prefeitura` | String | Identifier of the property in the municipality's own records. |
| `codigo_contribuinte` | String | Taxpayer code registered with the municipality for this property. |
| `regiao_id` | String | Internal identifier of the region the property belongs to. |
| `data_cadastro` | String | Date the tax record was registered in cloudSLIM. This is the date the source's start date filters on. |
| `data_ultima_alteracao` | String | Date the tax record was last changed in cloudSLIM. |
| `data_baixa` | String | Date the fiscal registration was deactivated. |
| `situacao` | String | Status of the property tax record. |
| `situacao_locacao` | String | Rental status of the property the tax record belongs to. |
| `natureza` | String | Nature of the property as classified by the municipality. |
| `zoneamento` | String | Municipal zoning classification of the property. |
| `area_terreno` | Number | Land area assessed by the municipality, in square metres. |
| `area_construida` | Number | Built area assessed by the municipality, in square metres. |
| `valor_venal_imovel` | Number | Assessed (venal) value of the property used to calculate the tax. |
| `aliquota` | Number | Tax rate applied to the assessed value, as a percentage. |
| `economia` | Number | Number of economic units (economias) the municipality counts on the fiscal registration. |
| `percentual` | Number | Share of the fiscal registration's total tax that falls on this property, as a percentage. |
| `total_valor` | Number | Total tax amount assessed for the fiscal registration. |
| `valor_rateado` | Number | Portion of the total tax apportioned to this property. |
| `total_data_vencimento` | String | Due date of the tax when paid in a single instalment. |
| `numero_parcela` | Integer | Number of instalments the tax is split into. |
| `mes_primeira_parcela` | String | Month of the year the first instalment is due. |
| `codigo_taxa` | String | Code of the charge type used to bill the tax to the contract. |
| `gerar_pagamento` | Boolean | Whether the agency pays the tax to the municipality on the owner's behalf. |
| `proprietario_pgto_total` | Boolean | Whether the owner paid the whole assessed amount. |
| `inquilino_pgto_total` | Boolean | Whether the tenant paid the whole assessed amount. |
| `nao_creditar_proprietario` | Boolean | Whether the amount must not be credited to the owner's statement. |
| `isento` | Boolean | Whether the property is exempt from the tax. |
| `recibo` | String | Receipt number recorded for the tax payment. |
| `observacao_do_iptu` | String | Free-text notes recorded about the property tax record. |
| `parcelas` | JSON String | Instalments of the tax as assessed by the municipality, as a JSON array of objects. |
| `iptu_parcelas` | JSON String | Instalments as billed by the agency, as a JSON array of objects. These are what reaches the owner's or tenant's statement. |
Brokerage deals closed by the agency, with the commission and how it is collected. Always read in full. CasaSoft's date filter on this endpoint returns a server error.
**Endpoint:** `/intermediacoes` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the brokerage deal in cloudSLIM. |
| `referencia` | String | Reference of the property the deal refers to, in the format 00000.000. |
| `endereco_imovel` | String | Address of the property the deal refers to. |
| `data_cadastro` | String | Date the deal was registered in cloudSLIM. CasaSoft's filter on this date is broken on their side, so every deal is read and the source's start date does not narrow it. |
| `negociador` | String | Name of the broker who closed the deal. |
| `comissao_negociador` | Number | Commission owed to the broker for the deal. |
| `valor_aluguel` | Number | Rent agreed in the deal, which the brokerage fee is calculated from. |
| `codigo_taxa` | String | Code of the charge type used to bill the brokerage fee. |
| `parcelas` | JSON String | Instalments the brokerage fee is collected in, as a JSON array of objects with their periods and amounts. |
| `observacao` | String | Free-text notes recorded about the deal. |
| `as_json` | JSON String | The deal exactly as cloudSLIM serializes it internally, as a JSON string. |
The agency's ledger: every charge and credit posted against a contract in a given month. Read one month per request, from the source's start date to the current month.
**Endpoint:** `/lancamentos` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the charge in cloudSLIM. |
| `referencia` | String | Reference of the contract the charge belongs to, in the format 00000.000.00. |
| `periodo` | String | Month the charge belongs to, as MM/YYYY. |
| `dia_periodo` | Integer | Day within the period the charge is assigned to. |
| `data_lancamento` | String | Date the charge was posted. |
| `codigo_taxa` | String | Code of the charge type, from 000 to 999. It identifies what is being charged: rent, condominium fee, insurance instalment and so on. |
| `historico` | String | Description of the charge, as it appears on the statement. |
| `complemento` | String | Additional description printed with the charge. |
| `participante` | String | Participant number, from 00 to 99, distinguishing co-owners or co-tenants sharing the same contract. |
| `inquilino_dcn` | String | How the charge affects the tenant: D debits it, C credits it, N leaves the tenant untouched. |
| `proprietario_dcn` | String | How the charge affects the owner: D debits it, C credits it, N leaves the owner untouched. |
| `comissao_dcn` | String | How the charge affects the agency's commission: D debits it, C credits it, N leaves it untouched, A excludes the charge from the commission calculation. |
| `valor_total` | Number | Total amount of the charge. |
| `valor_comissao` | Number | Portion of the charge that is the agency's commission. |
The templates behind repeated charges: what is billed every month, to whom, over which period.
**Endpoint:** `/TaxaFixas` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the recurring charge in cloudSLIM. |
| `referencia` | String | Reference of the contract or property the recurring charge belongs to. |
| `taxa` | String | Code of the charge type applied every month, from 000 to 999. |
| `grupo` | String | Grouping used to classify the recurring charge. |
| `historico` | String | Description posted with each occurrence of the charge. |
| `valor` | Number | Amount charged in each occurrence. |
| `fator` | Number | Factor the amount is calculated from, when the charge is not a fixed value: a percentage of the rent, for instance. |
| `periodo_inicial` | String | First month the charge is posted, as MM/YYYY. |
| `periodo_final` | String | Last month the charge is posted, as MM/YYYY. |
| `ocorrencia` | Integer | Number of times the charge has been posted so far. |
| `proporcional` | Boolean | Whether the amount is prorated when the contract covers only part of the month. |
| `participante` | String | Participant number, from 00 to 99, distinguishing co-owners or co-tenants sharing the same contract. |
| `proprietario_id` | String | Internal identifier of the owner the charge is assigned to. |
| `proprietario_dcn` | String | How the charge affects the owner: D debits it, C credits it, N leaves it untouched. |
| `inquilino_dcn` | String | How the charge affects the tenant: D debits it, C credits it, N leaves it untouched. |
| `comissao_dcn` | String | How the charge affects the agency's commission: D debits it, C credits it, N leaves it untouched. |
| `usuario` | String | User who created or last changed the recurring charge. |
Support tickets opened with the agency. Honours the source's start date.
**Endpoint:** `/tickets` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Ticket number, unique within the company. |
| `assunto` | String | Subject of the ticket. |
| `solicitacao` | String | Full text of what the requester asked for. |
| `situacao` | String | Current status of the ticket, such as open, in progress or closed. |
| `data_hora_cadastro` | String | Date and time the ticket was opened. This is the date the source's start date filters on. |
| `referencia` | String | Reference of the record the ticket is about: a property, a tenant contract or an owner, depending on referencia\_tipo. |
| `referencia_tipo` | String | Which kind of record the reference points at. |
| `solicitante_nome` | String | Name of the person who opened the ticket. |
| `solicitante_email` | String | Email address of the person who opened the ticket. |
| `solicitante_telefone` | String | Phone number of the person who opened the ticket. |
| `equipe` | String | Team or department the ticket was filed under. |
| `equipe_responsavel` | String | Team currently responsible for resolving the ticket. |
| `atribuido_para` | String | Agency user the ticket is currently assigned to. |
| `responsavel` | String | Agency user accountable for the ticket. |
| `privativa` | Boolean | Whether the ticket is private and visible only to agency staff. |
| `anexo` | JSON String | Attachments of the ticket, as a JSON array of objects with the file name and identifier. The files themselves are not extracted. |
Non-conformity records (RNCs) from the agency's quality process. Honours the source's start date.
**Endpoint:** `/rncs` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the non-conformity record in cloudSLIM. |
Index of the documents filed in cloudSLIM.
**Endpoint:** `/geds` · **Primary key:** `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique internal identifier of the document in cloudSLIM. |
## Implementation Notes
### API Limitations and Error Handling
* **Document files are not extracted**: The `geds` stream carries the index; the base64 payload cloudSLIM ships with it is dropped to prevent huge data loads.
* **Server Overloads**: cloudSLIM sometimes reports its own overload as a 500 server error rather than a typical 429 rate limit (e.g., `Maximum number of concurrent connections exceeded`). The connector recognizes this and backs off for at least 30 seconds before retrying, avoiding competition for scarce connections. The extraction waits and carries on; no data is lost, but the run takes longer. If this happens frequently, ask CasaSoft to raise the connection limit for the integration user, or schedule the source outside office hours.
* **Repeated Server Errors**: cloudSLIM answers `500` for both genuine hiccups and for endpoints that build invalid SQL. The connector treats any `5xx` error as transient initially. However, if the API returns three *consecutive* identical error bodies (that are not overload messages), the stream stops immediately. This fast-fails a broken request instead of spending the entire retry budget on the same error. The consecutive failure tally resets upon any successful request.
* **Request timeout**: A single page request is given **60 seconds** to complete instead of the SDK's default of 5 minutes. Across a full extraction of a real company, the slowest healthy cloudSLIM response measured 1.4 seconds, so a request that is still pending after five minutes is dead rather than slow. The shorter timeout stops the retry budget (up to 8 attempts) from being spent on dead requests, so a failing stream surfaces in minutes instead of sitting silent for around forty.
## Skills for agents
CasaSoft connector documentation as plain markdown, for use in AI agent contexts.
# Chargebee as a data source
Source: https://docs.nekt.com/sources/chargebee
Bring data from Chargebee to Nekt.
Chargebee is a subscription billing and revenue management platform that helps businesses automate recurring billing, manage subscriptions, handle invoicing, and optimize revenue operations across multiple payment gateways and regions.
![]()
## Configuring Chargebee as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Chargebee option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Chargebee API credentials. The following configurations are available:
* **API Key**: Your Chargebee API key. You can generate one from **Settings > Configure Chargebee > API Keys** in your Chargebee dashboard.
* **Site Name**: Your Chargebee site name, which is the subdomain of your Chargebee URL. For example, if your Chargebee URL is `acme.chargebee.com`, the site name is `acme`.
* **Start Date**: (Optional) The earliest date from which records will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Chargebee and their corresponding fields:
| Field | Type |
| - | - |
| id | string |
| first\_name | string |
| last\_name | string |
| email | string |
| phone | string |
| company | string |
| auto\_collection | string |
| net\_term\_days | integer |
| allow\_direct\_debit | boolean |
| created\_at | integer |
| updated\_at | integer |
| taxability | string |
| locale | string |
| pii\_cleared | string |
| channel | string |
| resource\_version | integer |
| deleted | boolean |
| registered\_for\_gst | boolean |
| customer\_type | string |
| business\_customer\_without\_vat\_number | boolean |
| primary\_payment\_source\_id | string |
| backup\_payment\_source\_id | string |
| promotional\_credits | integer |
| unbilled\_charges | integer |
| refundable\_credits | integer |
| excess\_payments | integer |
| preferred\_currency\_code | string |
| mrr | integer |
| cf\_custom\_fields | string |
| Field | Type |
| - | - |
| id | string |
| customer\_id | string |
| plan\_id | string |
| plan\_quantity | integer |
| plan\_unit\_price | integer |
| plan\_amount | integer |
| billing\_period | integer |
| billing\_period\_unit | string |
| plan\_free\_quantity | integer |
| status | string |
| trial\_start | integer |
| trial\_end | integer |
| current\_term\_start | integer |
| current\_term\_end | integer |
| next\_billing\_at | integer |
| created\_at | integer |
| updated\_at | integer |
| started\_at | integer |
| activated\_at | integer |
| cancelled\_at | integer |
| cancel\_reason | string |
| channel | string |
| resource\_version | integer |
| deleted | boolean |
| currency\_code | string |
| mrr | integer |
| exchange\_rate | number |
| base\_currency\_code | string |
| has\_scheduled\_changes | boolean |
| has\_scheduled\_advance\_invoices | boolean |
| due\_invoices\_count | integer |
| start\_date | integer |
| auto\_collection | string |
| remaining\_billing\_cycles | integer |
| Field | Type |
| - | - |
| id | string |
| document\_number | string |
| invoice\_id | string |
| subscription\_id | string |
| customer\_id | string |
| status | string |
| cancellation\_reason | string |
| payment\_status | string |
| order\_type | string |
| price\_type | string |
| order\_date | integer |
| shipping\_date | integer |
| created\_at | integer |
| updated\_at | integer |
| shipped\_at | integer |
| delivered\_at | integer |
| cancelled\_at | integer |
| total | integer |
| sub\_total | integer |
| tax | integer |
| discount | integer |
| amount\_paid | integer |
| amount\_adjusted | integer |
| refundable\_credits\_issued | integer |
| tracking\_id | string |
| tracking\_url | string |
| batch\_id | string |
| resource\_version | integer |
| deleted | boolean |
| currency\_code | string |
| Field | Type |
| - | - |
| id | string |
| customer\_id | string |
| subscription\_id | string |
| gateway\_account\_id | string |
| payment\_source\_id | string |
| payment\_method | string |
| gateway | string |
| type | string |
| date | integer |
| settled\_at | integer |
| exchange\_rate | number |
| currency\_code | string |
| amount | integer |
| amount\_unused | integer |
| base\_currency\_code | string |
| id\_at\_gateway | string |
| status | string |
| error\_code | string |
| error\_text | string |
| fraud\_flag | string |
| reference\_number | string |
| resource\_version | integer |
| updated\_at | integer |
| created\_at | integer |
| deleted | boolean |
# Chatwoot as a data source
Source: https://docs.nekt.com/sources/chatwoot
Bring data from Chatwoot to your Lakehouse.
Chatwoot is an open-source customer engagement platform that centralizes conversations from a website widget, WhatsApp, Instagram, Facebook, Telegram, email and a generic API channel into shared inboxes. The connector extracts those conversations and their full message history, the contacts behind them, and the account configuration (inboxes, agents, teams, labels and custom attributes), so you can analyse your customer conversations in your Lakehouse.
The connector works with **Chatwoot Cloud** and with **self-hosted** installations; the only difference is the address you inform in the Chatwoot URL setting.
## Configuring Chatwoot as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Chatwoot option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Chatwoot authenticates with a static **user access token**, sent in the `api_access_token` header. In Chatwoot, open your avatar in the bottom left corner, go to **Profile Settings**, scroll to **Access Token** and copy the value.
The token inherits the permissions of the user who owns it. An **administrator** token reaches every inbox, conversation and contact of the account; an **agent** token only reaches the inboxes that agent is assigned to, and Chatwoot denies access to the Agent Bots, Automation Rules and Inbox Members streams. Generate the token with an account administrator to sync the whole account.
The following configurations are available:
* **Access token**: the user access token copied from **Profile Settings > Access Token**. Required.
* **Account ID**: the numeric identifier of the account you want to sync. It is the number that appears in the application URL: in `app.chatwoot.com/app/accounts/1/dashboard`, the account ID is `1`. Required.
* **Chatwoot URL**: the address of your Chatwoot installation. Keep the default `https://app.chatwoot.com` for Chatwoot Cloud, or inform your own address when self-hosting, for example `https://chat.yourcompany.com`.
* **Initial sync date**: the earliest date from which records will be synced. It applies to the streams replicated incrementally: Conversations (by last activity) and CSAT Survey Responses (by creation date).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
The **Messages** stream is a child of **Conversations**: it fetches the message history of each conversation the run touched. Selecting it on an account with a long history makes the first extraction considerably longer, since every conversation is read from its first message.
If you don't see a stream you were expecting to find, please check if your access token has access to it. Chatwoot's Reporting Events and Audit Logs endpoints, for instance, exist only in the enterprise edition and are not covered by this connector.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Conversations and CSAT Survey Responses support INCREMENTAL; the remaining streams are full table. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Chatwoot and their corresponding fields. API reference: [developers.chatwoot.com](https://developers.chatwoot.com/api-reference/introduction).
The Chatwoot account itself (`GET /api/v1/accounts/{account_id}`). A single row carrying the account's locale, status, plan and feature flags.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the account, used as the primary key. |
| `name` | String | Display name of the account. |
| `locale` | String | Default language of the account, for example 'en' or 'pt\_BR'. |
| `domain` | String | Custom domain configured for the account's help center. |
| `support_email` | String | Email address used as the sender for the account's support emails. |
| `status` | String | Lifecycle status of the account, for example 'active' or 'suspended'. |
| `created_at` | Datetime | Timestamp when the account was created (UTC). |
| `latest_chatwoot_version` | String | Latest Chatwoot version available upstream, reported by self-hosted installations. |
| `features` | String | Feature flags enabled for the account, as a JSON object mapping each feature name to a boolean. |
| `settings` | String | Account-level settings, as a JSON object, including the auto-resolve delay and the message sent when a conversation is auto-resolved. |
| `custom_attributes` | String | Account metadata Chatwoot keeps outside the main columns, as a JSON object: plan name, subscription status, industry, company size and timezone. |
| `subscribed_features` | Array | Enterprise features included in the account's subscription. |
Users that belong to the account (`GET /api/v1/accounts/{account_id}/agents`), with their role and current presence.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the agent. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Full name of the agent. |
| `available_name` | String | Name shown to contacts in the conversation, which can differ from the agent's internal name. |
| `email` | String | Email address the agent signs in with. |
| `role` | String | Role of the agent in the account: 'administrator' or 'agent'. |
| `confirmed` | Boolean | Whether the agent has confirmed their email invitation. |
| `availability_status` | String | Current presence of the agent: 'online', 'busy' or 'offline'. |
| `auto_offline` | Boolean | Whether Chatwoot automatically marks the agent offline after a period of inactivity. |
| `provider` | String | Authentication provider of the agent account, for example 'email'. |
| `thumbnail` | String | URL of the agent's avatar image. |
| `custom_attributes` | String | Custom attributes stored on the agent, as a JSON object. |
Teams used to group agents and route conversations (`GET /api/v1/accounts/{account_id}/teams`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the team, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Name of the team. |
| `description` | String | Free-text description of what the team handles. |
| `allow_auto_assign` | Boolean | Whether assigning a conversation to this team also auto-assigns it to one of the team's agents. |
| `icon` | String | Emoji shown as the team's icon. |
| `icon_color` | String | Hex color of the team's icon. |
Membership rows linking each team to its agents (`GET /api/v1/accounts/{account_id}/teams/{team_id}/team_members`). **Child stream of Teams.**
Primary key: `team_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `team_id` | Integer | Identifier of the team the agent belongs to. |
| `id` | Integer | Unique identifier of the agent. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Full name of the agent. |
| `available_name` | String | Name shown to contacts in the conversation, which can differ from the agent's internal name. |
| `email` | String | Email address the agent signs in with. |
| `role` | String | Role of the agent in the account: 'administrator' or 'agent'. |
| `confirmed` | Boolean | Whether the agent has confirmed their email invitation. |
| `availability_status` | String | Current presence of the agent: 'online', 'busy' or 'offline'. |
| `auto_offline` | Boolean | Whether Chatwoot automatically marks the agent offline after a period of inactivity. |
| `provider` | String | Authentication provider of the agent account, for example 'email'. |
| `thumbnail` | String | URL of the agent's avatar image. |
| `custom_attributes` | String | Custom attributes stored on the agent, as a JSON object. |
The channels of the account: website widget, WhatsApp, Instagram, email, API and others (`GET /api/v1/accounts/{account_id}/inboxes`). Channel credentials that Chatwoot returns to administrators (IMAP/SMTP passwords, HMAC tokens, Twilio auth tokens) are deliberately **not** extracted.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the inbox, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Display name of the inbox. |
| `channel_type` | String | Channel class backing the inbox, for example 'Channel::WebWidget', 'Channel::Whatsapp', 'Channel::Email' or 'Channel::Api'. |
| `channel_id` | Integer | Identifier of the channel record backing the inbox. |
| `provider` | String | Provider used by the channel when it supports more than one, for example 'whatsapp\_cloud' or 'microsoft' for email. |
| `avatar_url` | String | URL of the inbox avatar shown to contacts. |
| `greeting_enabled` | Boolean | Whether a greeting message is sent when a conversation starts. |
| `greeting_message` | String | Greeting message sent when a conversation starts. |
| `working_hours_enabled` | Boolean | Whether business hours are configured for this inbox. |
| `working_hours` | String | Weekly business-hours schedule, as a JSON array of objects with the day of week, open and close times. |
| `timezone` | String | Timezone the inbox's business hours are expressed in. |
| `out_of_office_message` | String | Message sent to contacts who write outside business hours. |
| `enable_email_collect` | Boolean | Whether Chatwoot asks the contact for an email address. |
| `csat_survey_enabled` | Boolean | Whether a satisfaction survey is sent when a conversation is resolved. |
| `csat_config` | String | Satisfaction survey configuration, as a JSON object. |
| `enable_auto_assignment` | Boolean | Whether new conversations are auto-assigned to available agents. |
| `auto_assignment_config` | String | Auto-assignment configuration, as a JSON object, including the maximum number of conversations an agent can be assigned at once. |
| `allow_messages_after_resolved` | Boolean | Whether contacts can keep writing to a conversation after it is resolved. |
| `lock_to_single_conversation` | Boolean | Whether every message from a contact is appended to a single ongoing conversation instead of opening a new one. |
| `sender_name_type` | String | Which name outgoing messages are signed with: 'friendly' (agent name) or 'professional' (business name). |
| `business_name` | String | Business name used when messages are signed professionally. |
| `callback_webhook_url` | String | Webhook URL the channel provider calls back on. |
| `website_url` | String | Website the widget is installed on (website widget inboxes). |
| `widget_color` | String | Hex color of the website widget. |
| `welcome_title` | String | Title shown at the top of the website widget. |
| `welcome_tagline` | String | Tagline shown under the title of the website widget. |
| `reply_time` | String | Expected reply time advertised in the website widget: 'in\_a\_few\_minutes', 'in\_a\_few\_hours' or 'in\_a\_day'. |
| `allowed_domains` | String | Domains allowed to embed the website widget. |
| `pre_chat_form_enabled` | Boolean | Whether the widget asks the contact for details before chatting. |
| `pre_chat_form_options` | String | Pre-chat form configuration, as a JSON object. |
| `continuity_via_email` | Boolean | Whether website widget conversations continue by email once the contact leaves the page. |
| `email` | String | Address the inbox receives mail on (email inboxes). |
| `forward_to_email` | String | Address incoming mail should be forwarded to (email inboxes). |
| `phone_number` | String | Phone number of the inbox (WhatsApp, SMS and voice inboxes). |
| `medium` | String | Twilio medium of the inbox: 'sms' or 'whatsapp'. |
| `bot_name` | String | Name of the Telegram bot backing the inbox. |
| `page_id` | String | Facebook page identifier backing the inbox. |
| `instagram_id` | String | Instagram account identifier backing the inbox. |
| `inbox_identifier` | String | Public identifier of an API inbox, used by the client APIs to create contacts and conversations. |
| `webhook_url` | String | URL Chatwoot posts outgoing messages to (API inboxes). |
| `additional_attributes` | String | Channel-specific extra settings, as a JSON object. |
| `reauthorization_required` | Boolean | Whether the channel's authorization has expired and an administrator must reconnect it. |
| `help_center` | String | Help center portal linked to the inbox, as a JSON object with its name and slug. |
Membership rows linking each inbox to the agents assigned to it (`GET /api/v1/accounts/{account_id}/inbox_members/{inbox_id}`). **Child stream of Inboxes.**
Primary key: `inbox_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `inbox_id` | Integer | Identifier of the inbox the agent is assigned to. |
| `id` | Integer | Unique identifier of the agent. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Full name of the agent. |
| `available_name` | String | Name shown to contacts in the conversation, which can differ from the agent's internal name. |
| `email` | String | Email address the agent signs in with. |
| `role` | String | Role of the agent in the account: 'administrator' or 'agent'. |
| `confirmed` | Boolean | Whether the agent has confirmed their email invitation. |
| `availability_status` | String | Current presence of the agent: 'online', 'busy' or 'offline'. |
| `auto_offline` | Boolean | Whether Chatwoot automatically marks the agent offline after a period of inactivity. |
| `provider` | String | Authentication provider of the agent account, for example 'email'. |
| `thumbnail` | String | URL of the agent's avatar image. |
| `custom_attributes` | String | Custom attributes stored on the agent, as a JSON object. |
Labels available to tag conversations and contacts (`GET /api/v1/accounts/{account_id}/labels`). The `title` is what appears in the `labels` array of a conversation.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the label, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `title` | String | Title of the label; this is the value that appears in the 'labels' array of a conversation. |
| `description` | String | Free-text description of what the label means. |
| `color` | String | Hex color the label is displayed with. |
| `show_on_sidebar` | Boolean | Whether the label is pinned to the conversation sidebar. |
Saved replies agents insert with a short code (`GET /api/v1/accounts/{account_id}/canned_responses`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the canned response, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `short_code` | String | Short code an agent types to insert the response. |
| `content` | String | Message body inserted when the short code is used. |
| `created_at` | Datetime | Timestamp when the canned response was created (UTC). |
| `updated_at` | Datetime | Timestamp when the canned response was last updated (UTC). |
Definitions of the custom attributes available on contacts and conversations (`GET /api/v1/accounts/{account_id}/custom_attribute_definitions`). Both models are extracted; `attribute_key` is the key that appears inside the `custom_attributes` JSON of contacts and conversations.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the definition, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `attribute_model` | String | Which record the attribute is attached to: 'contact\_attribute' or 'conversation\_attribute'. |
| `attribute_key` | String | Machine key of the attribute, the key that appears inside the 'custom\_attributes' JSON of contacts and conversations. |
| `attribute_display_name` | String | Label shown for the attribute in the Chatwoot interface. |
| `attribute_display_type` | String | Input type of the attribute: text, number, currency, percent, link, date, list or checkbox. |
| `attribute_description` | String | Help text explaining what the attribute records. |
| `attribute_values` | String | Allowed values for a list attribute, as a JSON array of strings. |
| `default_value` | String | Value the attribute takes when none is set. |
| `regex_pattern` | String | Regular expression the value must match, when one is configured. |
| `regex_cue` | String | Hint shown to the agent when the value fails the regex. |
| `created_at` | Datetime | Timestamp when the definition was created (UTC). |
| `updated_at` | Datetime | Timestamp when the definition was last updated (UTC). |
Rules that react to conversation and message events (`GET /api/v1/accounts/{account_id}/automation_rules`). Requires an administrator token; the stream is skipped with a warning otherwise.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the automation rule, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Name of the automation rule. |
| `description` | String | Free-text description of what the rule does. |
| `event_name` | String | Event that triggers the rule: 'conversation\_created', 'conversation\_updated' or 'message\_created'. |
| `conditions` | String | Conditions evaluated when the event fires, as a JSON array of objects with attribute\_key, filter\_operator, values and query\_operator. |
| `actions` | String | Actions performed when the conditions match, as a JSON array of objects with action\_name and action\_params. |
| `active` | Boolean | Whether the rule is currently enabled. |
| `created_on` | Datetime | Timestamp when the rule was created (UTC). |
Agent bots registered on the account (`GET /api/v1/accounts/{account_id}/agent_bots`). Requires an administrator token; the stream is skipped with a warning otherwise. The bot access token and signing secret are deliberately **not** extracted.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the agent bot, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Name of the agent bot. |
| `description` | String | Free-text description of what the bot does. |
| `bot_type` | String | Type of the bot: 'webhook' or 'csml'. |
| `bot_config` | String | Bot-type-specific configuration, as a JSON object. |
| `outgoing_url` | String | URL Chatwoot posts conversation events to for webhook bots. |
| `thumbnail` | String | URL of the bot's avatar image. |
| `system_bot` | Boolean | Whether the bot ships with Chatwoot rather than being customer-created. |
The people behind the conversations (`GET /api/v1/accounts/{account_id}/contacts`). Chatwoot exposes no modification date on contacts, so this stream is replicated in full on every run.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the contact, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `name` | String | Display name of the contact. |
| `email` | String | Email address of the contact. |
| `phone_number` | String | Phone number of the contact, in E.164 format. |
| `identifier` | String | External identifier set by the system that created the contact, used to reconcile it with the customer's own database. |
| `blocked` | Boolean | Whether the contact has been blocked from writing to the account. |
| `availability_status` | String | Whether the contact is currently 'online' or 'offline'. |
| `thumbnail` | String | URL of the contact's avatar image. |
| `created_at` | Datetime | Timestamp when the contact was created (UTC). |
| `last_activity_at` | Datetime | Timestamp of the contact's most recent activity (UTC). |
| `additional_attributes` | String | Attributes Chatwoot collects automatically, as a JSON object: browser and device details, referrer, city, country and company name. |
| `custom_attributes` | String | Custom attributes set on the contact, as a JSON object keyed by the attribute keys listed in the custom\_attribute\_definitions stream. |
| `contact_inboxes` | String | Inboxes the contact can be reached through, as a JSON array of objects with the inbox and the source\_id that identifies the contact in it. |
Conversations held with contacts across every inbox (`GET /api/v1/accounts/{account_id}/conversations`), in every status and for every assignee. Replicated incrementally on `last_activity_at`, which Chatwoot bumps on any change to the conversation: a new message, a status or assignee change, a label.
Primary key: `id` · Replication key: `last_activity_at`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the conversation within the account, the number shown in the Chatwoot interface and used in the API paths. Used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `uuid` | String | Globally unique identifier of the conversation. |
| `inbox_id` | Integer | Identifier of the inbox the conversation arrived in. |
| `channel` | String | Channel class of the inbox the conversation arrived in, for example 'Channel::WebWidget' or 'Channel::Whatsapp'. |
| `status` | String | Current status of the conversation: 'open', 'resolved', 'pending' or 'snoozed'. |
| `priority` | String | Priority assigned to the conversation: 'low', 'medium', 'high' or 'urgent'. Empty when no priority is set. |
| `labels` | Array | Titles of the labels applied to the conversation. |
| `contact_id` | Integer | Identifier of the contact the conversation is held with. |
| `contact_name` | String | Display name of the contact the conversation is held with. |
| `contact_email` | String | Email address of the contact the conversation is held with. |
| `contact_phone_number` | String | Phone number of the contact the conversation is held with. |
| `contact_identifier` | String | External identifier of the contact, when the account sets one. |
| `assignee_id` | Integer | Identifier of the agent or agent bot the conversation is assigned to. Empty while the conversation is unassigned. |
| `assignee_name` | String | Name of the agent or agent bot the conversation is assigned to. |
| `assignee_email` | String | Email address of the agent the conversation is assigned to. |
| `assignee_type` | String | Whether the assignee is a human agent ('User') or an automation ('AgentBot'). |
| `team_id` | Integer | Identifier of the team the conversation is routed to. |
| `team_name` | String | Name of the team the conversation is routed to. |
| `created_at` | Datetime | Timestamp when the conversation was opened (UTC). |
| `updated_at` | Datetime | Timestamp when the conversation record was last written (UTC). |
| `last_activity_at` | Datetime | Timestamp of the last activity in the conversation (UTC): a message, a status change or an assignment. Used as the incremental replication key. |
| `first_reply_created_at` | Datetime | Timestamp of the first agent reply (UTC), the basis of the first response time metric. Empty when no agent has replied yet. |
| `waiting_since` | Datetime | Timestamp since when the conversation has been waiting for an agent reply (UTC). Empty when nothing is pending. |
| `snoozed_until` | Datetime | Timestamp until which the conversation stays snoozed (UTC). |
| `agent_last_seen_at` | Datetime | Timestamp when any agent last opened the conversation (UTC). |
| `assignee_last_seen_at` | Datetime | Timestamp when the assigned agent last opened the conversation (UTC). |
| `contact_last_seen_at` | Datetime | Timestamp when the contact last read the conversation (UTC). |
| `unread_count` | Integer | Number of incoming messages no agent has read yet. |
| `muted` | Boolean | Whether notifications for this conversation are muted. |
| `can_reply` | Boolean | Whether an agent can still reply; some channels close the reply window a fixed time after the contact's last message. |
| `hmac_verified` | Boolean | Whether the contact's identity was verified with the inbox's HMAC signature. |
| `sla_policy_id` | Integer | Identifier of the SLA policy applied to the conversation. |
| `last_message_id` | Integer | Identifier of the latest message that is not a system activity; the full history lives in the messages stream. |
| `last_message_content` | String | Body of the latest non-activity message in the conversation. |
| `last_message_type` | String | Direction of the latest non-activity message: 'incoming' (from the contact), 'outgoing' (from an agent) or 'template'. |
| `last_message_created_at` | Datetime | Timestamp of the latest non-activity message (UTC). |
| `additional_attributes` | String | Context Chatwoot captured when the conversation started, as a JSON object: browser, referrer and initiated-at details. |
| `custom_attributes` | String | Custom attributes set on the conversation, as a JSON object keyed by the attribute keys listed in the custom\_attribute\_definitions stream. |
The full message history of a conversation, including the system activity notes (`GET /api/v1/accounts/{account_id}/conversations/{conversation_id}/messages`). **Child stream of Conversations**: an incremental run only re-reads the conversations that saw activity since the last sync.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the message, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `conversation_id` | Integer | Identifier of the conversation the message belongs to. |
| `inbox_id` | Integer | Identifier of the inbox the message was exchanged in. |
| `content` | String | Text body of the message. |
| `message_type` | String | Direction of the message: 'incoming' (from the contact), 'outgoing' (from an agent or bot), 'activity' (a system note such as a status change) or 'template'. |
| `content_type` | String | How the message body should be rendered: 'text', 'input\_select', 'cards', 'form', 'article', 'incoming\_email', 'input\_csat' and others. |
| `status` | String | Delivery status of the message: 'sent', 'delivered', 'read' or 'failed'. |
| `private` | Boolean | Whether the message is a private note visible only to agents rather than a reply to the contact. |
| `created_at` | Datetime | Timestamp when the message was created (UTC). |
| `sender_id` | Integer | Identifier of the contact, agent or agent bot that sent the message. |
| `sender_type` | String | What kind of sender wrote the message: 'contact', 'user' (agent) or 'agent\_bot'. Empty for system activity messages. |
| `sender_name` | String | Display name of the sender. |
| `sender` | String | Full sender payload, as a JSON object; the shape depends on whether the sender is a contact, an agent or an agent bot. |
| `source_id` | String | Identifier of the message in the origin channel, for example the provider's message id on WhatsApp or the Message-ID of an email. |
| `content_attributes` | String | Structured payload attached to the message, as a JSON object holding email headers, CSAT answers, the items of an interactive message and the details of system activities. |
| `attachments` | String | Files sent with the message, as a JSON array of objects with the file type, size, extension and download URL. |
Answers contacts gave to the satisfaction survey sent when a conversation is resolved (`GET /api/v1/accounts/{account_id}/csat_survey_responses`). Only inboxes with `csat_survey_enabled` produce rows.
Primary key: `id` · Replication key: `created_at`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the survey response, used as the primary key. |
| `account_id` | Integer | Identifier of the Chatwoot account the record belongs to. |
| `conversation_id` | Integer | Identifier of the conversation the survey was sent for. |
| `message_id` | Integer | Identifier of the message that carried the survey. |
| `rating` | Integer | Score the contact gave, from 1 (very poor) to 5 (excellent). |
| `feedback_message` | String | Free-text comment the contact left with the rating. |
| `created_at` | Datetime | Timestamp when the survey response was recorded (UTC), used as the incremental replication key. |
| `contact_id` | Integer | Identifier of the contact who answered the survey. |
| `contact_name` | String | Display name of the contact who answered the survey. |
| `contact_email` | String | Email address of the contact who answered the survey. |
| `assigned_agent_id` | Integer | Identifier of the agent assigned to the rated conversation. |
| `assigned_agent_name` | String | Name of the agent assigned to the rated conversation. |
| `assigned_agent_email` | String | Email address of the agent assigned to the rated conversation. |
| `csat_review_notes` | String | Internal notes an agent added when reviewing the response. |
| `review_notes_updated_at` | Datetime | Timestamp when the review notes were last edited (UTC). |
| `review_notes_updated_by_id` | Integer | Identifier of the agent who last edited the review notes. |
| `review_notes_updated_by_name` | String | Name of the agent who last edited the review notes. |
# Data Model
**Conversations** is the core entity: every conversation belongs to a contact and to an inbox, and carries the messages exchanged in it. Agents, teams and labels describe who handled the conversation and how it was classified.
```mermaid theme={null}
graph TD;
subgraph "Conversations"
Conversations("Conversations");
Messages("Messages");
Csat("CSAT Survey Responses");
end
subgraph "People"
Contacts("Contacts");
Agents("Agents");
Teams("Teams");
TeamMembers("Team Members");
end
subgraph "Configuration"
Account("Account");
Inboxes("Inboxes");
InboxMembers("Inbox Members");
Labels("Labels");
CustomAttributes("Custom Attribute Definitions");
CannedResponses("Canned Responses");
AutomationRules("Automation Rules");
AgentBots("Agent Bots");
end
Messages -- "conversation_id" --> Conversations;
Csat -- "conversation_id" --> Conversations;
Conversations -- "contact_id" --> Contacts;
Conversations -- "inbox_id" --> Inboxes;
Conversations -- "assignee_id" --> Agents;
Conversations -- "team_id" --> Teams;
Conversations -- "labels" --> Labels;
Messages -- "sender_id" --> Contacts;
TeamMembers -- "team_id" --> Teams;
InboxMembers -- "inbox_id" --> Inboxes;
```
# Implementation Notes
### Incremental replication of conversations
Chatwoot offers no "modified since" filter on the conversations list, but it does return conversations sorted by last activity, newest first. An incremental extraction therefore reads from the most recent conversation and stops as soon as it walks past the point reached in the previous run. Because Chatwoot bumps `last_activity_at` on every change to a conversation (a new message, a status or assignee change, a label), no update is missed.
The **Messages** stream follows along: only the conversations that saw activity since the last run have their history re-read, and messages are matched on their identifier, so re-reading a conversation never duplicates rows.
### Timestamps and nested fields
* Chatwoot returns timestamps as Unix epochs and represents an absent timestamp as `0`. The connector converts them to proper datetimes in UTC and turns those zeros into empty values, so a conversation that was never opened by an agent shows an empty `agent_last_seen_at` rather than a date in 1970.
* Fields that hold a nested structure (`custom_attributes`, `additional_attributes`, `content_attributes`, `attachments`, `sender`, `working_hours` and others) are stored as **JSON strings**. Use your warehouse's JSON functions to read inside them. The `labels` field of a conversation is the exception: it is a plain array of label titles.
* `message_type` is translated from Chatwoot's internal number to its name: `incoming`, `outgoing`, `activity` or `template`.
### Credentials are never extracted
Chatwoot returns channel credentials to administrator tokens: IMAP and SMTP passwords, HMAC tokens, Twilio auth tokens and agent bot access tokens. None of them are part of the extracted schemas, so they never reach your Lakehouse.
### Rate limits
Chatwoot throttles requests per IP address (3000 requests per minute by default, configurable on self-hosted installations). The connector respects the limit, waits the interval Chatwoot asks for when it is reached, and retries automatically.
# Claude as a data source
Source: https://docs.nekt.com/sources/claude
Bring Claude usage and cost data to Nekt.
Claude (Anthropic) provides usage reporting and cost breakdowns for your organizations and API keys. This connector extracts those reports from the Anthropic Admin API and lands them in your Nekt Catalog as queryable tables.
This connector is not available for individual Claude accounts, only organization ones.
![]()
## Configuring Claude as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Claude option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an Anthropic Admin API key with access to usage and cost reporting for your organization. Once you have it, add the account access, then click **Next**.
The following configurations are available:
* **Admin API Key**: Anthropic Admin API key (starts with `sk-ant-admin...`).
* **Start Date**: Select the earliest date you want to extract data from.
Make sure the API key you use has the required Admin permissions for usage and cost reporting. Without it, the connector cannot extract data from the Anthropic Admin API.
### 2. Select streams
Choose which data streams you want to sync:
* **Messages Usage**: Token usage report (grouped by model, workspace, API key, service tier, and context window).
* **Cost Report**: Cost breakdown report (grouped by workspace and cost-item description).
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You can add a **prefix** to all tables at once.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: only new/updated buckets since the last successful run are extracted.
* Full table: every run fetches the full current state of the selected stream.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated.
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Claude and their corresponding fields:
Token usage report grouped by model, workspace, API key, service tier, and context window.
| Field | Type | Description |
| :- | :- | :- |
| `starting_at` | DateTime | Start of the time bucket. |
| `ending_at` | DateTime | End of the time bucket. |
| `model` | String | Model used. |
| `workspace_id` | String | Workspace ID. |
| `api_key_id` | String | API key ID. |
| `service_tier` | String | Service tier (for example, standard, batch, priority). |
| `context_window` | String | Context window bucket (for example, `0-200k` or `200k-1M`). |
| `uncached_input_tokens` | Integer | Uncached input tokens. |
| `cache_read_input_tokens` | Integer | Tokens read from cache. |
| `cache_creation_ephemeral_5m_input_tokens` | Integer | Tokens for 5-minute cache creation. |
| `cache_creation_ephemeral_1h_input_tokens` | Integer | Tokens for 1-hour cache creation. |
| `output_tokens` | Integer | Output tokens generated. |
| `web_search_requests` | Integer | Number of web search requests. |
Cost breakdown grouped by workspace and cost-item description (which includes model and token type).
| Field | Type | Description |
| :- | :- | :- |
| `starting_at` | DateTime | Start of the time bucket. |
| `ending_at` | DateTime | End of the time bucket. |
| `workspace_id` | String | Workspace ID. |
| `description` | String | Cost item description. |
| `amount` | String | Cost amount in cents (USD). |
| `currency` | String | Currency code (USD). |
| `cost_type` | String | Cost type (tokens, web\_search, or code\_execution). |
| `model` | String | Model name. |
| `token_type` | String | Token type (uncached\_input\_tokens, output\_tokens, etc.). |
| `service_tier` | String | Service tier (standard or batch). |
| `context_window` | String | Context window bucket (for example, `0-200k` or `200k-1M`). |
| `inference_geo` | String | Inference geography. |
The connector uses daily time buckets (`bucket_width = 1d`) and uses `starting_at` as the replication key for incremental sync.
## Use Cases for Data Analysis
This guide outlines a couple of common business intelligence use cases when consolidating Claude usage and cost data in your Catalog.
### 1. Token usage by model
Analyze how input/output tokens and web search activity vary by model over time.
```sql theme={null}
SELECT
model,
context_window,
SUM(uncached_input_tokens) AS uncached_input_tokens,
SUM(cache_read_input_tokens) AS cache_read_input_tokens,
SUM(cache_creation_ephemeral_5m_input_tokens) AS cache_creation_ephemeral_5m_input_tokens,
SUM(cache_creation_ephemeral_1h_input_tokens) AS cache_creation_ephemeral_1h_input_tokens,
SUM(output_tokens) AS output_tokens,
SUM(web_search_requests) AS web_search_requests
FROM
nekt_raw.claude_messages_usage
WHERE
starting_at >= current_timestamp - interval '30' day
GROUP BY
model,
context_window
ORDER BY
output_tokens DESC
```
```sql theme={null}
SELECT
model,
context_window,
SUM(uncached_input_tokens) AS uncached_input_tokens,
SUM(cache_read_input_tokens) AS cache_read_input_tokens,
SUM(cache_creation_ephemeral_5m_input_tokens) AS cache_creation_ephemeral_5m_input_tokens,
SUM(cache_creation_ephemeral_1h_input_tokens) AS cache_creation_ephemeral_1h_input_tokens,
SUM(output_tokens) AS output_tokens,
SUM(web_search_requests) AS web_search_requests
FROM
`nekt_raw.claude_messages_usage`
WHERE
starting_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY
model,
context_window
ORDER BY
output_tokens DESC
```
| model | context\_window | uncached\_input\_tokens | cache\_read\_input\_tokens | output\_tokens | web\_search\_requests |
| - | -: | -: | -: | -: | -: |
| claude-3-5-sonnet | 0-200k | 12,450,000 | 3,210,000 | 8,930,000 | 42,810 |
| claude-3-opus | 200k-1M | 7,110,000 | 1,905,000 | 4,620,000 | 18,120 |
### 2. Cost breakdown by token type
Understand which token categories dominate spend.
```sql theme={null}
SELECT
cost_type,
model,
token_type,
SUM(CAST(amount AS DOUBLE)) / 100.0 AS cost_usd
FROM
nekt_raw.claude_cost_report
WHERE
starting_at >= current_timestamp - interval '30' day
AND cost_type = 'tokens'
GROUP BY
cost_type,
model,
token_type
ORDER BY
cost_usd DESC
```
```sql theme={null}
SELECT
cost_type,
model,
token_type,
SUM(CAST(amount AS FLOAT64)) / 100.0 AS cost_usd
FROM
`nekt_raw.claude_cost_report`
WHERE
starting_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
AND cost_type = 'tokens'
GROUP BY
cost_type,
model,
token_type
ORDER BY
cost_usd DESC
```
| cost\_type | model | token\_type | cost\_usd |
| - | - | - | -: |
| tokens | claude-3-5-sonnet | output\_tokens | 124,531.20 |
| tokens | claude-3-5-sonnet | uncached\_input\_tokens | 88,102.77 |
| tokens | claude-3-opus | output\_tokens | 41,009.65 |
## Implementation Notes
The connector authenticates against the Anthropic Admin API using `admin_api_key` as the `x-api-key` request header.
Reports are fetched using daily buckets (`bucket_width = 1d`) and paginated through the Admin API using a `page` cursor when the response indicates more results.
Anthropic Admin API calls are rate-limited conservatively (shared limit of 1 call per 4 seconds) and retried using exponential backoff (up to 12 retries). The connector automatically handles HTTP 429 rate limit responses by respecting the `Retry-After` header when provided by the API, helping prevent transient failures during large backfills.
`cost_report.amount` is returned as a string representing cents (USD). Divide by `100` to get the value in USD.
## Skills for agents
Claude connector documentation as plain markdown, for use in AI agent contexts.
# CleverTap as a data source
Source: https://docs.nekt.com/sources/clevertap
Bring data from CleverTap to Nekt.
CleverTap is a customer engagement and retention platform that helps businesses analyze user behavior and deliver personalized experiences. It provides tools for segmentation, messaging, and analytics to drive user growth and retention.
![]()
## Configuring CleverTap as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **CleverTap** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **Account ID**: The account ID associated with your CleverTap project to authenticate against the API service. Check [CleverTap's documentation](https://developer.clevertap.com/docs/api-quickstart-guide#get-clevertap-account-credentials-to-authenticate-api-requests) to discover where you can find this parameter.
* **Passcode**: The passcode associated with your CleverTap project to authenticate against the API service.
* **Initial sync date**: Starting point for data extraction. Only records created or updated after this date will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings (if available):
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### 5. Check your new source
Click **Done** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
CleverTap connector documentation as plain markdown, for use in AI agent contexts.
# ClickHouse as a data source
Source: https://docs.nekt.com/sources/clickhouse
Bring data from ClickHouse to Nekt.
ClickHouse is a high-performance, column-oriented SQL database management system designed for online analytical processing (OLAP). It's known for its blazing-fast query performance on large datasets, making it ideal for real-time analytics, log processing, and business intelligence applications.
![]()
## Configuring ClickHouse as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the ClickHouse option from the list of connectors.
Click **Next** and you'll be prompted to add your database access.
### 1. Add database access
You'll need the following connection details to connect to your ClickHouse database:
* **Host**: The host address of your ClickHouse database. Do not include the protocol (http\:// or https\://).
* For ClickHouse Cloud: `your-instance.region.provider.clickhouse.cloud`
* For self-hosted: Your server hostname or IP address
* **Port**: The port used for connecting to your ClickHouse database.
* Default for HTTPS: `8443`
* Default for HTTP: `8123`
* Default for Native protocol: `9440` (secure) or `9000` (insecure)
* **Database**: The name of the database you want to extract data from. Default is `default`.
* **Username**: The username for accessing your ClickHouse database. Default is `default`.
* **Password**: The password for the specified user.
* **Batch Size**: The number of rows to fetch per batch during extraction. Default is `50000`. Adjust based on your table row sizes and memory constraints.
If you're using ClickHouse Cloud:
1. Log in to your [ClickHouse Cloud Console](https://clickhouse.cloud/)
2. Select your service from the dashboard
3. Click on **Connect** in the left sidebar
4. Choose **HTTPS** as the connection method
5. Copy the following details:
* **Host**: The hostname shown (e.g., `abc123.us-east1.gcp.clickhouse.cloud`)
* **Port**: Usually `8443` for HTTPS
* **Username**: Your configured username (default is `default`)
* **Password**: The password you set when creating the service
Make sure your ClickHouse Cloud service allows connections from Nekt's IP addresses. You may need to configure the IP Access List in your service settings.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams (tables/views) you want to sync. You can select entire databases or pick specific tables.
The connector will automatically discover all available tables and views in the specified database. System tables are excluded by default.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: Every time the extraction happens, we'll get only the new data since the last sync. This is efficient for large tables with a reliable timestamp or incrementing ID column.
* **Full table**: Every time the extraction happens, we'll get the current state of the data. This is useful for dimension tables or when you need complete accuracy.
For incremental syncs, you'll need to select a **Replication Key**: a column that indicates when a row was created or modified (e.g., `created_at`, `updated_at`, or an auto-incrementing `id`).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently your ClickHouse data is updated and how current you need your analytics to be.
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Implementation Notes
By default, the connector uses secure HTTPS connections to your ClickHouse database. This ensures your data is encrypted in transit.
**ClickHouse Cloud**: Always uses HTTPS on port 8443.
**Self-hosted**: Make sure your ClickHouse server is configured with SSL/TLS certificates for secure connections.
During large extractions, the ClickHouse server may close the connection before the full result set has been sent, typically because of a server-side constraint such as a socket write timeout (`SOCKET_TIMEOUT`, code 209, "Timeout exceeded while writing to socket") or a memory limit. When a proxy sits in front of the database, the aborted query can reach the connector as a seemingly complete but truncated response body.
To protect data integrity, the connector checks that every row carries the number of columns the result set declared, so a cut-off response is detected instead of being silently accepted. When that happens:
* The extraction for that stream fails immediately and the run is stopped. The connector cannot guarantee what the destination did with the rows it already received, so it makes no claims about the destination table's state.
* The replication bookmark does not advance, so the next run resumes from the same point.
* If ClickHouse's own error text (`Code: NNN. DB::Exception ...`) is present in the broken payload, it is captured and shown in the run logs to help you diagnose the server-side cause. Otherwise, check the server's `system.query_log` for the aborted query.
**Tip:** If you frequently run into timeouts or memory limit errors, reduce the **Batch Size** in the source configuration to lower the memory footprint of each request, or raise the relevant timeout on the ClickHouse server.
To prevent connection timeouts during large extractions, the connector reads each stream in windows of key values rather than in a single long-lived query. It slices the data by the stream's **Replication Key** (incremental syncs) or by its first **Primary Key** column (full table syncs), which on ClickHouse is usually the head of the sorting key, so the range filters hit the index.
* **Connection Stability**: Each window is fetched completely and its connection released before any record is handed downstream. Even if the destination is slow, ClickHouse never has an open response waiting on a client that is not reading it, so it cannot cut the connection due to a send timeout.
* **Value Ranges, Not Row Counts**: Windows are ranges of key values, never `LIMIT`/`OFFSET` slices. ClickHouse primary keys are not unique, so a row-count cursor could silently skip rows that share a key value; a value range cannot.
* **Saturated Windows**: If more rows share the *exact same* key value than fit in one window, they cannot be split into smaller ranges and are read in a single step. A warning is written to the run logs, and that part of the extraction temporarily uses more memory than the rest.
* **Incremental streams** are only ever windowed by their replication key. If the replication key is not among the selected columns, the stream is read in a single query.
For full-table streams without a usable primary key (such as views, materialized views, or tables with `ORDER BY tuple()`), the connector still slices the extraction into windows to prevent source timeouts. It chooses a fallback window column from the selected schema based on the following preference:
1. Timestamps (`DateTime`, `Date`)
2. Numbers (`Integer`, `Decimal`)
3. Strings (including `UUID` and `Enum`)
4. Booleans
Float types and complex containers (like Arrays or Maps) are never used for windowing. If a stream contains no comparable columns at all, it falls back to a single continuous query but fetches data in smaller chunks (capped at 2,000 rows per round trip) to ensure the network socket remains actively read and avoids triggering server timeouts.
ClickHouse is optimized for reading large amounts of data quickly. To get the best performance:
* **HTTP Streaming**: With the default HTTP driver, the connector streams query responses lazily instead of loading the whole result set into memory first. Memory usage is bounded to a single batch, which prevents out-of-memory failures when extracting very large tables.
* **Batch Size**: The default batch size of 50,000 rows works well for most use cases and also sets the target size of each extraction window. If you have very wide tables (many columns), consider reducing this value.
* **Incremental Syncs**: Use incremental syncs whenever possible. ClickHouse excels at filtering data by sorted columns (typically date-based).
* **Replication Key Selection**: Choose a column that ClickHouse can efficiently filter on like `updated_at` or a column that identifies a record has been modified.
ClickHouse DateTime columns don't accept ISO format strings with timezone suffixes in comparisons. The connector automatically handles this by converting bookmark values to ClickHouse-compatible format (`YYYY-MM-DD HH:MM:SS`).
This means you don't need to worry about timezone handling: the connector takes care of it automatically.
The following system databases are automatically excluded from discovery:
* `system`
* `INFORMATION_SCHEMA`
* `information_schema`
Only user-created databases and tables will be available for extraction.
ClickHouse data types are mapped to standard types for compatibility:
| ClickHouse Type | Output Type |
| - | - |
| Int8, Int16, Int32, Int64 | Integer |
| UInt8, UInt16, UInt32, UInt64 | Integer |
| Float32, Float64 | Number |
| Decimal | Number |
| String, FixedString | String |
| Date, Date32 | Date |
| DateTime, DateTime64 | DateTime |
| Bool | Boolean |
| UUID | String |
| Array | Array |
| Nullable(T) | Nullable of T |
## Skills for agents
ClickHouse connector documentation as plain markdown, for use in AI agent contexts.
# ClickUp as a data source
Source: https://docs.nekt.com/sources/clickup
Bring data from ClickUp to Nekt.
ClickUp is a cloud-based collaboration and project management tool that offers docs, reminders, goals, calendars, and an inbox. It's designed for teams of all sizes and industries.
![]()
## Configuring ClickUp as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the ClickUp option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your ClickUp Personal Access Token for this connection. Once you have it, add the account access, then click **Next**.
* **Access Token**: Your Personal Access Token to authenticate against the ClickUp API service.
* **Team ID**: The team ID you want to sync with this connector. You can obtain the team ID from your ClickUp URL. For example, in this URL -> `https://app.clickup.com/90132270444` the team ID will be `90132270444`.
* **Start Date**: The earliest record date to sync.
**Advanced Settings:**
* **List IDs** (optional): Specify ClickUp list IDs to sync. When provided, only these lists are fetched directly (bypassing the full space/folder hierarchy), which improves sync performance. If left empty, all lists across all spaces and folders will be fetched.
* **Comments List IDs** (optional): Specify the list IDs from which to extract task comments. Providing list IDs significantly improves sync performance and reduces API calls. If left empty, comments will be fetched from all tasks across all lists, which may be slow for large workspaces.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## API Limitations
The ClickUp API enforces rate limits per token, which is typically 100 requests per minute on most plans. This connector actively monitors your rate limit consumption via the `X-RateLimit-Remaining` and `X-RateLimit-Reset` API headers. When the remaining requests threshold approaches the limit, the connector will proactively pause extraction and wait until the rate limit window resets (up to 120 seconds). This built-in throttling ensures a reliable data extraction process without encountering `429 Too Many Requests` errors.
# Streams and Fields
Below you'll find all available data streams from ClickUp and their corresponding fields:
Stream for fetching space information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The unique identifier for the space. |
| `name` | String | The name of the space. |
| `color` | String | The color code for the space. |
| `private` | Boolean | Whether the space is private. |
| `avatar` | String | URL of the space avatar image. |
| `admin_can_manage` | Boolean | Whether admins can manage the space. |
| `statuses` | Array of Objects | List of status configurations for the space. |
| `multiple_assignees` | Boolean | Whether multiple assignees are allowed. |
| `features` | Object | Feature configurations for the space. |
| `archived` | Boolean | Whether the space is archived. |
Stream for fetching folder information within a specific space.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The unique identifier for the folder. |
| `name` | String | The name of the folder. |
| `orderindex` | Integer | The order index of the folder. |
| `override_statuses` | Boolean | Whether the folder overrides statuses. |
| `hidden` | Boolean | Whether the folder is hidden. |
| `space` | Object | The space this folder belongs to. |
| `task_count` | String | Number of tasks in the folder. |
| `archived` | Boolean | Whether the folder is archived. |
| `statuses` | Array of Objects | List of statuses for the folder. |
| `lists` | Array of Objects | Lists contained in the folder. |
| `permission_level` | String | Permission level for the folder. |
Stream for fetching list information. This includes both lists within folders and folderless lists.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The list ID. |
| `name` | String | Name of the list. |
| `orderindex` | Integer | Order index of the list. |
| `status` | Object | Status of the list (status name and color). |
| `priority` | Object | Priority of the list (color and priority level). |
| `assignee` | Object | Assignee of the list (id, username, color, initials, profilePicture). |
| `task_count` | Integer | Number of tasks in the list. |
| `due_date` | String | Due date of the list. |
| `start_date` | String | Start date of the list. |
| `folder` | Object | The folder this list belongs to. |
| `space` | Object | The space this list belongs to. |
| `archived` | Boolean | Whether the list is archived. |
| `override_statuses` | Boolean | Whether the list overrides statuses. |
| `permission_level` | String | Permission level for the list. |
Stream for fetching task information within a specific list.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The task ID. |
| `list_id` | String | The list ID. |
| `custom_id` | String | Custom ID for the task. |
| `custom_item_id` | Integer | Custom item ID for the task. |
| `name` | String | Name of the task. |
| `text_content` | String | Text content of the task. |
| `description` | String | Description of the task. |
| `status` | Object | Status information for the task. |
| `orderindex` | String | Order index of the task. |
| `date_created` | String | Creation date of the task. |
| `date_updated` | String | Last update date of the task. |
| `date_closed` | String | Closure date of the task. |
| `date_done` | String | Completion date of the task. |
| `archived` | Boolean | Whether the task is archived. |
| `creator` | Object | Creator information for the task. |
| `assignees` | Array of Objects | List of task assignees. |
| `group_assignees` | Array of Objects | List of group assignees. |
| `watchers` | Array of Objects | List of task watchers. |
| `checklists` | Array of Objects | Task checklists. |
| `tags` | Array of Objects | Task tags. |
| `parent` | String | Parent task ID. |
| `top_level_parent` | String | Top level parent task ID. |
| `priority` | Object | Task priority. |
| `due_date` | String | Due date of the task. |
| `start_date` | String | Start date of the task. |
| `points` | Integer | Task points. |
| `time_estimate` | Integer | Estimated time for the task. |
| `time_spent` | Integer | Time spent on the task. |
| `custom_fields` | Array of Objects | Custom fields for the task. |
| `dependencies` | Array of Objects | Task dependencies. |
| `linked_tasks` | Array of Objects | Linked tasks. |
| `locations` | Array of Objects | Task locations. |
| `team_id` | String | Team ID. |
| `url` | String | Task URL. |
| `sharing` | Object | Sharing settings for the task. |
| `permission_level` | String | Permission level for the task. |
| `list` | Object | List information for the task. |
| `project` | Object | Project information for the task. |
| `folder` | Object | Folder information for the task. |
| `space` | Object | Space information for the task. |
Stream for fetching user information from the team.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | The user ID. |
| `username` | String | The member's username. |
| `email` | String | The member's email address. |
| `color` | String | The member's color. |
| `initials` | String | The member's initials. |
| `profilePicture` | String | URL of the member's profile picture. |
| `profileInfo` | Object | Additional profile information for the member. |
Stream for fetching task comments. This is a child stream of Tasks.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The unique identifier for the comment. |
| `task_id` | String | The task ID this comment belongs to. |
| `comment` | Array of Objects | Structured comment content with text and attributes. |
| `comment_text` | String | Plain text content of the comment. |
| `user` | Object | User who made the comment (id, username, email, color, initials, profilePicture). |
| `resolved` | Boolean | Whether the comment has been resolved. |
| `assignee` | Object | User assigned to the comment (id, username, email, color, initials, profilePicture). |
| `group_assignee` | Object | Group assigned to the comment. |
| `assigned_by` | Object | User who assigned the comment (id, username, email, color, initials, profilePicture). |
| `reactions` | Array of Objects | Reactions to the comment. |
| `date` | String | Timestamp when the comment was created. |
| `reply_count` | Integer | Number of replies to the comment (0 if threaded). |
For large workspaces, use the **Comments List IDs** configuration to limit which lists are scanned for comments. This significantly improves sync performance.
Stream for fetching time entry information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The unique ID of the time entry. |
| `task` | Object | The task associated with this time entry. |
| `wid` | String | Workspace or team ID for this time entry. |
| `user` | Object | User information for this time entry. |
| `billable` | Boolean | Whether the time is billable. |
| `start` | String | Start timestamp in milliseconds. |
| `end` | String | End timestamp in milliseconds. |
| `duration` | String | Duration of the time entry. |
| `description` | String | Description of the time entry. |
| `tags` | Array of Objects | Tags assigned to the time entry (name, tag\_fg, tag\_bg, creator). |
| `source` | String | Source of the time entry. |
| `at` | String | Timestamp of time entry creation or update in milliseconds. |
| `is_locked` | Boolean | Whether the entry is locked. |
| `approval_id` | String | Approval ID for the time entry. |
| `task_location` | Object | Location of the task in the ClickUp hierarchy. |
| `task_tags` | Array of Objects | Tags assigned to the associated task (name, tag\_fg, tag\_bg, creator). |
| `task_url` | String | URL to the associated task. |
Stream for fetching the time spent by tasks in each status. This stream uses a bulk API endpoint to efficiently retrieve status duration data for multiple tasks.
| Field | Type | Description |
| :- | :- | :- |
| `task_id` | String | The task ID. |
| `status_last_updated` | Integer | The timestamp when the task status was last updated. |
| `current_status` | Object | Current status information including status name, color, and total time spent. |
| `status_history` | Array of Objects | Historical data of all statuses the task has been in, including status name, color, type, total time spent, and order index. |
The Time Entries stream is only available for users with the "Time Tracking" feature enabled in their ClickUp account. Also, only users with the "Admin" role can access this stream to get other users' time entries.
The **Time Entries** stream supports incremental sync using the `at` field (last update timestamp). The **Comments** stream uses full table sync and fetches comments from all tasks (or filtered by list IDs if configured).
## Skills for agents
ClickUp connector documentation as plain markdown, for use in AI agent contexts.
# Base de Clientes as a data source
Source: https://docs.nekt.com/sources/client-base
Bring data from Base de Clientes to Nekt.
Base de Clientes is a subscription and billing management platform for Brazilian merchants. It supports recurring billing, receivables, transfers, customer management, contracts, and reporting. Data is extracted via the Base de Clientes REST API using a Bearer token.
![]()
## Configuring Base de Clientes as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Base de Clientes** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Base de Clientes API credentials so Nekt can access your data.
The following configurations are available:
* **Auth Token**: (Required) Bearer token for the Base de Clientes API. Obtain it from [suporte@clientbase.com.br](mailto:suporte@clientbase.com.br). This token authenticates all API requests.
* **Start Date**: (Optional) Initial date for incremental sync. Used on the first run when there is no bookmark. Example: `2020-01-01T00:00:00Z`. Streams with incremental sync (billings, banks, recurrences, receivables, transfers, contracts, announcement\_portals) will filter records by `updated_at` greater than this date on the first extraction.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: Every time the extraction happens, we'll get only the new or updated data. Supported for **banks**, **billings**, **contracts**, **receivables**, **recurrences**, **transfers**, and **announcement\_portals** (they use `updated_at` as replication key).
* **Full table**: Every time the extraction happens, we'll get the current state of the data. Used for **bank\_accounts**, **customers**, **customer\_groups**, **custom\_fields**, **merchant\_users**, and **reports**.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Announcement Portals | `announcement_portals` | Announcement portal communications (comunicados) for the authenticated merchant. |
| Bank Accounts | `bank_accounts` | Bank accounts belonging to the authenticated merchant. |
| Banks | `banks` | Reference list of banks available in Base de Clientes. |
| Billings | `billings` | Invoices (faturas) issued by the authenticated merchant. |
| Contracts | `contracts` | Contracts (contratos) of the authenticated merchant. |
| Custom Fields | `custom_fields` | Custom field definitions configured for the merchant. |
| Customer Groups | `customer_groups` | Customer groups (grupos de clientes) of the merchant. |
| Customers | `customers` | Customers (clientes) of the authenticated merchant. |
| Merchant Users | `merchant_users` | Users belonging to the merchant account. |
| Receivables | `receivables` | Receivables (recebíveis) of the merchant. |
| Recurrences | `recurrences` | Recurring billing schedules (recorrências) of the merchant. |
| Reports | `reports` | Reports (relatórios) available to the merchant. |
| Transfers | `transfers` | Transfers (transferências) of the merchant. |
## Fields by stream
Below you'll find all available data streams from Base de Clientes and their corresponding fields:
List of banks available in Base de Clientes (incremental by updated\_at).
**Key Fields:**
* `uuid` - UUID do banco
* `compe` - Código COMPE
* `ispb` - Código ISPB
* `document` - CNPJ do banco
* `short_name` - Nome curto do banco
* `name` - Razão social do banco
* `url` - URL do banco
* `merchant_uuid` - UUID do merchant
* `created_at`, `updated_at` - Datas de criação e atualização
Bank accounts of the authenticated merchant.
**Key Fields:**
* `uuid` - UUID da conta bancária
* `agency`, `agency_dv` - Agência e dígito verificador
* `account`, `account_dv` - Número da conta e dígito
* `account_type` - Tipo da conta (ex: checking)
* `active` - Se a conta está ativa
* `merchant_uuid` - UUID do merchant
* `bank` - Objeto com dados do banco (uuid, compe, ispb, name, etc.)
* `created_at`, `updated_at` - Datas
Invoices of the merchant (incremental by updated\_at).
**Key Fields:**
* `uuid`, `merchant_uuid`, `customer_uuid` - Identificadores
* `amount_billed`, `amount_paid`, `discount_amount`, `interest_fine`, `interest_fee` - Valores (string)
* `status` - Status da fatura (ex: paid)
* `due_date`, `date_paid`, `expiration_date`, `competence_date`, `issue_date` - Datas
* `payment_type`, `discount_policy`, `interest_policy` - Políticas
* `recurrence_cycle`, `installment_cycle` - Ciclos
* `dirty`, `antecipated`, `issued`, `viewed`, `can_issue` - Flags
* `discounts` - Array de descontos (uuid, type, value, amount, policy, days, etc.)
* `tags` - Array de tags (objetos com uuid, name, color, etc.)
* `billing_items` - Array de itens (uuid, description, quantity, unit\_price, amount, product, etc.)
* `payments` - Array de pagamentos
* `customer`, `merchant` - Objetos aninhados
* `cancellation_reason`, `metadata`, `nfse` - Objetos adicionais
Contracts of the merchant (incremental by updated\_at).
**Key Fields:**
* `id`, `uuid` - Identificadores
* `title`, `description`, `message`, `content` - Textos
* `start_date`, `end_date`, `expiration_date` - Datas
* `days_notice` - Dias de aviso prévio
* `signed_at` - Data de assinatura
* `status` - Status (ex: current, waiting\_signatures)
* `merchant_id`, `customer_id`, `contract_model_id` - IDs de relacionamento
* `customer` - Objeto com dados do cliente
* `created_at`, `updated_at` - Datas
Customers of the merchant.
**Key Fields:**
* `uuid`, `merchant_uuid`, `customer_uuid` - Identificadores
* `name`, `nickname`, `document`, `email`, `phone` - Dados de contato
* `status`, `source` - Status e origem
* `email_block`, `category`, `category_dirty` - Configurações
* `metadata`, `custom_fields`, `customer_fields` - JSON strings
* `documents`, `tags`, `contracts`, `customer_info`, `archive_reasons` - JSON strings
* `addresses` - Array de endereços (uuid, street, number, city, state, postal\_code, etc.)
* `customer_groups` - Array de grupos (uuid, name, group\_color, active, icon, etc.)
* `created_at`, `updated_at` - Datas
Customer groups of the merchant.
**Key Fields:**
* `id`, `uuid` - Identificadores
* `name`, `active`, `icon`, `group_color` - Dados do grupo
* `merchant_uuid` - UUID do merchant
* `customers` - Array de clientes do grupo
* `created_at`, `updated_at` - Datas
Custom fields configuration of the merchant.
**Key Fields:**
* `uuid` - UUID do custom field
* `field_type` - Tipo do campo (ex: short\_text, radio)
* `field_configuration` - Configuração do campo (JSON string)
Users of the merchant.
**Key Fields:**
* `uuid`, `merchant_uuid` - Identificadores
* `name`, `email`, `phone` - Dados de contato
* `status`, `role` - Status e papel (ex: admin)
* `google_access_token`, `google_sub` - Integração Google
* `shortcuts`, `notification_preferences` - JSON strings
* `show_nps`, `show_notification_product_updates` - Preferências
* `virtual_assistant_notification_enabled`, `virtual_assistant_phone` - Assistente virtual
* `created_at`, `updated_at` - Datas
Receivables (recebíveis) of the merchant (incremental by updated\_at).
**Key Fields:**
* `uuid`, `billing_uuid`, `payment_uuid`, `transfer_uuid`, `customer_uuid`, `recurrence_uuid`, `merchant_uuid` - Identificadores
* `amount`, `gross_amount` - Valores (string)
* `status` - Status (ex: paid)
* `date_paid`, `expected_on` - Datas
* `installment` - Dados da parcela (JSON string)
* `billing` - Objeto com fatura associada (inclui customer, payments)
* `created_at`, `updated_at` - Datas
Recurring billing definitions (incremental by updated\_at).
**Key Fields:**
* `uuid`, `customer_uuid`, `merchant_uuid` - Identificadores
* `status`, `frequency`, `frequency_type`, `frequency_number` - Configuração
* `total_cycles`, `current_cycle`, `due_day` - Ciclos
* `first_due_date` - Primeira data de vencimento
* `amount`, `discount_amount`, `interest_fine`, `interest_fee` - Valores (string)
* `payment_type`, `discount_policy`, `interest_policy` - Políticas
* `credit_card_recurring`, `nfse_scheduled` - Flags
* `discounts`, `tags`, `billing_items`, `billings`, `recurrence_cards` - Arrays (JSON strings)
* `bonus`, `installment_plan`, `cancellation_reason` - Objetos/strings
* `customer` - Objeto com dados do cliente
* `created_at`, `updated_at` - Datas
Reports generated by the merchant.
**Key Fields:**
* `uuid`, `merchant_uuid` - Identificadores
* `model` - Modelo do relatório (ex: billing)
* `filters`, `filters_to_show` - Filtros (JSON strings)
* `status` - Status (ex: completed, generated)
* `downloaded` - Se foi baixado
* `created_at`, `updated_at` - Datas
Bank transfers of the merchant (incremental by updated\_at).
**Key Fields:**
* `uuid`, `bank_account_uuid`, `merchant_uuid` - Identificadores
* `amount`, `child_amount` - Valores (string)
* `description`, `status` - Descrição e status
* `transfer_date` - Data da transferência
* `transfer_type`, `transferable_type` - Tipos
* `transfer_position` - Posição da transferência
* `report_url` - URL do relatório
* `created_at`, `updated_at` - Datas
Announcements from the portal (comunicados) (incremental by updated\_at).
**Key Fields:**
* `uuid`, `merchant_uuid` - Identificadores
* `announcement_type`, `title`, `message` - Conteúdo
* `document_file` - Arquivo anexo (JSON string)
* `customers` - Array de clientes que visualizaram (JSON strings)
* `customer_groups` - Grupos de clientes (JSON strings)
* `created_at`, `updated_at` - Datas
# Data Model
The following diagram illustrates the main relationships between Base de Clientes streams. **Merchant** is the central entity; **customers** and **merchant\_users** belong to it. **Billings** and **recurrences** link to customers. **Receivables** tie billings to **transfers**. **Bank accounts** and **banks** support transfers. **Contracts** associate customers with contract models. **Customer groups** group customers. **Reports** and **announcement\_portals** are merchant-scoped.
```mermaid theme={null}
erDiagram
MerchantUsers ||--o{ Merchants : "belongs to"
Customers ||--o{ Merchants : "belongs to"
Billings }o--|| Customers : "customer_uuid"
Recurrences }o--|| Customers : "customer_uuid"
Receivables }o--|| Billings : "billing_uuid"
Receivables }o--o| Transfers : "transfer_uuid"
Transfers }o--|| BankAccounts : "bank_account_uuid"
BankAccounts }o--|| Banks : "bank"
Contracts }o--|| Customers : "customer_id"
CustomerGroups ||--o{ Customers : "customers"
BankAccounts }o--|| Banks : "bank"
Billings {
string uuid PK
string customer_uuid FK
string amount_billed
string status
datetime updated_at
}
Customers {
string uuid PK
string merchant_uuid FK
string name
string document
string status
}
Receivables {
string uuid PK
string billing_uuid FK
string transfer_uuid FK
string amount
string status
}
Recurrences {
string uuid PK
string customer_uuid FK
string amount
string status
datetime updated_at
}
Transfers {
string uuid PK
string bank_account_uuid FK
string amount
string status
}
```
# Implementation Notes
### Authentication
* The tap uses **Bearer token** authentication. The `auth_token` is sent in the `Authorization` header for all API requests.
### Incremental Sync
* Streams with `replication_key = "updated_at"` support incremental sync. On the first run (no bookmark), the tap uses `start_date` from config to filter records. On subsequent runs, the bookmark from state is used.
### Numeric Fields as Strings
* Several amount and value fields (`amount_billed`, `amount_paid`, `amount`, etc.) are stored as strings in the schema because the API can return them as strings or numbers. Use `CAST(column AS DOUBLE)` or equivalent in SQL for aggregations.
### JSON String Fields
* Fields like `metadata`, `custom_fields`, `tags`, `billing_items`, `product`, `paymentable`, and similar are serialized as JSON strings for compatibility with the data lake. Use `JSON_EXTRACT` or equivalent to parse them in queries.
## Skills for agents
Base de Clientes connector documentation as plain markdown, for use in AI agent contexts.
# Clínica nas Nuvens as a data source
Source: https://docs.nekt.com/sources/clinica-nas-nuvens
Bring data from Clínica nas Nuvens to Nekt.
Clínica nas Nuvens is a Brazilian cloud-based clinic management software designed for healthcare professionals. It provides tools for managing appointments, patients, procedures, and billing, helping clinics streamline their operations and improve patient care.
![]()
## Configuring Clínica nas Nuvens as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Clínica nas Nuvens option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your API credentials to access your Clínica nas Nuvens data. These credentials can be found in your Clínica nas Nuvens account under **Menu > Integrações > API**.
The following configurations are available:
* **Client ID**: The client ID to authenticate against the Clínica nas Nuvens API.
* **Client Secret**: The client secret to authenticate against the Clínica nas Nuvens API.
* **Token**: The token to authenticate against the Clínica nas Nuvens API.
* **Start Date**: The earliest record date to sync. If left empty, the full history is synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: FULL\_TABLE sync is used for all streams.
**Incremental sync is not supported.** The Clínica nas Nuvens API does not provide reliable filters by modification date, so all streams use full table sync. Each extraction will fetch the complete current state of the data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Clínica nas Nuvens and their corresponding fields:
Stream containing scheduled appointments and their details.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the appointment |
| `data` | String | Date of the scheduled appointment |
| `horaInicio` | String | Start time of the appointment |
| `horaFim` | String | End time of the appointment |
| `status` | String | Appointment status |
| `idPaciente` | Integer | ID of the patient |
| `emailPaciente` | String | Patient's email address |
| `telefoneCelularPaciente` | String | Patient's mobile phone number |
| `idOrigemPaciente` | Integer | ID of the patient's origin/source |
| `idLocalAgenda` | Integer | ID of the schedule location |
| `idPessoaExecutor` | Integer | ID of the person performing the appointment |
| `idTipoConsulta` | Integer | ID of the consultation type |
| `idTipoConvenio` | Integer | ID of the agreement/insurance type |
| `idRotulo` | Integer | ID of the label |
| `encaminhamento` | String | Referral information |
| `observacoes` | String | Observations about the appointment |
| `urlSalaEspera` | String | URL of the virtual waiting room |
| `procedimentos` | Array | Array of procedures for the appointment |
| `procedimentos[].id` | Integer | Procedure ID |
| `procedimentos[].nome` | String | Procedure name |
| `procedimentos[].idEspecialidade` | Integer | Specialty ID |
| `procedimentos[].idTipoProcedimento` | Integer | Procedure type ID |
| `procedimentos[].idPromocao` | Integer | Promotion ID |
| `procedimentos[].quantidade` | Integer | Quantity |
Stream containing patient records and their personal information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the patient |
| `nome` | String | Patient's full name |
| `nomeSocial` | String | Patient's social name |
| `ativo` | Boolean | Indicates whether the patient is active |
| `cpfcnpj` | String | Patient's CPF or CNPJ document number |
| `dataNascimento` | String | Patient's date of birth |
| `sexo` | String | Patient's biological sex |
| `genero` | String | Patient's gender |
| `idNacionalidade` | Integer | ID of the patient's nationality |
| `estrangeiro` | Boolean | Indicates if the patient is a foreigner |
| `numeroControle` | String | Patient's control number |
| `numeroIdentificacao` | String | Patient's identification number |
| `idOrigem` | Integer | Origin/source identifier |
| `contato.email` | String | Patient's email address |
| `contato.telefoneCelular` | String | Mobile phone number |
| `contato.telefoneResidencial` | String | Residential phone number |
| `contato.telefoneComercial` | String | Commercial phone number |
| `contato.telefoneRecados` | String | Message phone number |
| `contato.skype` | String | Skype username |
| `contato.ramalTelefoneResidencial` | String | Residential phone extension |
| `contato.ramalTelefoneComercial` | String | Commercial phone extension |
| `contato.ramalTelefoneRecados` | String | Message phone extension |
| `endereco.rua` | String | Street name |
| `endereco.numero` | String | Address number |
| `endereco.complemento` | String | Address complement |
| `endereco.bairro` | String | Neighborhood |
| `endereco.cep` | String | ZIP/postal code |
| `endereco.idCidade` | Integer | City ID (joins with Cidades stream) |
Lookup table containing city information with state details.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the city |
| `nome` | String | City name |
| `codIbge` | String | IBGE code (Brazilian official city code) |
| `estado.id` | Integer | State identifier |
| `estado.nome` | String | State name |
| `estado.sigla` | String | State abbreviation (e.g., SP, RJ, MG) |
Lookup table containing medical specialties.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the specialty |
| `nome` | String | Specialty name (e.g., Cardiologia, Dermatologia) |
Lookup table containing nationalities.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the nationality |
| `nacionalidade` | String | Nationality name (e.g., Brasileira, Portuguesa) |
Lookup table containing patient origin/source information. Tracks how patients discovered or were referred to the clinic.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the origin |
| `nome` | String | Origin name (e.g., "Indicação", "Google", "Convênio") |
| `ativo` | Boolean | Indicates if the origin is active |
Lookup table containing schedule/appointment origin information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the schedule origin |
| `nome` | String | Schedule origin name |
| `ativo` | Boolean | Indicates if the record is active |
Lookup table containing agreement/insurance types (convênios).
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the agreement type |
| `nome` | String | Agreement type name |
| `ativo` | Boolean | Indicates if the agreement type is active |
| `particular` | Boolean | Indicates if it's a private/self-pay option |
| `beneficio` | Boolean | Indicates benefit status |
Lookup table containing procedure types with associated specialties.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the procedure type |
| `nome` | String | Procedure type name |
| `ativo` | Boolean | Indicates if the procedure type is active |
| `especialidades` | Array | Array of specialties that can perform this procedure |
| `especialidades[].id` | Integer | Specialty ID |
| `especialidades[].nome` | String | Specialty name |
# Data Model
The following diagram illustrates the relationships between the core data streams in Clínica nas Nuvens. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
erDiagram
Agenda }o--|| Pacientes : "idPaciente"
Agenda }o--|| TipoConvenio : "idTipoConvenio"
Agenda }o--|| Origem : "idOrigemPaciente"
Agenda }o--|| Especialidades : "procedimentos.idEspecialidade"
Agenda }o--|| TipoProcedimento : "procedimentos.idTipoProcedimento"
Pacientes }o--|| Cidades : "endereco.idCidade"
Pacientes }o--|| Nacionalidades : "idNacionalidade"
Pacientes }o--|| Origem : "idOrigem"
TipoProcedimento }o--|| Especialidades : "especialidades.id"
Agenda {
int id PK
string data
string horaInicio
string horaFim
string status
int idPaciente FK
int idTipoConvenio FK
int idOrigemPaciente FK
int idLocalAgenda
int idPessoaExecutor
int idTipoConsulta
int idRotulo
string emailPaciente
string telefoneCelularPaciente
string observacoes
string encaminhamento
string urlSalaEspera
array procedimentos
}
Pacientes {
int id PK
string nome
string nomeSocial
string cpfcnpj
string dataNascimento
string sexo
string genero
boolean ativo
boolean estrangeiro
int idNacionalidade FK
int idOrigem FK
object contato
object endereco
}
Cidades {
int id PK
string nome
string codIbge
object estado
}
Especialidades {
int id PK
string nome
}
Nacionalidades {
int id PK
string nacionalidade
}
Origem {
int id PK
string nome
boolean ativo
}
TipoConvenio {
int id PK
string nome
boolean ativo
boolean particular
boolean beneficio
}
TipoProcedimento {
int id PK
string nome
boolean ativo
array especialidades
}
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Clínica nas Nuvens data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Appointment Volume Analysis
Track appointment volumes over time to understand clinic capacity and identify trends.
**Business Value:**
* Understand daily and weekly appointment patterns
* Identify peak hours and days for resource planning
* Monitor appointment status distribution
```sql theme={null}
WITH appointment_metrics AS (
SELECT
DATE(a.data) AS appointment_date,
DATE_FORMAT(DATE(a.data), '%W') AS day_of_week,
a.status,
COUNT(*) AS total_appointments,
COUNT(DISTINCT a.idPaciente) AS unique_patients
FROM
nekt_raw.clinica_nas_nuvens_agenda a
WHERE
DATE(a.data) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
DATE(a.data),
DATE_FORMAT(DATE(a.data), '%W'),
a.status
)
SELECT
appointment_date,
day_of_week,
status,
total_appointments,
unique_patients
FROM
appointment_metrics
ORDER BY
appointment_date DESC,
total_appointments DESC
```
```sql theme={null}
WITH appointment_metrics AS (
SELECT
DATE(a.data) AS appointment_date,
FORMAT_DATE('%A', DATE(a.data)) AS day_of_week,
a.status,
COUNT(*) AS total_appointments,
COUNT(DISTINCT a.idPaciente) AS unique_patients
FROM
`nekt_raw.clinica_nas_nuvens_agenda` a
WHERE
DATE(a.data) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
1, 2, 3
)
SELECT
appointment_date,
day_of_week,
status,
total_appointments,
unique_patients
FROM
appointment_metrics
ORDER BY
appointment_date DESC,
total_appointments DESC
```
| appointment\_date | day\_of\_week | status | total\_appointments | unique\_patients |
| - | - | - | - | - |
| 2024-11-27 | Wednesday | Confirmado | 45 | 42 |
| 2024-11-27 | Wednesday | Aguardando | 12 | 12 |
| 2024-11-27 | Wednesday | Cancelado | 3 | 3 |
| 2024-11-26 | Tuesday | Confirmado | 52 | 48 |
| 2024-11-26 | Tuesday | Aguardando | 8 | 8 |
### 2. Patient Origin Analysis
Understand where your patients are coming from to optimize marketing efforts.
**Business Value:**
* Identify most effective patient acquisition channels
* Measure ROI of marketing initiatives
* Optimize patient acquisition strategy
```sql theme={null}
SELECT
o.nome AS origem,
COUNT(DISTINCT p.id) AS total_patients,
COUNT(DISTINCT CASE WHEN p.ativo = true THEN p.id END) AS active_patients,
ROUND(
COUNT(DISTINCT CASE WHEN p.ativo = true THEN p.id END) * 100.0 /
NULLIF(COUNT(DISTINCT p.id), 0),
1
) AS active_rate_pct
FROM
nekt_raw.clinica_nas_nuvens_pacientes p
LEFT JOIN nekt_raw.clinica_nas_nuvens_origem o ON p.idOrigem = o.id
GROUP BY
o.nome
ORDER BY
total_patients DESC
```
```sql theme={null}
SELECT
o.nome AS origem,
COUNT(DISTINCT p.id) AS total_patients,
COUNT(DISTINCT CASE WHEN p.ativo = true THEN p.id END) AS active_patients,
ROUND(
SAFE_DIVIDE(
COUNT(DISTINCT CASE WHEN p.ativo = true THEN p.id END) * 100.0,
COUNT(DISTINCT p.id)
),
1
) AS active_rate_pct
FROM
`nekt_raw.clinica_nas_nuvens_pacientes` p
LEFT JOIN `nekt_raw.clinica_nas_nuvens_origem` o ON p.idOrigem = o.id
GROUP BY
o.nome
ORDER BY
total_patients DESC
```
| origem | total\_patients | active\_patients | active\_rate\_pct |
| - | - | - | - |
| Indicação | 1,245 | 1,102 | 88.5 |
| Google | 856 | 723 | 84.5 |
| Convênio | 634 | 589 | 92.9 |
| Instagram | 423 | 367 | 86.8 |
| Site | 312 | 256 | 82.1 |
### 3. Procedure and Specialty Analysis
Analyze which procedures and specialties are most in demand.
**Business Value:**
* Identify most popular procedures
* Understand specialty distribution
* Plan resource allocation based on demand
```sql theme={null}
WITH procedure_data AS (
SELECT
p.id AS procedure_id,
p.nome AS procedure_name,
p.idEspecialidade AS specialty_id,
p.quantidade AS quantity,
a.id AS appointment_id,
a.data AS appointment_date
FROM
nekt_raw.clinica_nas_nuvens_agenda a
CROSS JOIN UNNEST(a.procedimentos) AS t(p)
WHERE
DATE(a.data) >= CURRENT_DATE - INTERVAL '30' DAY
)
SELECT
e.nome AS specialty,
tp.nome AS procedure_type,
COUNT(DISTINCT pd.appointment_id) AS total_appointments,
SUM(pd.quantity) AS total_procedures
FROM
procedure_data pd
LEFT JOIN nekt_raw.clinica_nas_nuvens_especialidades e ON pd.specialty_id = e.id
LEFT JOIN nekt_raw.clinica_nas_nuvens_tipo_procedimento tp ON pd.procedure_id = tp.id
GROUP BY
e.nome,
tp.nome
ORDER BY
total_appointments DESC
```
```sql theme={null}
WITH procedure_data AS (
SELECT
p.id AS procedure_id,
p.nome AS procedure_name,
p.idEspecialidade AS specialty_id,
p.quantidade AS quantity,
a.id AS appointment_id,
a.data AS appointment_date
FROM
`nekt_raw.clinica_nas_nuvens_agenda` a,
UNNEST(a.procedimentos) AS p
WHERE
DATE(a.data) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
)
SELECT
e.nome AS specialty,
tp.nome AS procedure_type,
COUNT(DISTINCT pd.appointment_id) AS total_appointments,
SUM(pd.quantity) AS total_procedures
FROM
procedure_data pd
LEFT JOIN `nekt_raw.clinica_nas_nuvens_especialidades` e ON pd.specialty_id = e.id
LEFT JOIN `nekt_raw.clinica_nas_nuvens_tipo_procedimento` tp ON pd.procedure_id = tp.id
GROUP BY
e.nome,
tp.nome
ORDER BY
total_appointments DESC
```
| specialty | procedure\_type | total\_appointments | total\_procedures |
| - | - | - | - |
| Dermatologia | Consulta | 234 | 234 |
| Cardiologia | Consulta | 189 | 189 |
| Dermatologia | Procedimento Estético | 156 | 312 |
| Oftalmologia | Consulta | 145 | 145 |
| Pediatria | Consulta | 123 | 123 |
## Implementation Notes
### Data Quality Considerations
* The `Agenda` stream uses the `start_date` configuration to filter appointments by date.
* Patient records may have null values for optional contact and address fields.
* Use the lookup tables (Cidades, Especialidades, etc.) to enrich your analyses with descriptive names.
* To ensure data consistency and prevent pagination drift during long-running extractions, the end date (`dataFinal`) used in the API request filter is cached and locked to the exact date the extraction begins.
### Best Practices
* Always join with lookup tables to get descriptive names instead of just IDs.
* Consider patient privacy when analyzing personal data.
## Skills for agents
Clínica nas Nuvens connector documentation as plain markdown, for use in AI agent contexts.
# Clint as a data source
Source: https://docs.nekt.com/sources/clint
Bring data from Clint to Nekt.
Clint is a CRM and sales platform that helps teams manage contacts, deals, and groups. The connector extracts data from the Clint API (api.clint.digital) so you can centralize your commercial pipeline and contact data in your Lakehouse.
![]()
## Configuring Clint as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Clint option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Clint API token so Nekt can access your data. The token is sent in the `api-token` header for all API requests.
The following configurations are available:
* **API Token**: Your Clint API token. This is required and is sent in the `api-token` header. Obtain it from your Clint account or API settings.
* **Start Date**: (Optional) For streams that support incremental sync, only records past this date are synced on the first run when no bookmark exists: Deals and Messages filter by `updated_at`, Chats by `last_message_at`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched. The **Deals**, **Deal History**, **Chats**, and **Messages** streams support incremental sync.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog. **Contacts**, **Groups**, **Origins**, **Activities**, and **Channel Accounts** are typically synced as full table.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Clint and their corresponding fields:
Stream for Clint activities (tasks, calls, meetings, etc.). Uses full table sync. Requires the `activities:read` scope and the Activities API feature flag on your Clint account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Activity UUID |
| `title` | String | Activity title |
| `type` | String | Activity type (CALL, MAIL, SCHEDULE, TASK, MEETING, WHATSAPP, INSTAGRAM) |
| `content` | String | Activity description content (may contain HTML) |
| `template_id` | String | UUID of the activity template that generated this activity, when any |
| `completed` | Boolean | Whether the activity was completed |
| `completed_at` | DateTime | Timestamp when the activity was completed |
| `completed_by` | String | UUID of the user who completed the activity |
| `due_at` | DateTime | Due date of the activity |
| `custom` | Boolean | Whether the activity was created manually (not from a stage template) |
| `no_show` | Boolean | Whether the meeting was marked as no-show |
| `stage_id` | String | UUID of the pipeline stage the activity belongs to |
| `created_at` | DateTime | Activity creation timestamp |
| `updated_at` | DateTime | Last update timestamp |
| `deal` | Object | Deal the activity belongs to (`id`, `status`, `origin_id`, `stage_id`, `contact`) |
Stream for Clint channel accounts (WhatsApp/Instagram inboxes). Uses full table sync. Parent stream of Chats.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Channel account UUID |
| `created_at` | DateTime | Channel account creation timestamp |
| `name` | String | Display name of the channel account |
| `type` | String | Channel type (WHATSAPP\_OFFICIAL, WHATSAPP, INSTAGRAM) |
| `status` | String | Connection status (CONNECTED, DISCONNECTED, CANCELLED) |
| `avatar` | String | Avatar image URL |
| `identifier` | String | Phone number or channel identifier |
| `team_id` | String | UUID of the team that owns the channel account |
| `team` | Object | Team that owns the channel account (`id`, `name`) |
Stream for chats belonging to each channel account. Supports incremental sync via `last_message_at`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Chat UUID |
| `created_at` | DateTime | Chat creation timestamp |
| `contact_id` | String | UUID of the contact in the conversation |
| `user_id` | String | UUID of the user assigned to the chat |
| `status` | String | Chat status (OPEN, CLOSED, WAITING, SNOOZED, REOPENED) |
| `seen` | Boolean | Whether the chat has been seen |
| `unread` | Boolean | Whether the chat has unread messages |
| `replied` | Boolean | Whether the chat has been replied to |
| `unseen_count` | Integer | Number of unseen messages in the chat |
| `last_message_at` | DateTime | Timestamp of the last message; used for incremental sync |
| `last_response_at` | DateTime | Timestamp of the last response |
| `closed_at` | DateTime | Timestamp when the chat was closed |
| `last_status_at` | DateTime | Timestamp of the last status change |
| `first_response_at` | DateTime | Timestamp of the first response |
| `first_customer_message_at` | DateTime | Timestamp of the first message sent by the customer |
| `channel_account_id` | String | UUID of the parent channel account |
| `team_id` | String | UUID of the team handling the chat |
| `close_window_at` | DateTime | Timestamp when the messaging window closes |
Stream for Clint contacts (paginated list). Uses full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Contact UUID |
| `name` | String | Contact name |
| `email` | String | Contact e-mail |
| `username` | String | Username |
| `ddi` | String | International dialing code (DDI) |
| `phone` | String | Phone number without DDI |
| `fullPhone` | String | Full phone with DDI (country code) |
| `organization.id` | String | Organization ID (null if the contact is not linked to an organization) |
| `organization.name` | String | Organization name (null if not linked) |
| `organization.fields` | String | Custom organization fields (JSON string); null if not linked |
| `created_at` | DateTime | Created at |
| `updated_at` | DateTime | Last updated at |
| `tags` | Array | Array of contact tags, each with `id`, `name`, and `color` |
| `fields` | String | Custom contact fields, stored as a JSON object string |
Timeline events of each deal (the CRM panel's History tab). Child stream of Deals. Supports incremental sync via `date`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | History event UUID |
| `deal_id` | String | UUID of the deal the event belongs to |
| `category` | String | Event type classification (e.g. DEAL\_ARCHIVED, DEAL\_UNARCHIVED) |
| `date` | DateTime | Timestamp when the event occurred; used as replication key for incremental sync |
| `system` | Boolean | Whether the event was generated by the system |
| `user` | Object | User who performed the action (null for system events), with `id` and `name` |
| `body` | String | Event payload; shape varies by category, JSON object string |
| `summary` | String | Human-readable event summary in Portuguese, when available |
Stream for Clint deals. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Deal UUID |
| `origin_id` | String | Origin UUID |
| `stage_id` | String | Stage UUID |
| `stage` | String | Deal stage name |
| `status` | String | Deal status (OPEN, WON, LOST) |
| `value` | Number | Deal monetary value |
| `created_at` | DateTime | Created at |
| `updated_at` | DateTime | Last updated at (any change); used for incremental sync |
| `updated_stage_at` | DateTime | Last stage update at |
| `won_at` | DateTime | Won at |
| `lost_at` | DateTime | Lost at |
| `won_by` | String | Won by user UUID |
| `lost_by` | String | Lost by user UUID |
| `lost_status_id` | String | Lost status UUID |
| `user` | Object | Deal owner object: `id` (User UUID), `email`, `full_name` |
| `contact` | Object | Deal contact object: `id`, `name`, `email`, `phone`, `ddi`, `instagram` |
| `latest_meeting_datetime` | DateTime | Date and time of the latest meeting |
| `latest_meeting_link` | String | Link to the latest meeting |
| `fields` | String | Additional deal fields, stored as a JSON object string |
Stream for Clint groups (paginated list). Use for full table sync of groups.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Group UUID |
| `name` | String | Group name |
| `archived_at` | DateTime | Archived at |
| `archived_by` | String | Archived by user UUID |
Stream for chat messages exchanged within each conversation. Child stream of Chats. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Message UUID |
| `created_at` | DateTime | Message creation timestamp |
| `updated_at` | DateTime | Last update timestamp; used for incremental sync |
| `chat_id` | String | UUID of the parent chat |
| `user_id` | String | UUID of the user who sent the message (when sent by a user) |
| `content` | String | Text content of the message |
| `type` | String | Message author type (USER, CUSTOMER, EVENT, NOTE) |
| `sent` | Boolean | Whether the message was sent |
| `seen` | Boolean | Whether the message was seen |
| `delivered` | Boolean | Whether the message was delivered |
| `content_type` | String | Content type (TEXT, IMAGE, AUDIO, VIDEO, DOCUMENT, TEMPLATE) |
| `content_url` | String | URL of the media content, when any |
| `external_id` | String | Identifier of the message on the external channel (e.g. WhatsApp) |
| `content_object` | String | Structured content payload (shape varies by content type), JSON object string |
| `content_action` | String | Interactive content payload such as template components and buttons, JSON object string |
| `status` | String | Delivery status (QUEUED, SENT, DELIVERED, READ) |
| `source` | String | Source that produced the message (e.g. CHAT) |
Stream for Clint origins (paginated list). Use for full table sync of origins.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Origin UUID |
| `name` | String | Origin name |
| `group` | Object | Group info for the origin including `id` (Group UUID) and `name` |
| `stages` | Array | List of stages for the origin, each with `id`, `label`, `order`, and `type` |
| `archived_at` | DateTime | Archived at |
| `archived_by` | String | Archived by user UUID |
# Data Model
The following diagram illustrates the relationships between the core data streams in Clint. Deals reference contacts and origins, and carry their own history and activities; origins belong to groups; chats belong to channel accounts and contain messages.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Contacts("Contacts");
Deals("Deals");
DealHistory("Deal History");
Groups("Groups");
Origins("Origins");
Activities("Activities");
ChannelAccounts("Channel Accounts");
Chats("Chats");
Messages("Messages");
end
Deals -- "contact.id" --> Contacts;
Deals -- "origin_id" --> Origins;
DealHistory -- "deal_id" --> Deals;
Activities -- "deal.id" --> Deals;
Origins -- "group.id" --> Groups;
Chats -- "channel_account_id" --> ChannelAccounts;
Messages -- "chat_id" --> Chats;
```
# Implementation Notes
### Authentication & API
* The connector sends your API token in the `api-token` HTTP header for requests to `https://api.clint.digital/v1` and `https://api.clint.digital/v2`.
* Keep your API token secret and do not share it; it is stored securely in Nekt.
* API rate limits (500 requests/minute) are handled automatically by the connector's built-in backoff mechanism.
### Sync behavior
* **Contacts**, **Groups**, **Origins**, **Activities**, and **Channel Accounts**: No replication key; use FULL\_TABLE to sync the current state.
* **Deals**, **Chats**, and **Messages**: Use INCREMENTAL to fetch only records changed since the last run. On the first run (or when no bookmark exists), the optional **Start Date** config limits the initial extract (`updated_at_start` for Deals and Messages, `last_message_at_start` for Chats).
* **Deal History**: Child stream of Deals. The API endpoint has no date filters, so history is fully re-fetched per parent deal. However, since the Deals stream is incremental, only deals updated since the last run get their history re-fetched. The stream exposes `date` as a replication key to enable INCREMENTAL sync, ensuring events are upserted by primary key (`id`) instead of replacing the entire table.
* **Messages**: Child stream of Chats; only chats returned by the parent sync have their messages fetched.
* **Activities**: Requires the `activities:read` scope and the Activities API feature flag enabled on your account. If these are missing, the stream is gracefully skipped with a warning.
### API errors and skipped records
* **Persistent API errors**: The Clint API may sometimes persistently fail (e.g., returning 500 errors) when attempting to fetch data for records that are in a broken or deleted state (such as certain channel accounts or chats). Instead of crashing the entire sync run, the connector will skip these broken parent records after exhausting retries. A summary of any skipped record IDs is printed in the logs at the end of the run, allowing you to contact Clint support with these IDs for verification.
### Custom fields
* Contacts and Deals can have custom `fields` (key-value data). The tap serializes these as JSON strings in the `fields` column (and in `organization.fields` for Contacts) for compatibility with a fixed schema since fields can be dynamic.
* If a contact is not linked to an organization, the Clint API returns `null` for the whole `organization` object, so `organization.id`, `organization.name` and `organization.fields` are all null for that contact. The connector handles this case without failing the sync.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| A stream is reported as skipped at the end of the run | The Clint API returned an error for that stream; the others ran normally. | Check the list of skipped record IDs in the run log. If the failure is recurrent on the same stream, disable it temporarily and contact support with the source slug, the run ID, and the time with timezone. See [API errors and skipped records](#api-errors-and-skipped-records). |
| The activities stream returns no data | The activities read scope, or the feature itself, is not enabled on the account. | Grant the scope and enable the feature in Clint, then re-authorize. |
## Skills for agents
Clint connector documentation as plain markdown, for use in AI agent contexts.
# Colibri NCR as a data source
Source: https://docs.nekt.com/sources/colibri-ncr
Bring data from Colibri NCR to Nekt.
Colibri NCR is a comprehensive Point of Sale (POS) system designed for retail and food service businesses, widely used in Brazil. It helps manage sales, inventory, cash flow, and fiscal compliance. Integrating Colibri NCR data into your lakehouse provides valuable insights into your store's performance.
![]()
## Configuring Colibri NCR as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Colibri NCR option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Colibri NCR credentials for this connection.
The following configurations are available:
* **Client ID**: The client ID to authenticate against the API service.
* **Rede ID**: The rede ID to sync data from. You can find it in the Colibri NCR dashboard.
* **Start Date**: The earliest date from which records will be synced. This should be in DD-MM-YYYY format.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Colibri NCR and their corresponding fields:
Stream for managing cash register movements and transactions.
**Fields:**
* `idMovimentoCaixa`
* `redeId`
* `rede`
* `lojaId`
* `loja`
* `hora`
* `idAtendente`
* `codAtendente`
* `nomeAtendente`
* `vlDesconto`
* `vlAcrescimo`
* `vlTotalReceber`
* `vlTotalRecebido`
* `vlServicoRecebido`
* `vlConsumacaoRecebido`
* `vlTrocoContravale`
* `vlTrocoDinheiro`
* `vlTrocoRepique`
* `vlTaxaEntrega`
* `numPessoas`
* `operacaoId`
* `maquinaId`
* `nomeMaquina`
* `caminhoMaquina`
* `modoVendaId`
* `modoVendaNome`
* `meiosPagamento` (Array of Objects):
* `id`
* `codigo`
* `nome`
* `redeId`
* `valor`
* `bandeira`
* `tipoCartao`
* `idAtendente`
* `codAtendente`
* `nomeAtendente`
* `autorizacao`
* `nsu`
* `credenciadora`
* `turnoId`
* `cliente` (Object):
* `id`
* `nome`
* `identificacao`
* `email`
* `cpfCnpj`
* `turnoId`
* `consumidores` (Array of Objects):
* `documento`
* `tipo`
* `cancelado`
* `statusComprovante` (Object):
* `chave`
* `numero`
* `ressalva`
* `status`
* `dataContabil`
* `tipoId`
* `tipoDesc`
* `clientes` (Array of Objects):
* `id`
* `idPenduraPos`
* `vlTotal`
* `saldo`
* `operacaoId`
* `cancelado`
* `obs`
* `idCliente`
* `nomeCliente`
* `identificacaoCliente`
* `emailCliente`
* `cpfCnpjCliente`
Stream for sold items in transactions.
**Fields:**
* `idItemVenda`
* `redeId`
* `rede`
* `lojaId`
* `loja`
* `quantidade`
* `idMaterial`
* `codMaterial`
* `codMaterialStr`
* `descricao`
* `valorTotal`
* `valorUnitario`
* `codGrupo`
* `grupoNome`
* `comboId`
* `combo`
* `operacaoId`
* `atendenteId`
* `atendenteNome`
* `modoVendaId`
* `modoVendaNome`
* `motivoCancelamento` (Object):
* `id`
* `nome`
* `codigo`
* `consumidores` (Array of Objects):
* `documento`
* `tipo`
* `cancelado`
* `timestampLancamento`
* `dtLancamento`
* `horaLancamento`
* `tipoCancelamento`
Stream for fiscal receipts and invoices.
**Fields:**
* `denominacao`
* `valor`
* `maquinaId`
* `nfNumero`
* `nfSerie`
* `nfSubserie`
* `serieEcf`
* `contOrdemOperacaoEcf`
* `contCupomFiscalEcf`
* `contComprovanteDebCredEcf`
* `contRelatorioGerencialEcf`
* `contComprovanteNaoFiscalEcf`
* `consumidor`
* `desconto`
* `acrescimo`
* `chaveNfe`
* `statusNfe`
* `protocoloNfe`
* `dataContabil`
* `dataEmissao`
* `idOperacao`
* `troco`
* `dataMovimento`
* `cancelado`
* `detalhes` (Array of Objects):
* `ordem`
* `codigo`
* `descricao`
* `quantidade`
* `valor`
* `valorUnitario`
* `aliquota`
* `baseIcms`
* `codNcm`
* `prodOrigem`
* `cfop`
* `ctsIcms`
* `aliquotaIcmsIss`
* `ctsPis`
* `aliqPis`
* `cstCofins`
* `aliquotaCofins`
* `reducaoBcIcms`
* `desconto`
* `acrescimo`
* `cancelado`
* `motivoDesoneracaoIcms`
* `codigoBeneficioFiscal`
* `vlDesoneracaoIcms`
* `meios` (Array of Objects):
* `id`
* `descricao`
* `valor`
* `numAutorizacao`
* `credenciadora`
* `credenciadoraRede`
Stream for managing stores.
**Fields:**
* `idLoja`
* `nome`
Stream for managing work shifts.
**Fields:**
* `idTurno`
* `idFuncionario`
* `codFuncionario`
* `nomeFuncionario`
* `dataAbertura`
* `dataFechamento`
* `dataContabil`
* `idLoja`
* `nomeLoja`
* `idRede`
* `nomeRede`
* `conferencias` (Array of Objects):
* `idTurno`
* `idMeioPagamento`
* `codMeioPagamento`
* `nomeMeioPagamento`
* `valorDigitado`
Stream for managing payment methods.
**Fields:**
* `id`
* `rede`
* `sub_rede`
* `loja`
* `codigo`
* `descricao`
* `imagem`
* `troco_dinheiro`
* `troco_repique`
* `troco_contravale`
* `cartao`
* `tef`
* `multiplo`
* `cor`
* `ordem`
* `sistema`
Stream for managing price tables. This is a child stream of `Lojas`.
**Fields:**
* `id`
* `ativo`
* `dt_alt`
* `modo_venda`
* `nome`
* `dt_vigencia`
* `loja`
Stream for managing item prices within price tables. This is a child stream of `Tabelas Preco`.
**Fields:**
* `id`
* `ativo`
* `dt_alt`
* `preco`
* `tabela`
* `material`
Stream for ECF (Emissor de Cupom Fiscal) maps, related to fiscal printers in Brazil.
**Fields:**
* `id`
* `idLoja`
* `nomeLoja`
* `idRede`
* `nomeRede`
* `dataContabil`
* `cnpj`
* `inscricaoEstadual`
* `inscricaoMunicipal`
* `numeroEcf`
* `numeroSerieEcf`
* `dataReducaoZ`
* `contOrdemOperacao`
* `contComprovanteNaoFiscalEcf`
* `contReinicioOperacaoEcf`
* `contReducaoZEcf`
* `valorGrandeEcf`
* `valorVendaBruta`
* `detalhes` (Array of Objects):
* `idMapaEcf`
* `codTotalizador`
* `base`
* `percentual`
* `valor`
* `nuSerie`
* `contReducaoZEcf`
* `posicao`
* `dataContabil`
* `tipo`
* `totalizadores` (Array of Objects):
* `idMapaEcf`
* `contador`
* `valor`
* `operacao`
Stream for sales modes.
**Fields:**
* `id`
* `nome`
Stream for cancellation reasons.
**Fields:**
* `id`
* `rede_loja`
* `ativo`
* `dt_alt`
* `dt_alt_timezone`
* `codigo`
* `nome`
* `baixa_estoque`
* `rede`
* `sub_rede`
* `loja`
Stream for the status of an accounting period for a store.
**Fields:**
* `dataContabil`
* `redeId`
* `lojaId`
* `status`
## Skills for agents
Colibri NCR connector documentation as plain markdown, for use in AI agent contexts.
# Composio as a data source
Source: https://docs.nekt.com/sources/composio
Bring data from Composio to Nekt.
Composio is a tool-calling platform for AI agents. It gives agents authenticated access to
hundreds of third-party applications (Slack, GitHub, Gmail, Notion and others) by handling
the OAuth connections, exposing each application's actions as callable tools, and forwarding
the applications' events as triggers.
This connector brings two kinds of data to Nekt: the **catalog** of what Composio can do
(toolkits, tools, and trigger types) and the **activity** of your own project (authentication
configurations, connected accounts, active triggers, MCP servers, and the full execution and
trigger event logs).
## 1. Add your Composio access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Composio option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Key**: your Composio project API key. In the [Composio dashboard](https://app.composio.dev), go to **Settings → Project Settings** and copy the key from the **API Keys** section. See the [official documentation](https://docs.composio.dev/reference/v3/authentication) for details.
* **Initial sync date**: the starting point for the two log streams. Execution and trigger log entries older than this date are not synced. Leave it empty to bring the full history Composio still retains.
* **Toolkit slugs** (advanced): an optional list of toolkit slugs, such as `slack` or `github`. When filled, the `tools` and `trigger_types` catalog streams are narrowed to those toolkits. Leave it empty to sync the full Composio catalog, which spans every available toolkit and is considerably larger.
3. Click **Next**.
A Composio API key is scoped to a single **project**. To bring data from more than one project, create one Nekt source per project.
## 2. Select your Composio streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Composio data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Composio data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Available streams
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Toolkits | `toolkits` | Full table | Applications and services Composio can connect to. |
| Toolkit Categories | `toolkit_categories` | Full table | Categories used to group toolkits. |
| Tools | `tools` | Full table | Individual actions an agent can call, with their input and output schemas. |
| Trigger Types | `trigger_types` | Full table | Event types toolkits can emit, with their configuration and payload schemas. |
| Auth Configs | `auth_configs` | Full table | How your project authenticates against each toolkit. |
| Connected Accounts | `connected_accounts` | Full table | End-user connections between your project and an external service. |
| Trigger Instances | `trigger_instances` | Full table | Triggers your project has enabled on its connected accounts. |
| MCP Servers | `mcp_servers` | Full table | MCP servers your project exposes to AI assistants. |
| Action Execution Logs | `action_execution_logs` | Incremental | Every tool execution, with its status and duration. |
| Trigger Logs | `trigger_logs` | Incremental | Every trigger event Composio received and dispatched. |
Composio embeds live credentials in some API responses: the OAuth tokens of each connected account, the client secrets of your authentication configurations, and the shareable access URL of each MCP server. This connector removes all of them before writing to your Catalog, so no Composio credential ever reaches your Lakehouse.
## Streams and fields
The applications and services Composio can connect to. Deprecated toolkits are included so
rows never silently disappear from your Catalog.
**Key Fields:**
* `slug` - URL-friendly unique identifier of the toolkit, used to reference it everywhere else
* `name` - Human-readable name of the toolkit
* `type` - Toolkit provenance: `native` for Composio-managed toolkits, `custom` for one registered by your project
* `toolkit_id` - Legacy internal identifier, useful for cross-referencing older Composio data
**Description and Branding:**
* `description` - What the toolkit is for and which functionality it exposes
* `logo` - Image URL of the toolkit's branding
* `app_url` - Link to the toolkit's main application or service website
* `auth_guide_url` - Guide with authentication setup instructions
**Authentication:**
* `auth_schemes` - Authentication methods supported by the toolkit, such as `OAUTH2` or `API_KEY`
* `composio_managed_auth_schemes` - Methods for which Composio provides the OAuth application, so your project does not need its own credentials
* `no_auth` - Whether the toolkit can be used without any authentication
**Content and Metadata:**
* `categories` - Categories the toolkit belongs to, as a JSON array of objects with `id` and `name`
* `tools_count` - Number of tools available in the toolkit
* `triggers_count` - Number of trigger types available in the toolkit
* `version` - Version of the toolkit definition
* `created_at` - When the toolkit was added to the Composio catalog
* `updated_at` - When the toolkit definition was last modified
The categories used to group toolkits in the Composio catalog.
**Key Fields:**
* `id` - URL-friendly unique identifier of the category, used to filter toolkits
* `name` - Display name of the category
The individual actions an agent can call through Composio. The full catalog spans every
toolkit and is large; use the **Toolkit slugs** setting to narrow it down.
**Key Fields:**
* `slug` - Unique identifier of the tool, used when executing it
* `name` - Human-readable display name
* `description` - Explanation of the tool's functionality, as presented to the calling agent
* `human_description` - Human-friendly description, when the toolkit provides one
**Parent Toolkit:**
* `toolkit_slug` - Identifier of the toolkit that provides the tool
* `toolkit_name` - Name of that toolkit
* `toolkit_logo` - URL of that toolkit's logo image
**Schemas:**
* `input_parameters` - JSON Schema of the tool's input arguments, as a JSON string
* `output_parameters` - JSON Schema of the tool's return value, as a JSON string
**Access and Versioning:**
* `no_auth` - Whether the tool can be executed without a connected account
* `scopes` - Flat list of OAuth scopes the tool requires
* `scope_requirements` - Structured scope requirements, as a JSON object. Null means the tool is legacy and only exposes the flat scopes list
* `tags` - Tags used to categorize and filter the tool
* `version` - Current version of the tool definition
* `available_versions` - All versions of the tool that can be pinned
* `is_deprecated` - Whether the tool is deprecated and may be removed in the future
The event types toolkits can emit into Composio.
**Key Fields:**
* `slug` - Unique identifier of the trigger type
* `name` - Human-readable name of the trigger
* `description` - What the trigger does and when it fires
* `instructions` - Step-by-step instructions to set it up
* `type` - Trigger mechanism: `webhook` (event-based) or `poll` (scheduled check)
**Parent Toolkit:**
* `toolkit_slug` - Identifier of the toolkit that provides the trigger
* `toolkit_logo` - URL of that toolkit's logo image
**Schemas and Setup:**
* `config` - JSON Schema of the configuration required to set up the trigger, as a JSON string
* `payload` - JSON Schema of the payload delivered when the trigger fires, as a JSON string
* `requires_webhook_endpoint_setup` - Whether a Composio-issued webhook URL must first be registered with the upstream provider
* `version` - Version of the trigger type definition
How your project authenticates against each toolkit. Credential material is never synced.
**Key Fields:**
* `id` - Unique identifier of the authentication configuration
* `uuid` - Legacy UUID, kept for backward compatibility
* `name` - Display name of the configuration
* `type` - Whether it is a Composio `default` or a `custom` configuration created by your project
* `status` - Current status: `ENABLED` or `DISABLED`
**Toolkit and Scheme:**
* `toolkit_slug` - Toolkit the configuration authenticates against
* `toolkit_logo` - URL of that toolkit's logo image
* `auth_scheme` - Scheme used, such as `OAUTH2`, `API_KEY` or `BEARER_TOKEN`
* `is_composio_managed` - Whether Composio supplies the OAuth application, instead of your project bringing its own client credentials
**Usage and Access:**
* `no_of_connections` - Number of active connected accounts using the configuration
* `tools_for_connected_account_creation` - Tools used to derive the minimum OAuth scopes required
* `tools_available_for_execution` - Tools that accounts connected through this configuration are allowed to execute
* `expected_input_fields` - Fields the end user must fill in when connecting an account, as a JSON array
* `is_enabled_for_tool_router` - Whether the configuration is exposed to the Composio tool router
* `proxy_url` - URL of the authentication proxy, when one is used
**Audit:**
* `created_by` - Identifier of the user who created the configuration
* `created_at` - When the configuration was created
* `last_updated_at` - When the configuration was last modified
The end-user connections between your project and an external service, one row per user
per application. Access and refresh tokens are never synced.
**Key Fields:**
* `id` - Unique identifier of the connected account
* `toolkit_slug` - Toolkit the account is connected to
* `user_id` - End user the connection belongs to. Composio is phasing this field out of the API response, so it may come back empty
* `alias` - User-defined alias for the connected account
* `word_id` - Short, token-friendly identifier used to disambiguate multiple accounts of the same toolkit, such as `gmail_red-castle`
**Status:**
* `status` - Connection status: `INITIALIZING`, `INITIATED`, `ACTIVE`, `FAILED`, `EXPIRED`, `INACTIVE` or `REVOKED`
* `status_reason` - Explanation of the last status change, when the API provides one
* `is_disabled` - Whether the connection has been disabled
* `test_request_endpoint` - Endpoint Composio calls to verify the connection still works
**Authentication Configuration:**
* `auth_scheme` - Authentication scheme of the connection itself
* `auth_config_id` - Identifier of the configuration used to create the connection
* `auth_config_auth_scheme` - Authentication scheme of that configuration
* `auth_config_is_composio_managed` - Whether that configuration is managed by Composio
* `auth_config_is_disabled` - Whether that configuration is disabled
**Sharing:**
* `account_type` - Sharing model: `PRIVATE` (usable only by its owner) or `SHARED`
* `acl_config_for_shared` - Access control rules for shared connections, as a JSON object
**Audit:**
* `created_at` - When the connection was created
* `updated_at` - When the connection was last modified
The triggers your project has enabled on its connected accounts. Disabled instances are
included.
**Key Fields:**
* `id` - Unique identifier of the trigger instance
* `uuid` - Legacy UUID of the trigger instance
* `trigger_name` - Name of the trigger type this instance runs
* `version` - Version of the trigger instance definition
**Association:**
* `connected_account_id` - Connected account the trigger listens on
* `connected_account_uuid` - Legacy UUID of that connected account
* `user_id` - End user the trigger belongs to
**Configuration:**
* `trigger_config` - Configuration applied to the instance, as a JSON string
* `trigger_data` - Additional data attached to the instance
**Audit:**
* `created_at` - When the instance was created
* `updated_at` - When the instance was last modified
* `disabled_at` - When the instance was disabled, when applicable
The MCP servers your project exposes so AI assistants can reach its toolkits.
**Key Fields:**
* `id` - Unique identifier of the MCP server
* `name` - User-defined name of the server
* `server_instance_count` - Number of active user instances connected to the server
* `managed_auth_via_composio` - Whether authentication for the server is managed by Composio
**Scope:**
* `toolkits` - Toolkit slugs the server exposes
* `allowed_tools` - Tool slugs the server is allowed to execute
* `auth_config_ids` - Authentication configurations the server can use
* `toolkit_icons` - Mapping of each toolkit slug to its icon URL, as a JSON object
**Client Setup:**
* `command_cursor` - Command line instruction to connect the Cursor client
* `command_claude` - Command line instruction to connect the Claude client
* `command_windsurf` - Command line instruction to connect the Windsurf client
**Audit:**
* `created_at` - When the server was created
* `updated_at` - When the server configuration was last modified
Every tool execution Composio ran for your project, the stream to use for measuring agent
activity, success rates, and latency. Synced incrementally on `created_at`.
**Key Fields:**
* `id` - Unique identifier of the execution log entry
* `created_at` - Timestamp of the execution, used as the incremental replication key
* `status` - Outcome of the execution: `success` or `failed`
* `execution_time` - Duration of the execution in milliseconds
**What Ran:**
* `action_key` - Slug of the tool that was executed
* `app_key` - Slug of the toolkit the tool belongs to
* `app_name` - Display name of that toolkit
* `app_icon` - URL of that toolkit's icon image
**On Whose Behalf:**
* `connected_account_id` - Connected account used for the execution
* `entity_id` - End user on whose behalf the tool ran
**Result:**
* `minimal_response` - Truncated response body recorded for the execution
* `metadata` - Additional execution metadata, as a JSON object
Every trigger event Composio received from an upstream provider and dispatched to your
project. Synced incrementally on `created_at`.
**Key Fields:**
* `id` - Unique identifier of the trigger log entry
* `created_at` - When the event was received, used as the incremental replication key
* `updated_at` - When the log entry was last updated
* `status` - Outcome recorded for the trigger event
* `type` - Log entity type: `trigger` or `action`
**What Fired:**
* `trigger_name` - Name of the trigger type that fired
* `trigger_id` - Identifier of the trigger instance that fired
* `trigger_nano_id` - Nano ID of that trigger instance
* `app_name` - Name of the toolkit that emitted the event
* `provider` - Upstream provider that delivered the event
**Association:**
* `connection_id` - Connected account the event came from
* `client_id` - Composio client that owns the event
* `entity_id` - End user the event belongs to
**Payload:**
* `meta` - Full event detail, including the provider and client payloads, as a JSON object
## Notes and limitations
* **Log streams use internal endpoints.** `action_execution_logs` and `trigger_logs` read the endpoints that back the Log pages of the Composio dashboard. Composio publishes them in its public API document and accepts the regular project API key on them, but does not document their fields in the API reference, so they may change with less notice than the rest of the API.
* **Log history depends on your Composio plan.** The connector can only bring what Composio still retains for your project.
* **Organization-level data is not covered.** Endpoints such as the project list require a separate organization API key, which this connector does not use.
* **Rate limits.** Composio allows between 2,000 and 10,000 requests per minute depending on your plan. The connector throttles itself well below the lowest tier and retries rate-limited requests with exponential backoff, so a sync never competes with your own agents for quota.
# Conexa as a data source
Source: https://docs.nekt.com/sources/conexa
Bring data from Conexa to your Lakehouse.
Conexa is a Brazilian management platform for recurring billing, finance, and coworking spaces. The connector extracts your customers and the people linked to them, plans, contracts, sales and recurring sales, charges, expenses (bills), suppliers, accounts, and the settings around them, such as products, service categories, and receiving and payment methods. For Conexa Coworking accounts, it also extracts room bookings. With it you can analyze your recurring revenue, billing and collections, expenses, and customer base in your Lakehouse.
## Configuring Conexa as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Conexa option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Conexa authenticates with an Application Token and your account's subdomain. To create the token, an administrator goes to **Config > Integrações > API / Token** in Conexa, creates a token, and copies it.
Create the token with access to all units, so every unit's data is extracted. The connector only reads data.
The following configurations are available:
* **Subdomain**: the first part of the address you use to open Conexa. For `https://mycompany.conexa.app`, enter `mycompany`. Required.
* **Application Token**: the token created in **API / Token**. Required.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: all Conexa streams are FULL\_TABLE: Conexa does not let the connector ask only for records changed since the last sync, so every sync reads the whole table. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Limits and behavior
* Conexa allows 60 requests per minute per account, shared with any other integration that uses the same account. The connector reads 100 records per request and stays under that limit, so large accounts take longer to sync: about 6,000 records per minute.
* If the token cannot read a stream (for example, a module that is not part of your Conexa plan), that stream is skipped with a warning and its table is not updated. The sync fails only when none of the selected streams can be read.
* Fields marked "Conexa Coworking" are only filled for accounts of that product.
# Streams and Fields
Below you'll find all available data streams from Conexa and their corresponding fields. API reference: [Conexa API v2](https://documenter.getpostman.com/view/25182821/2s93RZMpcB).
Timestamps are stored in UTC. Objects and lists of objects (addresses, custom fields, quotas, complementary services) are stored as JSON text, and the most used values inside them also get their own columns (for example `cpf`, `cnpj`, `product_id`, `place_name`).
The units (branches) your Conexa account manages (`GET /companies`).
Primary key: `company_id`
| Field | Type | Description |
| :- | :- | :- |
| `company_id` | Integer | Unique identifier of the unit (company). |
| `trade_name` | String | Trade name of the unit. |
| `legal_name` | String | Legal name of the unit. |
| `cnpj` | String | CNPJ (Brazilian company taxpayer number) of the unit, digits only. |
| `address` | String | Address of the unit (JSON object): zipCode, street, number, neighborhood, additionalDetails, city and state (id, name and abbreviation). |
| `city` | String | City of the unit's address. |
| `state` | String | State (abbreviation, e.g. SP) of the unit's address. |
| `phone` | String | Phone number of the unit. |
| `notification_emails` | Array of strings | E-mail addresses that receive notifications. |
| `backup_emails` | Array of strings | Backup e-mail addresses. |
| `time_zone` | String | Time zone of the unit (e.g. America/Sao\_Paulo). |
| `notes` | String | Notes on the unit. |
| `active` | Boolean | Whether the unit is active. |
| `created_at` | Datetime | When the unit was created in Conexa (UTC). |
| `updated_at` | Datetime | When the unit was last updated in Conexa (UTC). |
Your customers, individuals or companies, with documents, contacts, address, tax withholding settings, and custom fields (`GET /customers`).
Primary key: `customer_id`
| Field | Type | Description |
| :- | :- | :- |
| `customer_id` | Integer | Unique identifier of the customer. |
| `company_id` | Integer | Identifier of the unit (company) the customer belongs to. |
| `name` | String | Legal name (legal entity) or full name (individual) of the customer. |
| `trade_name` | String | Trade name of the customer. |
| `first_name` | String | First name of the customer. |
| `pronunciation` | String | How the customer's name is pronounced. |
| `field_of_activity` | String | Line of business of the customer. |
| `notes` | String | Notes on the customer. |
| `is_juridical_person` | Boolean | Whether the customer is a legal entity (company). |
| `is_foreign` | Boolean | Whether the customer is foreign. |
| `cpf` | String | CPF (Brazilian individual taxpayer number) of an individual customer. |
| `cnpj` | String | CNPJ (Brazilian company taxpayer number) of a legal-entity customer. |
| `natural_person` | String | Data of an individual customer (JSON object): cpf, rg, birthDate, issuingAuthority, maritalStatus and profession. |
| `legal_person` | String | Data of a legal-entity customer (JSON object): cnpj, foundationDate, stateInscription and municipalInscription. |
| `foreign` | String | Data of a foreign customer (JSON object): document, birthDate and profession. |
| `address` | String | Address of the customer (JSON object): zipCode, street, number, neighborhood, additionalDetails, city and state (id, name and abbreviation). |
| `city` | String | City of the customer's address. |
| `state` | String | State (abbreviation, e.g. SP) of the customer's address. |
| `cell_number` | String | Mobile phone number. |
| `phones` | Array of strings | Contact phone numbers. |
| `emails` | Array of strings | E-mail addresses of the customer. |
| `emails_message` | Array of strings | E-mail addresses that receive messages for the customer. |
| `emails_financial_messages` | Array of strings | E-mail addresses that receive financial notices. |
| `website` | String | Website of the customer. |
| `has_login_access` | Boolean | Whether the customer can log in to the Conexa customer area. |
| `login` | String | Login of the customer area. |
| `automatically_issue_nfse` | String | When the service invoice (NFS-e) is issued automatically: whenGeneratingBilling, afterPaymentBilling or notIssue. |
| `notes_nfse` | String | Notes printed on the customer's service invoices (NFS-e). |
| `has_iss_retention` | Boolean | Whether the customer withholds ISS (municipal service tax). |
| `tax_deductions` | String | Taxes the customer withholds (JSON object of flags): iss, ir, pis, inss, csll and cofins. |
| `tags_id` | Array of integers | Identifiers of the tags assigned to the customer. |
| `is_blocked` | Boolean | Whether the customer is blocked. |
| `is_active` | Boolean | Whether the customer is active. |
| `extra_fields` | String | Custom (extra) fields of the customer (JSON array of objects with id, name and value). |
| `default_due_day` | Integer | Default due day of the month of the customer's charges. |
| `created_at` | Datetime | When the customer was registered in Conexa (UTC). |
| `is_networking_profile_visible` | Boolean | Whether the customer's profile is shown in the networking area of the customer area (Conexa Coworking). |
| `is_blocked_booking_customer_area` | Boolean | Whether the customer is prevented from booking from the customer area (Conexa Coworking). |
| `is_allowed_booking_outside_business_hours` | Boolean | Whether the customer can book outside business hours (Conexa Coworking). |
| `internet_plan` | String | Name of the customer's internet plan (Conexa Coworking, Mikrotik module). |
| `business_presentation` | String | Short presentation of the customer (Conexa Coworking). |
| `offered_services_products` | String | Services and products the customer offers (Conexa Coworking). |
| `reception_orientations` | String | Instructions for the front desk (Conexa Coworking). |
| `mailing_orientations` | String | Instructions for handling the customer's mail (Conexa Coworking). |
| `mailing_address` | String | Mailing address of the customer (JSON object, Conexa Coworking): zipCode, street, number, neighborhood, additionalDetails, landmark, city and state. |
| `extension_numbers` | Array of strings | Phone extensions of the customer (Conexa Coworking). |
| `dedicated_phones` | String | Dedicated phone lines of the customer (JSON array of objects with id and phone, Conexa Coworking). |
| `mailboxes` | Array of strings | Mailboxes (P.O. boxes) of the customer (Conexa Coworking). |
People linked to customers: partners, guarantors, and staff with access to the customer area (`GET /persons`).
Primary key: `person_id`
| Field | Type | Description |
| :- | :- | :- |
| `person_id` | Integer | Unique identifier of the person. |
| `customer_id` | Integer | Identifier of the customer the person is linked to. |
| `company_id` | Integer | Identifier of the unit (company) the person belongs to. |
| `name` | String | Name of the person. |
| `cpf` | String | CPF (Brazilian taxpayer number). |
| `rg` | String | RG (Brazilian identity card) number. |
| `issuing_authority` | String | Authority that issued the RG (e.g. SSP). |
| `birth_date` | Date | Date of birth. |
| `marital_status` | String | Marital status: single, married, divorced, widowed or not informed. |
| `sex` | String | Sex: I (not informed), M (male) or F (female). |
| `nationality` | String | Nationality. |
| `place_of_birth` | String | Place of birth. |
| `profession` | String | Profession. |
| `job_title` | String | Job title at the customer. |
| `resume` | String | Short résumé of the person. |
| `notes` | String | Notes on the person. |
| `is_foreign` | Boolean | Whether the person is foreign. |
| `is_company_partner` | Boolean | Whether the person signs on behalf of the customer company. |
| `is_guarantor` | Boolean | Whether the person is a guarantor of the customer. |
| `is_individual_customer` | Boolean | Whether the person is the customer itself (individual customer). |
| `has_login_access` | Boolean | Whether the person can log in to the Conexa customer area. |
| `is_active` | Boolean | Whether the person is active. |
| `cell_number` | String | Mobile phone number. |
| `phones` | Array of strings | Phone numbers. |
| `emails` | Array of strings | E-mail addresses. |
| `photo` | String | URL of the person's photo. |
| `address` | String | Address of the person (JSON object): zipCode, street, number, neighborhood, additionalDetails, city and state (id, name and abbreviation). |
| `can_receive_mail` | Boolean | Whether the person can receive mail for the customer (Conexa Coworking). |
| `url_linkedin` | String | LinkedIn profile URL (Conexa Coworking). |
| `url_instagram` | String | Instagram profile URL (Conexa Coworking). |
| `url_facebook` | String | Facebook profile URL (Conexa Coworking). |
| `url_twitter` | String | Twitter profile URL (Conexa Coworking). |
| `print_fee_id` | String | Identifier of the person in print billing files (Conexa Coworking, depends on configuration). |
| `access_id` | String | Identifier of the person in access control devices (Conexa Coworking, depends on configuration). |
| `use_facial_recognition_photo` | Boolean | Whether the person's photo is used by facial recognition devices (Conexa Coworking). |
The plans contracts are built on, with prices per billing periodicity and the quotas they include (`GET /plans`).
Primary key: `plan_id`
| Field | Type | Description |
| :- | :- | :- |
| `plan_id` | Integer | Unique identifier of the plan. |
| `company_id` | Integer | Identifier of the unit (company) the plan belongs to. |
| `name` | String | Name of the plan. |
| `description` | String | Description of the plan. |
| `is_active` | Boolean | Whether the plan is active. |
| `is_custom` | Boolean | Whether the plan was created as a custom plan while registering a contract. |
| `is_online_contracting_enabled` | Boolean | Whether the plan can be hired online. |
| `service_category_id` | Integer | Identifier of the service category. |
| `cost_center_id` | Integer | Identifier of the cost center. |
| `refund_value` | Number | Amount of the returnable deposit. |
| `membership_fee` | Number | Membership (sign-up) fee. |
| `fidelity_months` | Integer | Length of the loyalty (minimum term) period, in months. |
| `nfse_description` | String | Description printed on the service invoice (NFS-e). |
| `payment_periodicities` | String | Price of the plan per billing periodicity it offers (JSON object): monthly, bimonthly, quarterly, semester and yearly. |
| `product_quotas` | String | Quotas of services or items included in the plan (JSON array of objects with id, productId and quota). |
| `receipt_description` | String | Description printed on the rental receipt (Conexa Coworking). |
| `is_sms_enabled` | Boolean | Whether messages are also sent by SMS (Conexa Coworking). |
| `private_space_ids` | Array of integers | Identifiers of the private spaces of the plan (Conexa Coworking). |
| `discount_on_rooms` | Number | Discount (percentage) on meeting room bookings (Conexa Coworking). |
| `discount_on_workstation` | Number | Discount (percentage) on shared space bookings (Conexa Coworking). |
| `service_correspondence_quotas` | String | Front-desk service and mail quotas (JSON object, Conexa Coworking): whether they are limited, the limit and the price of each one above it. |
| `booking_models` | String | Booking models for shared space workstations (JSON array of objects: id, name, shareSpaceId, frequency, isActive, startHour, endHour, resume, daysOfWeek, monthlyType, createdAt, updatedAt; Conexa Coworking). |
| `hour_quotas` | String | Hour quotas in meeting rooms, shared spaces or room groups (JSON array of objects: id, name, spaceId, groupId, quantity, validityType, createdAt, updatedAt; Conexa Coworking). |
| `created_at` | Datetime | When the plan was created in Conexa (UTC). |
| `updated_at` | Datetime | When the plan was last updated in Conexa (UTC). |
Recurring agreements with your customers: plan, amount, billing frequency, start and end dates, and complementary services (`GET /contracts`).
Primary key: `contract_id`
| Field | Type | Description |
| :- | :- | :- |
| `contract_id` | Integer | Unique identifier of the contract. |
| `customer_id` | Integer | Identifier of the contracting customer. |
| `plan_id` | Integer | Identifier of the plan the contract follows. |
| `seller_id` | Integer | Identifier of the user (seller) responsible for the contract. |
| `creator_user_id` | Integer | Identifier of the user who created the contract. |
| `cost_center_id` | Integer | Identifier of the cost center. |
| `payment_frequency` | String | How often the contract is billed: Monthly, Bimonthly, Quarterly, Semester or Yearly. |
| `contract_summary` | String | Short description of the contract. |
| `start_date` | Date | Start date of the contract. |
| `end_date` | Date | End date of the contract, if ended. |
| `end_reason_id` | Integer | Identifier of the cancellation reason, if the contract was ended. |
| `fidelity_date` | Date | End date of the loyalty (minimum term) period. |
| `date_sales_generation` | Date | Date from which the contract generates sales. |
| `due_day` | Integer | Due day of the month of the contract's charges. |
| `had_prorata` | Boolean | Whether the first billing was calculated pro rata. |
| `is_active` | Boolean | Whether the contract is active. |
| `sales_quantity` | Integer | Number of sales the contract has generated. |
| `amount` | Number | Final amount of the contract. |
| `refund_amount` | Number | Amount of the returnable deposit. |
| `notes` | String | Notes on the contract. |
| `last_contractual_readjustment` | String | Last price readjustment of the contract (JSON object): index, date and percentage. |
| `complementary_services` | String | Complementary services of the contract (JSON array of objects): productOrServiceId, startDate, endDate, isActive, quantity, amount and notes. |
| `product_quotas` | String | Quotas of products (services or items) the customer can consume without being charged (JSON array of objects with productId and quantity). |
| `extra_fields` | String | Custom (extra) fields of the contract (JSON array of objects). |
| `discount_on_rooms` | Number | Discount (percentage) on meeting room bookings (Conexa Coworking). |
| `discount_on_workstation` | Number | Discount (percentage) on shared space bookings (Conexa Coworking). |
| `is_sms_enabled` | Boolean | Whether messages are also sent by SMS (Conexa Coworking). |
| `private_space_id` | Integer | Identifier of the private office or room linked to the contract (Conexa Coworking). |
| `booking_models` | String | Workstation quotas on predefined days and hours (JSON array of objects with personId and hourPlanId, Conexa Coworking). |
| `hour_plan_quota` | String | Hour packages of the contract (JSON array of objects with quantity, spaceId and groupId, Conexa Coworking). |
| `service_correspondence_quotas` | String | Front-desk service and mail quotas (JSON object, Conexa Coworking): whether they are limited, the limit and the price of each one above it. |
| `created_at` | Datetime | When the contract was created in Conexa (UTC). |
| `updated_at` | Datetime | When the contract was last updated in Conexa (UTC). |
The services and items you sell (`GET /products`).
Primary key: `product_id`
| Field | Type | Description |
| :- | :- | :- |
| `product_id` | Integer | Unique identifier of the product (service or item). |
| `company_id` | Integer | Identifier of the unit (company) the product belongs to. |
| `name` | String | Name of the service or item. |
| `description` | String | Description of the service or item. |
| `price` | Number | Price of the product. |
| `active` | Boolean | Whether the product is active. |
| `is_customer_consumable` | Boolean | Whether customers can consume the product from the customer area. |
| `is_default_product` | Boolean | Whether the product is a default product of the Conexa system. |
| `cost_center_id` | Integer | Identifier of the cost center. |
| `service_category_id` | Integer | Identifier of the service category. |
| `notifications_emails` | Array of strings | E-mail addresses notified when the product is sold. |
| `nfse_description` | String | Description printed on the service invoice (NFS-e). |
| `receipt_description` | String | Description printed on the rental receipt (Conexa Coworking). |
| `created_at` | Datetime | When the product was created in Conexa (UTC). |
| `updated_at` | Datetime | When the product was last updated in Conexa (UTC). |
Service categories and their tax settings for service invoices (`GET /serviceCategories`).
Primary key: `service_category_id`
| Field | Type | Description |
| :- | :- | :- |
| `service_category_id` | Integer | Unique identifier of the service category. |
| `name` | String | Name of the category. |
| `description` | String | Description of the category. |
| `is_active` | Boolean | Whether the category is active. |
| `companies` | String | Units the category belongs to (JSON array of objects with id and name). |
| `cnae_code` | String | CNAE code (Brazilian classification of economic activities). |
| `nbs_code` | String | NBS code (Brazilian nomenclature of services). |
| `municipal_service_code` | String | Service code of the municipality. |
| `municipal_tax_code` | String | Tax code of the municipality. |
| `municipal_benefit_code` | String | Municipal benefit code of the national NFS-e portal. |
| `operation_indicator_code` | String | Operation indicator code (cIndOp) of the tax reform. |
| `tax_classification_code` | String | Tax classification code (cClassTrib) of the tax reform. |
| `tax_situation_code` | String | Tax situation code (CST), usually the first 3 digits of cClassTrib. |
| `property_cib_code` | String | Property CIB code of the category (returned by Conexa, not documented). |
| `municipal_ibs_percentage` | Number | Municipal IBS rate (percentage, tax reform). |
| `state_ibs_percentage` | Number | State IBS rate (percentage, tax reform). |
| `cbs_percentage` | Number | CBS rate (percentage, tax reform). |
| `commission_percentage` | Number | Commission percentage (when the commission module is enabled). |
| `nfse_percentage` | Number | Share of the amount billed on a service invoice (percentage). |
| `receipt_percentage` | Number | Share of the amount billed on a receipt (percentage). |
| `tax_deductions` | String | Tax rates of the category (JSON object): iss, inss, pis, cofins, ir, csll, irp (public-company income tax) and minValueDarf (minimum DARF amount). |
Custom fields defined for customers, plans, and contracts (`GET /extraFields`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the extra field. |
| `name` | String | Name of the extra field. |
| `type` | String | Record type the field applies to: customer, plan or contract. |
| `show_customer_service` | Boolean | Whether the field is shown in the customer service information. |
| `created_at` | Datetime | When the extra field was created in Conexa (UTC). |
| `updated_at` | Datetime | When the extra field was last updated in Conexa (UTC). |
Every sale, whether registered by hand or generated by a contract or a recurring sale (`GET /sales`).
Primary key: `sale_id`
| Field | Type | Description |
| :- | :- | :- |
| `sale_id` | Integer | Unique identifier of the sale. |
| `status` | String | Status of the sale: paid, billed, cancelled, notBilled, deductedFromQuota, billedCancelled, billedNegociated or partiallyPaid. |
| `customer_id` | Integer | Identifier of the customer of the sale. |
| `requester_id` | Integer | Identifier of the person (of the customer) who requested the sale. |
| `seller_id` | Integer | Identifier of the user (seller) who registered the sale. |
| `contract_id` | Integer | Identifier of the contract that generated the sale, if any. |
| `recurring_sale_id` | Integer | Identifier of the recurring sale that generated the sale, if any. |
| `product_id` | Integer | Identifier of the product (service or item) sold. |
| `product_name` | String | Name of the product sold. |
| `product_company_id` | Integer | Identifier of the unit (company) the product sold belongs to. |
| `quantity` | Number | Quantity of the product in the sale. |
| `original_amount` | Number | Original amount of the sale, before the discount. |
| `discount_value` | Number | Discount given on the sale. |
| `amount` | Number | Final amount of the sale, after the discount. |
| `reference_date` | Datetime | Date and time the sale refers to (UTC). |
| `notes` | String | Notes on the sale. |
| `created_at` | Datetime | When the sale was created in Conexa (UTC). |
| `updated_at` | Datetime | When the sale was last updated in Conexa (UTC). |
Sales repeated on a schedule (`GET /recurringSales`).
Primary key: `recurring_sale_id`
| Field | Type | Description |
| :- | :- | :- |
| `recurring_sale_id` | Integer | Unique identifier of the recurring sale. |
| `customer_id` | Integer | Identifier of the customer of the recurring sale. |
| `requester_id` | Integer | Identifier of the person (of the customer) who requested it. |
| `seller_id` | Integer | Identifier of the user (seller) responsible for the recurring sale. |
| `creator_user_id` | Integer | Identifier of the user who created the recurring sale. |
| `product_id` | Integer | Identifier of the product (service or item) sold; empty when an hour package is sold instead. |
| `package_id` | Integer | Identifier of the hour package sold (Conexa Coworking); empty when a product is sold instead. |
| `frequency` | String | How often the sale repeats: daily, weekly, monthly, bimonthly, quarterly, semester or yearly. |
| `occurrence_quantity` | Integer | Number of occurrences planned for the recurring sale. |
| `generated_quantity` | Integer | Number of sales the recurring sale has generated so far. |
| `start_date` | Date | Date of the first occurrence. |
| `end_date` | Date | Date of the last occurrence (end of the recurring sale). |
| `quantity` | Number | Quantity of the product in each occurrence. |
| `amount` | Number | Amount of each occurrence. |
| `notes` | String | Notes on the recurring sale. |
| `last_adjustment_date` | Date | Date of the last price adjustment. |
| `recurring_sale_contract_id` | Integer | Identifier of the contract that generated the recurring sale, if any. |
| `is_active` | Boolean | Whether the recurring sale is active. |
| `is_calculate_pro_rata` | Boolean | Whether the first occurrence is charged pro rata (Conexa Coworking). |
| `created_at` | Datetime | When the recurring sale was created in Conexa (UTC). |
| `updated_at` | Datetime | When the recurring sale was last updated in Conexa (UTC). |
What your customers are billed, with amounts, due and payment dates, status, and bank slip details (`GET /charges`).
Primary key: `charge_id`
| Field | Type | Description |
| :- | :- | :- |
| `charge_id` | Integer | Unique identifier of the charge. |
| `company_id` | Integer | Identifier of the unit (company) the charge belongs to. |
| `customer_id` | Integer | Identifier of the customer charged. |
| `account_id` | Integer | Identifier of the account that receives the charge. |
| `receiving_method` | String | Name of the receiving method configured for the charge. |
| `type` | String | Type of the charge: loose, contractual, returnableDeposit, contractualPlan or contractualServices. |
| `status` | String | Status of the charge: unpaid, paid, negotiated, generatedByNegotiation, cancelled, denied, thirdPartyCompany, protested, juridical or excluded. |
| `origin` | String | Where the billing came from: default, onlineHiring or invoicing. |
| `amount` | Number | Amount of the charge. |
| `raw_amount` | Number | Gross amount, before discounts (raw\_amount - amount = discount). |
| `current_amount` | Number | Current amount, including interest and late fees. |
| `paid_amount` | Number | Amount paid. |
| `discount_amount` | Number | Discount granted on the charge. |
| `payment_discount_amount` | Number | Discount granted when the charge was paid. |
| `conditional_discount` | String | Summary of the conditional (early payment) discount. |
| `conditional_discount_type` | String | Type of the conditional discount: fixed or percentage. |
| `conditional_discount_fixed_amount` | Number | Fixed amount of the conditional discount. |
| `conditional_discount_percentage` | Number | Percentage of the conditional discount. |
| `conditional_discount_date` | Date | Last date on which the conditional discount applies. |
| `due_date` | Date | Due date of the charge. |
| `payment_date` | Date | Date the charge was paid (settled). |
| `competence_date` | Date | Competence (accrual) date of the charge. |
| `cancel_date` | Date | Date the charge was cancelled. |
| `installment_number` | Integer | Installment number, when the charge is paid in installments. |
| `total_installments_amount` | Integer | Total number of installments, when the charge is paid in installments. |
| `father_charge_id` | Integer | Identifier of the parent charge. |
| `original_charge_id` | Integer | Identifier of the original charge. |
| `sales_ids` | Array of integers | Identifiers of the sales billed in the charge. |
| `customer_views` | Integer | Number of times the customer viewed the charge. |
| `has_iss_retention` | Boolean | Whether the customer withholds ISS (municipal service tax). |
| `iss_amount` | Number | Amount of ISS withheld by the customer. |
| `tax_invoice_number` | Integer | Number of the service invoice (NFS-e) issued for the charge, if any. |
| `notes` | String | Notes on the charge. |
| `charge_url` | String | URL of the invoice. |
| `billet_digitable_line` | String | Typeable line of the bank slip (boleto), when billed by boleto. |
| `billet_url` | String | URL of the bank slip (boleto) PDF. |
| `receipt_amount` | Number | Amount of the invoice/receipt (Conexa Coworking). |
| `raw_receipt_amount` | Number | Gross amount of the invoice/receipt (Conexa Coworking). |
| `receipt_description` | String | Additional description of the invoice/receipt (Conexa Coworking). |
| `receipt_code` | String | Code of the invoice/receipt (Conexa Coworking). |
| `created_at` | Datetime | When the charge was issued (UTC). |
| `updated_at` | Datetime | When the charge was last updated in Conexa (UTC). |
How charges are issued (`GET /invoicingMethods`).
Primary key: `invoicing_method_id`
| Field | Type | Description |
| :- | :- | :- |
| `invoicing_method_id` | Integer | Unique identifier of the invoicing method. |
| `company_id` | Integer | Identifier of the unit (company) the method belongs to. |
| `name` | String | Name of the invoicing method. |
| `type` | String | Type of the invoicing method: billet (bank slip) or others. |
| `is_active` | Boolean | Whether the invoicing method is active. |
How payments are received, with their fees and settlement times (`GET /receivingMethods`).
Primary key: `receiving_method_id`
| Field | Type | Description |
| :- | :- | :- |
| `receiving_method_id` | Integer | Unique identifier of the receiving method. |
| `name` | String | Name of the receiving method. |
| `max_installments` | Integer | Maximum number of installments. |
| `credit_days` | Integer | Days until the amount received is credited. |
| `is_installment_fee` | Boolean | Whether the fee is charged per installment. |
| `transaction_fee` | Number | Fixed fee per transaction. |
| `transaction_rate` | Number | Fee as a percentage of the transaction amount. |
| `account_id` | Integer | Identifier of the account it credits. |
| `cost_center_id` | Integer | Identifier of the cost center of the fee expense. |
| `bill_category_id` | Integer | Identifier of the expense category of the fee. |
| `bill_subcategory_id` | Integer | Identifier of the expense subcategory of the fee. |
| `payment_method_id` | Integer | Identifier of the payment method of the fee expense. |
| `supplier_id` | Integer | Identifier of the supplier of the fee expense. |
| `is_active` | Boolean | Whether the receiving method is active. |
| `created_at` | Datetime | When the receiving method was created in Conexa (UTC). |
| `updated_at` | Datetime | When the receiving method was last updated in Conexa (UTC). |
How expenses are paid (`GET /paymentMethods`).
Primary key: `payment_method_id`
| Field | Type | Description |
| :- | :- | :- |
| `payment_method_id` | Integer | Unique identifier of the payment method. |
| `name` | String | Name of the payment method. |
| `max_installments` | Integer | Maximum number of installments. |
| `account_id` | Integer | Identifier of the account assigned to the payment method. |
| `is_active` | Boolean | Whether the payment method is active. |
| `created_at` | Datetime | When the payment method was created in Conexa (UTC). |
| `updated_at` | Datetime | When the payment method was last updated in Conexa (UTC). |
Your expenses, with supplier, category, amounts, and due and payment dates (`GET /bills`).
Primary key: `bill_id`
| Field | Type | Description |
| :- | :- | :- |
| `bill_id` | Integer | Unique identifier of the expense. |
| `parent_bill_id` | Integer | Identifier of the parent expense (partial settlement). |
| `original_bill_id` | Integer | Identifier of the original expense (partial settlement). |
| `recurrent_bill_id` | Integer | Identifier of the fixed or recurring expense that generated it. |
| `charge_id` | Integer | Identifier of the charge that generated the expense, if any. |
| `installment_id` | Integer | Identifier of the installment plan, when paid in installments. |
| `installment_number` | Integer | Installment number, when paid in installments. |
| `company_id` | Integer | Identifier of the unit (company) the expense belongs to. |
| `supplier_id` | Integer | Identifier of the supplier. |
| `account_id` | Integer | Identifier of the account the expense is paid from. |
| `category_id` | Integer | Identifier of the expense category. |
| `subcategory_id` | Integer | Identifier of the expense subcategory. |
| `payment_method_id` | Integer | Identifier of the payment method. |
| `status` | String | Status of the expense: unpaid, cancelled, paid, received or scheduled. |
| `type` | String | Type of the expense: fixed or loose. |
| `description` | String | Description of the expense. |
| `amount` | Number | Amount of the expense. |
| `paid_amount` | Number | Amount paid. |
| `due_date` | Date | Due date of the expense. |
| `competence_date` | Date | Competence (accrual) date of the expense. |
| `payment_date` | Date | Date the expense was paid (settled). |
| `document_date` | Date | Issue date of the expense document. |
| `document_number` | String | Number of the expense document. |
| `digitable_line` | String | Typeable line of the expense's bank slip (boleto). |
| `is_reconciled` | Boolean | Whether the expense is reconciled with the bank statement. |
| `cac_is_included` | Boolean | Whether the expense is counted in the customer acquisition cost (CAC). |
| `cac_percentage` | Number | Share of the expense counted in the customer acquisition cost (percentage). |
| `cost_centers` | String | Cost centers the expense is split across (JSON array of objects with id and percentage). |
| `created_at` | Datetime | When the expense was created in Conexa (UTC). |
| `updated_at` | Datetime | When the expense was last updated in Conexa (UTC). |
Expense categories (`GET /billCategories`).
Primary key: `bill_category_id`
| Field | Type | Description |
| :- | :- | :- |
| `bill_category_id` | Integer | Unique identifier of the expense category. |
| `name` | String | Name of the expense category. |
| `is_active` | Boolean | Whether the category is active. |
| `subcategories` | String | Subcategories of the category (JSON array of objects with id and name). |
Expense subcategories and the income statement (DRE) group each one is reported under (`GET /billSubcategories`).
Primary key: `bill_subcategory_id`
| Field | Type | Description |
| :- | :- | :- |
| `bill_subcategory_id` | Integer | Unique identifier of the expense subcategory. |
| `name` | String | Name of the expense subcategory. |
| `bill_category_id` | Integer | Identifier of the expense category it belongs to. |
| `dre_category` | String | Income statement (DRE) group the subcategory is reported under. |
| `accounting_code` | Integer | Code of the subcategory in the accounting software. |
| `is_active` | Boolean | Whether the subcategory is active. |
Your suppliers (`GET /suppliers`).
Primary key: `supplier_id`
| Field | Type | Description |
| :- | :- | :- |
| `supplier_id` | Integer | Unique identifier of the supplier. |
| `name` | String | Name or trade name of the supplier. |
| `legal_name` | String | Legal name of a legal-entity supplier. |
| `type` | String | Type of the supplier: legalPerson (company) or naturalPerson (individual). |
| `cpf` | String | CPF (Brazilian individual taxpayer number) of an individual supplier. |
| `cnpj` | String | CNPJ (Brazilian company taxpayer number) of a legal-entity supplier. |
| `natural_person` | String | Data of an individual supplier (JSON object): cpf, rg and issuingAuthority. |
| `legal_person` | String | Data of a legal-entity supplier (JSON object): legalName, cnpj, stateInscription and municipalInscription. |
| `field_of_activity` | String | Line of business of the supplier. |
| `notes` | String | Notes on the supplier. |
| `cell_number` | String | Mobile phone number. |
| `website` | String | Website of the supplier. |
| `phones` | Array of strings | Phone numbers. |
| `emails` | Array of strings | E-mail addresses. |
| `contact_person_names` | Array of strings | Names of the contact people at the supplier. |
| `address` | String | Address of the supplier (JSON object): zipCode, street, number, neighborhood, additionalDetails, city and state (id, name and abbreviation). |
| `origin` | String | Where the supplier was registered from. |
| `is_active` | Boolean | Whether the supplier is active. |
| `created_at` | Datetime | When the supplier was created in Conexa (UTC). |
| `updated_at` | Datetime | When the supplier was last updated in Conexa (UTC). |
Cost centers (`GET /costCenters`).
Primary key: `cost_center_id`
| Field | Type | Description |
| :- | :- | :- |
| `cost_center_id` | Integer | Unique identifier of the cost center. |
| `name` | String | Name of the cost center. |
| `created_at` | Datetime | When the cost center was created in Conexa (UTC). |
| `updated_at` | Datetime | When the cost center was last updated in Conexa (UTC). |
Bank and cash accounts money moves through (`GET /accounts`).
Primary key: `account_id`
| Field | Type | Description |
| :- | :- | :- |
| `account_id` | Integer | Unique identifier of the account. |
| `company_id` | Integer | Identifier of the unit (company) that owns the account. |
| `name` | String | Name of the account. |
| `movement_block_until` | Date | Date until which entries in the account are blocked. |
| `automatic_bank_reconciliation` | Boolean | Whether the account has automatic bank reconciliation. |
| `accounting_code` | Integer | Code of the account in the accounting software. |
| `is_active` | Boolean | Whether the account is active. |
| `synced_at` | Datetime | When the account was last synchronized with the bank (UTC). |
| `created_at` | Datetime | When the account was created in Conexa (UTC). |
| `updated_at` | Datetime | When the account was last updated in Conexa (UTC). |
Meeting room bookings (`GET /room/bookings`). Only for Conexa Coworking accounts with the module enabled by Conexa's support; otherwise the stream is skipped with a warning. Access passwords are not extracted.
Primary key: `booking_id`
| Field | Type | Description |
| :- | :- | :- |
| `booking_id` | Integer | Unique identifier of the booking. |
| `sale_id` | Integer | Identifier of the sale generated by the booking. |
| `place_id` | Integer | Identifier of the room booked. |
| `place_name` | String | Name of the room booked. |
| `customer_id` | Integer | Identifier of the customer responsible for the booking. |
| `person_id` | Integer | Identifier of the person responsible for the booking. |
| `recurring_booking_id` | Integer | Identifier of the recurring booking it belongs to, if any. |
| `status` | String | Billing status of the booking: notBilled, billed, billedCancelled, partiallyPaid, paid, cancelled or deductedFromQuota. |
| `origin` | String | Where the booking was made: managerArea, app, api, broker or customerArea. |
| `start_time` | Datetime | When the booking starts (UTC). |
| `final_time` | Datetime | When the booking ends (UTC). |
| `completed` | Boolean | Whether the booking was completed. |
| `is_active` | Boolean | Whether the booking is active. |
| `is_billed` | Boolean | Whether the booking was billed. |
| `is_online_payment` | Boolean | Whether the booking was paid online. |
| `device_access_released` | Boolean | Whether access through devices was released for the booking. |
| `cancellation_reason` | String | Reason for the cancellation, if cancelled. |
| `room_temperature` | Number | Temperature set for the room (°C). |
| `visitors` | String | Visitors of the booking (JSON array of objects with id, name and email). |
| `notes` | String | Notes on the booking. |
| `created_at` | Datetime | When the booking was created in Conexa (UTC). |
| `updated_at` | Datetime | When the booking was last updated in Conexa (UTC). |
# Connectif as a data source
Source: https://docs.nekt.com/sources/connectif
Bring data from Connectif to Nekt.
Connectif is a marketing automation platform for e-commerce that unifies customer data to power personalized
journeys across email, SMS, push notifications, and on-site content. This connector extracts the store's
configuration entities (workflows, coupon sets, contact fields, custom integrations) through the Connectif REST
API, and the bulk data (contacts and their activities) through Connectif's asynchronous export system.
## 1. Add your Connectif access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Connectif option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Key**: a Connectif API key in the `keyId:keySecret` format. In Connectif, go to **Store Settings -> API Keys**, click **Create new API Key**, give it a name and select its permission scopes. Copy the generated key right away. For more information, check the official [documentation](https://api-docs.connectif.cloud/connectif-api/guides/authentication).
* **Start Date** (advanced): earliest activity date to sync in the activities streams. Defaults to 30 days before the first run.
* **Activity Types** (advanced): restrict the activities export to specific activity types (e.g. `purchase`, `login`, `register`). Leave empty to sync all types.
* **Segment ID** (advanced): optionally restrict the contacts and activities exports to contacts of a given Connectif segment.
* **Export Version** (advanced): the Connectif export format version (`v1` to `v6`), which affects the CSV column layout of the contacts and activities streams. Defaults to `v6`.
The API key must include **read** scope for the resources you want to sync (contacts, workflows, coupons, store,
integrations) and **read + write** scope for **Exports**: the connector creates export jobs to extract contacts
and activities in bulk. No IP allowlisting is required.
3. Click **Next**.
## 2. Select your Connectif streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Discovering the streams runs a real Connectif export to detect the columns of the `contacts` and `activities`
streams, so this step can take a few minutes on accounts with many contacts.
2. Click **Next**.
## 3. Configure your Connectif data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Connectif data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Available streams
| Stream | Slug | Description |
| - | - | - |
| Store | `store` | Details of the Connectif store the API key belongs to. |
| Workflows | `workflows` | Automation workflows configured in the store, with status and lifecycle dates. |
| Coupon Sets | `coupon_sets` | Coupon sets with availability and assignment counters. |
| Contact Fields | `contact_fields` | Standard and custom contact field definitions. |
| Custom Integrations | `custom_integrations` | Custom integrations configured in the store. |
| Custom Event Types | `custom_event_types` | Custom event types defined under each custom integration. |
| Contacts | `contacts` | All contacts of the store, extracted through a full contacts export. |
| Activities | `activities` | Contact activities (events), extracted incrementally in date windows of up to 31 days. |
| Activity Products | `activity_products` | Product lines that accompany the activities export (e.g. purchase basket items), when present. |
The columns of the `contacts`, `activities`, and `activity_products` streams are discovered at runtime from the
CSV files produced by the Connectif export, so they depend on your account and on the selected **Export Version**.
All export columns arrive as text and column names are normalized to snake\_case. Purchase data is available
through the `activities` stream; filter it with the `purchase` activity type if that's all you need.
## Fields by stream
Below you'll find all available data streams from Connectif and their corresponding fields:
Details of the Connectif store. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the store
* `name` - Display name of the store
* `alias` - Alias of the store
* `image_url` - URL of the store logo image
* `organization_id` - Identifier of the organization that owns the store
* `organization_name` - Name of the organization that owns the store
* `created_at` - Timestamp when the store was created
* `updated_at` - Timestamp of the last store update
Automation workflows configured in the store. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the workflow
* `name` - Display name of the workflow
* `description` - Description of the workflow
* `tags` - Tags assigned to the workflow
* `folder` - Folder that contains the workflow
* `status` - Current status (`draft`, `active`, `paused`, `finalized`, `scheduled`)
* `created_at` - Timestamp when the workflow was created
* `updated_at` - Timestamp of the latest update
* `activated_at` - Timestamp when the workflow was activated
* `finalized_at` - Timestamp when the workflow was finalized
* `last_paused_at` - Timestamp of the latest pause
* `last_resumed_at` - Timestamp of the latest resume
Coupon sets available in the store. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the coupon set
* `name` - Display name of the coupon set
* `code` - Coupon code; only present for coupon sets of type single
* `type` - Coupon set type: `single` (one shared code) or `multiple` (one code per contact)
* `available` - Number of coupons still available; null for single coupon sets
* `assigned` - Number of coupons already assigned to contacts (updated in batches)
* `quantity` - Total quantity of coupons in the set; null for single coupon sets
* `created_at` - Timestamp when the coupon set was created
* `updated_at` - Timestamp of the last update
* `used_in_workflows` - Workflows that use this coupon set (JSON string)
Contact field definitions (standard and custom). **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the contact field
* `name` - Name of the contact field
* `type` - Data type (`text`, `integer`, `decimal`, `boolean`, `date`, `radio`, `checkbox`, `phone`, `email`)
* `required` - Whether every contact must have a value for this field
* `read_only` - Whether the field value cannot be modified
* `custom` - Whether the field is a custom field created by the store members
* `status` - Processing status of the field
* `default_value` - Default value of the field
* `max_length` - Maximum length allowed; only for text fields
* `default_country_code` - Default country code; only for phone fields
* `min_date` / `max_date` - Valid date range; only for date fields
* `min_value` / `max_value` - Valid value range; only for integer or decimal fields
* `options` - Allowed options for checkbox and radio fields (JSON string)
Custom integrations configured in the store. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the custom integration
* `name` - Display name of the custom integration
* `description` - Description of the custom integration
* `created_at` - Timestamp when the custom integration was created
* `color` - Hexadecimal color assigned in the Connectif UI
* `image_uri` - Image URI; null when no custom image is set
* `events_enabled` - Whether events are enabled for this integration
Custom event types defined under each custom integration. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the custom event type
* `custom_integration_id` - Identifier of the parent custom integration
* `name` - Display name of the custom event type
* `alias` - Alias of the custom event type
* `icon` - Icon of the custom event type
* `fields` - Field definitions of the event type (JSON string)
* `category` - Category of the custom event type
* `created_at` - Timestamp when the event type was created
* `updated_at` - Timestamp of the last update
* `webhook_url` - URL of the associated webhook; only when category is send
* `webhook_headers` - Headers sent with the webhook (JSON string); only when category is send
* `receive_type` - How events are received (`webhook`, `api`, `web/mobile-app`)
All contacts of the store, extracted through a full contacts export. **Sync:** Full table.
Columns are discovered at runtime from the export CSV and mirror the contact fields of your account
(standard fields plus any custom fields), so the exact list depends on your store and the selected
**Export Version**. All columns arrive as text, with names normalized to snake\_case.
Contact activities (events), extracted incrementally in date windows of up to 31 days. **Sync:** Incremental. **Replication key:** `export_window_end`.
Columns are discovered at runtime from the export CSV and depend on your account and the selected
**Export Version**; all columns arrive as text. The connector adds the `export_window_end` column (the end
timestamp of the export window that produced the record) as the incremental replication key.
The export can be filtered with the **Activity Types** setting. Available types:
`contact-created`, `email-sent`, `email-sent-alert`, `email-unsubscribed`, `email-opened`,
`email-link-clicked`, `email-bounce`, `web-content-opened`, `web-link-clicked`, `web-form-submitted`,
`purchase`, `contact-identified`, `login`, `register`, `search`, `newsletter-subscribe`,
`product-searched`, `cart-abandoned`, `cart-product-added`, `cart-product-removed`,
`push-notification-opened`, `push-notification-link-clicked`, `push-notification-subscribed`,
`push-notification-unsubscribed`, `push-notification-migrate-subscription`, `push-notification-error`,
`contact-entered-segment`, `contact-exited-segment`, `contact-merged`, `sms-sent`, `sms-sent-alert`,
`sms-bounce`, `sms-unsubscribed`, `facebook-contact-sent`, `coupon-assigned`, `product-visited`,
`web-page-visited`, `contact-added-to-gads-audience`, `contact-removed-from-gads-audience`,
`shared-data-received`.
Product lines that accompany the activities export. **Sync:** Incremental. **Replication key:** `export_window_end`.
When the activities export includes product-level detail (e.g. purchase basket items), Connectif ships it
as a companion products file inside the same export. This stream emits those rows, with columns discovered
at runtime; it yields no records when your account's export contains no products file.
## Skills for agents
Connectif connector documentation as plain markdown, for use in AI agent contexts.
# Conta Simples as a data source
Source: https://docs.nekt.com/sources/conta-simples
Bring data from Conta Simples to Nekt.
Conta Simples is a Brazilian digital banking platform designed for businesses, offering corporate credit cards, banking services, and expense management tools. This connector extracts financial transaction data, card information, user data, and category details from your Conta Simples account into your Nekt Catalog.
## Configuring Conta Simples as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button in the top-right corner, then select Conta Simples from the list of connectors.
Click **Next** and add your access details.
### 1. Add account access
The following configuration fields are available:
* **API Key**: API Key obtained from Conta Simples Internet Banking. Check [Conta Simples developer documentation](https://developers.contasimples.com/br/comece-aqui/quickstart) for instructions on generating your credentials.
* **API Secret**: API Secret obtained from Conta Simples Internet Banking.
* **Start Date**: The earliest date from which statement data will be synced.
After configuring access, click **Next**.
### 2. Select streams
Choose which streams you want to sync. For faster extractions, select only the streams you need.
> Tip: You can search streams by name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how data should appear in your Catalog, including layer, table naming, and sync type.
* **Layer**: Select the destination layer where extracted tables will be created.
* **Table name**: Use the suggested name or customize it. You can also apply a prefix to all tables.
* **Sync Type**: Choose between `INCREMENTAL` and `FULL_TABLE`.
When you're done, click **Next**.
### 4. Configure data source
Add a clear description for this source so your team can identify it easily.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), choose how often data should be extracted (for example, daily, weekly, or at fixed times).
Optionally, configure additional settings:
* Delta Log Retention, which controls how long old table states are kept. Learn more [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync), to periodically complement incremental runs with a complete refresh.
When you are ready, click **Next** to finalize setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, trigger a run manually from the source actions.
To see data in the [Catalog](https://app.nekt.ai/catalog), at least one source run must complete successfully.
# Streams and Fields
Below are the currently available streams in the Conta Simples connector and their main fields.
Credit card transaction records from the `/statements/v1/credit-card` endpoint. Supports incremental sync via `transactionDate`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique transaction identifier. |
| `operation` | String | Transaction direction: `CASH_IN` or `CASH_OUT`. |
| `transactionDate` | DateTime | Date and time of the transaction. |
| `status` | String | Transaction status: `PENDING`, `PROCESSED`, or `CANCELED`. |
| `isCanceled` | Boolean | Whether the transaction was canceled. |
| `type` | String | Transaction type (e.g., `PURCHASE`, `REFUND`, `IOF`). |
| `merchant` | String | Name of the merchant or establishment. |
| `amountBrl` | Number | Transaction amount in BRL. |
| `exchangeRateUsd` | Number | USD exchange rate (for international transactions). |
| `card_id` | String | Card identifier. |
| `card_maskedNumber` | String | Masked card number (e.g., `**** **** **** 1234`). |
| `card_responsibleName` | String | Name of the cardholder. |
| `card_responsibleEmail` | String | Email of the cardholder. |
| `card_type` | String | Card type: `VIRTUAL` or `PHYSICAL`. |
| `category_id` | String | Category identifier. |
| `category_name` | String | Category name. |
| `costCenter_id` | String | Cost center identifier. |
| `costCenter_name` | String | Cost center name. |
| `isConciled` | Boolean | Whether the transaction has been reconciled. |
| `installment` | Integer | Installment number. |
| `notes` | String | User-added notes. |
| `attachments` | String | JSON-encoded list of attached files. |
Banking account transaction records from the `/statements/v1/banking` endpoint. Supports incremental sync via `transactionDate`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique transaction identifier. |
| `companyId` | String | Company identifier. |
| `status` | Integer | Transaction status code (1=Cancelled, 2=Processed, 3=Pending). |
| `statusDescription` | String | Human-readable status description. |
| `accountId` | Integer | Bank account identifier. |
| `transactionDate` | DateTime | Date and time of the transaction. |
| `description` | String | Transaction description. |
| `notes` | String | User-added notes. |
| `brlAmount` | Number | Amount in BRL. |
| `usdAmount` | Number | Amount in USD. |
| `usdExchangeRate` | Number | USD exchange rate. |
| `totalTransactionAmount` | Number | Total transaction amount. |
| `iofAmount` | Number | IOF tax amount. |
| `feeServiceAmount` | Number | Service fee amount. |
| `mccCode` | Integer | Merchant Category Code. |
| `mccDescription` | String | MCC description. |
| `sourceDestinationName` | String | Name of the source or destination. |
| `placeEstablishment` | String | Establishment location. |
| `transactionType_id` | Integer | Transaction type identifier. |
| `transactionType_description` | String | Transaction type description. |
| `category_id` | String | Category identifier. |
| `category_description` | String | Category description. |
| `costCenter_id` | String | Cost center identifier. |
| `costCenter_description` | String | Cost center description. |
| `user_id` | String | User identifier. |
| `user_email` | String | User email. |
| `conciliation_conciled` | Boolean | Whether the transaction has been reconciled. |
| `customCategory_id` | Integer | Custom category identifier. |
| `customCategory_name` | String | Custom category name. |
| `attachments` | String | JSON-encoded list of attached files. |
Credit card listing from the `/credit-cards/v1/cards` endpoint. Supports incremental sync via `updatedAt`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique card identifier. |
| `maskedNumber` | String | Masked card number (e.g., `**** **** **** 1234`). |
| `cardHolderName` | String | Cardholder name. |
| `responsible_name` | String | Name of the responsible person. |
| `responsible_email` | String | Email of the responsible person. |
| `type` | String | Card type: `PHYSICAL` or `VIRTUAL`. |
| `createdAt` | DateTime | Card creation date. |
| `updatedAt` | DateTime | Last update date. |
| `status` | String | Card status: `ACTIVATED`, `BLOCKED`, `CANCELLED`, or `INACTIVATED`. |
| `expirationDate` | DateTime | Card expiration date. |
| `name` | String | Assigned card name. |
| `purpose` | String | Card purpose (e.g., `FREE`, `TRIP`, `ADS`, `SOFTWARE`). |
| `costCenter_id` | String | Cost center identifier. |
| `costCenter_name` | String | Cost center name. |
Expense categories from the `/categories/v1/categories` endpoint. Supports incremental sync via `updatedAt`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Category identifier. |
| `createdAt` | DateTime | Creation date. |
| `updatedAt` | DateTime | Last update date. |
| `companyId` | String | Company identifier. |
| `name` | String | Category name. |
| `type_id` | Integer | Category type identifier. |
| `type_type` | String | Category type: `CUSTOM` or `DEFAULT`. |
| `establishments` | String | JSON-encoded list of linked establishments. |
User accounts from the `/users/v1/users` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | User identifier. |
| `email` | String | User email. |
| `name` | String | User name. |
| `phone` | String | Phone number. |
| `active` | Boolean | Whether the user is active. |
| `role_id` | String | Role identifier. |
| `role_name` | String | Role name. |
User roles from the `/users/v1/roles` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Role identifier. |
| `name` | String | Role name. |
Pending user invitations from the `/users/v1/invites` endpoint.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Invite identifier. |
| `email` | String | Invited user email. |
| `role_id` | String | Assigned role identifier. |
| `role_name` | String | Assigned role name. |
| `status` | String | Invite status. |
| `createdAt` | DateTime | Invite creation date. |
## Implementation Notes
### Date Windowing
The Conta Simples API enforces a maximum of 62 days per query for statement endpoints. The connector automatically splits larger date ranges into 62-day windows and paginates within each window.
### Authentication
This connector uses OAuth2 client credentials flow. Tokens are automatically refreshed every 30 minutes.
# ContaAzul as a data source
Source: https://docs.nekt.com/sources/contaazul
Bring data from ContaAzul to Nekt.
ContaAzul is a financial management platform designed for small businesses in Brazil. It provides tools for invoicing, expense tracking, cash flow management, and integration with banks and accountants to simplify business finances. The ContaAzul API enables access to your financial data including payables, receivables, sales, products, services, and contacts.
![]()
## Configuring ContaAzul as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the ContaAzul option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your ContaAzul data. Click on the **ContaAzul Authorization** button and log in with your ContaAzul account. Grant the necessary permissions for the company you want to extract data from.
The following configurations are available:
* **Start Date**: The earliest date from which records will be synced. Used as the starting point for incremental streams and as a date filter for financial events.
Make sure your ContaAzul account has the necessary permissions to access the data you want to extract. The connector requires access to financial, sales, product, and contact APIs depending on which streams you select.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Despesas (Payables) | `despesas` | Accounts payable / expense records. |
| Receitas (Receivables) | `receitas` | Accounts receivable / revenue records. |
| Parcelas (Installments) | `parcelas` | Detailed installment data for both payables and receivables. |
| Contas Financeiras (Financial Accounts) | `contas_financeiras` | Bank accounts and financial accounts. |
| Categorias (Categories) | `categorias` | Financial categories. |
| Categorias DRE (Income Statement Categories) | `categorias_dre` | Hierarchical income statement categories. |
| Centros de Custos (Cost Centers) | `centros_de_custos` | Cost centers. |
| Produtos (Products) | `produtos` | Product catalog with inventory. |
| Servico (Services) | `servico` | Service catalog. |
| Pessoas Lista (People List) | `pessoas_lista` | List of contacts. |
| Pessoas Detalhes (People Details) | `pessoas_detalhes` | Detailed contact information. |
| Suppliers | `suppliers` | Supplier records. |
| Vendas Lista (Sales List) | `vendas_lista` | List of sales with summary information. |
| Vendas Detalhes (Sales Details) | `vendas_detalhes` | Full sale details. |
| Vendas Itens (Sales Items) | `vendas_itens` | Individual line items for each sale. |
## Fields by stream
Below you'll find all available data streams from ContaAzul and their corresponding fields:
Accounts payable / expense records with status, amounts, and due dates.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `descricao` | String | Description of the expense |
| `data_vencimento` | Date | Due date |
| `status` | String | Payment status |
| `total` | Number | Total amount |
| `nao_pago` | Number | Unpaid amount |
| `pago` | Number | Paid amount |
| `data_criacao` | DateTime | Creation date |
| `data_alteracao` | DateTime | Last modification date (replication key) |
Accounts receivable / revenue records with status, amounts, and due dates.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `descricao` | String | Description of the receivable |
| `data_vencimento` | Date | Due date |
| `status` | String | Payment status |
| `total` | Number | Total amount |
| `nao_pago` | Number | Unpaid amount |
| `pago` | Number | Paid amount |
| `data_criacao` | DateTime | Creation date |
| `data_alteracao` | DateTime | Last modification date (replication key) |
Detailed installment data for both payables (despesas) and receivables (receitas). Includes payment breakdowns, financial account info, billing requests, and attachments. This is a child stream fetched for each parent expense or receivable record.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Installment identifier |
| `evento` | Object | Parent financial event with competence date, payment conditions, reference, apportionment details, and type |
| `versao` | Integer | Record version |
| `referencia` | String | Reference code |
| `indice` | Integer | Installment index |
| `conciliado` | Boolean | Whether the installment is reconciled |
| `status` | String | Installment status |
| `valor_pago` | Number | Amount paid |
| `perda` | Object | Loss details with date and value |
| `nao_pago` | Number | Unpaid amount |
| `data_vencimento` | String | Due date |
| `data_pagamento_previsto` | String | Expected payment date |
| `descricao` | String | Description |
| `nota` | String | Notes |
| `conta_financeira` | Object | Financial account details (bank, name, type, branch, number) |
| `id_conta_financeira` | String | Financial account ID |
| `valor_composicao` | Object | Value breakdown with penalty, interest, gross value, discount, fee, and net value |
| `metodo_pagamento` | String | Payment method |
| `nsu` | String | NSU (unique sequential number) |
| `baixa_agendada` | Boolean | Whether write-off is scheduled |
| `baixas` | Array | Array of write-off records with payment date, value composition, financial account, and attachments |
| `anexos` | Array | Array of attachments with name, description, type, and URL |
| `solicitacoes_cobrancas` | Array | Array of billing requests with status, values, due date, billing notifications, and payment details |
| `fatura` | Object | Invoice details with number, RPS, and invoice type |
| `data_alteracao` | String | Last modification date (replication key) |
| `valor_total_liquido` | Number | Total net value |
| `renegociacao` | Object | Renegotiation details with ID and value |
Bank accounts and financial accounts registered in ContaAzul.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Account identifier |
| `banco` | String | Bank name |
| `codigo_banco` | Integer | Bank code |
| `nome` | String | Account name |
| `ativo` | Boolean | Whether the account is active |
| `tipo` | String | Account type |
| `conta_padrao` | Boolean | Whether this is the default account |
| `possui_config_boleto_bancario` | Boolean | Whether boleto (bank slip) configuration is set up |
| `agencia` | String | Branch number |
| `numero` | String | Account number |
Financial categories used to classify transactions.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Category identifier |
| `versao` | Integer | Record version |
| `nome` | String | Category name |
| `categoria_pai` | String | Parent category ID |
| `tipo` | String | Category type |
| `entrada_dre` | String | Income statement (DRE) entry |
| `considera_custo_dre` | Boolean | Whether it's considered a cost in the income statement |
Hierarchical income statement (DRE) categories with sub-items and linked financial categories.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Category identifier |
| `descricao` | String | Description |
| `codigo` | String | Category code |
| `posicao` | Integer | Position in the statement |
| `indica_totalizador` | Boolean | Whether this is a totalizer row |
| `representa_soma_custo_medio` | Boolean | Whether it represents average cost sum |
| `subitens` | Array | Array of sub-items with their own ID, description, code, position, and linked financial categories |
| `categorias_financeiras` | Array | Array of linked financial categories with ID, code, name, and active status |
Cost centers used to allocate expenses and revenues.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Cost center identifier |
| `codigo` | String | Cost center code |
| `nome` | String | Cost center name |
| `ativo` | Boolean | Whether it is active |
Product catalog with inventory, pricing, and variant information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Product identifier |
| `id_legado` | Integer | Legacy product ID |
| `nome` | String | Product name |
| `codigo_sku` | String | SKU code |
| `codigo_ean` | String | EAN barcode |
| `tipo` | String | Product type |
| `status` | String | Product status |
| `estoque` | Number | Current stock quantity |
| `valor_venda` | Number | Sale price |
| `custo_medio` | Number | Average cost |
| `filhos` | Array | Array of product variants/children with their own ID, name, SKU, stock, pricing, and parent reference |
| `variacao` | Integer | Variation identifier |
| `nivel_estoque` | String | Stock level indicator |
| `estoque_minimo` | Number | Minimum stock threshold |
| `estoque_maximo` | Number | Maximum stock threshold |
| `movimentado` | Boolean | Whether the product has had stock movement |
| `id_pai` | String | Parent product ID (for variants) |
| `integracao_ecommerce_ativa` | Boolean | Whether e-commerce integration is active |
Service catalog with pricing and type information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Service identifier |
| `id_servico` | Integer | Legacy service ID |
| `nome` | String | Service name |
| `codigo` | String | Service code |
| `descricao` | String | Service description |
| `preco` | Number | Service price |
| `custo` | Number | Service cost |
| `status` | String | Service status |
| `tipo_servico` | String | Service type |
List of contacts (customers, suppliers, etc.) with basic information and address.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier |
| `nome` | String | Contact name |
| `documento` | String | Document number (CPF/CNPJ) |
| `inscricao_estadual` | String | State registration number |
| `email` | String | Email address |
| `telefone` | String | Phone number |
| `ativo` | Boolean | Whether the contact is active |
| `id_legado` | Integer | Legacy ID |
| `uuid_legado` | String | Legacy UUID |
| `perfis` | Array | Array of profile types (e.g. customer, supplier) |
| `tipo_pessoa` | String | Person type (individual or company) |
| `endereco` | Object | Address with zip code, street, number, complement, neighborhood, city, state, and country |
Detailed contact information including financial summaries, multiple addresses, additional contacts, tax data, and custom fields. This is a child stream; one detail record is fetched per contact in the People List.
| Field | Type | Description |
| :- | :- | :- |
| `uuid` | String | Unique identifier |
| `nome` | String | Contact name |
| `documento` | String | Document number (CPF/CNPJ) |
| `atrasos_recebimentos` | Number | Overdue receivables amount |
| `recebimentos_mes_atual` | Number | Current month receivables |
| `atrasos_pagamentos` | Number | Overdue payables amount |
| `pagamentos_mes_atual` | Number | Current month payables |
| `email` | String | Email address |
| `telefone_comercial` | String | Business phone |
| `celular` | String | Mobile phone |
| `criado_em` | DateTime | Creation date |
| `codigo` | String | Contact code |
| `perfis` | Array | Array of profile types |
| `pessoas_legado` | Array | Array of legacy person records with ID, UUID, and profile |
| `enderecos` | Array | Array of addresses with zip code, street, number, complement, neighborhood, city, state, and country |
| `observacoes_gerais` | String | General notes |
| `optante_simples_nacional` | Boolean | Whether opted into Simples Nacional tax regime |
| `orgao_publico` | Boolean | Whether it is a public institution |
| `nome_empresa` | String | Company name |
| `tipo_pessoa` | String | Person type (individual or company) |
| `lembretes_vencimento` | Array | Array of due date reminders with email and status |
| `informacoes_adicionais` | Array | Array of custom fields with label, value, and type |
| `outros_contatos` | Array | Array of additional contacts with name, email, phone, and office |
| `informacoes_fiscais` | Array | Array of tax information with state registration, national registration, and SUFRAMA registration |
Supplier records with contact information and address.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Supplier identifier |
| `name` | String | Supplier name |
| `company_name` | String | Company name |
| `email` | String | Email address |
| `business_phone` | String | Business phone |
| `mobile_phone` | String | Mobile phone |
| `person_type` | String | Person type (NATURAL or LEGAL) |
| `document` | String | Document number (CPF/CNPJ) |
| `identity_document` | String | Identity document (RG) |
| `state_registration_number` | String | State registration number |
| `state_registration_type` | String | State registration type |
| `city_registration_number` | String | City registration number |
| `date_of_birth` | DateTime | Date of birth |
| `notes` | String | Notes |
| `foreignId` | String | Foreign/external ID |
| `contactName` | String | Contact person name |
| `address` | Object | Address with street, number, complement, zip code, neighborhood, city, and state |
List of sales with summary information, customer data, and status.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Sale identifier |
| `total` | Number | Total sale amount |
| `id_legado` | Integer | Legacy sale ID |
| `data` | String | Sale date |
| `criado_em` | DateTime | Creation date |
| `data_alteracao` | DateTime | Last modification date (replication key) |
| `tipo` | String | Sale type |
| `itens` | String | Items summary |
| `condicao_pagamento` | Boolean | Payment condition flag |
| `numero` | Integer | Sale number |
| `cliente` | Object | Customer info with ID, name, email, phone, address, city, state, country, and zip code |
| `situacao` | Object | Status with status code, description, and date |
| `versao` | Integer | Record version |
| `status_email` | Object | Email notification status with status and sent date |
Full sale details including customer, financial event, operation type, payment conditions, value breakdown, and pending status. This is a child stream; one detail record is fetched per sale in the Sales List.
| Field | Type | Description |
| :- | :- | :- |
| `sales_id` | String | Sale identifier (primary key) |
| `data_alteracao_venda` | DateTime | Sale last modification date (replication key) |
| `data_alteracao` | DateTime | Record modification date |
| `cliente` | Object | Customer with UUID, person type, document, and name |
| `evento_financeiro` | Object | Linked financial event with ID |
| `notificacao` | Object | Email notification with reference ID, recipient, sent date, opened date, and status |
| `natureza_operacao` | Object | Operation type with UUID, operation type, template, label, financial impact, and stock change |
| `venda` | Object | Detailed sale data with ID, status, legacy ID, negotiation type, sale number, category ID, commitment date, discount settings, value composition (gross, discount, freight, taxes, insurance, net), payment conditions (type, account, installments, card payment details), item counts, notes, and pending status |
Individual line items for each sale. This is a child stream; items are fetched per sale in the Sales List.
| Field | Type | Description |
| :- | :- | :- |
| `sales_id` | String | Parent sale identifier (part of primary key) |
| `id` | String | Item record identifier (part of primary key) |
| `id_item` | String | Product/service identifier |
| `nome` | String | Item name |
| `descricao` | String | Item description |
| `tipo` | String | Item type (product or service) |
| `quantidade` | Number | Quantity |
| `valor` | Number | Item value |
| `data_alteracao_venda` | DateTime | Sale last modification date (replication key) |
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating ContaAzul data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Cash Flow Overview
Analyze payables vs. receivables to understand your cash flow position.
**Business Value:**
* Monitor outstanding payables and receivables
* Identify overdue payments
* Track payment efficiency
```sql theme={null}
WITH payables AS (
SELECT
'PAYABLE' AS type,
status,
COUNT(*) AS record_count,
SUM(total) AS total_amount,
SUM(nao_pago) AS unpaid_amount,
SUM(pago) AS paid_amount
FROM
nekt_raw.contaazul_despesas
GROUP BY
status
),
receivables AS (
SELECT
'RECEIVABLE' AS type,
status,
COUNT(*) AS record_count,
SUM(total) AS total_amount,
SUM(nao_pago) AS unpaid_amount,
SUM(pago) AS paid_amount
FROM
nekt_raw.contaazul_receitas
GROUP BY
status
)
SELECT * FROM payables
UNION ALL
SELECT * FROM receivables
ORDER BY
type, record_count DESC
```
```sql theme={null}
WITH payables AS (
SELECT
'PAYABLE' AS type,
status,
COUNT(*) AS record_count,
SUM(total) AS total_amount,
SUM(nao_pago) AS unpaid_amount,
SUM(pago) AS paid_amount
FROM
`nekt_raw.contaazul_despesas`
GROUP BY
status
),
receivables AS (
SELECT
'RECEIVABLE' AS type,
status,
COUNT(*) AS record_count,
SUM(total) AS total_amount,
SUM(nao_pago) AS unpaid_amount,
SUM(pago) AS paid_amount
FROM
`nekt_raw.contaazul_receitas`
GROUP BY
status
)
SELECT * FROM payables
UNION ALL
SELECT * FROM receivables
ORDER BY
type, record_count DESC
```
| type | status | record\_count | total\_amount | unpaid\_amount | paid\_amount |
| - | - | - | - | - | - |
| PAYABLE | PAGO | 456 | 234,567.89 | 0.00 | 234,567.89 |
| PAYABLE | PENDENTE | 89 | 45,230.00 | 45,230.00 | 0.00 |
| PAYABLE | VENCIDO | 12 | 8,900.00 | 8,900.00 | 0.00 |
| RECEIVABLE | PAGO | 780 | 567,890.12 | 0.00 | 567,890.12 |
| RECEIVABLE | PENDENTE | 134 | 98,450.00 | 98,450.00 | 0.00 |
| RECEIVABLE | VENCIDO | 23 | 15,670.00 | 15,670.00 | 0.00 |
### 2. Sales Performance by Customer
Analyze sales volume and revenue by customer to identify top accounts.
**Business Value:**
* Identify highest-value customers
* Track sales trends by customer
* Support customer segmentation
```sql theme={null}
SELECT
v.cliente.nome AS customer_name,
v.cliente.email AS customer_email,
COUNT(*) AS total_sales,
SUM(v.total) AS total_revenue,
ROUND(AVG(v.total), 2) AS avg_sale_value,
MIN(v.data) AS first_sale,
MAX(v.data) AS last_sale
FROM
nekt_raw.contaazul_vendas_lista v
WHERE
v.situacao.status != 'CANCELADO'
GROUP BY
v.cliente.nome, v.cliente.email
ORDER BY
total_revenue DESC
LIMIT 20
```
```sql theme={null}
SELECT
v.cliente.nome AS customer_name,
v.cliente.email AS customer_email,
COUNT(*) AS total_sales,
SUM(v.total) AS total_revenue,
ROUND(AVG(v.total), 2) AS avg_sale_value,
MIN(v.data) AS first_sale,
MAX(v.data) AS last_sale
FROM
`nekt_raw.contaazul_vendas_lista` v
WHERE
v.situacao.status != 'CANCELADO'
GROUP BY
v.cliente.nome, v.cliente.email
ORDER BY
total_revenue DESC
LIMIT 20
```
| customer\_name | customer\_email | total\_sales | total\_revenue | avg\_sale\_value | first\_sale | last\_sale |
| - | - | - | - | - | - | - |
| Empresa ABC Ltda | [contato@abc.com.br](mailto:contato@abc.com.br) | 45 | 123,456.78 | 2,743.48 | 2025-01-15 | 2026-03-08 |
| Distribuidora XYZ | [vendas@xyz.com.br](mailto:vendas@xyz.com.br) | 32 | 89,234.50 | 2,788.58 | 2025-03-20 | 2026-03-10 |
| Comércio Silva | [silva@email.com](mailto:silva@email.com) | 28 | 67,890.00 | 2,424.64 | 2025-02-01 | 2026-02-28 |
| Tech Solutions SA | [compras@tech.com.br](mailto:compras@tech.com.br) | 19 | 45,670.00 | 2,403.68 | 2025-06-10 | 2026-03-05 |
### 3. Product Inventory Status
Monitor product stock levels and identify items that need replenishment.
**Business Value:**
* Identify low-stock products before they run out
* Track inventory turnover
* Optimize purchasing decisions
```sql theme={null}
SELECT
nome AS product_name,
codigo_sku AS sku,
status,
estoque AS current_stock,
estoque_minimo AS min_stock,
estoque_maximo AS max_stock,
valor_venda AS sale_price,
custo_medio AS avg_cost,
CASE
WHEN estoque <= estoque_minimo THEN 'CRITICAL'
WHEN estoque <= estoque_minimo * 1.5 THEN 'LOW'
ELSE 'OK'
END AS stock_status
FROM
nekt_raw.contaazul_produtos
WHERE
status = 'ATIVO'
AND estoque_minimo IS NOT NULL
ORDER BY
estoque ASC
```
```sql theme={null}
SELECT
nome AS product_name,
codigo_sku AS sku,
status,
estoque AS current_stock,
estoque_minimo AS min_stock,
estoque_maximo AS max_stock,
valor_venda AS sale_price,
custo_medio AS avg_cost,
CASE
WHEN estoque <= estoque_minimo THEN 'CRITICAL'
WHEN estoque <= estoque_minimo * 1.5 THEN 'LOW'
ELSE 'OK'
END AS stock_status
FROM
`nekt_raw.contaazul_produtos`
WHERE
status = 'ATIVO'
AND estoque_minimo IS NOT NULL
ORDER BY
estoque ASC
```
| product\_name | sku | status | current\_stock | min\_stock | max\_stock | sale\_price | avg\_cost | stock\_status |
| - | - | - | - | - | - | - | - | - |
| Cabo USB-C 1m | SKU-001 | ATIVO | 5 | 10 | 100 | 29.90 | 12.50 | CRITICAL |
| Carregador 20W | SKU-042 | ATIVO | 12 | 10 | 80 | 89.90 | 38.00 | LOW |
| Película Protetora | SKU-103 | ATIVO | 8 | 15 | 200 | 19.90 | 5.20 | CRITICAL |
| Mouse Sem Fio | SKU-078 | ATIVO | 45 | 20 | 150 | 79.90 | 32.00 | OK |
## Implementation Notes
Several streams follow a parent-child pattern where a list endpoint provides summary records, and a detail endpoint is called for each individual record:
* **Pessoas Lista → Pessoas Detalhes**: The people list provides basic contact info; the detail stream fetches full contact data per person.
* **Vendas Lista → Vendas Detalhes**: The sales list provides summary data; the detail stream fetches full sale info per sale.
* **Vendas Lista → Vendas Itens**: Line items are fetched per sale from the sales list. This stream is fully paginated to ensure all line items are captured correctly.
* **Despesas → Parcelas**: Installment details are fetched for each expense.
* **Receitas → Parcelas**: Installment details are fetched for each receivable.
The connector supports incremental sync for:
* **Despesas**: Uses `data_alteracao` as the replication key
* **Receitas**: Uses `data_alteracao` as the replication key
* **Vendas Lista / Detalhes / Itens**: Uses `data_alteracao` as the replication key
Streams without a replication key (Categorias, Categorias DRE, Centros de Custos, Contas Financeiras, Produtos, Pessoas, Servico, Suppliers) are always synced as full table.
The ContaAzul API enforces rate limits. The connector automatically handles `429 Too Many Requests` responses by waiting 60 seconds before retrying. If you experience frequent rate limit errors, consider reducing the number of streams synced concurrently or adjusting your trigger frequency.
The connector automatically manages OAuth access token refresh to prevent authentication failures:
* Access tokens are refreshed proactively 5 minutes before expiration to avoid race conditions
* Refresh tokens are stored securely and rotated automatically
* Basic authentication headers are derived from your client ID and secret
ContaAzul data contains nested structures. When querying:
* Use dot notation to access nested fields (e.g., `cliente.nome` in the sales list)
* Use `UNNEST` to flatten arrays (e.g., `parcelas` in sales details, `filhos` in products)
Example for product variants:
```sql theme={null}
-- AWS Athena
SELECT p.nome, v.nome AS variant_name, v.estoque, v.valor_venda
FROM contaazul_produtos p
CROSS JOIN UNNEST(p.filhos) AS t(v)
-- GCP BigQuery
SELECT p.nome, v.nome AS variant_name, v.estoque, v.valor_venda
FROM `contaazul_produtos` p, UNNEST(p.filhos) AS v
```
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `429 Too Many Requests` | You hit Conta Azul's rate limit. | Extraction waits and retries on its own. If it is recurrent, reduce concurrency by scheduling sources that share the account at different times. |
| A listing returns empty | Some listings require the date range to be filled in the stream settings. | Fill in the date range on the stream. |
| Permission error on a module | The application does not have access to that module on the account. | Enable the module in the application's permissions in Conta Azul and re-authorize. |
## Skills for agents
ContaAzul connector documentation as plain markdown, for use in AI agent contexts.
# Convenia as a data source
Source: https://docs.nekt.com/sources/convenia
Bring data from Convenia to Nekt.
Convenia is an HR management platform designed for Brazilian companies. It offers tools for managing employee records, payroll, benefits, vacations, and HR processes, helping organizations streamline their human resources operations with eSocial compliance.
![]()
## Configuring Convenia as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Convenia option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an API Token to connect to Convenia:
* **API Token**: Your Convenia API access token. You can generate this in your Convenia account settings under the API integrations section.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. The available streams include employees, employee details, dismissed employees, dismissed employee details, vacation solicitations, and salary history.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Employees | `employees` | Main employee list with comprehensive employee data. |
| Employee Details | `employee_details` | Extended employee information (child of Employees stream). |
| Dismissed Employees | `dismissed_employees` | Terminated employees with dismissal details. |
| Dismissed Employee Details | `dismissed_employee_details` | Extended information for terminated employees (child of Dismissed Employees stream). |
| Employee Vacations Solicitations | `employee_vacations_solicitations` | Employee vacation requests and approvals. |
| Employee Salary History | `employees_salary_history` | Salary change history. |
## Fields by stream
Below you'll find all available data streams from Convenia and their corresponding fields:
Stream containing the main employee list with comprehensive employee data.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique employee identifier |
| `name` | String | First name |
| `last_name` | String | Last name |
| `social_name` | String | Social/preferred name |
| `email` | String | Corporate email |
| `status` | String | Employee status |
| `registration` | String | Employee registration number |
| `hiring_date` | String | Hiring date |
| `birth_date` | String | Birth date |
| `salary` | String | Current salary |
| `document` | Object | Main documents (`cpf`, `pis`) |
| `cpf` | Object | CPF details (`id`, `cpf`) |
| `rg` | Object | Identity document details (`number`, `emission_date`, etc.) |
| `ctps` | Object | Work card details (`number`, `serial_number`, etc.) |
| `driver_license` | Object | Driver's license details (`category`, `validate_date`, etc.) |
| `electoral_card` | Object | Voter registration details (`section`, `zone`, etc.) |
| `reservist` | Object | Military service details |
| `address` | Object | Employee address details |
| `department` | Object | Department (`id`, `name`) |
| `cost_center` | Object | Cost center (`id`, `name`) |
| `job` | Object | Job position (`id`, `name`) |
| `supervisor` | Object | Direct supervisor (`id`, `name`, `last_name`) |
| `contact_information` | Object | Contact details (phones, personal email) |
| `bank_accounts` | Array | Array of bank accounts (bank, agency, account, pix) |
| `emergency_contacts` | Array | Array of emergency contacts |
| `intern` | Object | Internship information |
| `educations` | Array | Array of education records |
| `nationalities` | Array | Array of nationalities |
| `foreign` | Object | Foreign employee information |
| `disability` | Object | Disability information |
| `experience_period` | Object | Trial period info |
| `aso` | Array | Array of occupational health exams |
| `annotations` | Array | Array of employee annotations |
Stream with extended employee information (child of Employees stream).
| Field | Type | Description |
| :- | :- | :- |
| `mother_name` | String | Mother's name |
| `father_name` | String | Father's name |
| `gender` | String | Gender |
| `gender_identity_id` | String | Gender identity ID |
| `gender_identity` | Object | Gender identity (`id`, `name`) |
| `ethnicity_id` | String | Ethnicity ID |
| `ethnicity` | Object | Ethnicity (`id`, `name`) |
| `marital_status_id` | String | Marital status ID |
| `natural_from_state_uf` | String | Birth state |
| `natural_from_city_name` | String | Birth city |
| `first_job` | Boolean | Whether this is their first job |
| `alternative_email` | String | Alternative email |
| `phone` | String | Phone number |
| `cellphone` | String | Mobile phone |
| `relationship_id` | String | Employment relationship ID |
| `relationship` | Object | Employment relationship (`id`, `name`) |
| `salary_type` | Object | Salary type (`id`, `name`) |
| `team` | Object | Team assignment (`id`, `name`) |
| `time_tracking` | Boolean | Whether time tracking is enabled |
| `work_period` | Object | Work period info |
| `dismissal` | Object | Dismissal info if applicable (`id`, `date`) |
| `documents` | Object | Consolidated document information |
| `benefits` | Array | Array of employee benefits |
| `custom_fields` | Array | Array of custom fields |
*Plus all fields from the Employees stream.*
Stream containing terminated employees with dismissal details.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Employee ID |
| `corporate_email` | String | Corporate email |
| `dismissal` | Object | Dismissal details (`date`, `type`, `termination_notice`, `motive`, etc.) |
Stream with extended information for terminated employees (child of Dismissed Employees stream).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the dismissed employee in Convenia |
| `name` | String | First name of the dismissed employee |
| `last_name` | String | Last name of the dismissed employee |
| `birth_date` | Date | Date of birth of the dismissed employee |
| `hiring_date` | Date | Date the employee was hired, i.e. the start of the employment contract |
| `gender` | String | Gender recorded for the employee |
| `marital_status_id` | String | Identifier of the employee's marital status in Convenia |
| `time_tracking` | Boolean | Whether the employee was subject to time tracking |
| `department` | Object | Department the employee belonged to at the time of dismissal |
| `cost_center` | Object | Cost center the employee was allocated to at the time of dismissal |
| `job` | Object | Job position the employee held at the time of dismissal |
| `team` | Object | Team the employee belonged to at the time of dismissal |
| `supervisor` | Object | Employee who supervised the dismissed employee |
| `dismissal` | Object | Summary of the dismissal process, matching the parent record |
| `relationship_id` | Integer | Identifier of the employment relationship type |
| `relationship` | Object | Type of employment relationship the employee had with the company |
| `ethnicity_id` | Integer | Identifier of the employee's ethnicity |
| `ethnicity` | Object | Ethnicity declared by the employee |
| `gender_identity_id` | Integer | Identifier of the employee's gender identity |
| `gender_identity` | Object | Gender identity declared by the employee |
| `intern` | Object | Internship details (includes dates, institution details, and `internship_supervisor`) |
| `work_period` | Object | Working period configuration of the employee |
Stream for vacation requests and approvals (child of Employees stream).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Solicitation ID |
| `employee_id` | String | Employee ID |
| `period_id` | String | Vacation period ID |
| `collective_id` | String | Collective vacation ID (if applicable) |
| `justification` | String | Request justification |
| `status` | String | Request status |
| `starts_at` | String | Vacation start date |
| `finishes_at` | String | Vacation end date |
| `requested_days` | Number | Number of requested days |
| `spent_balance` | Number | Balance spent |
| `paid_leave` | Number | Paid leave days |
| `allowance` | Boolean | Vacation allowance |
| `thirteenth` | Boolean | Include 13th salary advance |
| `available_steps` | Object | Available workflow steps |
| `supervisor_approvals` | Array | Array of supervisor approvals |
| `hr_approvals` | Array | Array of HR approvals |
| `sent_to_accountant` | Boolean | Sent to accountant flag |
| `accounting_started_at` | String | Accounting start date |
| `hr_approvals_started_at` | String | HR approval start date |
| `supervisor_approvals_started_at` | String | Supervisor approval start date |
| `signature_started_at` | String | Signature start date |
| `files` | Array | Array of attached files |
| `canceled` | Object | Cancellation info |
| `created_at` | String | Request creation date |
| `updated_at` | String | Last update date |
Stream for salary change history (child of Employees stream).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Salary history record ID |
| `salary` | Number | Salary value |
| `description` | String | Change description |
| `is_active` | Boolean | Whether this is the active salary |
| `date_from` | String | Effective start date |
| `date_to` | String | Effective end date |
| `relationship_id` | String | Employment relationship ID |
| `relationship` | Object | Employment relationship (`id`, `name`) |
| `department_id` | String | Department ID |
| `department` | Object | Department (`id`, `name`) |
| `team_id` | String | Team ID |
| `team` | Object | Team (`id`, `name`) |
| `job_description_id` | String | Job description ID |
| `job` | Object | Job position (`id`, `name`) |
| `cost_center_id` | String | Cost center ID |
| `cost_center` | Object | Cost center (`id`, `name`) |
| `motive_id` | String | Change motive ID |
| `motive` | Object | Change motive (`id`, `name`) |
| `created_at` | String | Record creation date |
| `updated_at` | String | Last update date |
## Data Model
The following diagram illustrates the relationships between the Convenia data streams:
```mermaid theme={null}
graph TD;
subgraph "Employee Data"
Employees("Employees");
EmployeeDetails("Employee Details");
DismissedEmployees("Dismissed Employees");
DismissedEmployeeDetails("Dismissed Employee Details");
end
subgraph "HR Processes"
VacationsSolicitations("Vacation Solicitations");
SalaryHistory("Salary History");
end
Employees -- "id" --> EmployeeDetails;
Employees -- "id" --> VacationsSolicitations;
Employees -- "id" --> SalaryHistory;
DismissedEmployees -- "id" --> Employees;
DismissedEmployees -- "id" --> DismissedEmployeeDetails;
```
## Notes
* **Employee Details** is a child stream of **Employees** and provides extended information for each employee
* **Vacation Solicitations** and **Salary History** are also child streams, fetched per employee
* **Dismissed Employees** contains only terminated employees with their dismissal details
* **Dismissed Employee Details** is a child stream of **Dismissed Employees** and provides extended information for each terminated employee. Some employees might not have detailed records returned by the API; in those cases, they are skipped in the detailed stream but remain in the parent stream
* All dates follow Brazilian format conventions
## Skills for agents
Convenia connector documentation as plain markdown, for use in AI agent contexts.
# Convert as a data source
Source: https://docs.nekt.com/sources/convert
Bring data from Convert to Nekt.
Convert is an A/B testing and personalization platform that helps businesses optimize their websites and improve conversion rates. It provides tools for creating and running experiments, analyzing results, and delivering personalized experiences to users.
## Configuring Convert as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Convert option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Convert API credentials for this connection.
* **Key ID**: The API key ID used to authenticate against the API service. Check [Convert's documentation](https://api.convert.com/doc/v2/#tag/API-KEY-Authentication) for more details.
* **Key Secret**: The API key secret used to authenticate against the API service.
* **Account ID**: The account ID to sync.
* **Project ID**: The project ID to sync.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Convert and their corresponding fields:
Stream for fetching a list of experiences.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | The unique identifier for the experience. |
| `name` | String | The name of the experience. |
| `description` | String | The description of the experience. |
| `objective` | String | The objective of the experience. |
| `type` | String | The type of the experience. |
| `status` | String | The status of the experience. |
| `start_time` | Integer | The start time of the experience. |
| `end_time` | Integer | The end time of the experience. |
| `global_js` | String | The global JavaScript for the experience. |
| `global_css` | String | The global CSS for the experience. |
| `url` | String | The URL of the experience. |
| `environment` | String | The environment of the experience. |
| `project` | Integer | The project ID associated with the experience. |
| `primary_goal` | Integer | The primary goal ID for the experience. |
| `traffic_distribution` | Number | The traffic distribution for the experience. |
| `settings` | Object | The settings for the experience. |
| `version` | Integer | The version of the experience. |
| `integrations` | Array of Objects | The integrations for the experience. |
| `key` | String | The key for the experience. |
| `screenshots_info` | Array of Objects | Information about screenshots for the experience. |
| `site_area` | String | The site area for the experience. |
| `concurrency_key` | String | The concurrency key for the experience. |
Stream for fetching experience details.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | The unique identifier for the experience. |
| `name` | String | The name of the experience. |
| `description` | String | The description of the experience. |
| `objective` | String | The objective of the experience. |
| `type` | String | The type of the experience. |
| `status` | String | The status of the experience. |
| `start_time` | Integer | The start time of the experience. |
| `end_time` | Integer | The end time of the experience. |
| `global_js` | String | The global JavaScript for the experience. |
| `global_css` | String | The global CSS for the experience. |
| `url` | String | The URL of the experience. |
| `environment` | String | The environment of the experience. |
| `project` | Object | The project associated with the experience. |
| `primary_goal` | Integer | The primary goal ID for the experience. |
| `traffic_distribution` | Number | The traffic distribution for the experience. |
| `settings` | Object | The settings for the experience. |
| `version` | Integer | The version of the experience. |
| `integrations` | Array of Objects | The integrations for the experience. |
| `key` | String | The key for the experience. |
| `screenshots_info` | Array of Objects | Information about screenshots for the experience. |
| `site_area` | String | The site area for the experience. |
| `concurrency_key` | String | The concurrency key for the experience. |
| `stats` | Object | The statistics for the experience. |
| `min_running_timestamp` | Integer | The minimum running timestamp for the experience. |
| `max_running_timestamp` | Integer | The maximum running timestamp for the experience. |
| `variations` | Array of Objects | The variations for the experience. |
| `goals` | Array of Objects | The goals for the experience. |
| `locations` | Array of Objects | The locations for the experience. |
| `tags` | Array of Strings | The tags for the experience. |
# Data Model
The following diagram illustrates the relationship between the core data streams in Convert. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
erDiagram
EXPERIENCE_LIST {
integer id PK
integer project
string name
}
EXPERIENCE_DETAILS {
integer id PK
string name
object project
}
EXPERIENCE_LIST ||--|| EXPERIENCE_DETAILS : "id"
```
## Skills for agents
Convert connector documentation as plain markdown, for use in AI agent contexts.
# Corp as a data source
Source: https://docs.nekt.com/sources/corp
Bring data from Corp to Nekt.
The CORP connector enables seamless data extraction from the CORP platform directly into your Nekt Lakehouse, simplifying the process of importing critical insurance-related data. This integration ensures that you have up-to-date information from CORP ready for analysis.
## Configuring Corp as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Corp** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Corp credentials to authorize Nekt to access your data.
The following configurations are available:
* **User**: Your user login for accessing the Corp system.
* **Password**: Your user password for accessing the Corp system.
* **Initial sync date**: Starting point for data extraction. Only records created or updated after this date will be synced. This will help to avoid processing unnecessary data.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings (if available):
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### 5. Check your new source
Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Corp connector documentation as plain markdown, for use in AI agent contexts.
# Criteo as a data source
Source: https://docs.nekt.com/sources/criteo
Bring data from Criteo to Nekt.
Criteo is a global technology company that specializes in digital performance marketing. Their platform enables advertisers to run personalized retargeting campaigns across the web, driving conversions and maximizing return on ad spend (ROAS). Criteo's AI-powered solutions help businesses reach customers at the right moment with relevant product recommendations.
![]()
## Configuring Criteo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Criteo option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Criteo API credentials to authenticate.
To connect Criteo to Nekt, you'll need to create an API application in the Criteo Developer Portal and obtain your **Client ID** and **Client Secret**. Follow these steps:
### Prerequisites
Before you begin, ensure you have:
* A Criteo account with advertiser access
* Access to the [Criteo Developer Portal](https://developers.criteo.com/)
* A Partner Account and Organization set up in Criteo
### Step-by-Step Guide
Go to [developers.criteo.com](https://developers.criteo.com/) and sign in with your Criteo account credentials. If you don't have access yet, you'll need to request it from your Criteo account manager.
Once logged in, navigate to the **My Apps** page. This is where you'll create and manage your Criteo API applications.
Click the **Create a New App** button in the top right corner of the page. This will start the application creation wizard.
Provide the following information for your application:
* **App Name**: Give your application a clear, identifiable name (e.g., "Nekt Data Integration")
* **Description**: (Optional) Add a description of the application's purpose
* **App Image**: (Optional) Upload an image for your application
API applications cannot be deleted once created. Make sure to use a clear and identifiable name, as this helps Criteo support teams assist you more effectively.
Choose **Client Credentials** as the authentication method. This is the recommended method for server-to-server integrations like Nekt.
**Client Credentials** authentication is ideal for automated data pipelines because it doesn't require user interaction for each API call. It uses your Client ID and Client Secret to generate access tokens automatically.
Choose the Criteo service your application will interact with:
* Select **C-Growth and Marketing Solutions** for accessing campaign statistics and marketing data
Select the domains (permissions) your application needs. For Nekt integration, you'll need at minimum:
* **Analytics**: Set to **Read** - Required to generate campaign statistics reports
You may also enable:
* **Campaigns**: Set to **Read** if you want to access campaign configuration data
* **Audiences**: Set to **Read** if you want to access audience data
After selecting the appropriate domains, click **Activate App**.
Once activated, you cannot change the app name, description, image, or domain scopes. Review your selections carefully before activating.
After activation, you'll be redirected to your application details page. Here you'll find:
* **Client ID**: Copy this value for the Nekt configuration
* **Client Secret**: Click to reveal and copy this value
Store your Client Secret securely. You won't be able to view it again after leaving this page. If you lose it, you'll need to generate a new one.
You need to authorize your app to access the advertiser accounts you want to sync:
1. On the application details page, go to the top right corner and click **Generate new URL**
2. A consent URL will appear on the page. Copy it using the button on the right
3. **If you have admin access** to the Criteo advertiser account: Open this URL in a new browser tab
4. **If someone else has admin access**: Send them the consent URL
5. When accessing the URL, scroll to the bottom of the page to the **Portfolio access** section
6. Select the advertiser accounts you want to grant access to
7. Click **Approve** to authorize the application
This authorization step is required for the API to access advertiser data. Without it, your API calls will fail even with valid credentials.
For more detailed information about Criteo's API setup, visit the official [Criteo API Documentation](https://developers.criteo.com/marketing-solutions/docs/create-your-app).
The following configurations are required:
* **Client ID**: The client ID from your Criteo API application. See the [Obtaining Your Criteo API Credentials](#obtaining-your-criteo-api-credentials) section above for detailed instructions.
* **Client Secret**: The client secret from your Criteo API application. See the [Obtaining Your Criteo API Credentials](#obtaining-your-criteo-api-credentials) section above for detailed instructions.
* **Advertiser ID**: The unique identifier of the advertiser account you want to sync data from. You can find this in your Criteo Management Center.
* **Currency**: The currency code for reporting (e.g., USD, EUR, BRL). This determines how monetary values are displayed in your reports.
* **Timezone**: The timezone for organizing your data. This affects how daily aggregations are calculated.
* **Start Date**: The earliest date from which records will be synced. Records from this date onward will be extracted.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Criteo and their corresponding fields:
Daily performance report for campaigns, including impressions, clicks, conversions, revenue, and ROAS metrics with various attribution windows.
**Dimensions:**
* `Day` - Date of the statistics
* `AdvertiserId` - Unique identifier of the advertiser
* `Advertiser` - Name of the advertiser
* `AdsetId` - Unique identifier of the ad set
* `Adset` - Name of the ad set
* `CategoryId` - Unique identifier of the category
* `Category` - Name of the category
* `Device` - Device type (Desktop, Mobile, Tablet)
* `Os` - Operating system
* `Currency` - Currency code for monetary values
**Basic Performance Metrics:**
* `Clicks` - Number of clicks
* `Displays` - Number of ad impressions
* `ViewableDisplays` - Number of viewable ad impressions
* `NonViewableDisplays` - Number of non-viewable ad impressions
* `UntrackableDisplays` - Number of untrackable displays
* `AdvertiserCost` - Total advertising spend
* `QualifiedVisits` - Number of qualified visits
* `Visits` - Total number of visits
* `CostPerVisit` - Cost per visit
* `BounceRate` - Bounce rate (0-1)
* `PotentialDisplays` - Number of potential display opportunities
* `OverallCompetitionWin` - Auction win rate
**Sales Metrics (by Attribution Window):**
* `SalesClientAttribution` - Sales with client-defined attribution
* `SalesAllClientAttribution` - All sales with client attribution
* `SalesPc30d` - Sales post-click 30 days
* `SalesAllPc30d` - All sales post-click 30 days
* `SalesPv24h` - Sales post-view 24 hours
* `SalesAllPv24h` - All sales post-view 24 hours
* `SalesPc30dPv24h` - Sales post-click 30 days / post-view 24 hours
* `SalesAllPc30dPv24h` - All sales PC 30d / PV 24h
* `SalesPc1d` - Sales post-click 1 day
* `SalesAllPc1d` - All sales post-click 1 day
* `SalesPc7d` - Sales post-click 7 days
* `SalesAllPc7d` - All sales post-click 7 days
**Revenue Metrics (by Attribution Window):**
* `RevenueGeneratedClientAttribution` - Revenue with client attribution
* `RevenueGeneratedAllClientAttribution` - All revenue with client attribution
* `RevenueGeneratedPc30d` - Revenue post-click 30 days
* `RevenueGeneratedAllPc30d` - All revenue post-click 30 days
* `RevenueGeneratedPv24h` - Revenue post-view 24 hours
* `RevenueGeneratedAllPv24h` - All revenue post-view 24 hours
* `RevenueGeneratedPc30dPv24h` - Revenue PC 30d / PV 24h
* `RevenueGeneratedAllPc30dPv24h` - All revenue PC 30d / PV 24h
* `RevenueGeneratedPc1d` - Revenue post-click 1 day
* `RevenueGeneratedAllPc1d` - All revenue post-click 1 day
* `RevenueGeneratedPc7d` - Revenue post-click 7 days
* `RevenueGeneratedAllPc7d` - All revenue post-click 7 days
**User & Reach Metrics:**
* `ExposedUsers` - Number of users served an ad
* `Audience` - Potential users who could be served an ad
* `Reach` - Share of potential users served an ad
**Average Cart Metrics (by Attribution Window):**
* `AverageCartClientAttribution` - Average cart with client attribution
* `AverageCartAllClientAttribution` - All average cart with client attribution
* `AverageCartPc30d` - Average cart post-click 30 days
* `AverageCartAllPc30d` - All average cart PC 30 days
* `AverageCartPv24h` - Average cart post-view 24 hours
* `AverageCartAllPv24h` - All average cart PV 24 hours
* `AverageCartPc30dPv24h` - Average cart PC 30d / PV 24h
* `AverageCartAllPc30dPv24h` - All average cart PC 30d / PV 24h
* `AverageCartPc1d` - Average cart post-click 1 day
* `AverageCartAllPc1d` - All average cart PC 1 day
* `AverageCartPc7d` - Average cart post-click 7 days
* `AverageCartAllPc7d` - All average cart PC 7 days
**Rate Metrics:**
* `ClickThroughRate` - Click-through rate (CTR)
* `ConversionRateClientAttribution` - Conversion rate with client attribution
* `ConversionRateAllClientAttribution` - All conversion rate with client attribution
* `ConversionRatePc30d` - Conversion rate post-click 30 days
* `ConversionRateAllPc30d` - All conversion rate PC 30 days
* `ConversionRatePv24h` - Conversion rate post-view 24 hours
* `ConversionRateAllPv24h` - All conversion rate PV 24 hours
* `ConversionRatePc30dPv24h` - Conversion rate PC 30d / PV 24h
* `ConversionRateAllPc30dPv24h` - All conversion rate PC 30d / PV 24h
* `ConversionRatePc1d` - Conversion rate post-click 1 day
* `ConversionRateAllPc1d` - All conversion rate PC 1 day
* `ConversionRatePc7d` - Conversion rate post-click 7 days
* `ConversionRateAllPc7d` - All conversion rate PC 7 days
**Cost of Sale (eCOS) Metrics:**
* `ECosClientAttribution` - Cost of sale with client attribution
* `ECosAllClientAttribution` - All cost of sale with client attribution
* `ECosPc30d` - Cost of sale post-click 30 days
* `ECosAllPc30d` - All cost of sale PC 30 days
* `ECosPv24h` - Cost of sale post-view 24 hours
* `ECosAllPv24h` - All cost of sale PV 24 hours
* `ECosPc30dPv24h` - Cost of sale PC 30d / PV 24h
* `ECosAllPc30dPv24h` - All cost of sale PC 30d / PV 24h
* `ECosPc1d` - Cost of sale post-click 1 day
* `ECosAllPc1d` - All cost of sale PC 1 day
* `ECosPc7d` - Cost of sale post-click 7 days
* `ECosAllPc7d` - All cost of sale PC 7 days
**Cost Per Order Metrics:**
* `CostPerOrderClientAttribution` - Cost per order with client attribution
* `CostPerOrderAllClientAttribution` - All cost per order with client attribution
* `CostPerOrderPc30d` - Cost per order post-click 30 days
* `CostPerOrderAllPc30d` - All cost per order PC 30 days
* `CostPerOrderPv24h` - Cost per order post-view 24 hours
* `CostPerOrderAllPv24h` - All cost per order PV 24 hours
* `CostPerOrderPc30dPv24h` - Cost per order PC 30d / PV 24h
* `CostPerOrderAllPc30dPv24h` - All cost per order PC 30d / PV 24h
* `CostPerOrderPc1d` - Cost per order post-click 1 day
* `CostPerOrderAllPc1d` - All cost per order PC 1 day
* `CostPerOrderPc7d` - Cost per order post-click 7 days
* `CostPerOrderAllPc7d` - All cost per order PC 7 days
* `Cpc` - Cost per click
* `ECpm` - Effective cost per 1,000 impressions
**ROAS Metrics (by Attribution Window):**
* `RoasClientAttribution` - ROAS with client attribution
* `RoasAllClientAttribution` - All ROAS with client attribution
* `RoasPc30d` - ROAS post-click 30 days
* `RoasAllPc30d` - All ROAS post-click 30 days
* `RoasPv24h` - ROAS post-view 24 hours
* `RoasAllPv24h` - All ROAS post-view 24 hours
* `RoasPc30dPv24h` - ROAS PC 30d / PV 24h
* `RoasAllPc30dPv24h` - All ROAS PC 30d / PV 24h
* `RoasPc1d` - ROAS post-click 1 day
* `RoasAllPc1d` - All ROAS post-click 1 day
* `RoasPc7d` - ROAS post-click 7 days
* `RoasAllPc7d` - All ROAS post-click 7 days
**Advertiser Value Metrics:**
* `AdvertiserValue` - Advertiser-defined value
* `AdvertiserAllValue` - All advertiser value
* `CostOfAdvertiserValue` - Cost of advertiser value
* `CostOfAdvertiserValueAll` - All cost of advertiser value
**App Install Metrics:**
* `PostInstallSales` - Sales after app install
* `AppInstalls` - Number of app installs
* `PostInstallConversionRate` - Post-install conversion rate
* `PostInstallCostOfSale` - Post-install cost of sale
* `PostInstallOrderValue` - Post-install order value
* `CostPerInstall` - Cost per app install
* `InstallRate` - App install rate
* `PostInstallCostPerOrder` - Post-install cost per order
* `PostInstallRoas` - Post-install ROAS
**Omnichannel Metrics:**
* `OmnichannelRoasClientAttribution` - Omnichannel ROAS with client attribution
* `OmnichannelRoasPc30d` - Omnichannel ROAS post-click 30 days
* `OmnichannelRoasAllPc30d` - All omnichannel ROAS PC 30 days
* `OmnichannelRoasPv24h` - Omnichannel ROAS post-view 24 hours
* `OmnichannelRoasAllPv24h` - All omnichannel ROAS PV 24 hours
* `OmnichannelRevenueClientAttribution` - Omnichannel revenue with client attribution
* `OmnichannelRevenuePc30d` - Omnichannel revenue post-click 30 days
* `OmnichannelRevenueAllPc30d` - All omnichannel revenue PC 30 days
* `OmnichannelRevenuePv24h` - Omnichannel revenue post-view 24 hours
* `OmnichannelRevenueAllPv24h` - All omnichannel revenue PV 24 hours
* `OmnichannelsalesClientAttribution` - Omnichannel sales with client attribution
* `OmnichannelSalesPc30d` - Omnichannel sales post-click 30 days
* `OmnichannelSalesAllPc30d` - All omnichannel sales PC 30 days
* `OmnichannelSalesPv24h` - Omnichannel sales post-view 24 hours
* `OmnichannelSalesAllPv24h` - All omnichannel sales PV 24 hours
**Offline/Store Metrics:**
* `RoasOfflinePc30d` - Offline ROAS post-click 30 days
* `RoasOfflinePv24h` - Offline ROAS post-view 24 hours
* `SalesOfflinePc30d` - Offline sales post-click 30 days
* `SalesOfflinePv24h` - Offline sales post-view 24 hours
* `RevenueGeneratedOfflinePc30d` - Offline revenue post-click 30 days
* `RevenueGeneratedOfflinePv24h` - Offline revenue post-view 24 hours
# Data Model
The Criteo connector exposes a single stream of aggregated campaign statistics. Records are unique by the combination of date, advertiser, ad set, category, device and operating system, and the stream is replicated incrementally on `Day`.
```mermaid theme={null}
erDiagram
CAMPAIGN_STATISTICS {
string Day PK
string AdvertiserId PK
string AdsetId PK
string CategoryId PK
string Device PK
string Os PK
string Advertiser
string Adset
string Currency
string AdvertiserCost
string Clicks
string Displays
string Visits
string ClickThroughRate
}
```
## Implementation Notes
### Attribution Windows Explained
Criteo provides metrics with different attribution windows to help you understand how ads influence conversions:
| Suffix | Meaning |
| - | - |
| `Pc1d` | Post-click 1 day: Conversions within 1 day of a click |
| `Pc7d` | Post-click 7 days: Conversions within 7 days of a click |
| `Pc30d` | Post-click 30 days: Conversions within 30 days of a click |
| `Pv24h` | Post-view 24 hours: Conversions within 24 hours of viewing an ad |
| `Pc30dPv24h` | Combined post-click 30 days and post-view 24 hours |
| `ClientAttribution` | Using your custom attribution model configured in Criteo |
### Data Granularity
The Campaign Statistics stream provides daily aggregated data broken down by:
* Day
* Advertiser
* Ad Set
* Category
* Device type
* Operating System
This allows for detailed analysis of campaign performance across different dimensions.
### Currency and Timezone
* **Currency**: All monetary values (spend, revenue, ROAS, etc.) are reported in the currency you configure. Make sure to select the same currency used in your Criteo account for accurate reporting.
* **Timezone**: Affects how daily data is aggregated. Choose the timezone that matches your reporting needs.
## Skills for agents
Criteo connector documentation as plain markdown, for use in AI agent contexts.
# CRM Educacional as a data source
Source: https://docs.nekt.com/sources/crm-educacional
Bring data from CRM Educacional to your Lakehouse.
CRM Educacional is the admissions CRM used by Brazilian educational institutions to run their selection processes end to end: capturing leads, taking applications, charging the registration fee, scheduling and grading exams, ranking candidates and calling them for enrolment. The connector extracts that whole funnel: leads, registrations, selection processes, offers, payments, evaluations, classifications and exam schedules, so you can measure capture and conversion in your Lakehouse.
Every institution runs on its own address, so the connector asks for the URL of your CRM in addition to the credentials of an integration user.
CRM Educacional sells its software under the **Wakeme** name. This connector reads the integration API of **Wakeme PRO** and **Wakeme Essencial**. **Wakeme One** is a different system, with its own application and API, and is not covered by this connector.
## Configuring CRM Educacional as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the CRM Educacional option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
CRM Educacional authenticates with the **user name and password of an integration user** (*Usuário de Integração*), which the institution creates inside its own CRM. The connector exchanges them for a temporary token on every run, so nothing has to be copied or renewed by hand.
Create the integration user in the CRM before setting the source up. The vendor documents the steps under **Usuário de Integração** in its API documentation. A regular Dynamics CRM user also works, but it has to be declared as such in the **Authentication type** setting below.
The integration user's permissions decide what the extraction can read. Give it access to the entities you want to sync: a user without access to an entity makes that stream fail rather than come back empty.
The following configurations are available:
* **API URL**: the address of your institution's CRM Educacional API, for example `https://yourschoolapi.crmeducacional.com`. You can find it in the CRM under **Configurações > CRM Educacional > aba Captação > "Endereço da Api Ficha de Inscrição"**. Required.
* **Username**: the user name of the integration user. Required.
* **Password**: the password of that user. Required.
* **Authentication type**: which kind of user is signing in: `APICRMEducacional` for an integration user (recommended) or `CRM` for a Dynamics CRM user. Defaults to `APICRMEducacional`.
* **Initial sync date**: the earliest record change to sync. Records modified before this date are not read. Defaults to 2020-01-01.
* **Page size**: how many records are requested per page (FetchXML count). Maximum 5000, the limit the CRM enforces. Defaults to 1000. CRM Educacional's API may serve a fixed page size regardless of this value; the connector follows the size actually served.
* **Requests per minute**: how fast the connector may call the API. CRM Educacional documents 50 requests per minute per user, which is the default. Lower it if the same integration user is shared with another integration.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Candidate Registrations and Leads are the largest streams on an institution's account. The first extraction reads the whole history from the initial sync date; the runs after it only read what changed.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: every stream supports INCREMENTAL, replicated on the timestamp of the last change to the record. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Leads | `leads` | Incremental | Prospective students captured by the institution |
| Candidate Registrations | `candidate_registrations` | Incremental | Applications to a selection process, with their status and payment situation |
| Selection Processes | `selection_processes` | Incremental | The intakes candidates apply to |
| Offers | `offers` | Incremental | Course/campus/shift offers inside a selection process, with seats and price |
| Payments | `payments` | Incremental | Registration-fee charges raised against an application |
| Evaluations | `evaluations` | Incremental | Scores per evaluation factor of a registration |
| Classifications | `classifications` | Incremental | Ranking of a candidate in a call for a given offer |
| Exam Schedules | `exam_schedules` | Incremental | Slots a scheduled exam is booked into |
| Accounts | `accounts` | Incremental | Institutional accounts and campuses |
The identifiers that join these tables are the CRM's own GUIDs: `cad_clientepotencialid` on a registration points at `leadid`, `cad_concursoid` at a selection process, `cad_ofertaid` at an offer, and `cad_inscricaocandidatoid` on payments, evaluations and classifications points back at the registration.
## Fields by stream
Below you'll find all available data streams from CRM Educacional and their corresponding fields:
Every prospective student registered in the CRM, whether or not they went on to apply. This is the top of the admissions funnel: where the person came from, how to reach them, and which selection process and offer they are currently engaged with.
| Field | Type | Description |
| :- | :- | :- |
| `leadid` | String | Unique identifier (GUID) of the lead in the CRM |
| `fullname` | String | Full name of the lead, as displayed in the CRM |
| `firstname` | String | First name of the lead |
| `lastname` | String | Last name of the lead |
| `emailaddress1` | String | Primary email address of the lead |
| `mobilephone` | String | Mobile phone number of the lead, in the format the CRM stored it |
| `cad_cpf` | String | CPF (Brazilian individual taxpayer number) of the lead |
| `cad_rg` | String | RG (Brazilian identity card number) of the lead |
| `new_sexo` | String | Gender recorded for the lead |
| `cad_datanascimento` | Timestamp | Date of birth of the lead |
| `cad_necessidadesespeciais` | String | Special needs declared by the lead, when any were declared |
| `cad_concursoatual` | String | Identifier of the selection process the lead is currently engaged with |
| `cad_concursodeorigem` | String | Identifier of the selection process the lead originally came from |
| `cad_ofertaatual` | String | Identifier of the offer (course/intake) the lead is currently interested in |
| `cad_dataalteracaocadastrooferta` | Timestamp | Timestamp of the last change to the lead's registration data or chosen offer |
| `createdon` | Timestamp | Timestamp the lead was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the lead, used as the incremental replication key (incremental replication key) |
The heart of the funnel: one row per application. It carries the lead who applied, the selection process and the offers chosen, and where the application stands: its status in the process, whether the fee was paid, and whether the candidate has been evaluated.
| Field | Type | Description |
| :- | :- | :- |
| `cad_inscricaocandidatoid` | String | Unique identifier (GUID) of the registration |
| `cad_numeroinscricao` | String | Registration number shown to the candidate and used by the academic system |
| `cad_clientepotencialid` | String | Identifier of the lead who applied |
| `cad_situacao` | String | Status of the candidate within the selection process (e.g. registered, called, withdrawn) |
| `cad_concursoid` | String | Identifier of the selection process the candidate applied to |
| `cad_ofertaid` | String | Identifier of the offer chosen as first option |
| `cad_2opcaooferta` | String | Identifier of the offer chosen as second option, when the process allows one |
| `cad_3opcaooferta` | String | Identifier of the offer chosen as third option, when the process allows one |
| `coc_situacaopagamento` | String | Payment status of the registration fee (e.g. paid, approved, exempt) |
| `cad_situacaoavaliacao` | String | Status of the candidate's evaluation |
| `cad_salaid` | String | Identifier of the room assigned to the candidate for the exam |
| `cad_conta` | String | Identifier of the account (institution or campus) the registration belongs to |
| `cad_inscriesid` | String | Identifier of the exam schedule ('exam booked on') linked to the registration |
| `cad_dadosobrigatorioscoletados` | String | Whether all mandatory data has been collected from the candidate |
| `cad_trainee` | String | Whether the registration is flagged as trainee |
| `cad_linguaestrangeira` | String | Foreign language chosen by the candidate |
| `createdon` | Timestamp | Timestamp the registration was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the registration, used as the incremental replication key (incremental replication key) |
The intakes candidates apply to (a traditional entrance exam, an ENEM-based intake, a scheduled exam, graduate admissions, and so on), with the academic term they admit into and the date of the exam.
| Field | Type | Description |
| :- | :- | :- |
| `cad_concursoid` | String | Unique identifier (GUID) of the selection process |
| `cad_codigo` | String | Code of the selection process, the identifier used when integrating with the academic system |
| `cad_nome` | String | Name of the selection process |
| `cad_periodoletivo` | String | Academic term the selection process admits into (e.g. 2026/1) |
| `cad_tipodeconcurso` | String | Type of selection process (traditional, ENEM, scheduled, PROUNI, graduate, free courses, interest form, transfer, second degree, medicine) |
| `cad_dataprova` | Timestamp | Date of the entrance exam |
| `cad_horaprova` | String | Time of day the entrance exam starts |
| `cad_minutosdaprova` | Integer | Duration of the entrance exam in minutes |
| `cad_tamanhoanexo` | String | Maximum size allowed for files attached to a registration in this process |
| `createdon` | Timestamp | Timestamp the selection process was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the selection process, used as the incremental replication key (incremental replication key) |
One course/campus/shift combination offered inside a selection process, with its seat counts, its price in instalments and the codes that tie it to the institution's academic system.
| Field | Type | Description |
| :- | :- | :- |
| `cad_ofertaid` | String | Unique identifier (GUID) of the offer |
| `cad_nome` | String | Name of the offer, usually the course name |
| `cad_identificadoroferta` | String | Identifier of the offer used when integrating with the academic system |
| `cad_concursoid` | String | Identifier of the selection process this offer belongs to |
| `cad_categoriaid` | String | Identifier of the category the offer is classified under |
| `cad_categoria` | String | Category code of the offer |
| `cad_ano` | Integer | Year the offer admits into |
| `cad_periodo` | Integer | Academic period (term) the offer admits into |
| `cad_modalidade` | String | Delivery mode of the course (e.g. on-campus, distance learning) |
| `cad_regime` | String | Academic regime of the course (e.g. semester, yearly) |
| `cad_semestreofertado` | String | Semester being offered |
| `cad_turnoofertado` | String | Shift being offered (morning, afternoon, evening) |
| `cad_turnoequivalente` | String | Equivalent shift in the academic system |
| `cad_codturno` | String | Shift code in the academic system |
| `cad_codcampus` | String | Campus code in the academic system |
| `cad_codcurriculo` | String | Curriculum code in the academic system |
| `cad_codcurso` | String | Course code in the academic system |
| `cad_codescola` | String | School code in the academic system |
| `cad_currequiv` | String | Equivalent curriculum code in the academic system |
| `cad_formaingr` | String | Admission type code (how the candidate enters the course) |
| `cad_subformaingr` | String | Admission subtype code |
| `cad_codigoocde` | String | OCDE classification code of the course |
| `cad_codigodomec` | String | MEC (Brazilian Ministry of Education) code of the course |
| `cad_datainicial` | Timestamp | Date the offer becomes available |
| `cad_datafinal` | Timestamp | Date the offer stops being available |
| `cad_quantidadevagas` | Integer | Total number of seats in the offer |
| `cad_vagasdisponiveis` | Integer | Seats still available in the CRM |
| `cad_vagasdisponveissga` | Integer | Seats still available according to the academic system (SGA) |
| `cad_minimovagaspreenchidas` | Integer | Minimum number of filled seats required for the offer to run |
| `cad_quantidadeparcelas` | Integer | Number of instalments the offer's price can be split into |
| `cad_valorparcela` | Float | Value of each instalment |
| `cad_formulanotafinal` | String | Formula used to calculate the candidate's final score in this offer |
| `cad_idareainteresse` | String | Identifier of the area of interest the offer belongs to |
| `cad_servicoeducacionalid` | String | Identifier of the educational service linked to the offer |
| `cad_usuariointegracao` | String | Integration user that last wrote the offer from an external system |
| `createdon` | Timestamp | Timestamp the offer was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the offer, used as the incremental replication key (incremental replication key) |
The registration-fee charges raised against an application: the boleto or card transaction, when it was due, when it was processed and when it was paid.
| Field | Type | Description |
| :- | :- | :- |
| `cad_pagamentoid` | String | Unique identifier (GUID) of the payment |
| `cad_name` | String | Name of the payment record in the CRM |
| `cad_valor` | Float | Amount charged |
| `cad_datavencimento` | Timestamp | Due date of the charge |
| `cad_datapagamento` | Timestamp | Date the charge was paid |
| `cad_boletodatacredito` | Timestamp | Date the boleto was credited to the institution |
| `cad_datadeprocessamento` | Timestamp | Date the payment was processed by the CRM |
| `statuscode` | String | Status of the payment record (e.g. active, inactive) |
| `cad_isento` | String | Whether the candidate was exempted from paying this charge |
| `cad_formapagamento` | String | Payment method used (e.g. boleto, credit card) |
| `cad_urlboleto` | String | URL where the boleto can be downloaded |
| `cad_numeroboleto` | String | Number of the boleto issued for the charge |
| `cad_numerolotearquivoretorno` | String | Number of the bank return file batch the payment was reconciled in |
| `cad_nomeoperadoracartao` | String | Name of the card operator that processed the payment |
| `cad_idtransacao` | String | Identifier of the transaction in the CRM |
| `cad_idtransacaooperadoracartao` | String | Identifier of the transaction at the card operator |
| `cad_contacorrente` | String | Bank account the payment was credited to |
| `cad_agencia` | String | Bank branch the payment was credited to |
| `cad_numeroparcelasselecionado` | Integer | Number of instalments the candidate chose |
| `cad_inscricaocandidatoid` | String | Identifier of the registration the charge belongs to |
| `cad_precoid` | String | Identifier of the price record the charge was raised from |
| `createdon` | Timestamp | Timestamp the payment was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the payment, used as the incremental replication key (incremental replication key) |
One score per evaluation factor of a registration: the essay, the objective test, and any other factor the process is composed of.
| Field | Type | Description |
| :- | :- | :- |
| `cad_avaliacaoid` | String | Unique identifier (GUID) of the evaluation |
| `cad_valor` | Float | Score the candidate obtained in this evaluation |
| `cad_inscricaocandidatoid` | String | Identifier of the registration the score belongs to |
| `cad_fatoravaliacaoid` | String | Identifier of the evaluation factor the score was given for (e.g. essay, objective test) |
| `createdon` | Timestamp | Timestamp the evaluation was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the evaluation, used as the incremental replication key (incremental replication key) |
Where a candidate ranked in a call (admission round) for a given offer, with the final score used to rank them.
| Field | Type | Description |
| :- | :- | :- |
| `cad_classificacaoid` | String | Unique identifier (GUID) of the classification |
| `cad_inscricaocandidatoid` | String | Identifier of the registration that was ranked |
| `cad_ofertaid` | String | Identifier of the offer the candidate was ranked for |
| `cad_chamadaid` | String | Identifier of the call (admission round) this classification belongs to |
| `cad_posicao` | Integer | Position the candidate reached in the ranking |
| `cad_situacao` | String | Status of the candidate in the call (e.g. called, waiting list) |
| `cad_pontuacaofinal` | Float | Final score used to rank the candidate |
| `createdon` | Timestamp | Timestamp the classification was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the classification, used as the incremental replication key (incremental replication key) |
The slots a scheduled exam is booked into, with the date, time, room and unit of the exam.
| Field | Type | Description |
| :- | :- | :- |
| `cad_calendarioprovaid` | String | Unique identifier (GUID) of the exam schedule |
| `cad_data` | Timestamp | Date of the scheduled exam |
| `cad_hora` | String | Hour of the scheduled exam |
| `cad_minutos` | Integer | Minutes past the hour the scheduled exam starts |
| `cad_sala` | String | Room the exam takes place in |
| `cad_categoria` | String | Unit or category the exam belongs to |
| `createdon` | Timestamp | Timestamp the exam schedule was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the exam schedule, used as the incremental replication key (incremental replication key) |
The institutional accounts (institution or campus) registrations belong to, with the permissions configured for each.
| Field | Type | Description |
| :- | :- | :- |
| `accountid` | String | Unique identifier (GUID) of the account in the CRM |
| `name` | String | Name of the account, usually the institution or campus |
| `accountcategorycode` | String | Category the account is classified under |
| `cad_permitiruploadprova` | String | Whether the account allows exam files to be uploaded |
| `cad_permitirrelatorios` | String | Whether the account allows reports to be generated |
| `cad_permitirarquivoretorno` | String | Whether the account allows bank return files to be processed |
| `createdon` | Timestamp | Timestamp the account was created in the CRM |
| `modifiedon` | Timestamp | Timestamp of the last change to the account, used as the incremental replication key (incremental replication key) |
## Implementation Notes
### Pagination
The gateway rewrites the `` element before forwarding it to Dynamics and drops `page`, `count`, `top`, and `paging-cookie` on the way. Therefore, the connector sends the page number in the `pagina` HTTP header. The CRM Educacional API gateway usually serves a fixed number of rows (e.g., 250) regardless of the **Page size** configuration, and the connector follows the size actually served. The pagination walk stops on `morerecords=0`, on an empty page, or on a page shorter than the first one served. If two consecutive pages return the same primary keys, the stream fails the run instead of loading the same rows forever.
### Incremental Sync
Every stream replicates incrementally on `modifiedon`, the timestamp Dynamics stamps on every row. The connector uses the `ge` filter (which compares timestamps down to the second), rather than comparing dates only, preventing the extraction from re-reading the entire day of the bookmark on every run. Rows are ordered by `createdon` plus the entity's primary key so a page boundary cannot silently drop a record if data changes mid-sync.
# CSV as a data source
Source: https://docs.nekt.com/sources/csv
Bring data from CSV to Nekt.
CSV (Comma-Separated Values) is a simple file format used to store tabular data, such as a spreadsheet or database. Each line represents a row of data, with values separated by commas, making it a widely supported format for data exchange between different applications and systems.
![]()
## Step 1: Select your file
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the **Add source** button located on the top right of your screen. Then, select the **CSV** option from the list of connectors.
2. Click **Next** and you'll be prompted to describe your data source. Give it a name and upload your CSV.
3. As soon as you upload it, optionally, you have the chance to edit the file's name. You can't have more than one file with the same name, but you can have the same file uploaded more than once with different names if you ever need it.
4. The next step consists in letting us know more about your new table:
* Inform how you want it to be named in the catalog;
* Specify what's the delimiter being used in your CSV. If you're not sure about it, you can open your CSV in a text program and see how your data is being separated;
* Inform the name of the columns used as unique key.
5. Click **Next**.
## Step 2: Set your new table
Now that you've specified how's your csv structure, define how you want your table in the catalog to be.
1. Select one column to work as primary key (select among the ones informed previously) and which fields from your csv you want to bring.
2. Click **Next** to finalize.
## Step 3: Check your new source!
Your CSV source was added! Now, for you to be able to see it on your datalake, you have to wait for the pipeline to run.
1. Click in **Done** and you'll see the new source in your sources list.
Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> Let us know through our Slack if you face any blocker and we'll be happy to help!
## Skills for agents
CSV connector documentation as plain markdown, for use in AI agent contexts.
# Cube as a data source
Source: https://docs.nekt.com/sources/cube
Bring data from Cube to Nekt.
[Cube](https://cube.dev) is a semantic layer that sits in front of a data warehouse or database, letting teams model "cubes" of measures and dimensions on top of their data. This connector is fully generic: it has no knowledge of any specific data model, and instead discovers a client's cubes and views at runtime, from whichever Cube Core or Cube Cloud deployment it's pointed at.
![]()
## Configuring Cube as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Cube option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Cube exposes two separate APIs, and this connector needs credentials for both: the **Meta API** (used to discover the data model) and the **SQL API** (used to extract data).
The following configurations are available:
* **Meta API Base URL**: Base URL of Cube's REST API, used for schema discovery. For example, `https://my-deployment.cubecloudapp.dev` for Cube Cloud, or `http://localhost:4000` for a self-hosted Cube Core. Do not include the `/cubejs-api/v1/meta` suffix.
* **Authorization Header**: Value sent as the `Authorization` header when calling the Meta API. Use the raw Cube API token, or `Bearer ` if your deployment issues a JWT that requires the Bearer scheme.
* **SQL API Host**: Hostname of Cube's SQL API (Postgres-compatible protocol). For Cube Cloud, this is a different host than the Meta API's, typically something like `my-deployment.sql.gcp-us-central1.cubecloudapp.dev`.
* **SQL API Port**: (Optional, default `5432`) Port of Cube's SQL API. Self-hosted Cube Core deployments set this via `CUBEJS_PG_SQL_PORT` and commonly use a different port.
* **Database**: (Optional, default `cube`) Database name for the SQL API connection. Cube accepts any non-empty string here; it doesn't correspond to a real database.
* **SQL API User**: Username for the SQL API connection (`CUBEJS_SQL_USER`).
* **SQL API Password**: Password for the SQL API connection (`CUBEJS_SQL_PASSWORD`).
* **Use SSL**: (Optional, default enabled) Whether to require SSL for the SQL API connection.
* **Start Date**: (Optional) Earliest value to sync for cubes that expose a time dimension. Cubes without a time dimension are always synced in full.
Cube Cloud typically serves the Meta API and SQL API on different hosts. Make sure you provide the SQL-specific host, not the Meta API's.
Once you're done, click **Next**.
### 2. Select streams
The connector automatically discovers every cube and view exposed by the Meta API. Choose which ones you want to sync.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer, a folder to organize it, a name for each table, and the type of sync.
Sync type is decided automatically per cube, not configured manually: cubes with a `time`-typed dimension sync **incrementally** on that dimension; cubes without one sync as **full table**.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization.
Define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers) based on how often you need data updated.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Streams and Fields
Cube is a semantic-layer connector: streams are dynamically discovered from your Cube deployment's own data model. Each cube or view becomes a stream, with one field per measure and dimension it defines.
### Supported Data Types
| Cube Type | Mapped Type |
| - | - |
| string | String |
| number | Number |
| boolean | Boolean |
| time | DateTime |
### Primary Keys and Replication
* **Primary key**: Cube's Meta API doesn't expose an explicit primary-key flag for regular cubes, so the full set of dimensions is used as the primary key: every row is a `GROUP BY` of all dimensions, so that combination is the row's natural key.
* **Replication**: If a cube has a `time`-typed dimension, it's synced incrementally, filtered on that dimension using the **Start Date** (or the bookmark from a previous run). Cubes without a time dimension are always synced in full.
Data freshness for incremental streams also depends on the cube's own pre-aggregation refresh configuration on the Cube deployment itself; this connector has no control over that.
## Known Limitations
* Cube's SQL API decides on its own, per query, whether to answer from cache/pre-aggregations or push the query down to the underlying data source. This connector sends plain, portable SQL and does not force or detect either path.
* Cube Cloud's separate Metadata API exposes richer schema information (an explicit primary-key flag, table lineage), but it requires a JWT obtained through the Cube Cloud Control Plane API and is Cube-Cloud-only. It is not used, so that discovery behaves identically on Cube Core and Cube Cloud.
* Very large result sets may require Cube's `CUBESQL_STREAM_MODE` to be enabled on the deployment side.
# cubeSQL as a data source
Source: https://docs.nekt.com/sources/cubesql
Bring data from cubeSQL to Nekt.
[cubeSQL](https://www.sqlabs.com/cubesql/) is a client/server SQL database engine from SQLabs, built on top of SQLite. It's commonly used to add multi-user, network access to SQLite data, and communicates over its own lightweight protocol rather than a standard database wire protocol. This connector is fully generic: it discovers a database's tables and columns at runtime, so it works against any cubeSQL server regardless of schema.
cubeSQL (SQLabs) is unrelated to [Cube](https://cube.dev), the semantic layer product supported by Nekt's separate **Cube** connector. The two share a name but nothing else, so make sure you're picking the right one.
![]()
## Configuring cubeSQL as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the cubeSQL option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **Host**: Hostname or IP address of the cubeSQL server.
* **Port**: (Optional, default `4430`) Port of the cubeSQL server.
* **User**: Username for the cubeSQL connection.
* **Password**: Password for the cubeSQL connection.
* **Database**: Name of the cubeSQL database to sync, as shown in the Admin app's Databases list.
* **Start Date**: (Optional) Earliest value to sync for streams using incremental replication (used as the lower bound before any bookmark exists). Only applies to streams whose replication key is selected in step 3.
Once you're done, click **Next**.
### 2. Select streams
The connector automatically discovers every table in the configured database. Choose which ones you want to sync.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer, a folder to organize it, a name for each table, and the type of sync.
* **Sync Type**: choose between INCREMENTAL and FULL TABLE.
* Incremental: every time the extraction happens, only rows with a value greater than the last synced value of the chosen replication key are fetched.
* Full table: every time the extraction happens, the current state of the table is fetched in full.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization.
Define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers) based on how often you need data updated.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Streams and Fields
cubeSQL is a database connector: streams are dynamically discovered from your database's own tables. Each table becomes a stream, with one field per column.
### Supported Data Types
Column types are inferred from cubeSQL's reported column type using SQLite's own type-affinity rules, plus a few common naming conventions:
| Declared column type contains | Mapped Type |
| - | - |
| `BOOL` | Boolean |
| `DATE`, `DATETIME`, `TIMESTAMP` | DateTime |
| `INT` | Integer |
| `CHAR`, `CLOB`, `TEXT` | String |
| `REAL`, `FLOA`, `DOUB` | Number |
| anything else (including `BLOB` or no declared type) | String |
### Primary Keys and Replication
* **Primary key**: taken from the table's own primary key definition in cubeSQL.
* **Replication**: every stream defaults to full table sync. To sync a table incrementally, select a replication key column (typically an `updated_at`-style column) in step 3 of the source configuration.
## Known Limitations
* cubeSQL has no native change-data-capture mechanism, so log-based replication isn't available: only full table and incremental (via a replication key column).
* The connector requires a licensed (registered) cubeSQL server. Unregistered/demo cubeSQL servers reject all standard SQL statements (`SELECT`, `CREATE TABLE`, etc.), accepting only cubeSQL's own administrative commands, and won't work with this connector.
# Customer.io as a data source
Source: https://docs.nekt.com/sources/customer-io
Bring data from Customer.io to Nekt.
Customer.io is a powerful customer engagement platform that enables businesses to send targeted messages across multiple channels. It provides tools for managing customer data, tracking user activities, and automating personalized marketing campaigns through email, push notifications, and SMS.
![]()
## Configuring Customer.io as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Customer.io option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Customer.io API token for this connection. This can be obtained from your Customer.io account settings (or through [this link](https://docs.customer.io/integrations/api/authentication)).
Once you have it, add the account access and click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. The available streams include activities, customers, transactional messages, metrics, and message details.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Customer.io and their corresponding fields:
Stream for tracking customer activities and interactions with your messages and campaigns.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the activity |
| `customer_id` | String | ID of the customer who performed the activity |
| `customer_identifiers` | Object | Object containing customer identification details |
| `customer_identifiers.email` | String | Customer's email address |
| `customer_identifiers.id` | String | Customer ID (the identifier you assigned to the person) |
| `customer_identifiers.cio_id` | String | Customer.io assigned ID |
| `type` | String | Type of activity performed |
| `timestamp` | Integer | Unix timestamp when the activity occurred |
| `data` | String | Activity data as JSON string (structure varies by activity type) |
| `delivery_id` | String | ID of the message delivery associated with this activity |
| `delivery_type` | String | Type of delivery (email, push, etc.) |
Stream containing basic customer identification information.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The customer's unique system ID |
| `email` | String | The customer's email address |
| `cio_id` | String | The Customer.io assigned ID |
Stream for managing transactional messages configured in your Customer.io account.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the transactional message |
| `name` | String | Name of the transactional message |
| `created_at` | Integer | Unix timestamp when the message was created |
| `updated_at` | Integer | Unix timestamp when the message was last updated |
| `type` | String | Type of transactional message |
| `status` | String | Current status of the transactional message |
| `template_id` | String | ID of the template used |
| `description` | String | Description of the message |
| `send_to_unsubscribed` | Boolean | Whether this message is sent to unsubscribed users |
| `link_tracking` | Boolean | Whether link tracking is enabled |
| `open_tracking` | Boolean | Whether open tracking is enabled |
| `hide_message_body` | Boolean | Whether the message body is hidden in the interface |
| `queue_drafts` | Boolean | Whether draft messages are queued |
| `trigger_name` | String | Name of the trigger associated with this message |
Stream providing metrics and performance data for your transactional messages over time.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `transactional_id` | String | ID of the transactional message these metrics belong to |
| `attempted` | Array | Array of message attempt counts over time |
| `bounced` | Array | Array of bounced message counts over time |
| `clicked` | Array | Array of clicked message counts over time |
| `human_clicked` | Array | Array of human-generated clicks over time |
| `prefetch_clicked` | Array | Array of machine/prefetch clicks over time |
| `converted` | Array | Array of conversion counts over time |
| `created` | Array | Array of message creation counts over time |
| `deferred` | Array | Array of deferred message counts over time |
| `delivered` | Array | Array of delivered message counts over time |
| `drafted` | Array | Array of drafted message counts over time |
| `failed` | Array | Array of failed delivery counts over time |
| `opened` | Array | Array of opened message counts over time |
| `human_opened` | Array | Array of human-generated opens over time |
| `prefetch_opened` | Array | Array of machine/prefetch opens over time |
| `sent` | Array | Array of sent message counts over time |
| `spammed` | Array | Array of spam report counts over time |
| `suppressed` | Array | Array of suppressed message counts over time |
| `undeliverable` | Array | Array of undeliverable message counts over time |
| `topic_unsubscribed` | Array | Array of topic unsubscribe counts over time |
| `unsubscribed` | Array | Array of unsubscribe counts over time |
Stream providing detailed information about individual transactional message deliveries. This stream retrieves data from the last year.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the message delivery |
| `transactional_id` | String | ID of the parent transactional message |
| `deduplicate_id` | String | ID used for deduplication |
| `msg_template_id` | String | ID of the message template used |
| `action_id` | String | ID of the action that triggered this message |
| `customer_id` | String | ID of the customer who received the message |
| `identifiers` | Object | Object containing customer identification details |
| `identifiers.email` | String | Customer's email address |
| `identifiers.id` | String | Customer ID (the identifier you assigned to the person) |
| `identifiers.cio_id` | String | Customer.io assigned ID |
| `recipient` | String | Email address or identifier of the recipient |
| `subject` | String | Subject line of the message |
| `metrics` | Object | Object containing message performance metrics |
| `metrics.bounced` | Integer | Timestamp when the message bounced |
| `metrics.clicked` | Integer | Timestamp when the message was clicked |
| `metrics.human_clicked` | Integer | Human-generated click count (reliable from April 20, 2025) |
| `metrics.prefetch_clicked` | Integer | Machine-generated click count (reliable from April 20, 2025) |
| `metrics.converted` | Integer | Timestamp when conversion occurred |
| `metrics.created` | Integer | Timestamp when the message was created |
| `metrics.delivered` | Integer | Timestamp when the message was delivered |
| `metrics.drafted` | Integer | Timestamp when the message was drafted |
| `metrics.dropped` | Integer | Timestamp when the message was dropped |
| `metrics.failed` | Integer | Timestamp when delivery failed |
| `metrics.opened` | Integer | Timestamp when the message was opened |
| `metrics.human_opened` | Integer | Human-generated open count (reliable from March 20, 2025) |
| `metrics.prefetch_opened` | Integer | Machine-generated open count (reliable from March 20, 2025) |
| `metrics.sent` | Integer | Timestamp when the message was sent |
| `metrics.spammed` | Integer | Timestamp when marked as spam |
| `metrics.undeliverable` | Integer | Timestamp when marked undeliverable |
| `metrics.unsubscribed` | Integer | Timestamp when user unsubscribed |
| `created` | Integer | Unix timestamp when the message was created |
| `failure_message` | String | Error message if delivery failed |
| `newsletter_id` | String | ID of associated newsletter (if applicable) |
| `content_id` | String | ID of the content used |
| `campaign_id` | String | ID of associated campaign (if applicable) |
| `broadcast_id` | String | ID of associated broadcast (if applicable) |
| `type` | String | Type of message (email, push, SMS, etc.) |
| `forgotten` | Boolean | Whether the message has been forgotten/deleted |
## Skills for agents
Customer.io connector documentation as plain markdown, for use in AI agent contexts.
# CV CRM as a data source
Source: https://docs.nekt.com/sources/cvcrm
Bring data from CV CRM to your Lakehouse.
CV CRM is a customer relationship management system designed specifically for Brazilian businesses. It provides tools for managing customer data, sales pipelines, and business relationships, helping companies organize and optimize their customer interactions and sales processes.
![]()
## Configuring CV CRM as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **CV CRM** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your CV CRM credentials to authorize Nekt to access your data.
The following configurations are available:
* **Email**: The email address used to authenticate against the API service. This should be a valid email address with enough permissions to access the CV CRM data you want to extract. The email should be in the format `user@example.com`.
* **API token**: The authentication token required to access the API service. Please check [this documentation](https://desenvolvedor.cvcrm.com.br/docs/getting-started#etapa-2-gerando-o-token-de-acesso) to find more information about how to generate the token.
* **Company subdomain**: The subdomain identifier for your company account. For example, if your CRM URL is `https://mycompany.cvcrm.com.br`, then your company subdomain would be `mycompany`.
* **Start date**: The date from which you want to begin synchronizing data. This defines the earliest point in time for historical extraction. Any data created or modified before this date will not be included in the synchronization.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* **Delta Log Retention**: Determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**: Determine when to execute an additional full sync. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Leads | `leads` | Sales leads. |
| Additional fields | `additional_fields` | Custom/additional field definitions configured in the CRM. |
| Sales | `sales` | Sales (vendas). |
| Reservations | `reservas` | Property reservations/bookings. |
| Reservation flags | `reservas_flags` | Flags/markers attached to reservations. |
| Pre-registrations | `pre_cadastros` | Pre-registration records. |
| Lead visits | `leads_visitas` | Visits scheduled or made for leads. |
| Brokers | `corretores` | Real-estate brokers. |
| Real-estate agencies | `imobiliarias` | Real-estate agencies. |
| People | `pessoas` | People/persons (clients and contacts). |
| Developments | `empreendimentos` | Real-estate developments/projects. |
| Activation campaigns | `campanhas_ativacao` | Lead activation campaigns. |
| Units | `unidades` | Units within developments. |
| Contract terminations | `distratos` | Contract terminations (distratos) of reservations. |
| Lead interactions | `leads_interacoes` | Interactions logged against leads. |
| Lead tasks | `leads_tarefas` | Tasks associated with leads. |
| Lead status history | `leads_historico_situacoes` | History of lead status changes. |
| Commissions | `comissoes` | Sales commissions. |
| Commission details | `comissoes_detalhes` | Detailed commissions from the administrative API, including payer, beneficiaries, payment batches and payment records. |
| Service tickets | `atendimentos` | Customer service tickets/interactions. |
| Assistance requests | `assistencias` | After-sales assistance/support requests. |
| Lead conversions | `leads_conversoes` | Lead conversion records. |
| Transfers | `repasses` | Financial transfers (repasses) tied to reservations. |
| Lead workflow times | `leads_workflow_tempo` | Time spent by leads in each workflow status. |
| Reservation conditions | `reservas_condicoes` | Payment/contract conditions for reservations. |
| Reservation contracts | `reservas_contratos` | Contracts linked to reservations. |
| Reservation Sienge sync | `reservas_sienge` | Reservation records synced with Sienge (ERP). |
| Lead brokers | `leads_corretores` | Broker assignments for leads. |
| Unit status | `unidades_situacao` | Current status of units. |
| Pre-registration status history | `pre_cadastros_historico_situacoes` | History of pre-registration status changes. |
| Lead distribution | `distribuicao_leads` | Lead distribution/queue assignment logs. |
| Developments (CVbot) | `empreendimentos_cvbot` | Developments exposed via the CVbot endpoint. |
| Development units (CVbot) | `empreendimentos_unidades_cvbot` | Units per development exposed via the CVbot endpoint. |
| People professional data | `pessoas_profissional` | Professional/employment data for people. |
| Unit typologies | `tipologias_unidades` | Unit typologies (floor-plan types). |
| Reservation associates | `reservas_associados` | People associated with reservations. |
| Administrative users | `usuarios_administrativos` | Administrative (back-office) users. |
| Surveys | `pesquisas` | Surveys. |
| Survey questions | `pesquisas_perguntas` | Questions belonging to surveys. |
| Survey answers | `pesquisas_respostas` | Answers submitted to surveys. |
| Assistance items | `assistencias_itens` | Line items within assistance requests. |
| Assistance workflow times | `assistencias_workflow_tempo` | Time spent by assistance requests in each workflow status. |
| Assistance item workflow times | `assistencias_itens_workflow_tempo` | Time spent by assistance items in each workflow status. |
| Assistance visit workflow times | `assistencias_visitas_workflow_tempo` | Time spent by assistance visits in each workflow status. |
| Service ticket answers | `atendimentos_respostas` | Answers/responses within service tickets. |
| Service ticket workflow times | `atendimentos_workflow_tempo` | Time spent by service tickets in each workflow status. |
| Service ticket interactions | `atendimentos_interacoes` | Interactions logged within service tickets. |
| Unit prices | `unidades_precos` | Prices/valuations for units. |
| Service teams | `atendimentos_times` | Customer service teams. |
| Service team members | `atendimentos_times_integrantes` | Members belonging to service teams. |
| Service ticket tasks | `atendimentos_tarefas` | Tasks associated with service tickets. |
| Reservation workflow times | `reservas_workflow_tempo` | Time spent by reservations in each workflow status. |
| Transfer status history | `repasses_historico_situacoes` | History of transfer (repasse) status changes. |
## Fields by stream
Below you'll find all available data streams from CV CRM and their corresponding key fields:
Stream name: `leads`.
**Endpoint:**
* `/cvdw/leads`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idlead` | String | Lead identifier |
| `idsituacao` | String | Situation identifier |
| `situacao` | String | Situation description |
| `submotivo_cancelamento` | String | Cancellation sub-reason |
Stream name: `additional_fields`.
**Endpoint:**
* `/cvdw/campos_adicionais`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idcampo` | String | Field identifier |
| `campo_nome` | String | Field name |
| `idfuncionalidade` | String | Feature identifier |
Stream name: `sales`.
**Endpoint:**
* `/cvdw/vendas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idreserva` | String | Reservation identifier |
| `data_reserva` | String | Reservation date |
| `data_venda` | String | Sale date |
Stream name: `reservas`.
**Endpoint:**
* `/cvdw/reservas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idreserva` | String | Reservation identifier |
| `codigointerno` | String | Internal code |
| `numero_venda` | String | Sale number |
Stream name: `reservas_flags`.
**Endpoint:**
* `/cvdw/reservas/registros/flags`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idreserva` | String | Reservation identifier |
| `data_criacao_reserva` | String | Reservation creation date |
| `empreendimento` | String | Enterprise/Project name |
Stream name: `pre_cadastros`.
**Endpoint:**
* `/cvdw/precadastros`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idprecadastro` | String | Pre-registration identifier |
| `codigointerno` | String | Internal code |
| `idsituacao` | String | Situation identifier |
Stream name: `leads_visitas`.
**Endpoint:**
* `/cvdw/leads/visitas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtarefa` | String | Task identifier |
| `data_cad` | String | Registration date |
| `data` | String | Date |
Stream name: `corretores`.
**Endpoint:**
* `/cvdw/corretores`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `ativo_login` | Boolean | Login active status |
| `idcorretor` | String | Broker identifier |
| `idimobiliaria` | String | Real estate identifier |
Stream name: `imobiliarias`.
**Endpoint:**
* `/cvdw/imobiliarias`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `ativo_painel` | Boolean | Panel active status |
| `idimobiliaria` | String | Real estate identifier |
| `data_cad` | String | Registration date |
Stream name: `pessoas`.
**Endpoint:**
* `/cvdw/pessoas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idpessoa` | String | Person identifier |
| `idpessoa_int` | String | Internal person identifier |
| `idpessoa_legado` | String | Legacy person identifier |
Stream name: `empreendimentos`.
**Endpoint:**
* `/cadastros/empreendimentos`
**Sync Strategy:**
* Full table (no replication key configured)
* Primary key(s): `idempreendimento`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `idempreendimento` | String | Enterprise/Project identifier |
| `idempreendimento_int` | String | Internal enterprise/project identifier |
| `referencia_externa` | String | External reference |
| `nome` | String | Name |
| `regiao` | String | Region |
| `cidade` | String | City |
Stream name: `campanhas_ativacao`.
**Endpoint:**
* `/cvdw/campanhas_ativacao`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idcampanha_lead` | String | Lead campaign identifier |
| `idcampanha` | String | Campaign identifier |
| `campanha_nome` | String | Campaign name |
Stream name: `unidades`.
**Endpoint:**
* `/cvdw/unidades`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `idempreendimento` | String | Enterprise/Project identifier |
| `idempreendimento_int` | String | Internal enterprise/project identifier |
| `nome_empreendimento` | String | Enterprise/Project name |
| `idtipo_empreendimento` | String | Enterprise/Project type identifier |
Stream name: `distratos`.
**Endpoint:**
* `/cvdw/distratos`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idreserva` | String | Reservation identifier |
| `aprovada` | Boolean | Approved status |
| `data_cad` | String | Registration date |
Stream name: `leads_interacoes`.
**Endpoint:**
* `/cvdw/leads/interacoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idinteracao` | String | Interaction identifier |
| `idlead` | String | Lead identifier |
| `data_cad` | String | Registration date |
Stream name: `leads_tarefas`.
**Endpoint:**
* `/cvdw/leads/tarefas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtarefa` | String | Task identifier |
| `data_cad` | String | Registration date |
| `data` | String | Date |
Stream name: `leads_historico_situacoes`.
**Endpoint:**
* `/cvdw/leads/historico/situacoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idhistorico` | String | History identifier |
| `idlead` | String | Lead identifier |
| `data_cad` | String | Registration date |
Stream name: `comissoes`.
**Endpoint:**
* `/cvdw/comissoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idcomissao` | String | Commission identifier |
| `situacao` | String | Situation |
| `idsituacao` | String | Situation identifier |
Stream name: `comissoes_detalhes`.
**Endpoint:**
* `/financeiro/comissoes`
**Sync Strategy:**
* Full table (no replication key configured)
* Primary key(s): `idcomissao_cv`
Unlike the standard `comissoes` stream, this endpoint exposes the payer, the beneficiaries with their payment schedule, payment batches, payment document metadata, and payment records.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `idcomissao_cv` | Integer | Unique identifier of the commission in CVCRM |
| `idcomissao_int` | String | Internal (integration) code of the commission |
| `data_cad` | DateTime | Timestamp when the commission was registered |
| `idsituacao` | Integer | Identifier of the commission status |
| `nome_situacao` | String | Name of the commission status |
| `idreserva_cv` | Integer | Identifier of the associated reservation in CVCRM |
| `numero_venda` | String | Sale number associated with the commission |
| `idempreendimento_cv` | Integer | Identifier of the development in CVCRM |
| `empreendimento` | String | Name of the development |
| `idunidade_cv` | Integer | Identifier of the unit in CVCRM |
| `unidade` | String | Name of the unit |
| `idpessoa_cv` | Integer | Identifier of the client (person) in CVCRM |
| `valor_comissao` | Number | Total monetary value of the commission |
| `valor_comissao_tipo_comissao` | Number | Portion of the value classified as commission |
| `valor_comissao_tipo_premio` | Number | Portion of the value classified as bonus/award |
| `idcorretor_cv` | Integer | Identifier of the broker in CVCRM |
| `corretor_nome` | String | Name of the broker |
| `idimobiliaria_cv` | Integer | Identifier of the real estate agency in CVCRM |
| `imobiliaria_nome` | String | Name of the real estate agency |
| `data_cancelamento` | DateTime | Timestamp when the commission was canceled, if applicable |
| `data_finalizacao` | DateTime | Timestamp when the commission was finalized, if applicable |
| `pagador` | Object | Details of the person or company paying the commission |
| `beneficiarios` | Array | Beneficiaries of the commission with their payment schedules |
| `documentos_pagamento` | Array | Metadata of documents attached to payments (file contents are not extracted) |
| `lotes_pagamento` | Array | Payment batches linked to the commission |
| `mensagens` | Array | Internal messages posted on the commission |
| `pagamentos` | Array | Payment records of the commission |
Stream name: `atendimentos`.
**Endpoint:**
* `/cvdw/atendimentos`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idatendimento` | String | Service/Attendance identifier |
| `protocolo` | String | Protocol |
| `data_cad` | String | Registration date |
Stream name: `assistencias`.
**Endpoint:**
* `/cvdw/assistencias`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idassistencia` | String | Assistance identifier |
| `idcliente` | String | Customer identifier |
| `documento_cliente` | String | Customer document |
Stream name: `leads_conversoes`.
**Endpoint:**
* `/cvdw/leads/conversoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idlead_conversao` | String | Lead conversion identifier |
| `idlead` | String | Lead identifier |
| `nome` | String | Name |
Stream name: `repasses`.
**Endpoint:**
* `/cvdw/repasses`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `data_cad` | String | Registration date |
| `ultima_atualizacao` | String | Last update |
| `data_modificacao_situacao` | String | Situation modification date |
| `idreserva` | String | Reservation identifier |
Stream name: `leads_workflow_tempo`.
**Endpoint:**
* `/cvdw/leads/workflow/tempo`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtempo` | String | Time identifier |
| `idlead` | String | Lead identifier |
| `idsituacao` | String | Situation identifier |
Stream name: `reservas_condicoes`.
**Endpoint:**
* `/cvdw/reservas/condicoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idreservascondicoes` | String | Reservation conditions identifier |
| `idreserva` | String | Reservation identifier |
| `serie` | String | Series |
Stream name: `reservas_contratos`.
**Endpoint:**
* `/cvdw/reservas/contratos`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idreservacontrato` | String | Reservation contract identifier |
| `idreserva` | String | Reservation identifier |
| `data_cad` | String | Registration date |
Stream name: `reservas_sienge`.
**Endpoint:**
* `/cvdw/reservas/sienge`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `data_cad` | String | Registration date |
| `idsienge_reserva` | String | Sienge reservation identifier |
| `idreserva` | String | Reservation identifier |
Stream name: `leads_corretores`.
**Endpoint:**
* `/cvdw/leads/corretores`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `idlead, idcorretor, data_cad`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idlead` | String | Lead identifier |
| `data_cad` | String | Registration date |
| `idcorretor` | String | Broker identifier |
Stream name: `unidades_situacao`.
**Endpoint:**
* `/cvdw/unidades/situacao`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `idempreendimento` | String | Enterprise/Project identifier |
| `idempreendimento_int` | String | Internal enterprise/project identifier |
| `nome_empreendimento` | String | Enterprise/Project name |
| `idtipo_empreendimento` | String | Enterprise/Project type identifier |
Stream name: `pre_cadastros_historico_situacoes`.
**Endpoint:**
* `/cvdw/precadastro/historico/situacoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idhistorico` | String | History identifier |
| `idprecadastro` | String | Pre-registration identifier |
| `data_cad` | String | Registration date |
Stream name: `distribuicao_leads`.
**Endpoint:**
* `/cvdw/distribuicao/leads`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `id_log` | String | Log identifier |
| `recebido_em` | String | Received at |
| `id_fila` | String | Queue identifier |
Stream name: `empreendimentos_cvbot`.
**Endpoint:**
* `/cvbot/empreendimentos`
**Sync Strategy:**
* Full table (no replication key configured)
* Primary key(s): `idempreendimento`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `idempreendimento` | String | Enterprise identifier |
| `nome` | String | Name |
| `endereco` | String | Address |
| `cidade` | String | City |
| `estado` | String | State |
| `data_entrega` | String | Delivery date |
Stream name: `empreendimentos_unidades_cvbot`.
**Endpoint:**
* `/cvbot/empreendimentos/{idempreendimento}/unidades`
**Sync Strategy:**
* Full table (no replication key configured)
* Primary key(s): `idunidade`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `idunidade` | String | Unit identifier |
| `nome` | String | Name |
| `area_privativa` | String | Private area |
| `vagas_garagem` | String | Garage spaces |
| `andar` | String | Floor |
| `area_comum` | String | Common area |
Stream name: `pessoas_profissional`.
**Endpoint:**
* `/cvdw/pessoas/profissional`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idpessoa` | String | Person identifier |
| `idpessoa_int` | String | Internal person identifier |
| `trabalho_nome_empresa` | String | Work company name |
Stream name: `tipologias_unidades`.
**Endpoint:**
* `/configuracoes/tipologias-unidades`
**Sync Strategy:**
* Full table (no replication key configured)
* Primary key(s): `idtipologia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `idtipologia` | String | Typology identifier |
| `nome` | String | Name |
| `ativo_painel` | Boolean | Panel active status |
| `data_cad` | String | Registration date |
Stream name: `reservas_associados`.
**Endpoint:**
* `/cvdw/reservas/associados`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idassociado` | String | Associate identifier |
| `idreserva` | String | Reservation identifier |
| `idpessoa` | String | Person identifier |
Stream name: `usuarios_administrativos`.
**Endpoint:**
* `/cvdw/usuarios_administrativos`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idusuario` | String | User identifier |
| `ativo_painel` | Boolean | Panel active status |
| `tipo_login` | String | Login type |
Stream name: `pesquisas`.
**Endpoint:**
* `/cvdw/pesquisas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idpesquisa` | String | Survey identifier |
| `data_inicio` | String | Start date |
| `data_fim` | String | End date |
Stream name: `pesquisas_perguntas`.
**Endpoint:**
* `/cvdw/pesquisas/perguntas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idpesquisa_pergunta` | String | Survey question identifier |
| `idpergunta` | String | Question identifier |
| `nome_pergunta` | String | Question name |
Stream name: `pesquisas_respostas`.
**Endpoint:**
* `/cvdw/pesquisas/respostas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `data_resposta` | String | Response date |
| `idpesquisa` | String | Survey identifier |
| `idpergunta` | String | Question identifier |
Stream name: `assistencias_itens`.
**Endpoint:**
* `/cvdw/assistencias/itens`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idassistencia_item` | String | Assistance item identifier |
| `idassistencia` | String | Assistance identifier |
| `item` | String | Item |
Stream name: `assistencias_workflow_tempo`.
**Endpoint:**
* `/cvdw/assistencias/workflow/tempo`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtempo` | String | Time identifier |
| `idassistencia` | String | Assistance identifier |
| `idsituacao` | String | Situation identifier |
Stream name: `assistencias_itens_workflow_tempo`.
**Endpoint:**
* `/cvdw/assistencias/itens/workflow/tempo`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtempo` | String | Time identifier |
| `idassistencia` | String | Assistance identifier |
| `idsituacao` | String | Situation identifier |
Stream name: `assistencias_visitas_workflow_tempo`.
**Endpoint:**
* `/cvdw/assistencias/visitas/workflow/tempo`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtempo` | String | Time identifier |
| `idvisita` | String | Visit identifier |
| `idsituacao` | String | Situation identifier |
Stream name: `atendimentos_respostas`.
**Endpoint:**
* `/cvdw/atendimentos/respostas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idresposta` | String | Response identifier |
| `idatendimento` | String | Service/Attendance identifier |
| `idusuario` | String | User identifier |
Stream name: `atendimentos_workflow_tempo`.
**Endpoint:**
* `/cvdw/atendimentos/workflow/tempo`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtempo` | String | Time identifier |
| `idatendimento` | String | Service/Attendance identifier |
| `idsituacao` | String | Situation identifier |
Stream name: `atendimentos_interacoes`.
**Endpoint:**
* `/cvdw/atendimentos/interacoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idinteracao` | String | Interaction identifier |
| `idatendimento` | String | Service/Attendance identifier |
| `idusuario` | String | User identifier |
Stream name: `unidades_precos`.
**Endpoint:**
* `/cvdw/unidades/precos`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idunidade` | String | Unit identifier |
| `unidade` | String | Unit |
| `valor_avaliacao` | String | Evaluation value |
Stream name: `atendimentos_times`.
**Endpoint:**
* `/cvdw/atendimentos/times`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtime` | String | Team identifier |
| `idgestor` | String | Manager identifier |
| `gestor` | String | Manager name |
Stream name: `atendimentos_times_integrantes`.
**Endpoint:**
* `/cvdw/atendimentos/times/integrantes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idintegrante` | String | Member identifier |
| `idtime` | String | Team identifier |
| `time` | String | Team name |
Stream name: `atendimentos_tarefas`.
**Endpoint:**
* `/cvdw/atendimentos/tarefas`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idatendimento` | String | Service/Attendance identifier |
| `protocolo` | String | Protocol |
| `data_cad` | String | Registration date |
Stream name: `reservas_workflow_tempo`.
**Endpoint:**
* `/cvdw/reservas/workflow/tempo`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Reference ID |
| `referencia_data` | String | Date reference for incremental sync |
| `ativo` | Boolean | Active status |
| `idtempo` | String | Time identifier |
| `idreserva` | String | Reservation identifier |
| `idsituacao` | String | Situation identifier |
| `situacao` | String | Situation name |
| `sigla` | String | Acronym |
| `tempo` | String | Time |
| `data_cad` | String | Registration date |
Stream name: `repasses_historico_situacoes`.
**Endpoint:**
* `/cvdw/repasses/historico/situacoes`
**Sync Strategy:**
* Incremental replication key: `referencia_data`
* Primary key(s): `referencia`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `referencia` | String | Unique reference identifier of the record. |
| `referencia_data` | DateTime | Timestamp of the last reference update. |
| `ativo` | String | Indicates whether the record is active. |
| `idhistorico` | Integer | Unique identifier of the history entry. |
| `idrepasse` | Integer | Identifier of the associated transfer. |
| `idusuario` | Integer | Identifier of the user who made the change. |
| `usuario` | String | Name of the user who made the change. |
| `idcorrespondente` | Integer | Identifier of the correspondent bank associated with the change. |
| `correspondente` | String | Name of the correspondent bank associated with the change. |
| `data_cad` | DateTime | Timestamp when the record was created. |
| `de` | String | Identifier of the previous status. |
| `para` | String | Identifier of the new status. |
| `de_nome` | String | Name of the previous status. |
| `para_nome` | String | Name of the new status. |
## Implementation Notes
### Empty Response Handling
The CV CRM connector safely handles `204 No Content` and empty body responses from the API, particularly on paginated endpoints. This prevents data pipeline failures when specific data streams or pages return no data during the synchronization window.
Some endpoints (for example `Empreendimentos Unidades CVbot` or `Unidades`) signal "no data" with an HTTP `400 Bad Request` whose body is `{"error": "Nenhum dado encontrado!"}` (for instance, an *empreendimento* with no units). The connector recognizes this payload and treats it as an empty page instead of a fatal error, so the sync continues without failing.
### Module Not Enabled (HTTP 403)
CV CRM answers `403 Forbidden` with the message `Módulo não está habilitado` on endpoints that belong to a module your account has not contracted. The connector recognizes this payload and **skips the stream** instead of failing the run: the stream yields no records, a warning is logged and the corresponding table is left untouched (it is *not* emptied), while every other selected stream keeps syncing. To stop seeing this warning, either enable the module with CV CRM or unselect the stream in your source settings.
Any other `403 Forbidden` (for example a credential without access to the CVDW APIs, or a request refused outright by CV CRM) is a real failure and still aborts the run, with the API's own reason surfaced in the run logs. In that case, check the connection's **Email** and **API token** and whether the account is allowed to use the CV CRM API; if so, contact CV CRM support.
### Commission Details Sync Behavior
The `comissoes_detalhes` stream always runs as a full refresh (FULL\_TABLE). Commissions are frequently updated in place (status changes, new payment records), and the API's `a_partir_de` date filter only covers the initial registration date, so a full sync ensures that all subsequent state changes to existing commissions are accurately captured.
### Financeiro Pagination
The `/financeiro/comissoes` endpoint returns pagination metadata (`total`, `limit`, `offset`, `totalConteudo`) as string values. The connector coerces these values to integers and advances the offset by the number of records the API actually returned, so pagination stays correct even if the server caps the requested page size.
### Transfer Status History Sync Behavior
For the `repasses_historico_situacoes` stream, CV CRM applies the incremental filter (`a_partir_data_referencia`) to the parent transfer (repasse), not to the individual history row, and returns the complete status history for that transfer. As a result, rows may be returned that are older than the incremental filter date. The connector safely handles this by updating the state bookmark to the maximum timestamp seen across all records at the end of the sync, without assuming chronological sorting.
## Skills for agents
CV CRM connector documentation as plain markdown, for use in AI agent contexts.
# D4Sign as a data source
Source: https://docs.nekt.com/sources/d4sign
Bring electronic signature data from D4Sign to Nekt.
D4Sign is a Brazilian electronic signature platform that provides legal validity to documents, complying with Brazilian regulations. It allows users to send, sign, and manage documents securely online.
![]()
## Configuring D4Sign as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the D4Sign option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You will need your D4Sign **Token API** and **Crypt Key** (if enabled on your account). To find your keys, log in to your D4Sign account and navigate to the 'Dev API' menu.
The following configurations are available:
* **Token API**: Your D4Sign API Key.
* **Crypt Key**: Your D4Sign Crypt Key. This is only required if it is enabled in your account settings.
The D4Sign API has a default limit of **10 requests per hour**, which is not enough for most use cases. To increase this limit, you need to contact D4Sign support directly to request a higher limit.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from D4Sign and their corresponding fields:
Stream for managing documents.
**Key Fields:**
* `uuidDoc` - Unique identifier for the document.
* `nameDoc` - Name of the document.
* `type` - MIME type of the document.
* `size` - Size of the document in bytes.
* `pages` - Number of pages in the document.
* `uuidSafe` - Unique identifier for the safe where the document is stored.
* `safeName` - Name of the safe.
* `statusId` - ID of the document's status.
* `statusName` - Name of the status (e.g., "Aguardando assinaturas").
* `statusComment` - Optional comment related to the status.
* `whoCanceled` - Indicates who canceled the document, if applicable.
Stream for retrieving the list of signers for a document. This is a child stream of `Documents`.
**Key Fields:**
* `uuidDoc` - Unique identifier for the document.
* `nameDoc` - Name of the document.
* `type` - MIME type of the document.
* `size` - Size of the document in bytes.
* `pages` - Number of pages in the document.
* `uuidSafe` - Unique identifier for the safe where the document is stored.
* `safeName` - Name of the safe.
* `statusId` - ID of the document's status.
* `statusName` - Name of the status.
* `statusComment` - Optional comment related to the status.
* `whoCanceled` - Indicates who canceled the document, if applicable.
* `list` - A JSON string containing a complete list of signers and their details.
Stream for retrieving per-signer signature links. This is a child stream of `Documents`.
Signature links only exist while the document is waiting for signatures (status ID "3"). In any other phase, the API does not provide a signature link.
**Key Fields:**
* `uuidDoc` - UUID of the document the signer belongs to.
* `key_signer` - Signer key as returned by the D4Sign API (base64-encoded signer ID).
* `signer_id` - Signer ID obtained by base64-decoding `key_signer`, used to request the signature link.
* `email` - Signer email address.
* `user_name` - Signer display name.
* `signed` - Whether the signer has already signed the document ("1") or not ("0").
* `link` - Signature link the signer can use to sign the document. Null when the link is not available (e.g., the signer already signed or the link could not be retrieved).
Stream for listing all users in the domain.
**Key Fields:**
* `nome` - Name of the user.
* `email` - Email address of the user.
* `status` - Status of the user (e.g., "1" for active).
* `apenas_cofre_compartilhado` - Indicates if the user only has access to a shared safe ("0" for No, "1" for Yes).
## Skills for agents
D4Sign connector documentation as plain markdown, for use in AI agent contexts.
# Dafiti as a data source
Source: https://docs.nekt.com/sources/dafiti
Bring data from Dafiti Seller Center to your Lakehouse.
Dafiti is one of the largest fashion and lifestyle marketplaces in Brazil. Sellers manage their catalog, orders, and payouts in **Dafiti Seller Center**. The connector extracts your orders and order items, your catalog with stock and prices, and your financial statements and transactions, so you can analyze sales, margins, and payouts in your Lakehouse.
## Configuring Dafiti as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Dafiti option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Dafiti Seller Center authenticates API access with an **OAuth application** that you create inside your own Seller Center account. It gives you an **Application ID** and an **Application Secret**.
Sign in to Dafiti Seller Center and open **Settings** → **Integration Management** → **OAuth Applications**.
Click **Add Application** (top right corner), give it a name such as `Nekt`, and save.
Copy the **Application ID** and the **Application Secret** shown for the new application.
The OAuth application is linked to the Seller Center user who creates it. If that user is deactivated or deleted, the connection stops working. Create the application with a user that will stay active and whose role can see orders, catalog, and finance: areas the user cannot access are skipped during extraction.
The following configurations are available:
* **Application ID**: the Application ID of the OAuth application. Required.
* **Application Secret**: the Application Secret shown together with the Application ID. Required.
* **Start Date**: the earliest date of orders and financial transactions to extract. Leave it empty to extract the full history.
* **Lookback Days**: how many days are re-read before the last sync point on orders and financial transactions, so records changed while a sync was running are never missed. The default is 7.
* **Window Days**: the size, in days, of each date range requested from Dafiti. The default of 30 fits most accounts.
* **Requests Per Second**: how fast the connector may call Dafiti. Seller Center allows 30 requests per second per seller, and that budget is shared with every other integration connected to your account (ERP, hub, marketplace manager). The default of 5 leaves room for them.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Order Items come from the same requests as Orders, so selecting them adds no extraction time. Product Stocks, on the other hand, makes one request per product set, so on large catalogs it is the slowest stream; deselect it if you do not need stock levels.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Orders, Order Items, and Finance Transactions support INCREMENTAL. The catalog streams, Finance Statements, and Seller are extracted in full on every run. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Things to know
* **Customer data is masked by Dafiti.** Restricted fields of the billing and shipping addresses, such as email and phone, come masked from the API.
* **Transactions follow their statement.** A transaction belongs to the open statement (`statement_id` 0) until the statement closes. The connector re-reads the open statement and the statements closed since the last sync, so `statement_id` is updated when a statement closes.
* **Only active categories.** The category tree returned by Dafiti contains active categories only.
# Streams and Fields
Below you'll find all available data streams from Dafiti and their corresponding fields. API reference: [Seller Center REST API](https://api.sellercenter.com.br/docs/).
Marketplace orders (`GET /v2/orders`). Every sync reads the orders created or with an item changed since the last sync, re-reading the last 7 days by default.
Primary key: `id` · Replication key: `updated_at`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the order. |
| `uuid` | String | Unique string identifier of the order. |
| `number` | String | Order number shown to the seller and the customer. |
| `region_id` | String | Order identifier on the shop (region) side. |
| `seller_id` | Integer | Identifier of the seller. |
| `source` | String | Channel the order came from. |
| `customer_first_name` | String | First name of the customer. |
| `customer_last_name` | String | Last name of the customer. |
| `national_registration_number` | String | Registration number (CPF/CNPJ) of the customer. |
| `billing_address` | String | Billing address of the customer; restricted fields come masked (JSON object) |
| `shipping_address` | String | Shipping address of the customer; restricted fields come masked (JSON object) |
| `payment_method` | String | Payment method used for the order. |
| `currency` | String | Currency of the order amounts. |
| `item_count` | Integer | Number of items in the order. |
| `packed_items_count` | Integer | Number of items already packed. |
| `unit_price_sum_with_fees` | Number | Sum of the unit prices of the items, fees included. |
| `grand_total` | Number | Sum of the prices paid for the items, fees included. |
| `shipping_fee_sst_sum` | Number | Sum of the shipping fee SST tax. |
| `status_list` | String | Number of items in each item status, keyed by status (JSON object) |
| `order_item_ids` | Array of Integer | Identifiers of the items of the order. |
| `invoice_required` | Boolean | Whether an invoice is required for the order. |
| `payout_pending` | Boolean | Whether the order's payout transactions are on hold. |
| `gift_option` | Boolean | Whether the order was marked as a gift. |
| `gift_message` | String | Gift message left by the customer. |
| `voucher_code` | String | Voucher code applied to the order. |
| `voucher_type` | String | Type of the voucher applied to the order. |
| `delivery_info` | String | Additional delivery information. |
| `remarks` | String | Additional remarks about the order. |
| `extra_attributes` | String | Additional attributes set by the seller. |
| `shipment_provider_type` | String | Shipment provider type across all items (standard, express, ...) |
| `shipment_provider_pre_selected` | Boolean | Whether the shipment provider was pre-selected. |
| `exchange_by_order_id` | String | Identifier of the order that replaces this one in an exchange. |
| `exchange_for_order_id` | String | Identifier of the original order this one exchanges. |
| `printed` | Boolean | Whether all items of the order were printed. |
| `can_generate_invoice` | Boolean | Whether an invoice can be generated for the order. |
| `can_generate_shipping_label` | Boolean | Whether a shipping label can be generated for the order. |
| `target_to_ship` | Datetime | Promised shipment date, when the promised-date feature is enabled. |
| `pending_since` | Datetime | Date and time when the order's items first became pending. |
| `address_updated_at` | Datetime | Date and time when the address was last updated. |
| `created_at` | Datetime | Date and time when the order was created. |
| `updated_at` | Datetime | Date and time when the order was last updated, used as the incremental replication key. |
One row per item of each order, with price, discounts, commission, shipment and invoice data. Expanded from the order payload, so it adds no extraction time.
Primary key: `id` · Replication key: `order_updated_at`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the order item. |
| `uuid` | String | Unique string identifier of the order item. |
| `src_id` | String | Identifier of the order item on the shop side. |
| `order_id` | Integer | Identifier of the order. |
| `order_number` | String | Number of the order. |
| `seller_id` | Integer | Identifier of the seller. |
| `status` | String | Item status (pending, ready\_to\_ship, shipped, delivered, canceled, returned, ...) |
| `is_processable` | Boolean | Whether the item can be processed by the seller (not consignment or awaiting fulfilment) |
| `failure_reason_type` | String | Type of the failure reason (canceled, returned, failed, ...) |
| `failure_reason_name` | String | Name of the failure reason. |
| `failure_reason_description` | String | Description of the failure reason. |
| `failure_reason_details` | String | Additional details of the failure reason. |
| `product_name` | String | Name of the product sold. |
| `product_sku` | String | Shop SKU of the product sold. |
| `product_seller_sku` | String | Seller SKU of the product sold. |
| `product_variation` | String | Variation (usually size) of the product sold. |
| `product_image_url` | String | URL of the product image. |
| `unit_price` | Number | Price of the item. |
| `paid_price` | Number | Price paid by the customer for the item. |
| `price_after_discount` | Number | Price of the item after discounts. |
| `discount_amount` | Number | Discount applied to the item. |
| `tax_amount` | Number | Tax amount of the item. |
| `tax_percent` | Number | Tax percentage of the item. |
| `paid_commission` | Number | Commission paid on the item. |
| `shipping_fee` | Number | Shipping fee of the item. |
| `shipping_service_cost` | Number | Shipping service cost of the item. |
| `shipping_voucher_amount` | Number | Shipping voucher amount applied to the item. |
| `wallet_credits` | Number | Wallet credits used by the customer on the item. |
| `store_credits` | Number | Store credits used on the item. |
| `sales_due_amount` | Number | Sales amount due for the item. |
| `vouchers` | String | Discounts applied to the item, with the share funded by the seller (JSON array of objects) |
| `shipment_type` | String | Shipment type (warehouse, dropshipping, crossdocking, ...) |
| `shipment_method` | String | Where the item is delivered (home, pickup, ...) |
| `crossdocking_delivery_type` | String | Crossdocking delivery type (send\_to\_warehouse or pickup) |
| `shipment_provider_name` | String | Name of the shipment provider. |
| `shipment_provider_type` | String | Type of the shipment provider (standard, express, ...) |
| `pre_shipment_provider_name` | String | Name of the pre-defined shipment provider. |
| `shipment_provider_preselected` | Boolean | Whether the shipment provider was pre-selected. |
| `shipment_provider_product` | String | Product of the shipment provider. |
| `shipment_weight` | Number | Weight of the shipment. |
| `tracking_code` | String | Tracking code of the shipment. |
| `pre_tracking_code` | String | Pre-defined tracking code of the shipment. |
| `invoice_number` | String | Number of the invoice of the item. |
| `invoice_access_key` | String | Access key of the invoice (NF-e) of the item. |
| `in_transit` | Boolean | Whether the item is in transit. |
| `premium` | Boolean | Whether the item is premium. |
| `is_hybrid` | Boolean | Whether the product is a hybrid product. |
| `is_outlet` | Boolean | Whether the item carries the outlet flag. |
| `printed` | Boolean | Whether the item has been printed. |
| `manifest_status` | String | Manifest status of the item. |
| `is_pickup_request_sent` | Boolean | Whether a pickup request was sent for the item. |
| `pending_package_id` | Integer | Pending package associated with the item. |
| `warehouse_name` | String | Warehouse the item ships from. |
| `purchase_order_src_id` | Integer | Purchase order identifier on the shop side. |
| `purchase_order_number` | String | Purchase order number. |
| `purchase_delivery_info` | String | Delivery information of the purchase order. |
| `extra_attributes` | String | Extra attributes of the item, such as installment fee and delivery type (JSON object) |
| `product_set_attributes` | String | Attributes of the product set of the item (JSON object) |
| `actions` | Array of String | Next actions allowed on the item. |
| `target_to_ship_at` | Datetime | Promised shipment date, when the promised-date feature is enabled. |
| `last_status_changed_at` | Datetime | Date and time when the item status last changed. |
| `created_at` | Datetime | Date and time when the item was created. |
| `updated_at` | Datetime | Date and time when the item was last updated. |
| `order_updated_at` | Datetime | Last update of the parent order, used as the incremental replication key. |
Every product set of your catalog, in any status (`GET /v2/product-sets?status=all`). A product set groups the variations (sizes) of a product.
Primary key: `id` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the product set. |
| `uuid` | String | Unique string identifier of the product set. |
| `src_id` | String | Identifier in the shop system; empty while the set is not synchronised. |
| `name` | String | Product set name shown to customers. |
| `parent_sku` | String | SKU of the product acting as the main one of the set. |
| `seller_sku` | String | SKU of the product set on the seller's side. |
| `description` | String | Product description visible to end customers. |
| `brand_id` | Integer | Identifier of the brand of the product set. |
| `primary_category_id` | Integer | Identifier of the main category of the product set. |
| `categories` | Array of Integer | Identifiers of the additional categories the set is listed in. |
| `browse_nodes` | Array of Integer | Identifiers of the browse nodes the set is listed in. |
| `attribute_set_id` | Integer | Attribute set that defines which attributes apply to the product set. |
| `attributes` | String | Attribute values of the product set, keyed by attribute ID (JSON object) |
| `size_system` | Integer | Size system used by the product set. |
| `seller_id` | Integer | Identifier of the seller. |
| `status` | String | Product set status (active, inactive or deleted) |
| `sin` | String | Shared identifier when the set is a hybrid product sold by several sellers. |
| `created_at` | Datetime | Date and time when the product set was created. |
| `updated_at` | Datetime | Date and time when the product set was last updated. |
Every sellable variation (usually one per size) of your product sets (`GET /v2/products`).
Primary key: `id` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the product. |
| `uuid` | String | Unique string identifier of the product. |
| `product_set_id` | Integer | Identifier of the product set this variation belongs to. |
| `src_id` | String | Identifier in the shop system; empty while the product is not synchronised. |
| `shop_sku` | String | SKU on the shop's side, usually visible to customers. |
| `seller_sku` | String | SKU on the seller's side, unique across the seller's products. |
| `name` | String | Product name. |
| `variation` | String | Variation name, usually the size (e.g. M, 39) |
| `product_identifier` | String | Barcode of the product, such as EAN or Code-39. |
| `sin` | String | Shared identifier when the same product is sold by several sellers. |
| `seller_id` | Integer | Identifier of the seller. |
| `approval_status` | String | Quality-control approval status (pending, approved or rejected) |
| `status` | String | Product status (active, inactive or deleted) |
| `tax_class_id` | Integer | Tax class of this variation. |
| `shipment_type_id` | Integer | Shipment type of this variation (1 warehouse, 2 dropshipping, 3 crossdocking) |
| `duplicate_group_id` | Integer | Group identifier shared by products detected as duplicates. |
| `updated_by_user_id` | Integer | Identifier of the Seller Center user who last updated the product. |
| `src_updated_at` | Datetime | Date and time when the product was last synchronised with the shop. |
| `created_at` | Datetime | Date and time when the product was created. |
| `updated_at` | Datetime | Date and time when the product was last updated. |
Current stock of every product, including consignment and warehouse breakdowns. Seller Center only exposes stock per product set, so this stream makes one request per product set.
Primary key: `product_set_id`, `seller_sku` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `product_set_id` | Integer | Identifier of the product set. |
| `seller_sku` | String | SKU of the product on the seller's side. |
| `shop_sku` | String | SKU of the product on the shop's side. |
| `name` | String | Product name. |
| `quantity` | Integer | Sellable stock in the seller's own warehouse. |
| `reserved_stock` | Integer | Stock reserved by pending or ready-to-ship dropshipping/crossdocking items. |
| `pre_verification_stock` | Integer | Stock waiting for pre-verification. |
| `available` | Integer | Available stock: own quantity plus sellable consignment, minus reservations. |
| `sellable_stock` | Integer | Sellable consignment stock (received, canceled, returned and failed) |
| `non_sellable_stock` | Integer | Non-sellable consignment stock (quarantined and defective) |
| `consignment_received` | Integer | Received consignment stock. |
| `consignment_quarantined` | Integer | Quarantined consignment stock. |
| `consignment_defective` | Integer | Defective consignment stock. |
| `consignment_canceled` | Integer | Canceled consignment stock. |
| `consignment_returned` | Integer | Returned consignment stock. |
| `consignment_failed` | Integer | Failed consignment stock. |
| `warehouses` | String | Stock per warehouse when multi-warehouse is enabled (JSON array of objects) |
Regular and sale price of every product, one row per product and country (`GET /v2/prices`).
Primary key: `product_id`, `country` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `product_id` | Integer | Identifier of the product (variation) |
| `country` | String | Country the price applies to. |
| `product_set_id` | Integer | Identifier of the product set. |
| `price` | Number | Regular price of the product. |
| `sale_price` | Number | Sale price, when a sale is configured. |
| `sale_start_date` | Datetime | Start of the sale price period. |
| `sale_end_date` | Datetime | End of the sale price period. |
| `status` | String | Whether the product is sold in this country (e.g. active) |
Brands available to your seller account (`GET /v2/brands`).
Primary key: `id` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the brand. |
| `uuid` | String | Unique string identifier of the brand. |
| `src_id` | String | Identifier of the brand in the shop system. |
| `name` | String | Brand name. |
| `status` | String | Brand status (active, inactive or deleted) |
| `is_active` | Boolean | Whether the brand is active. |
| `is_approved` | Boolean | Whether the brand is approved for use in the marketplace. |
| `is_restricted` | Boolean | Whether the brand requires prior approval before it can be used. |
| `countries` | String | Comma-separated country codes where the brand is available. |
| `created_at` | Datetime | Date and time when the brand was created. |
| `updated_at` | Datetime | Date and time when the brand was last updated. |
The active category tree, flattened to one row per category (`GET /v2/category/tree`).
Primary key: `id` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the category. |
| `uuid` | String | Unique string identifier of the category. |
| `parent_id` | Integer | Identifier of the parent category. |
| `name` | String | Category name. |
| `attribute_set_id` | Integer | Attribute set that applies to products of this category. |
| `path_ids` | Array of Integer | Identifiers of the categories from the root down to this one. |
| `path` | Array of String | Names of the categories from the root down to this one. |
| `depth` | Integer | Depth of the category in the tree (0 for roots) |
Financial transactions of your account (item prices, commissions, shipping fees, refunds, and adjustments) for both marketplace and consignment statements (`GET /v2/finance/transactions`).
Primary key: `id` · Replication key: `date`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the transaction. |
| `date` | Datetime | Date of the transaction, used as the incremental replication key. |
| `number` | String | Transaction number. |
| `type_id` | Integer | Identifier of the transaction type. |
| `type_name` | String | Name of the transaction type (e.g. commission, item price) |
| `group_name` | String | Name of the transaction group. |
| `tre_statement_group_id` | Integer | Transaction Rule Engine statement group identifier. |
| `amount` | Number | Amount of the transaction. |
| `vat_amount` | Number | VAT amount of the transaction. |
| `wht_amount` | Number | Withholding tax amount of the transaction. |
| `currency` | String | Currency of the transaction (ISO 4217) |
| `country` | String | Country of the transaction (ISO 3166-1 alpha-2) |
| `source` | String | Where the transaction was created (sellercenter, web, csv) |
| `comment` | String | Comment attached to the transaction. |
| `statement_id` | Integer | Statement the transaction belongs to; 0 while the statement is still open. |
| `statement_type` | String | Type of the statement of the transaction (marketplace or consignment) |
| `order_item_id` | Integer | Order item the transaction refers to, if any. |
| `order_number` | String | Order number the transaction refers to, if any. |
Account statements: the payout periods with their balances, due dates, and payment status (`GET /v2/finance/statements`).
Primary key: `id`, `type` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the statement. |
| `type` | String | Statement type (marketplace or consignment) |
| `number` | String | Statement number. |
| `seller_id` | Integer | Identifier of the seller. |
| `start_date` | Datetime | Start of the period covered by the statement. |
| `end_date` | Datetime | End of the period covered by the statement. |
| `due_date` | Datetime | Date the statement payout is due. |
| `paid_at` | Datetime | Date and time the statement was paid. |
| `paid` | Boolean | Whether the statement has been paid. |
| `opening_balance` | Number | Balance at the start of the period. |
| `closing_balance` | Number | Balance at the end of the period. |
| `payout_amount` | Number | Amount paid out to the seller. |
| `guarantee_deposit` | Number | Guarantee deposit held from the payout. |
| `currency` | String | Currency of the statement (ISO 4217) |
| `country` | String | Country of the statement (ISO 3166-1 alpha-2) |
| `note` | String | Note attached to the statement. |
| `commission_invoice` | String | URL of the commission invoice document of the statement. |
The seller account the OAuth application belongs to (`GET /v2/seller`).
Primary key: `id` · Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique numeric identifier of the seller. |
| `uuid` | String | Unique string identifier of the seller. |
| `src_id` | String | Identifier of the seller in the Dafiti shop system. |
| `short_code` | String | Short code of the seller. |
| `status` | String | Seller status (active, inactive or deleted) |
| `name` | String | Seller display name. |
| `company_name` | String | Legal name of the seller company. |
| `email` | String | Contact email of the seller. |
| `tax_class` | String | Tax class of the seller. |
| `verified` | Boolean | Whether the seller account has been verified. |
| `listed` | Boolean | Whether the seller's products are currently listed in the shop. |
| `delist_reasons` | String | Reasons why the seller is delisted, such as holiday mode or order limits (JSON array of objects) |
| `profile_fields` | String | Approved values of the seller profile fields, with their display values (JSON array of objects) |
| `agreements_accepted` | Boolean | Whether the seller has accepted the current marketplace agreements. |
| `created_at` | Datetime | Date and time when the seller account was created. |
| `updated_at` | Datetime | Date and time when the seller account was last updated. |
# Databricks as a data source
Source: https://docs.nekt.com/sources/databricks
Bring data from Databricks to Nekt.
Databricks is a cloud-based data analytics and machine learning platform built on Apache Spark. It enables organizations to process large volumes of data, build data pipelines, and collaborate on analytics and AI projects in a unified environment.
## 1. Add your Databricks access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Databricks option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access:
* **Databricks instance:** The workspace instance name of your Databricks deployment. For detailed instructions, click [here](https://docs.databricks.com/en/workspace/workspace-details.html#workspace-instance-names-urls-and-ids).
* **Personal access token:** Databricks personal access token authentication (PATs). For detailed instructions, click [here](https://docs.databricks.com/en/dev-tools/auth/pat.html).
* **SQL Warehouse ID:** A SQL warehouse is a compute resource that lets you run SQL commands on data objects within Databricks SQL. We recommend using a serverless SQL warehouse with size 2X-Small and auto-stop enabled after 5 minutes of inactivity. For detailed instructions, click [here](https://docs.databricks.com/en/admin/sql/serverless.html).
3. Click **Next**.
## 2. Select your Databricks streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Databricks data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Databricks data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Databricks connector documentation as plain markdown, for use in AI agent contexts.
# Datadog as a data source
Source: https://docs.nekt.com/sources/datadog
Bring data from Datadog to Nekt.
Datadog is a monitoring and analytics platform for cloud-scale applications, providing monitoring of servers, databases, tools, and services through a SaaS-based data analytics platform. It offers comprehensive logging, metrics, and tracing capabilities to help organizations monitor their infrastructure and applications in real-time.
![]()
## Configuring Datadog as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Datadog option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide API credentials from your Datadog account to authenticate the connection.
The following configurations are available:
* **API Key**: The API key to authenticate against the API service. You can find this in your Datadog account under Organization Settings > API Keys. Check [Datadog's documentation](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys) for more details.
* **App Key**: The application key to authenticate against the API service. You can find this in your Datadog account under Organization Settings > Application Keys. For RUM streams, it needs the `rum_apps_read` permission. Check [Datadog's documentation](https://docs.datadoghq.com/account_management/api-app-keys/#application-keys) for more details.
* **Start Date** (optional): Starting point for data extraction. Leave empty to backfill the last 30 days on the first run (Datadog RUM retention).
* **Logs query** (optional): The query to search for logs (applies to the logs stream only). You can use Datadog's query syntax to filter by service, host, tags, or any other attributes. If not provided, all logs will be retrieved.
* **Datadog site** (optional): Datadog region your account lives in. Check the URL you use to open Datadog: `app.datadoghq.com` is US1 (default), `us5.datadoghq.com` is US5, `app.datadoghq.eu` is EU, and so on. Default is `datadoghq.com`.
* **RUM query filter** (optional): Optional extra RUM search query applied to every RUM stream, e.g., `@application.name:my-app`. Leave empty to extract all RUM applications.
* **Enable RUM resource events** (optional): Adds the `rum_resource` stream (network requests, assets). This is by far the highest-volume RUM event type and Datadog keeps it for only 15 days, so enable it as soon as you decide you need it.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. The Datadog connector exposes streams for **logs** and **RUM (Real User Monitoring) events** (actions, views, sessions, errors, and optionally resources).
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Datadog and their corresponding fields:
Stream for retrieving log events from Datadog with their associated metadata and attributes.
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique identifier for each log event |
| `timestamp` | String | When the log event occurred |
| `type` | String | Type of log event |
| `attributes` | Object | Additional log metadata including host, service, message, status, tags, and custom attributes |
**Attributes Object Contains:**
* `host` - The host where the log originated
* `service` - The service name
* `message` - The log message content
* `status` - Status of the log event
* `tags` - Array of tags associated with the log
* `attributes` - Array of custom attributes with key-value pairs
* `timestamp` - Timestamp within the attributes object
**Primary Key:** `id`
**Replication Key:** `timestamp`
Stream for retrieving Real User Monitoring (RUM) user action events like clicks, taps, and custom actions.
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique identifier of the RUM event. |
| `event_type` | String | RUM event type (e.g., action). |
| `timestamp` | Datetime | Time the event was recorded by Datadog (ISO 8601, UTC). Used as the incremental bookmark. |
| `service` | String | Name of the application or service that generated the event. |
| `tags` | Array | Tags attached to the event (key:value strings). |
| `application_id` | String | Identifier of the RUM application that emitted the event. |
| `application_name` | String | Name of the RUM application that emitted the event. |
| `session_id` | String | Identifier of the user session the event belongs to. |
| `view_id` | String | Identifier of the view (page/screen) the event belongs to. |
| `usr_id` | String | Identifier of the end user, when set through the RUM SDK. |
| `date_ns` | Integer | Event date as reported by the RUM SDK, in nanoseconds since epoch. |
| `attributes` | String | Full event payload as delivered by Datadog, serialized as a JSON string. All duration fields are in nanoseconds. |
**Primary Key:** `id`
**Replication Key:** `timestamp`
Stream for retrieving RUM page view and screen events.
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique identifier of the RUM event. |
| `event_type` | String | RUM event type (e.g., view). |
| `timestamp` | Datetime | Time the event was recorded by Datadog (ISO 8601, UTC). Used as the incremental bookmark. |
| `service` | String | Name of the application or service that generated the event. |
| `tags` | Array | Tags attached to the event (key:value strings). |
| `application_id` | String | Identifier of the RUM application that emitted the event. |
| `application_name` | String | Name of the RUM application that emitted the event. |
| `session_id` | String | Identifier of the user session the event belongs to. |
| `view_id` | String | Identifier of the view (page/screen) the event belongs to. |
| `usr_id` | String | Identifier of the end user, when set through the RUM SDK. |
| `date_ns` | Integer | Event date as reported by the RUM SDK, in nanoseconds since epoch. |
| `attributes` | String | Full event payload as delivered by Datadog, serialized as a JSON string. All duration fields are in nanoseconds. |
**Primary Key:** `id`
**Replication Key:** `timestamp`
Stream for retrieving RUM user session events.
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique identifier of the RUM event. |
| `event_type` | String | RUM event type (e.g., session). |
| `timestamp` | Datetime | Time the event was recorded by Datadog (ISO 8601, UTC). Used as the incremental bookmark. |
| `service` | String | Name of the application or service that generated the event. |
| `tags` | Array | Tags attached to the event (key:value strings). |
| `application_id` | String | Identifier of the RUM application that emitted the event. |
| `application_name` | String | Name of the RUM application that emitted the event. |
| `session_id` | String | Identifier of the user session the event belongs to. |
| `view_id` | String | Identifier of the view (page/screen) the event belongs to. |
| `usr_id` | String | Identifier of the end user, when set through the RUM SDK. |
| `date_ns` | Integer | Event date as reported by the RUM SDK, in nanoseconds since epoch. |
| `attributes` | String | Full event payload as delivered by Datadog, serialized as a JSON string. All duration fields are in nanoseconds. |
**Primary Key:** `id`
**Replication Key:** `timestamp`
Stream for retrieving RUM front-end error events.
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique identifier of the RUM event. |
| `event_type` | String | RUM event type (e.g., error). |
| `timestamp` | Datetime | Time the event was recorded by Datadog (ISO 8601, UTC). Used as the incremental bookmark. |
| `service` | String | Name of the application or service that generated the event. |
| `tags` | Array | Tags attached to the event (key:value strings). |
| `application_id` | String | Identifier of the RUM application that emitted the event. |
| `application_name` | String | Name of the RUM application that emitted the event. |
| `session_id` | String | Identifier of the user session the event belongs to. |
| `view_id` | String | Identifier of the view (page/screen) the event belongs to. |
| `usr_id` | String | Identifier of the end user, when set through the RUM SDK. |
| `date_ns` | Integer | Event date as reported by the RUM SDK, in nanoseconds since epoch. |
| `attributes` | String | Full event payload as delivered by Datadog, serialized as a JSON string. All duration fields are in nanoseconds. |
**Primary Key:** `id`
**Replication Key:** `timestamp`
Stream for retrieving RUM network resources (XHR, fetch, assets). This stream is high volume and requires enabling explicitly via the "Enable RUM resource events" option.
| Field | Type | Description |
| - | - | - |
| `id` | String | Unique identifier of the RUM event. |
| `event_type` | String | RUM event type (e.g., resource). |
| `timestamp` | Datetime | Time the event was recorded by Datadog (ISO 8601, UTC). Used as the incremental bookmark. |
| `service` | String | Name of the application or service that generated the event. |
| `tags` | Array | Tags attached to the event (key:value strings). |
| `application_id` | String | Identifier of the RUM application that emitted the event. |
| `application_name` | String | Name of the RUM application that emitted the event. |
| `session_id` | String | Identifier of the user session the event belongs to. |
| `view_id` | String | Identifier of the view (page/screen) the event belongs to. |
| `usr_id` | String | Identifier of the end user, when set through the RUM SDK. |
| `date_ns` | Integer | Event date as reported by the RUM SDK, in nanoseconds since epoch. |
| `attributes` | String | Full event payload as delivered by Datadog, serialized as a JSON string. All duration fields are in nanoseconds. |
**Primary Key:** `id`
**Replication Key:** `timestamp`
# Data Model
The following diagram shows the structure of the Datadog logs and RUM streams:
```mermaid theme={null}
erDiagram
LOGS {
string id PK
string timestamp
string type
object attributes
}
RUM_ACTION {
string id PK
datetime timestamp
string event_type
string session_id
string view_id
string attributes
}
RUM_VIEW {
string id PK
datetime timestamp
string event_type
string session_id
string view_id
string attributes
}
RUM_SESSION {
string id PK
datetime timestamp
string event_type
string session_id
string attributes
}
RUM_ERROR {
string id PK
datetime timestamp
string event_type
string session_id
string view_id
string attributes
}
RUM_RESOURCE {
string id PK
datetime timestamp
string event_type
string session_id
string view_id
string attributes
}
```
# Implementation Notes
### Data Quality Considerations
* The logs and RUM streams both use `timestamp` as the replication key for incremental syncs.
* Custom log attributes and tags provide additional context for filtering and analysis.
* For RUM streams, the `attributes` column preserves the full nested event payload exactly as delivered by Datadog, serialized as a JSON string. This includes details like `action.target.name`, `view.url`, and `error.message`.
* Duration fields found inside the RUM `attributes` payload are delivered in nanoseconds.
* RUM event timestamps are re-serialized with fixed millisecond precision (`.mmmZ`). Datadog omits the fractional part when the milliseconds are zero, and mixed precision would break the chronological ordering the incremental bookmark relies on.
* By default, leaving the **Start Date** empty will backfill the last 30 days of RUM history (matching Datadog's standard retention period).
### API Limits & Performance
* Use the `query` and `rum_query` parameters to filter events at the API level, heavily reducing extraction time and data volume.
* Consider the volume of events when setting extraction frequency. The `rum_resource` stream in particular represents an extremely high volume of data (every network request and loaded asset), which is why it requires manual opt-in. Datadog typically retains resource events for only 15 days.
* Datadog's API has rate limits that the connector respects automatically, retrying on HTTP 429 (Too Many Requests). If Datadog responds with partial timeout statuses, the connector will retry those pages seamlessly.
* The connector also retries transient server-side timeouts (HTTP 408) that Datadog's search endpoints may return when querying large datasets or wide time windows, instead of failing the run.
# Digisac as a data source
Source: https://docs.nekt.com/sources/digisac
Bring data from Digisac to your Lakehouse.
Digisac is a Brazilian multichannel customer-service platform that centralizes WhatsApp, Telegram, Instagram, Facebook, e-mail and other channels into a single service desk, with departments, chatbots, campaigns, satisfaction surveys and a sales funnel. The connector extracts the service history (tickets, messages and contacts) along with the account configuration and the sales funnel, so you can analyse your customer service in your Lakehouse.
Every Digisac account runs on its own address, so the connector asks for the URL of your platform in addition to the access token.
## Configuring Digisac as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Digisac option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Digisac authenticates with a **personal access token**, sent as a Bearer token. To generate one, open the **user menu** in Digisac, go to **Conta > API > Tokens de acesso pessoal** and click **Gerar token**. Give the token a name, choose an expiration period and click **Gerar token** again.
Digisac only shows the token once, in the window that opens right after it is generated. Copy it before closing that window; it cannot be retrieved afterwards, and a new token has to be generated instead.
The expiration period is your choice: 30, 90 or 180 days, 1 year, or **Não expirar**. Pick **Não expirar**: with any other option the source stops extracting on the expiry date and the token has to be replaced by hand.
The token inherits the permissions of the user who created it. Generate it from a user with **administrator** permissions so the sync can reach every module; an operator token is denied access to part of the platform.
The following configurations are available:
* **Digisac URL**: the address of your Digisac platform, the same one you use to log in, for example `https://mycompany.digisac.chat`. Required.
* **Access token**: the personal access token copied from **Conta > API > Tokens de acesso pessoal**. Required.
* **Initial sync date**: the earliest date from which records will be synced. It applies only to the streams replicated incrementally: Contacts, Messages, Tickets, Campaigns and Sales Opportunities. Leave it empty to extract the full history.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
**Messages** is by far the largest stream on a Digisac account: it holds every message exchanged on every channel. The first extraction reads the whole history, so it takes considerably longer than the others; the runs after it only read what changed.
Some streams depend on modules that may not be enabled on your plan: WhatsApp Templates requires an official WhatsApp Business API connection, and Sales Funnels, Sales Opportunities and the satisfaction surveys require those modules to be active. When a module is unavailable the stream is skipped with a warning in the run log instead of failing the extraction.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Contacts, Messages, Tickets, Campaigns and Sales Opportunities support INCREMENTAL; the remaining streams are full table. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Digisac and their corresponding fields. API reference: [Digisac API documentation](https://documenter.getpostman.com/view/24605757/2sA3BhfaDg).
Answers contacts gave to the satisfaction surveys (`GET /api/v1/answers`), one row per answered survey, carrying the NPS or CSAT score itself.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the answer. |
| `value` | String | Score or option the contact answered with. |
| `questionId` | String | Identifier of the survey that was answered. |
| `contactId` | String | Identifier of the contact who answered. |
| `ticketId` | String | Identifier of the ticket the survey was sent for. |
| `userId` | String | Identifier of the agent who handled the rated service. |
| `departmentId` | String | Identifier of the department that handled the rated service. |
| `serviceId` | String | Identifier of the connection the survey was sent on. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Chatbots configured on the account (`GET /api/v1/bots`). Only the bot's identification is extracted; the conversation flow itself stays in the platform.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the bot. |
| `name` | String | Name of the bot. |
| `settings` | String | Bot settings as a JSON object; `botCreatedVersion` identifies which builder version the bot was created with. |
| `archivedAt` | Datetime | Timestamp when the bot was archived, or null while active. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Mass-message campaigns sent from the account (`GET /api/v1/campaigns`).
Primary key: `id`
Replication key: `createdAt` (incremental sync).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the campaign. |
| `name` | String | Name of the campaign. |
| `status` | String | Current status of the campaign, such as scheduled or sent. |
| `scheduledAt` | Datetime | Timestamp the campaign was scheduled to go out. |
| `serviceId` | String | Identifier of the connection used to send the campaign. |
| `createdById` | String | Identifier of the user who created the campaign. |
| `sentById` | String | Identifier of the user who triggered the send. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Opportunities on the sales funnels (`GET /api/v1/cards`). Read one funnel at a time, as a child of Sales Funnels.
Primary key: `id`
Replication key: `updatedAt` (incremental sync).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the opportunity. |
| `title` | String | Title of the opportunity. |
| `description` | String | Free-text description of the opportunity. |
| `value` | Number | Monetary value attached to the opportunity. |
| `order` | Integer | Position of the opportunity within its funnel stage. |
| `pipelineId` | String | Identifier of the funnel the opportunity belongs to. |
| `pipelineStageId` | String | Identifier of the funnel stage the opportunity sits in. |
| `contactId` | String | Identifier of the contact the opportunity refers to. |
| `ownerId` | String | Identifier of the user responsible for the opportunity. |
| `statusId` | String | Identifier of the outcome status, such as won or lost. |
| `reasonId` | String | Identifier of the reason recorded for the outcome. |
| `organization` | String | Organization recorded on the opportunity. |
| `organizationSegment` | String | Market segment recorded for the organization. |
| `originChannel` | String | Channel the opportunity originated from. |
| `originCampaign` | String | Campaign the opportunity originated from. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
People and groups the account exchanges messages with (`GET /api/v1/contacts`). A contact is a number, e-mail address or Telegram user on one specific connection, so the same person reached through two connections is two contacts; `personId` is what ties them together.
Primary key: `id`
Replication key: `updatedAt` (incremental sync).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the contact. |
| `name` | String | Name of the contact as resolved by Digisac: the name saved on the platform when there is one, otherwise the name published by the messaging channel. |
| `internalName` | String | Name given to the contact inside Digisac by the team. |
| `alternativeName` | String | Name published by the messaging channel itself, such as the WhatsApp profile name or the formatted phone number. |
| `data` | String | Channel-specific identification of the contact, as a JSON object. For WhatsApp connections it carries the phone number in the `number` key. |
| `note` | String | Free-text note the team recorded about the contact. |
| `status` | String | Presence reported by the messaging channel, such as online or offline. |
| `serviceId` | String | Identifier of the connection (WhatsApp, Telegram, e-mail…) this contact belongs to. |
| `personId` | String | Identifier of the person this contact was grouped under, linking the same individual across connections. |
| `defaultDepartmentId` | String | Department that receives this contact's tickets by default. |
| `defaultUserId` | String | Agent who receives this contact's tickets by default. |
| `lastMessageId` | String | Identifier of the most recent message exchanged. |
| `lastMessage` | String | Most recent message exchanged with the contact, as a JSON object. Only populated when the API is asked to include it. |
| `lastMessageAt` | Datetime | Timestamp of the most recent message in either direction. |
| `lastContactMessageAt` | Datetime | Timestamp of the most recent message sent by the contact. |
| `unread` | Integer | Number of unread messages from this contact. |
| `unsubscribed` | Boolean | Whether the contact opted out of receiving campaign messages. |
| `isMe` | Boolean | Whether the contact is the account's own connection number. |
| `isGroup` | Boolean | Whether the contact is a group conversation. |
| `isBroadcast` | Boolean | Whether the contact is a broadcast list. |
| `isSilenced` | Boolean | Whether notifications from this contact are muted. |
| `isMyContact` | Boolean | Whether the contact is saved in the address book of the connected device. |
| `hadChat` | Boolean | Whether any conversation ever took place with this contact. |
| `visible` | Boolean | Whether the contact is shown in the platform's chat list. Hidden contacts are usually system or service entries. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Definitions of the extra fields the account added to contact records (`GET /api/v1/custom-fields`). This stream carries the field definitions, not their values.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the custom field. |
| `name` | String | Label of the custom field. |
| `type` | String | Data type of the field, such as text. |
| `allowed` | String | Kind of record the field can be attached to, such as contacts. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Departments that tickets are routed to, such as Support or Sales (`GET /api/v1/departments`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the department. |
| `name` | String | Name of the department. |
| `distributionId` | String | Identifier of the ticket-distribution rule applied to the department. |
| `archivedAt` | Datetime | Timestamp when the department was archived, or null while active. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Rules that spread incoming tickets across the agents of a department (`GET /api/v1/distribution`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the distribution rule. |
| `name` | String | Name of the distribution rule. |
| `maxNum` | Integer | Maximum number of simultaneous tickets per agent. |
| `timeToRedistribute` | Integer | Minutes a ticket waits in an offline agent's queue before being redistributed. |
| `distributeQueue` | Boolean | Whether tickets are handed out from a queue rather than assigned directly. |
| `redistributeAll` | Boolean | Whether every ticket is redistributed, not only new ones. |
| `redistributeAssignedTickets` | Boolean | Whether unanswered assigned tickets are redistributed. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Dates the chatbot treats as non-working days (`GET /api/v1/holiday`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the holiday. |
| `name` | String | Name of the holiday. |
| `from` | Datetime | Timestamp the holiday period starts at. |
| `to` | Datetime | Timestamp the holiday period ends at. |
| `message` | String | Message sent to contacts who write during the holiday. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Reusable interactive messages: button and list menus sent to contacts (`GET /api/v1/interactive-messages`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the interactive message. |
| `name` | String | Name identifying the interactive message internally. |
| `interactive` | String | Definition of the interactive content (type, header, body, footer and the buttons or list rows), as a JSON object. |
| `file` | String | File attached to the interactive message, as a JSON object. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Every message exchanged on the account, in both directions (`GET /api/v1/messages`). Not every record is a chat message: Digisac writes system entries onto the same timeline (`type` = `ticket` when a ticket opens or closes, for example), which is why `text` is frequently null.
Primary key: `id`
Replication key: `updatedAt` (incremental sync).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the message. |
| `type` | String | Kind of entry: `chat` for a message exchanged with the contact, `comment` for an internal note, and `ticket` for the system entries Digisac writes when a ticket opens, closes or is transferred. |
| `text` | String | Text of the message. Null for attachments without a caption and for the system entries. |
| `timestamp` | Datetime | Timestamp of the message on the conversation timeline: this is the one to use for conversation analysis, since `createdAt` records when Digisac stored it. |
| `isFromMe` | Boolean | Whether the message was sent by the account (an agent or a bot) rather than received from the contact. |
| `isFromBot` | Boolean | Whether the message was produced by a chatbot. |
| `isFromSync` | Boolean | Whether the message was imported from the connected device's history rather than sent or received through Digisac. |
| `sent` | Boolean | Whether the message was successfully handed to the channel. |
| `visible` | Boolean | Whether the message is shown in the conversation. |
| `obfuscated` | Boolean | Whether the message content was masked for privacy in the interface. |
| `origin` | String | What produced the message, such as `user`, `bot`, `ticket` or `campaign`. |
| `data` | String | Channel-specific metadata of the message, as a JSON object. For WhatsApp it carries the delivery receipt in the `ack` key; for system entries it describes the event, for example `ticketClose`. |
| `files` | String | Attachments sent with the message, as a JSON array of objects. |
| `quotedMessage` | String | Message this one replies to, as a JSON object. |
| `quotedMessageId` | String | Identifier of the message this one replies to. |
| `contactId` | String | Identifier of the contact the conversation belongs to. |
| `serviceId` | String | Identifier of the connection that carried the message. |
| `fromId` | String | Identifier of the contact that sent the message, when it came from outside. |
| `toId` | String | Identifier of the recipient contact, when applicable. |
| `userId` | String | Identifier of the agent who sent the message. |
| `botId` | String | Identifier of the chatbot that produced the message. |
| `campaignId` | String | Identifier of the campaign that produced the message. |
| `ticketId` | String | Identifier of the ticket the message belongs to. |
| `ticketUserId` | String | Agent assigned to the ticket at the moment the message was written. |
| `ticketDepartmentId` | String | Department the ticket belonged to at the moment the message was written. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Organizations: groupings of people used to scope third-party access (`GET /api/v1/organizations`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the organization. |
| `name` | String | Name of the organization. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
People: the grouping that ties a contact's several channels together (`GET /api/v1/people`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the person. |
| `name` | String | Full name of the person. |
| `document` | String | Identification document recorded for the person, such as CPF. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Sales funnels configured on the account (`GET /api/v1/pipelines`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the sales funnel. |
| `name` | String | Name of the sales funnel. |
| `type` | String | Kind of funnel, such as sales. |
| `goBack` | Boolean | Whether opportunities may be moved back to a previous stage, or only forward. |
| `stages` | String | Stages that make up the funnel, as a JSON array of objects. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Satisfaction surveys (NPS/CSAT) configured to run at the end of a service (`GET /api/v1/questions`). This stream carries the survey definitions; the scores are in Survey Answers.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the survey. |
| `name` | String | Name of the survey. |
| `type` | String | Kind of survey, either NPS or CSAT. |
| `questionMessage` | String | Question sent to the contact. |
| `successMessage` | String | Message sent after a valid answer is received. |
| `invalidMessage` | String | Message sent when the contact answers with something invalid. |
| `duration` | Integer | How long the survey stays open for an answer. |
| `tries` | Integer | How many times the survey is re-sent after an invalid answer. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Canned replies agents can insert into a conversation (`GET /api/v1/quick-replies`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the quick reply. |
| `text` | String | Body of the quick reply. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Access profiles that define what a user may do on the platform (`GET /api/v1/roles`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the role. |
| `name` | String | Internal name of the role. |
| `displayName` | String | Name of the role as shown in the interface. |
| `isAdmin` | Boolean | Whether the role grants administrative access. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Scheduled actions: future ticket openings and agent notifications (`GET /api/v1/schedule`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the scheduled action. |
| `scheduledAt` | Datetime | Timestamp the action is scheduled to run at. |
| `message` | String | Message to be sent when the schedule runs. |
| `notes` | String | Internal note recorded with the schedule. |
| `openTicket` | Boolean | Whether a ticket is opened when the schedule runs. |
| `notificateUser` | Boolean | Whether the assigned agent is notified when it runs. |
| `contactId` | String | Identifier of the contact the schedule refers to. |
| `departmentId` | String | Identifier of the department the schedule is aimed at. |
| `userId` | String | Identifier of the agent the schedule is aimed at. |
| `files` | String | Files to be sent with the message, as a JSON array. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Connections: the messaging channels plugged into the account (`GET /api/v1/services`). One record per WhatsApp number, Telegram bot, e-mail box or other channel; every contact and message points at one of these through `serviceId`.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the connection. |
| `name` | String | Name of the connection. |
| `type` | String | Channel of the connection, such as whatsapp, telegram or email. |
| `botId` | String | Identifier of the chatbot answering on this connection. |
| `defaultDepartmentId` | String | Department that receives tickets opened on this connection by default. |
| `archivedAt` | Datetime | Timestamp when the connection was archived, or null while active. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Labels used to classify contacts (`GET /api/v1/tags`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the tag. |
| `label` | String | Display name of the tag. |
| `backgroundColor` | String | Hexadecimal color the tag is drawn with in the interface. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Closing subjects agents pick when they end a ticket (`GET /api/v1/ticket-topics`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the closing subject. |
| `name` | String | Name of the closing subject. |
| `archivedAt` | Datetime | Timestamp when the subject was archived, or null while it is available to agents. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Service tickets (`GET /api/v1/tickets`), one per conversation an agent handled with a contact. A ticket opens when a conversation starts and closes when an agent ends the service, so this is the stream that carries handling time, routing and closing subject.
Primary key: `id`
Replication key: `updatedAt` (incremental sync).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the ticket. |
| `protocol` | String | Protocol number shown to the contact and used to look the service up in the platform. |
| `isOpen` | Boolean | Whether the ticket is still open. Closed tickets keep an `endedAt`. |
| `origin` | String | What opened the ticket, such as a message from the contact, an agent, a bot or a campaign. |
| `startedAt` | Datetime | Timestamp when the service started. |
| `endedAt` | Datetime | Timestamp when the service was closed, or null while it is open. |
| `contactId` | String | Identifier of the contact being served. |
| `userId` | String | Identifier of the agent the ticket is assigned to. |
| `departmentId` | String | Identifier of the department handling the ticket. |
| `botId` | String | Identifier of the chatbot that handled the ticket. |
| `serviceId` | String | Identifier of the connection the conversation arrived on. |
| `firstMessageId` | String | Identifier of the message that opened the ticket. |
| `lastMessageId` | String | Identifier of the last message on the ticket. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
Agents and administrators who log in to the platform (`GET /api/v1/users`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the user. |
| `name` | String | Full name of the user. |
| `email` | String | Login e-mail of the user. |
| `phoneNumber` | String | Phone number recorded for the user. |
| `status` | String | Availability of the agent on the platform, such as online, away or offline. |
| `active` | Boolean | Whether the user account is active. |
| `isSuperAdmin` | Boolean | Whether the user has full administrative access. |
| `isFirstLogin` | Boolean | Whether the user has yet to log in for the first time. |
| `isActiveInternalChat` | Boolean | Whether the user has the internal chat enabled. |
| `language` | String | Interface language chosen by the user, such as pt-BR. |
| `timetableId` | String | Identifier of the working-hours schedule assigned to the user. |
| `archivedAt` | Datetime | Timestamp when the user was archived, or null while still in use. |
| `data` | String | Additional user settings, as a JSON object. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
WhatsApp message templates (HSM) approved for the official connection (`GET /api/v1/whatsapp-business-templates`). Only accounts with a WhatsApp Business API connection have templates.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the template. |
| `name` | String | Name of the template as registered with WhatsApp. |
| `internalName` | String | Name shown for the template inside Digisac. |
| `components` | String | Header, body, footer and buttons that make up the template, as a JSON array of objects. |
| `messageType` | String | Kind of template content, such as text\_only. |
| `language` | String | Language the template was approved in, such as pt\_BR. |
| `category` | String | Category the template was submitted under, such as MARKETING or UTILITY. |
| `status` | String | Approval status returned by WhatsApp. |
| `namespace` | String | WhatsApp namespace the template belongs to. |
| `serviceId` | String | Identifier of the connection the template belongs to. |
| `accountId` | String | Identifier of the Digisac account the record belongs to. |
| `createdAt` | Datetime | Timestamp when the record was created in Digisac. |
| `updatedAt` | Datetime | Timestamp when the record was last updated in Digisac. |
| `deletedAt` | Datetime | Timestamp when the record was soft-deleted, or null while it is active. |
# Dinamize as a data source
Source: https://docs.nekt.com/sources/dinamize
Bring data from Dinamize to Nekt.
Dinamize is an email marketing platform that provides comprehensive tools for managing email campaigns, contact lists, and marketing automation. It enables businesses to create, send, and track email campaigns, manage subscriber lists, segment contacts, and analyze marketing performance through detailed transaction and engagement data.
## Configuring Dinamize as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Dinamize option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Dinamize credentials to access your data.
The following configurations are required:
* **User**: Your Dinamize username (Nome do usuário)
* **Password**: Your Dinamize password (Senha do usuário)
* **Client Code**: Your Dinamize client code (Código do cliente)
The following configuration is optional:
* **Start Date**: (Optional) The earliest date from which records will be synced. If not provided, all available data will be synced. This is particularly useful for streams with replication keys to limit the initial sync window.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Dinamize and their corresponding fields:
Stream for managing contact information, including email addresses, names, custom fields, and contact preferences.
**Identifiers:**
* `code` - Unique identifier for the contact (primary key)
* `external_code` - External code of the contact
**Contact Information:**
* `email` - Contact email address
* `name` - Contact name
* `custom_fields` - Custom fields object (cmp4, cmp5, cmp6...)
**Dates:**
* `insert_date` - Registration date (replication key)
* `rest_date` - Rest date (contacts will not receive emails until the end of the specified day)
Stream for managing contact lists, which group contacts for campaign targeting and segmentation.
**Identifiers:**
* `code` - Unique identifier for the contact list (primary key)
* `user_code` - User code of the contact list owner
**List Information:**
* `title` - Contact list title
* `type` - Key field for duplicate verification: EM (email) | CE (external code)
* `status` - Contact list status: LI (released) | EA (under review) | BL (blocked)
**Metrics:**
* `total_contacts` - Total number of contacts
* `total_valid` - Total number of valid contacts
**Dates:**
* `insert_date` - Registration date (replication key)
* `rest_date` - Rest date (contacts in the list will not receive emails until the end of the specified day)
Stream for managing email sends (envios), including campaign dispatches, scheduling, and send statistics.
**Identifiers:**
* `code` - Unique identifier for the send (primary key)
* `campaign_code` - Campaign code
* `contact-list_code` - Contact list code
* `contact-list_title` - Contact list title
* `filter_code` - Filter code
* `message_code` - Message code
**Send Information:**
* `title` - Send title
* `subject` - Email subject
* `sender_name` - Sender name
* `sender_email` - Sender email
* `reply_to` - Reply-to email
* `message_type` - Message type (CAD, URL)
* `url` - Message URL (if send type is 'URL')
* `action_mark` - Verification code to confirm 'URL'
* `action_type` - Send type (PT: Point, IC: Incremental)
* `status` - Send status (PD: Pending, AG: Scheduled, EN: Sending, EP: Partially sending, FN: Finished)
**Metrics:**
* `total_contacts` - Total contacts separated for the send
* `total_sent` - Total sent
**Dates:**
* `date_create` - Registration date (replication key)
* `date_start` - Start date
* `date_end` - End date
Stream for managing email marketing campaigns.
**Identifiers:**
* `code` - Unique identifier for the campaign (primary key)
**Campaign Information:**
* `title` - Campaign title
**Dates:**
* `insert_date` - Registration date (replication key)
Stream for managing email message templates and content.
**Identifiers:**
* `code` - Unique identifier for the message (primary key)
**Message Information:**
* `title` - Message title
**Dates:**
* `insert_date` - Registration date (replication key)
Stream for tracking consumption/usage transactions, providing billing and usage data for email sends and platform usage.
**Identifiers:**
* `action_code` - Send code (part of composite primary key)
* `dt_transaction` - Transaction date (part of composite primary key, replication key)
**Transaction Information:**
* `description` - Consumption description
* `value` - Transaction value
Stream for managing custom fields that can be associated with contacts for segmentation and personalization.
**Identifiers:**
* `code` - Unique identifier for the field (primary key)
* `code_name` - Internal name for reference in other API calls
* `field-group_code` - Field group code
* `field-group_name` - Field group title
**Field Information:**
* `title` - Field title
* `type` - Field type (LV: List of Values, DH: Date/time, DT: Date, VC: Short text, TXT: Long text, INT: Integer, FLT: Decimal, PHN: Phone number, DVC: Application token)
* `is_required` - Whether the field is required
* `is_uniquevalue` - For 'List of Values' type fields, whether it's unique (true) or multiple (false)
* `is_searchable` - Field applicability
**Dates:**
* `insert_date` - Registration date (replication key)
Stream for managing field groups that organize custom fields into logical categories.
**Identifiers:**
* `code` - Unique identifier for the field group (primary key)
**Group Information:**
* `title` - Field group title
**Dates:**
* `insert_date` - Registration date (replication key)
Stream for managing segmentation filters used to create dynamic contact lists and target specific contact groups.
**Identifiers:**
* `code` - Unique identifier for the filter (primary key)
**Filter Information:**
* `title` - Filter title
* `type` - Operator between rules (AND, OR)
**Dates:**
* `insert_date` - Registration date (replication key)
Stream for managing predefined value lists used in fields of type "List of Values".
**Identifiers:**
* `code` - Unique identifier for the value in the list of values (primary key)
**Value Information:**
* `value` - The value
Stream for tracking email opt-outs and unsubscribes, including the reason and origin of the opt-out request.
**Identifiers:**
* `email` - Contact email (primary key)
**Optout Information:**
* `reason_description` - Opt-out reason
* `origin` - Opt-out origin (C: Manually included, P: Requested by email owner, D: Spam complaint)
**Dates:**
* `request_date` - Opt-out request date (replication key)
Stream for managing user accounts and user information.
**Identifiers:**
* `code` - Unique identifier for the user (primary key)
**User Information:**
* `email` - User email
* `name` - User name
* `language` - Language
* `timezone` - Timezone
**Dates:**
* `insert_date` - Registration date (replication key)
Stream for managing teams and organizational hierarchy.
**Identifiers:**
* `code` - Unique identifier for the team (primary key)
* `top_team` - Parent team code
**Team Information:**
* `name` - Team name
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Dinamize data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Campaign Performance Overview
Analyze email campaign performance by tracking send statistics, delivery rates, and engagement metrics across all campaigns.
**Business Value:**
* Identify which campaigns have the highest delivery rates
* Track send volumes over time
* Monitor campaign status distribution
* Analyze send patterns by campaign type
```sql theme={null}
SELECT
a.campaign_code,
a.title AS action_title,
a.action_type,
a.status,
a.contact-list_title,
CAST(a.total_contacts AS INTEGER) AS total_contacts,
CAST(a.total_sent AS INTEGER) AS total_sent,
CASE
WHEN CAST(a.total_contacts AS INTEGER) > 0
THEN CAST(a.total_sent AS INTEGER) * 100.0 / CAST(a.total_contacts AS INTEGER)
ELSE 0
END AS send_rate,
a.date_create,
a.date_start,
a.date_end
FROM
nekt_raw.dinamize_actions a
WHERE
a.date_create >= CURRENT_DATE - INTERVAL '30' DAY
ORDER BY
a.date_create DESC
```
```sql theme={null}
SELECT
a.campaign_code,
a.title AS action_title,
a.action_type,
a.status,
a.contact-list_title,
CAST(a.total_contacts AS INT64) AS total_contacts,
CAST(a.total_sent AS INT64) AS total_sent,
SAFE_DIVIDE(
CAST(a.total_sent AS INT64),
CAST(a.total_contacts AS INT64)
) * 100 AS send_rate,
a.date_create,
a.date_start,
a.date_end
FROM
`nekt_raw.dinamize_actions` a
WHERE
DATE(a.date_create) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
ORDER BY
a.date_create DESC
```
| campaign\_code | action\_title | action\_type | status | contact-list\_title | total\_contacts | total\_sent | send\_rate | date\_create | date\_start | date\_end |
| - | - | - | - | - | - | - | - | - | - | - |
| CAM001 | Welcome Campaign | PT | FN | Newsletter Subscribers | 1000 | 985 | 98.5 | 2024-11-15 10:00:00 | 2024-11-15 10:00:00 | 2024-11-15 12:00:00 |
| CAM002 | Monthly Newsletter | IC | EN | Customer Base | 5000 | 3200 | 64.0 | 2024-11-20 08:00:00 | 2024-11-20 08:00:00 | NULL |
### 2. Contact List Analysis
Analyze contact list health, growth, and composition to understand your subscriber base.
**Business Value:**
* Track contact list growth over time
* Monitor list status and health
* Identify lists with high valid contact rates
* Understand list ownership and distribution
```sql theme={null}
SELECT
cl.code,
cl.title,
cl.type,
cl.status,
cl.user_code,
CAST(cl.total_contacts AS INTEGER) AS total_contacts,
CAST(cl.total_valid AS INTEGER) AS total_valid,
CASE
WHEN CAST(cl.total_contacts AS INTEGER) > 0
THEN CAST(cl.total_valid AS INTEGER) * 100.0 / CAST(cl.total_contacts AS INTEGER)
ELSE 0
END AS validity_rate,
cl.insert_date,
u.name AS owner_name
FROM
nekt_raw.dinamize_contact_lists cl
LEFT JOIN nekt_raw.dinamize_users u ON cl.user_code = u.code
WHERE
cl.status = 'LI'
ORDER BY
CAST(cl.total_contacts AS INTEGER) DESC
```
```sql theme={null}
SELECT
cl.code,
cl.title,
cl.type,
cl.status,
cl.user_code,
CAST(cl.total_contacts AS INT64) AS total_contacts,
CAST(cl.total_valid AS INT64) AS total_valid,
SAFE_DIVIDE(
CAST(cl.total_valid AS INT64),
CAST(cl.total_contacts AS INT64)
) * 100 AS validity_rate,
cl.insert_date,
u.name AS owner_name
FROM
`nekt_raw.dinamize_contact_lists` cl
LEFT JOIN `nekt_raw.dinamize_users` u ON cl.user_code = u.code
WHERE
cl.status = 'LI'
ORDER BY
CAST(cl.total_contacts AS INT64) DESC
```
| code | title | type | status | user\_code | total\_contacts | total\_valid | validity\_rate | insert\_date | owner\_name |
| - | - | - | - | - | - | - | - | - | - |
| LIST001 | Newsletter Subscribers | EM | LI | USR001 | 5000 | 4850 | 97.0 | 2024-01-15 10:00:00 | João Silva |
| LIST002 | Customer Base | CE | LI | USR002 | 3200 | 3100 | 96.9 | 2024-02-20 14:30:00 | Maria Santos |
### 3. Transaction and Consumption Analysis
Analyze platform usage and consumption patterns to understand costs and usage trends.
**Business Value:**
* Track platform usage and consumption over time
* Identify high-consumption periods
* Monitor transaction values and trends
* Correlate usage with campaign activity
```sql theme={null}
SELECT
DATE(t.dt_transaction) AS transaction_date,
COUNT(*) AS transaction_count,
SUM(CAST(t.value AS DECIMAL(10,2))) AS total_value,
AVG(CAST(t.value AS DECIMAL(10,2))) AS avg_value,
COUNT(DISTINCT t.action_code) AS unique_actions
FROM
nekt_raw.dinamize_transactions t
WHERE
DATE(t.dt_transaction) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
DATE(t.dt_transaction)
ORDER BY
transaction_date DESC
```
```sql theme={null}
SELECT
DATE(t.dt_transaction) AS transaction_date,
COUNT(*) AS transaction_count,
SUM(CAST(t.value AS FLOAT64)) AS total_value,
AVG(CAST(t.value AS FLOAT64)) AS avg_value,
COUNT(DISTINCT t.action_code) AS unique_actions
FROM
`nekt_raw.dinamize_transactions` t
WHERE
DATE(t.dt_transaction) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
DATE(t.dt_transaction)
ORDER BY
transaction_date DESC
```
| transaction\_date | transaction\_count | total\_value | avg\_value | unique\_actions |
| - | - | - | - | - |
| 2024-11-27 | 45 | 1250.50 | 27.79 | 12 |
| 2024-11-26 | 38 | 980.00 | 25.79 | 10 |
| 2024-11-25 | 52 | 1450.75 | 27.90 | 15 |
### 4. Opt-out Analysis
Track and analyze email opt-outs to understand subscriber churn and identify patterns in unsubscribes.
**Business Value:**
* Monitor opt-out trends over time
* Identify common opt-out reasons
* Track opt-out origins (manual, user request, spam complaint)
* Correlate opt-outs with campaign sends
```sql theme={null}
SELECT
DATE(o.request_date) AS optout_date,
o.origin,
o.reason_description,
COUNT(*) AS optout_count,
COUNT(DISTINCT o.email) AS unique_emails
FROM
nekt_raw.dinamize_optout_emails o
WHERE
DATE(o.request_date) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
DATE(o.request_date),
o.origin,
o.reason_description
ORDER BY
optout_date DESC,
optout_count DESC
```
```sql theme={null}
SELECT
DATE(o.request_date) AS optout_date,
o.origin,
o.reason_description,
COUNT(*) AS optout_count,
COUNT(DISTINCT o.email) AS unique_emails
FROM
`nekt_raw.dinamize_optout_emails` o
WHERE
DATE(o.request_date) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
DATE(o.request_date),
o.origin,
o.reason_description
ORDER BY
optout_date DESC,
optout_count DESC
```
## Implementation Notes
### Data Quality Considerations
* Most streams use incremental replication based on date fields (`insert_date`, `date_create`, `dt_transaction`, `request_date`)
* The `start_date` configuration parameter can be used to limit the initial sync window for streams with replication keys
* Some fields like `total_contacts`, `total_sent`, and `value` are stored as strings and may need casting to numeric types in queries
* The `custom_fields` field in contacts is an object type and may contain dynamic fields (cmp4, cmp5, cmp6, etc.)
* Contact lists and contacts support a `rest_date` field that prevents emails from being sent until the end of the specified day
### API Limits & Performance
* The API uses POST requests with pagination
* Rate limiting is handled automatically with retry logic (error code 240024)
* For faster extractions, select only the streams necessary for your analysis
* Streams with replication keys (incremental sync) will only fetch new records since the last sync or the configured start date
* The `teams` and `list_of_values` streams are full table syncs (no replication key)
## Skills for agents
Dinamize connector documentation as plain markdown, for use in AI agent contexts.
# Dito as a data source
Source: https://docs.nekt.com/sources/dito
Bring cashback and customer profile data from Dito to Nekt.
Dito is a Brazilian CRM and customer engagement platform focused on retail, combining unified consumer profiles, behavior events, campaigns and a cashback product. This connector extracts **cashback rewards, cashback balances and consumer profiles** from Dito's public APIs.
The Dito API has **no bulk listing**: every read endpoint works one customer at a time. The connector fetches data for the exact list of **Customer IDs** you provide in the configuration; it cannot discover your customer base by itself. Keep in mind that each sync makes one request per customer per stream, so very large lists lead to proportionally long runs.
## Prerequisites: Service Account
All Dito APIs require a **Service Account** (SA), a generic user registered by **Dito's support team** with a non-personal corporate e-mail (e.g. `integracao@yourbrand.com`). There is no self-service way to create it. Ask your Dito account team to:
1. Create a **Service Account** for your brand;
2. Assign it the **Cashback Rewards Admin** role (required to list cashbacks and balances);
3. Grant it access to the **Get Profile** endpoint of the Profiles API (permissions are granted per endpoint);
4. Confirm your **brand label** (the identifier Dito uses for your brand in API calls).
## 1. Add your Dito access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Dito option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **Service Account E-mail**: the e-mail of the Service Account created by Dito's support team.
* **Service Account Password**: the password of that Service Account.
* **Brand**: the brand label provided by Dito for your account.
* **Customer IDs**: the customer identifiers you want to fetch data for, one per line. Use the **same identifiers your systems send to Dito** (typically your internal customer ID or CPF).
* **Store ID** (advanced): optionally filter cashback data to a single store, using the same store identifier sent to Dito.
* **Page Size** (advanced): number of rewards fetched per page (default 100).
3. Click **Next**.
## 2. Select your Dito streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If a customer in your list doesn't exist in Dito, the connector logs a warning and skips it; the sync never fails because of an unknown customer.
2. Click **Next**.
## 3. Configure your Dito data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: all Dito streams are extracted as FULL\_TABLE: the API offers no date filtering, so each run fetches the current data for every configured customer. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Dito data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Available streams
| Stream | Slug | Description |
| - | - | - |
| Customer Rewards | `customer_rewards` | Every cashback reward generated for each configured customer, with status, values, dates and originating campaign. |
| Customer Balances | `customer_balances` | Snapshot of each customer's calculated cashback balance at extraction time. |
| Profiles | `profiles` | Registration data of each customer's unified consumer profile. |
All date/time values returned by the Cashback API are in **Brasilia time (GMT-3)**. The `custom_data` field of profiles is delivered as a **JSON string**; parse it downstream (for example in a Query) when you need the individual attributes.
## Fields by stream
Below you'll find all available data streams from Dito and their corresponding fields:
Every cashback reward generated for each configured customer. **Sync:** Full table. **Primary key:** `customer_id`, `id`.
* `customer_id` - Identifier of the customer the reward belongs to, as provided in the Customer IDs configuration
* `id` - Unique identifier of the reward
* `status` - Current status of the reward (available, locked, used, expired or undefined)
* `value` - Maximum amount that can be redeemed for the reward
* `balance` - Remaining balance of the reward available to be redeemed
* `rate` - Maximum percentage of the purchase value that the reward value can be applied to
* `created_at` - Timestamp when the reward was created (GMT-3)
* `available_at` - Timestamp when the reward becomes available for use (GMT-3)
* `due_date` - Expiration date of the reward (GMT-3)
* `canceled_at` - Timestamp when the reward was canceled, if applicable (GMT-3)
* `order_id` - Identifier of the purchase transaction that generated the reward
* `store_id` - Identifier of the store where the purchase that generated the reward was made
* `campaign_id` - Identifier of the campaign that generated the reward
* `campaign_name` - Name of the campaign that generated the reward
* `campaign_description` - Description of the campaign that generated the reward
Current cashback balance breakdown for each configured customer, a snapshot taken at extraction time. **Sync:** Full table. **Primary key:** `customer_id`.
* `customer_id` - Identifier of the customer, as provided in the Customer IDs configuration
* `available_balance` - Amount of cashback currently available to be consumed
* `unavailable_balance` - Amount of cashback that is unavailable because it is still in downtime
* `next_purchase_balance` - Amount of cashback available for the next purchase based on redemption rules
* `next_purchase_first_due_date` - First expiration date among the rewards that make up the next purchase balance (GMT-3)
* `campaign_id` - Identifier of the campaign related to the next purchase balance
* `campaign_name` - Name of the campaign related to the next purchase balance
* `campaign_description` - Description of the campaign related to the next purchase balance
Registration data of each customer's unified consumer profile. **Sync:** Full table. **Primary key:** `customer_id`.
* `customer_id` - Identifier of the customer, as provided in the Customer IDs configuration
* `id` - Unique identifier of the profile within the brand, as returned by Dito
* `brand` - Brand identifier the profile belongs to
* `name` - Full name of the consumer
* `email` - E-mail address of the consumer
* `phone` - Phone number of the consumer
* `gender` - Gender of the consumer
* `birthday` - Birth date of the consumer
* `create_time` - Timestamp when the profile was created in Dito
* `time_zone` - Time zone of the consumer
* `address_street` - Street of the consumer's address
* `address_city` - City of the consumer's address
* `address_state` - State of the consumer's address
* `address_country` - Country of the consumer's address
* `address_postal_code` - Postal code of the consumer's address
* `custom_data` - Brand-specific custom attributes of the profile (JSON string; each key holds a value and its format)
Campaigns, journeys and messaging data ("réguas") are **not available** through Dito's read APIs. Dito only pushes that information outward via its self-service outbound webhooks. If you need campaign/journey events in Nekt, talk to us about setting up a webhook-based ingestion instead.
# Microsoft Dynamics 365 CRM as a data source
Source: https://docs.nekt.com/sources/dynamics-crm
Bring data from Microsoft Dynamics 365 CRM (Dataverse) to Nekt.
Microsoft Dynamics 365 CRM (Customer Engagement: Sales, Customer Service, Field Service) stores its data in Microsoft Dataverse. This connector reads that data through the Dataverse Web API (OData v4), so it works across the standard CRM entities as well as any custom entities and fields configured in your environment.
## Before you start
Dynamics 365 authenticates server-to-server with OAuth2 client credentials, but Dataverse also requires the application to be granted access **inside your environment**. Your Dynamics 365 environment administrator needs to complete these steps once:
In the Azure portal, register a new application and create a **client secret**. Note the **Directory (tenant) ID**, the **Application (client) ID**, and the secret value.
In the Power Platform admin center, open your environment and go to **Settings → Users + permissions → Application users → New app user**. Add the registered application and assign it a **security role** that grants read access to the tables you want to sync.
In Power Apps, open **Settings → Developer resources** and copy the **Web API endpoint** host, e.g. `https://myorg.crm2.dynamics.com`.
Without the Application User and security role (step 2), the connector receives an authorization error even with valid credentials. Only your environment administrator can grant this access.
## Configuring Dynamics 365 CRM as a source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button on the top right, then select Dynamics 365 CRM from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Provide the credentials gathered above:
* **Organization URL**: Your Dataverse organization (Web API) URL, without a trailing path, for example `https://myorg.crm2.dynamics.com`. The region segment varies by data center (`crm` for North America, `crm2` for South America, `crm4` for Europe, and so on).
* **Tenant ID**: The Microsoft Entra ID (Azure AD) directory (tenant) ID.
* **Client ID**: The Application (client) ID of the Entra app registration.
* **Client Secret**: The client secret of the Entra app registration.
* **Entities**: The list of Dataverse entity **logical names** to sync (for example `account`, `contact`, `lead`, `opportunity`). Each entity becomes a stream. Leave it empty to use the standard CRM entities, or add your own, including custom entities.
* **Start Date**: (Optional) The earliest `modifiedon` timestamp to sync for incremental streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. Each stream corresponds to one of the configured entities. For faster extractions, select only the streams relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired Layer where the data will be placed, a Folder to organize it, a name for each table, and the type of sync.
* **Layer**: choose between the existing Layers on your catalog. This is where your extracted tables appear once the extraction runs successfully.
* **Folder**: a Folder can be created inside the selected Layer to group all tables created from this source.
* **Table name**: a name is suggested, but you can customize it. You can also add a **prefix** to all tables at once.
* **Sync Type**: choose between INCREMENTAL and FULL\_TABLE.
* Incremental: each extraction fetches only records changed since the last run, based on the `modifiedon` column.
* Full table: each extraction fetches the current state of the entity.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Optionally, you can define additional settings such as Delta Log Retention and an additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync) to complement the incremental extractions.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking the arrow button. Once executed, your data appears in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Streams are discovered dynamically from your environment. For each entity you configure, the connector reads its Dataverse attribute metadata and builds the table schema from it, so the columns always match your organization's actual configuration, including custom fields.
The standard entities synced by default are:
Companies and organizations you do business with. Primary key `accountid`; incremental on `modifiedon`.
People associated with accounts. Primary key `contactid`; incremental on `modifiedon`.
Prospects and unqualified sales opportunities. Primary key `leadid`; incremental on `modifiedon`.
Qualified, potential revenue-generating deals. Primary key `opportunityid`; incremental on `modifiedon`.
You can sync any other Dataverse entity, such as `incident` (cases), `task`, `activitypointer`, `systemuser`, or your own custom entities, by adding its logical name to the **Entities** setting.
## How columns are represented
Because the schema is derived from Dataverse metadata, a few column conventions are worth noting:
* **Lookup and owner columns** appear as the related record's GUID under the key `__value` (for example, the account's primary contact is `_primarycontactid_value`).
* **Choice / option set columns** (such as `statecode` and `statuscode`) are returned as their integer option code.
* **Currency columns** include a companion `_base` column holding the value converted to the organization's base currency.
Incremental sync captures new and updated records via the `modifiedon` column. Deletes are not captured.
# Microsoft Dynamics Finance and Operations as a data source
Source: https://docs.nekt.com/sources/dynamics-fo
Bring data from Microsoft Dynamics Finance and Operations to Nekt.
Microsoft Dynamics 365 Finance and Operations (Finance, Supply Chain Management, Commerce, and the other finance and operations apps) is Microsoft's ERP. This connector reads its **data entities** (customers, vendors, products, sales and purchase orders, financial data, and any other public entity, including custom ones) through the environment's OData endpoint, `/data`.
## Before you start
The connector signs in as an application (Microsoft Entra ID client credentials), and Finance and Operations must know that application. Your administrators need to complete these steps once:
In the Azure portal, open **Microsoft Entra ID → App registrations → New registration** and create an application. Under **Certificates & secrets**, create a **client secret** and copy its **value** (not its ID). Note the **Directory (tenant) ID** and the **Application (client) ID** from the app's overview page. Client secrets expire; note the expiry date and update the connection before it.
In Finance and Operations, go to **System administration → Setup → Microsoft Entra applications** and select **New**. Enter the **Client ID** from step 1, a name, and a **User ID**: an enabled user whose security roles can read the data entities you want to sync, in every company you want to sync. A dedicated service account with read-only roles is recommended.
Copy the address of your environment from the browser, for example `https://contoso.operations.dynamics.com`.
Without step 2, Microsoft Entra ID issues a token but Finance and Operations refuses every request. The connector reads only what the linked user is allowed to see.
## Configuring Microsoft Dynamics Finance and Operations as a source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button on the top right, then select Microsoft Dynamics Finance and Operations from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
* **Environment URL**: The address of your environment, for example `https://contoso.operations.dynamics.com`. Pasting the full address from the browser also works.
* **Tenant ID**: The Directory (tenant) ID of your Microsoft Entra ID tenant.
* **Client ID**: The Application (client) ID of the app registration.
* **Client secret**: The client secret value of the app registration.
* **Entities**: The data entities to sync, by their collection name (the name in the OData URL), for example `CustomersV3` or `SalesOrderHeadersV2`. Each becomes a stream. For each entity you can optionally set an **Incremental field**, a **Filter** (an OData `$filter` expression) and the **Fields** to read. Leave it empty to sync the standard set listed below.
* **Companies**: The company (legal entity) codes to sync, for example `usmf`. Leave empty to sync every company the linked user can access. The codes are checked when the sync starts; an unknown code stops the sync with the list of available companies.
Advanced settings:
* **Initial sync date**: For incremental entities, only records changed after this date are read on the first run.
* **All companies**: On by default. When off, Finance and Operations returns only the default company of the linked user (it must stay on when **Companies** is set).
* **Default incremental fields**: Field names used as the incremental field of any entity that has them. Defaults to `SysModifiedDateTime`.
* **Page size**: Records per request when an entity is read by key ranges (default 5000).
* **Pagination**: **Automatic** (default) reads entities with a single numeric key by key ranges (fast and correct while data changes) and the others by following the pages Finance and Operations returns.
* **Incremental lookback (minutes)**: Each incremental sync re-reads records changed this many minutes before the previous position (default 60), so postings that are saved late are not missed.
* **Request timeout**: How long to wait for each page before retrying (default 300 seconds).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. Each stream corresponds to one entity. For faster extractions, select only the streams relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired Layer where the data will be placed, a Folder to organize it, a name for each table, and the type of sync.
* **Layer**: choose between the existing Layers on your catalog. This is where your extracted tables appear once the extraction runs successfully.
* **Folder**: a Folder can be created inside the selected Layer to group all tables created from this source.
* **Table name**: a name is suggested, but you can customize it. You can also add a **prefix** to all tables at once.
* **Sync Type**: choose between INCREMENTAL and FULL\_TABLE.
* Incremental: available for entities with an incremental field; each extraction fetches only records changed since the last run.
* Full table: each extraction fetches the whole entity.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Optionally, you can define additional settings such as Delta Log Retention and an additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync) to complement the incremental extractions.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking the arrow button. Once executed, your data appears in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Streams are built from your environment's metadata: one stream per entity, named in snake\_case (`CustomersV3` becomes `customers_v3`). Columns keep the entity's field names, and the entity key, which includes `dataAreaId` (the company) for company-specific entities, becomes the table's primary key. Custom entities and custom fields are included as long as the entity is public (exposed through OData).
When **Entities** is empty, these standard entities are offered (an entity that does not exist in your version is skipped, and the run log lists which ones):
`LegalEntities` (companies) and `MainAccounts` (chart of accounts).
General journal entries and account entries (`GeneralJournalEntryBiEntities`, `GeneralJournalAccountEntryBiEntities`). Synced incrementally on `SysModifiedDateTime`.
`CustTransBiEntities` and `VendTransBiEntities`. Synced incrementally on `SysModifiedDateTime`.
`CustomersV3`, `VendorsV2` and `ReleasedProductsV2`, per company.
`SalesOrderHeadersV2`, `SalesOrderLines` and posted sales invoice headers (`SalesInvoiceHeadersV4`, or `SalesInvoiceHeadersV2` on versions before 10.0.40).
`PurchaseOrderHeadersV2` and `PurchaseOrderLinesV2`. By design these entities do not include purchase return orders.
In step 2 you choose which of these streams to sync.
You can sync any other public entity by adding its collection name to **Entities**. To find names, open **System administration → Workspaces → Data management → Data entities** in Finance and Operations (the **Public collection name** column), or open `/data` in the browser while signed in.
## How columns are represented
* **Enumerations** (such as `NoYes` or `SalesStatus`) are stored as their name, for example `Yes` or `Invoiced`.
* **Dates** are stored as UTC timestamps. Finance and Operations represents an empty date as `1900-01-01`; those values are stored as empty (null).
* **Amounts** are numbers in the transaction currency of the record.
## Incremental sync
Most entities do not expose a date that changes when a record is updated, so they are read in full on every run. Entities that do (typically the `*BiEntities`, with `SysModifiedDateTime`) are synced incrementally by default, re-reading the last 60 minutes on each run (**Incremental lookback**). `SysModifiedDateTime` tracks changes to the entity's main table only; changes that affect only related tables, and deleted records, are not detected. Use FULL\_TABLE (or an additional Full Sync) where that matters.
If you change an entity's **Filter**, **Incremental field** or the **Companies**, its next sync starts again from the **Initial sync date**, so the history of the new scope is loaded too.
## Limits
Finance and Operations protects its service with request limits. When a limit is reached, the connector waits the time Finance and Operations asks for and continues automatically; no data is lost, but the run takes longer. You can give this integration a lower throttling priority in Finance and Operations (**System administration → Setup → Throttling priority mapping**) so it yields to interactive users.
Large entities (for example general ledger entries) can take a long time on the first sync. Microsoft recommends its bulk export options (such as Azure Synapse Link) for tables of many millions of rows; with this connector, prefer an incremental field, **Filters**, or **Fields** to limit what is read.
The connector supports Finance and Operations in the Microsoft public cloud. Sovereign clouds (for example US Government) and on-premises deployments (AD FS) are not supported.
# DynamoDB as a data source
Source: https://docs.nekt.com/sources/dynamodb
Bring data from DynamoDB to Nekt.
Amazon DynamoDB is a fully managed NoSQL database service provided by AWS that offers seamless scalability and high performance. It's designed for applications that need consistent, single-digit millisecond performance at any scale, making it ideal for mobile, web, gaming, and IoT applications.
By connecting DynamoDB to your Nekt catalog, you can centralize your NoSQL data alongside other data sources, enabling comprehensive analysis and data integration across your organization.
This guide will walk you through the process of setting up DynamoDB as a data source, including the necessary AWS permissions configuration and connection settings. It's important to note we support incremental sync for this source, making the whole process more efficient. For that to work, you need to have a valid replication key in your documents (either a date or timestamp field).
![]()
## Configuring DynamoDB as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **DynamoDB** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You need to define some permissions to allow Nekt to access your DynamoDB tables. Check the instructions below:
**Create role and add custom policy**
* Open the AWS Console using the account that hosts the DynamoDB table you'd like to extract data from.
* Enter the IAM (Identity and Access Management) service page.
* In the left panel under **Access management**, select **Roles**.
* Click **Create role**.
* In **Trusted entity type**, select **Custom trust policy**.
* Change the **Principal** value to:
```json theme={null}
{
"AWS": "arn:aws:iam::{AWS_ACCOUNT_ID}:role/nekt-ecs-task-role"
}
```
The variable must be replaced by the ID of the AWS account where the Nekt workspace is deployed. In order to find this information, click the dropdown in the top-right corner of the AWS console page after you log in with your account, then click on the "copy" icon to the right of your **Account ID**.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::{AWS_ACCOUNT_ID}:role/nekt-ecs-task-role"
},
"Action": "sts:AssumeRole"
}
]
}
```
* Click **Next**.
* Click **Next** again without adding any permission. We'll create an inline policy further on.
* Under **Role details**, you can type any **Role name** as you want. Example: `nekt-dynamodb-source`.
* Optionally, type a **Description**. Example: `Used by Nekt to extract data from DynamoDB`.
* Click **Create role** .
You should get redirected back to the IAM roles page and a message of success should appear at the top.
**Add permissions to role**
* Open the role you just created by clicking **View role** in the right end of the green banner that appeared at the top of the page. If you dismissed it already, you can type the given **Role name** in the search field and then select it when it appears in the list.
* Within **Permission policies (0)**, click the **Add permissions**.
* Select **Create inline policy**.
* Within **Policy editor**, select `JSON`.
* Paste the following policy in the **Policy editor**:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:BatchGetItem",
"dynamodb:ConditionCheckItem",
"dynamodb:DescribeContributorInsights",
"dynamodb:Scan",
"dynamodb:ListTagsOfResource",
"dynamodb:Query",
"dynamodb:DescribeTimeToLive",
"dynamodb:PartiQLSelect",
"dynamodb:DescribeTable",
"dynamodb:GetItem",
"dynamodb:DescribeContinuousBackups",
"dynamodb:DescribeKinesisStreamingDestination",
"dynamodb:DescribeTableReplicaAutoScaling"
],
"Resource": "{DYNAMO_DB_ARN}"
}
]
}
```
* Replace the `{DYNAMO_DB_ARN}` part with the ARN of the DynamoDB table which you'd like to extract data from.
* If you don't know it, open a new tab in your browser and enter the DynamoDB service page.
* In the left panel, click **Tables**.
* Open the desired table by clicking on its name in the list.
* Under **General information**, click **Additional info**.
* Click the "copy" icon just below **Amazon Resource Name (ARN)**. A tool tip should appear containing the message `ARN copied`.
* Go back to the IAM policy editor that you left open in the previous tab and paste the ARN you just copied over `{DYNAMO_DB_ARN}`.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:BatchGetItem",
"dynamodb:ConditionCheckItem",
"dynamodb:DescribeContributorInsights",
"dynamodb:Scan",
"dynamodb:ListTagsOfResource",
"dynamodb:Query",
"dynamodb:DescribeTimeToLive",
"dynamodb:PartiQLSelect",
"dynamodb:DescribeTable",
"dynamodb:GetItem",
"dynamodb:DescribeContinuousBackups",
"dynamodb:DescribeKinesisStreamingDestination",
"dynamodb:DescribeTableReplicaAutoScaling"
],
"Resource": "arn:aws:dynamodb:{AWS_REGION}:{AWS_ACCOUNT_ID}:table/{TABLE_NAME}"
}
]
}
```
* Click **Next**.
* Under **Policy details** type any Policy name as you want. Example: `nekt-dynamodb-source-policy`.
* Click **Create policy**.
You should get redirected back to the role you just created and a message of success should appear at the top.
When setting up the connector at Nekt, you will use the ARN of the role you've just created.
The following configurations are available:
* **Table names:** Provide the name of the tables you want to extract. Write them exactly as you see in your Dynamo DB.
* **Assume role ARN (AWS):** The ARN role you've just created in the permissions setup step.
* **Extraction mode:** Choose how records are extracted. **Envelope** extracts each item as a JSON document stored in a single column, while **Infer schema** (default) tries to infer a fixed schema based on a sample of records. Only use Infer schema if the records in your table have consistent types and fields across all documents.
* **Infer schema sample size:** (Default: `500`, only applies to Infer schema mode) Defines how many records you want to use to infer your table's schema. The more consistent your schema is, the smaller your sample can be.
* **Incremental lookback window (days):** (Default: `7`) For tables replicated incrementally, how many days to re-scan before the last saved replication key value. Protects against DynamoDB's eventually consistent reads silently skipping records that were mid-write during a previous scan. Set to `0` to disable.
Additionally, the following advanced configurations are available:
* **Per-table Scan overrides:** Map a table name to boto3 Scan kwargs that override the default when reading that table. Example: `{"MyTable": {"ConsistentRead": true, "ProjectionExpression": "id, updated_at"}}`.
* **Incremental GSI partition key values:** For a table with a Global Secondary Index (GSI) whose sort key matches its configured incremental replication key, provide the full list of partition key values on that index (e.g. shard values like `SHARD#1,SHARD#2,SHARD#3`). When set, incremental extraction Querys this GSI once per value instead of Scanning the whole table. DynamoDB has no way to enumerate these values automatically, so this list must be kept up to date manually; the connector warns if the data contains a value missing from it.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For DynamoDB, these correspond to the tables you configured in the previous step.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
If you define INCREMENTAL as the sync type for your table, you will have to add
an incremental key. Incremental keys must be of `integer` (epoch seconds or milliseconds)
or `string` types (formatted as ISO8601 dates, using either `T` or a space to separate
date and time). To ensure consistency between extractions,
make sure the field you select represents the last modification date of a document.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
The DynamoDB connector dynamically exposes streams corresponding to your selected tables. The shape of each stream depends on the configured **Extraction mode**.
Each selected DynamoDB table is mapped to a stream whose schema is inferred from a sample of records.
* The schema is inferred based on a sample of records from each table (size controlled by **Infer schema sample size**).
* The properties captured match the fields present in the sampled documents.
* Primary keys come from the DynamoDB table's key schema.
* If you are running incremental syncs, make sure the configured replication key is present in all records.
Each item is serialized into a standardized envelope, which is robust to inconsistent document shapes.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `_hash_id` | String | Unique identifier for the item, generated by hashing the primary keys of the DynamoDB record. |
| `document` | String | A JSON string containing the full serialized DynamoDB item. |
If an incremental replication key is configured, it is also exposed as its own top-level column alongside `_hash_id` and `document`.
**Serialization Notes:**
The entire DynamoDB item is flattened into a single JSON string within the `document` field. During this process, specific native data types are serialized to ensure proper formatting downstream:
* **Decimals:** Automatically converted to standard numeric formats (`int` or `float`).
* **Dates/Timestamps:** Serialized to ISO 8601 standard strings.
* **Sets:** Converted to standard JSON Arrays (lists).
* **Bytes:** Decoded into UTF-8 strings.
# Implementation Notes
### Eventual Consistency and Incremental Syncs
To optimize AWS costs and reduce Read Capacity Units (RCU) usage, the connector performs **eventually consistent reads** by default.
When using incremental syncs, a trailing lookback window (configurable via the **Incremental lookback window (days)** option) is applied to your replication key. This ensures that records that were mid-write during previous scans are caught in subsequent runs and not silently skipped, safely bridging the gap introduced by eventual consistency.
For `string` replication keys, the lookback shifts only the calendar date: the time of day, fractional seconds, timezone, and the original date/time separator (`T` per ISO 8601, or a space) are preserved verbatim, so the shifted value always matches the exact format used by your source table.
### Query-based incremental extraction via GSI
By default, incremental syncs Scan the whole table and filter on the replication key. If a table has a Global Secondary Index whose sort key matches the configured replication key, the connector auto-detects it; when you also configure **Incremental GSI partition key values**, incremental runs Query that index once per partition key value instead of Scanning the entire table, reading only changed records. The auto-detected GSI must have a projection type of `ALL`: a GSI only carries the attributes it projects, so Querying one that projects less than `ALL` would return incomplete records; the connector detects this and falls back to Scan instead. If a matching GSI exists but the partition key values aren't configured, the connector also keeps using Scan and logs a hint. The very first run for a stream (no bookmark yet) always uses Scan. If `dynamodb:Query` is denied on the index ARN (a separate IAM resource from the table ARN), the connector falls back to a full-table Scan and reports it in the end-of-run summary.
# Transformation example: extracting fields from the document
When using **Envelope mode**, the `document` column stores all event attributes as a single JSON string. To analyze specific properties in [Explorer](https://app.nekt.ai/explorer) or downstream models, you can parse the JSON and expose the keys as separate columns.
```sql theme={null}
SELECT
_hash_id,
json_extract_scalar(document, '$.item_name') AS item_name,
json_extract_scalar(document, '$.status') AS status,
CAST(json_extract_scalar(document, '$.price') AS DOUBLE) AS price
FROM
nekt_raw.dynamodb_your_table
```
Replace `nekt_raw.dynamodb_your_table` with your actual layer and table name. Use `json_extract_scalar()` for string properties; for numeric properties use `CAST(json_extract_scalar(document, '$.key') AS DOUBLE)`.
```sql theme={null}
SELECT
_hash_id,
JSON_VALUE(document, '$.item_name') AS item_name,
JSON_VALUE(document, '$.status') AS status,
SAFE_CAST(JSON_VALUE(document, '$.price') AS FLOAT64) AS price
FROM
`nekt_raw.dynamodb_your_table`
```
Replace `nekt_raw.dynamodb_your_table` with your actual dataset and table name. Use `JSON_VALUE()` for string properties; for numeric properties use `SAFE_CAST(JSON_VALUE(document, '$.key') AS FLOAT64)`.
## Skills for agents
DynamoDB connector documentation as plain markdown, for use in AI agent contexts.
# E-goi as a data source
Source: https://docs.nekt.com/sources/e-goi
Bring data from E-goi to Nekt.
E-goi is a multichannel marketing automation platform. It manages contact lists and segments, sends email, SMS, push, and web push campaigns, and runs marketing automations. This connector reads your lists, contacts, campaigns, and the delivery and engagement report of each campaign through the E-goi Marketing API (v3).
## Before you start
The connection uses your E-goi **API key**. The API is available on every E-goi plan except **GoSend**.
Sign in to E-goi, open the user menu in the top right corner, and click **Integrations**. Copy the value under **API Key**.
E-goi counts API usage against your plan: each plan includes a monthly number of API requests, and your account also has an hourly limit shared by every integration that uses it. This connector keeps its usage low (see [API usage](#api-usage) below), but the requests it makes still count toward your quota.
## Configuring E-goi as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the E-goi option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **API Key**: The key from **Integrations → API Key** in E-goi.
* **Start Date**: The earliest campaign send whose report is read on the first sync. Lists, contacts, campaigns, and the other catalogs are always read in full, whatever this date. Each campaign report costs one API request, so on an account with a long history, leaving this empty makes the first sync read one report per campaign ever sent.
* **Report Lookback Days**: (Default: 30) How many days of already-sent campaigns have their report read again on every sync. Opens, clicks, bounces, and unsubscriptions keep arriving for weeks after a send; this window keeps recent reports up to date. A larger window costs more API requests on every sync.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Keep the campaign report streams on **INCREMENTAL**. Each sync only reads the reports of recently sent campaigns, so a full-table sync would replace the table with those recent reports alone.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## How syncing works
**Catalogs** (lists, contacts, list fields, segments, campaigns, automations, tags, users, senders, and the suppression list) are read in full on every sync. E-goi offers no way to ask only for contacts changed since a date, and a full read costs one request per 1,000 records.
**Campaign reports** sync **incrementally** on `campaign_sent_at`, the time the campaign went out. Each report stream reads only the campaigns of its channel. On the first sync, those are the campaigns sent since **Start Date**. On every later sync, they are the campaigns sent since the latest send already read minus **Report Lookback Days**. Each report is stored under its `campaign_hash`, so reading it again updates its counters, and reports of older campaigns keep their last values.
## API usage
A daily sync of an account with a few lists and a few thousand contacts typically makes a few dozen requests, plus one per campaign inside the report window. What drives the count:
* one request per 1,000 records of each catalog (contacts are the largest);
* one request per campaign report, only for campaigns inside the report window;
* nothing per contact: contact activity and per-contact statistics are not extracted, because they would cost one request per contact on every sync.
When E-goi reports that the hourly limit of your account is spent, the sync pauses until the limit resets and then continues, and the run log says so. If your plan's API quota is used up, the sync stops with a message saying so. It works again once the quota renews or you add requests to your plan.
## Data notes
* E-goi sends dates without a time zone (for example `2026-09-01 10:30:00`). They are stored as UTC, unchanged.
* Empty dates, which E-goi sends as `0000-00-00 00:00:00`, are stored as null.
* Nested data, such as contact custom fields (`extra`), report breakdowns (`by_date`, `by_url`, and so on), segment rules, and field options, is stored as JSON text.
* Custom field values in `contacts.extra` are keyed by `field_id`. The `list_fields` stream gives the name and format of each one.
## Streams
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Lists | `lists` | Full table | Contact lists of the account. |
| Contacts | `contacts` | Full table | Contacts of every list, with their channel statuses, subscription history, custom fields and tags. |
| List Fields | `list_fields` | Full table | Base and custom fields of every list. Use it to name the `field_id` keys in `contacts.extra`. |
| Segments | `segments` | Full table | Segments of every list: automatic, rule-based (saved) and tag segments. |
| Campaigns | `campaigns` | Full table | Campaigns of every channel, with their status and send dates. |
| Campaign Groups | `campaign_groups` | Full table | Groups used to organize campaigns. |
| Email Campaign Reports | `email_campaign_reports` | Incremental | Delivery and engagement report of each sent email campaign, with daily, hourly, location, domain, link and email-client breakdowns. |
| SMS Campaign Reports | `sms_campaign_reports` | Incremental | Delivery report of each sent SMS campaign, with a breakdown per mobile network. |
| Push Campaign Reports | `push_campaign_reports` | Incremental | Delivery and open report of each sent mobile push campaign, with a breakdown per operating system. |
| Web Push Campaign Reports | `web_push_campaign_reports` | Incremental | Delivery, open and click report of each sent web push campaign, with device, operating system, browser and URL breakdowns. |
| Automations | `automations` | Full table | Marketing automations of the account. |
| Tags | `tags` | Full table | Tags available to label contacts. |
| Users | `users` | Full table | E-goi users of the account and their permission profiles. |
| Email Senders | `email_senders` | Full table | Email addresses campaigns can be sent from, and their approval status. |
| Cellphone Senders | `cellphone_senders` | Full table | SMS senders (numbers or alphanumeric names) and their approval status. |
| Suppression List | `suppression_list` | Full table | Emails, domains and numbers E-goi never sends to again (unsubscribes, bounces, manual blocks). |
## Fields by stream
Contact lists of the account.
**Key:** `list_id` · **Sync:** full table
**Fields:**
* `list_id` - Unique identifier of the list
* `internal_name` - Internal name of the list, only visible inside E-goi
* `public_name` - Public name of the list, shown to contacts (e.g. in forms and emails)
* `status` - Status of the list (active or blocked)
* `group_id` - Identifier of the list group the list belongs to
* `created` - Date and time the list was created (as sent by E-goi, without time zone)
* `updated` - Date and time the list was last updated (as sent by E-goi, without time zone)
Contacts of every list, with their channel statuses, subscription history, custom fields and tags.
**Key:** `list_id`, `contact_id` · **Sync:** full table
**Fields:**
* `list_id` - Identifier of the E-goi contact list the record belongs to
* `contact_id` - Unique identifier of the contact in the list (10 hexadecimal characters)
* `status` - Status of the contact in the list (active, inactive, removed or unconfirmed)
* `consent` - Consents given by the contact
* `consent_date` - Date and time the contact gave consent
* `subscription_method` - How the contact joined the list (manual, form, imported, referral or api)
* `subscription_date` - Date and time the contact joined the list
* `subscription_form` - Identifier of the form the contact subscribed through
* `unsubscription_method` - How the contact left the list (manual, form, unsubscribe\_link, bounce or api)
* `unsubscription_reason` - Reason given for unsubscribing (not\_interested, lack\_of\_time, email\_address\_change, spam or other)
* `unsubscription_observation` - Free-text comment left when unsubscribing
* `unsubscription_date` - Date and time the contact unsubscribed (empty while subscribed)
* `change_date` - Date and time the contact was last modified
* `first_name` - First name of the contact
* `last_name` - Last name of the contact
* `birth_date` - Birth date of the contact
* `language` - Language of the contact (en, pt, br, es, de, hu or fr)
* `email` - Email address of the contact
* `email_status` - Status of the email channel for the contact (active or inactive)
* `cellphone` - Mobile number of the contact (country code and number split by '-')
* `cellphone_status` - Status of the SMS channel for the contact (active or inactive)
* `phone` - Landline number of the contact (country code and number split by '-')
* `phone_status` - Status of the voice channel for the contact (active or inactive)
* `push_token_android` - Android push tokens of the contact (JSON array of objects with app\_id and token)
* `push_token_ios` - iOS push tokens of the contact (JSON array of objects with app\_id and token)
* `extra` - Custom (extra) fields of the contact (JSON array of objects with field\_id, format, value and, for phone fields, status). Field names are in the list\_fields stream
* `tags` - Identifiers of the tags attached to the contact (see the tags stream)
Base and custom fields of every list. Use it to name the `field_id` keys in `contacts.extra`.
**Key:** `list_id`, `field_id` · **Sync:** full table
**Fields:**
* `list_id` - Identifier of the E-goi contact list the record belongs to
* `field_id` - Identifier of the field: the field name for base fields (e.g. email), a number for extra fields (matches field\_id in contacts.extra)
* `type` - Field type: base (built into E-goi) or extra (custom field)
* `name` - Name of the field
* `format` - Format of the field (date, text, cellphone, phone, number, email, options or countries)
* `unique` - Whether the field value must be unique within the list
* `options` - Options of an 'options' field (JSON array of objects with option\_id and one label per language)
Segments of every list: automatic, rule-based (saved) and tag segments.
**Key:** `list_id`, `segment_id` · **Sync:** full table
**Fields:**
* `list_id` - Identifier of the E-goi contact list the record belongs to
* `segment_id` - Unique identifier of the segment
* `name` - Name of the segment
* `type` - Type of segment: auto (created by E-goi), saved (rule-based) or tag
* `match` - How the rules of a saved segment combine: all or any
* `invert` - Whether a saved segment selects the contacts that do NOT match its rules
* `rules` - Rules of a saved segment (JSON array of objects with field\_id, operator and value)
* `created` - Date and time the segment was created (tag segments)
* `updated` - Date and time the segment was last updated (tag segments)
Campaigns of every channel, with their status and send dates.
**Key:** `campaign_hash` · **Sync:** full table
**Fields:**
* `campaign_hash` - Unique hash of the campaign, used to fetch its report
* `campaign_id` - Numeric identifier of the campaign
* `channel` - Channel of the campaign (email, push, sms, smart\_sms, voice or web\_push)
* `status` - Status of the campaign (draft, canceled, queued, paused, sending or sent)
* `internal_name` - Internal name of the campaign, only visible inside E-goi
* `title` - Subject of the campaign (email subject line)
* `list_id` - Identifier of the contact list the campaign was sent to
* `group_id` - Identifier of the campaign group (see the campaign\_groups stream)
* `automation_id` - Identifier of the automation that sends the campaign, when it is part of one
* `created_by` - Identifier of the E-goi user who created the campaign
* `processed_messages` - Total number of messages processed for the campaign
* `thumbnail` - URL of the campaign thumbnail image
* `created` - Date and time the campaign was created
* `updated` - Date and time the campaign was last updated
* `start_date` - Date and time the campaign started sending
* `end_date` - Date and time the campaign finished sending
* `schedule_enabled` - Whether the campaign was scheduled
* `schedule_date` - Date and time the campaign was scheduled to send
Groups used to organize campaigns.
**Key:** `group_id` · **Sync:** full table
**Fields:**
* `group_id` - Unique identifier of the group
* `name` - Name of the campaign group
Delivery and engagement report of each sent email campaign, with daily, hourly, location, domain, link and email-client breakdowns.
**Key:** `campaign_hash` · **Sync:** incremental on `campaign_sent_at`
**Fields:**
* `campaign_hash` - Hash of the campaign the report belongs to (see the campaigns stream)
* `campaign_id` - Numeric identifier of the campaign
* `list_id` - Identifier of the contact list the campaign was sent to
* `campaign_status` - Status of the campaign when the report was read
* `campaign_sent_at` - When the campaign went out (start date, else schedule or end date); used as the incremental replication key
* `sends` - Total number of messages sent
* `opens` - Total number of opens (a contact can open more than once)
* `unique_opens` - Number of contacts who opened the email
* `clicks` - Total number of link clicks
* `unique_clicks` - Number of contacts who clicked a link
* `hard_bounces` - Number of permanent delivery failures
* `soft_bounces` - Number of temporary delivery failures
* `complaints` - Number of spam complaints
* `unsubscriptions` - Number of unsubscriptions caused by the campaign
* `by_date` - Daily stats (JSON array of objects with date in DD-MM-YYYY, sends, opens, clicks, hard\_bounces, soft\_bounces, complaints and unsubscriptions)
* `by_weekday` - Stats per day of the week (JSON array; weekday 0 is Sunday)
* `by_hour` - Stats per hour of the day (JSON array; hour 00-23)
* `by_location` - Stats per location (JSON array of objects with location, region, country and counters)
* `by_domain` - Stats per recipient email domain (JSON array)
* `by_url` - Clicks per link of the email (JSON array of objects with url and clicks)
* `by_reader` - Opens per email client (JSON array of objects with reader, type and opens)
Delivery report of each sent SMS campaign, with a breakdown per mobile network.
**Key:** `campaign_hash` · **Sync:** incremental on `campaign_sent_at`
**Fields:**
* `campaign_hash` - Hash of the campaign the report belongs to (see the campaigns stream)
* `campaign_id` - Numeric identifier of the campaign
* `list_id` - Identifier of the contact list the campaign was sent to
* `campaign_status` - Status of the campaign when the report was read
* `campaign_sent_at` - When the campaign went out (start date, else schedule or end date); used as the incremental replication key
* `destinations` - Number of destinations of the campaign
* `sends` - Total number of messages sent
* `delivered` - Number of messages delivered
* `error` - Number of messages that failed
* `invalid` - Number of messages to invalid numbers
* `pending` - Number of messages still pending delivery
* `by_network` - Stats per mobile network (JSON array of objects with network, sends, delivered, error, invalid and pending)
Delivery and open report of each sent mobile push campaign, with a breakdown per operating system.
**Key:** `campaign_hash` · **Sync:** incremental on `campaign_sent_at`
**Fields:**
* `campaign_hash` - Hash of the campaign the report belongs to (see the campaigns stream)
* `campaign_id` - Numeric identifier of the campaign
* `list_id` - Identifier of the contact list the campaign was sent to
* `campaign_status` - Status of the campaign when the report was read
* `campaign_sent_at` - When the campaign went out (start date, else schedule or end date); used as the incremental replication key
* `sends` - Total number of notifications sent
* `opens` - Number of notifications opened
* `delivered` - Number of notifications delivered
* `received` - Number of notifications received
* `bounces` - Number of notifications that bounced
* `error` - Number of notifications that failed
* `by_operating_system` - Stats per operating system (JSON array of objects with operating\_systems and counters)
Delivery, open and click report of each sent web push campaign, with device, operating system, browser and URL breakdowns.
**Key:** `campaign_hash` · **Sync:** incremental on `campaign_sent_at`
**Fields:**
* `campaign_hash` - Hash of the campaign the report belongs to (see the campaigns stream)
* `campaign_id` - Numeric identifier of the campaign
* `list_id` - Identifier of the contact list the campaign was sent to
* `campaign_status` - Status of the campaign when the report was read
* `campaign_sent_at` - When the campaign went out (start date, else schedule or end date); used as the incremental replication key
* `sends` - Total number of notifications sent
* `opens` - Number of notifications opened
* `clicks` - Number of notifications clicked
* `bounces` - Number of notifications that bounced
* `by_device` - Stats per device type (JSON array)
* `by_operating_system` - Stats per operating system and version (JSON array)
* `by_browser` - Stats per browser and version (JSON array)
* `by_url` - Stats per website URL (JSON array)
Marketing automations of the account.
**Key:** `automation_id` · **Sync:** full table
**Fields:**
* `automation_id` - Unique identifier of the automation
* `title` - Title of the automation
* `list_id` - Identifier of the contact list the automation runs on
* `status` - Status of the automation (active or inactive)
* `created_by` - Identifier of the E-goi user who created the automation
* `allow_multiple_contacts` - Whether a contact can enter the automation more than once
* `contact_limit` - Maximum number of contacts the automation accepts (0 means no limit)
* `created` - Date the automation was created
* `updated` - Date the automation was last updated
Tags available to label contacts.
**Key:** `tag_id` · **Sync:** full table
**Fields:**
* `tag_id` - Unique identifier of the tag
* `name` - Name of the tag
* `color` - Color of the tag (hex code)
E-goi users of the account and their permission profiles.
**Key:** `user_id` · **Sync:** full table
**Fields:**
* `user_id` - Unique identifier of the user
* `username` - Login name of the user
* `is_admin` - Whether the user is an administrator of the account
* `first_name` - First name of the user
* `last_name` - Last name of the user
* `email` - Email address of the user
* `phone` - Phone number of the user
* `profile_image` - URL of the user's profile image
* `status` - Status of the user (active or inactive)
* `created` - Date and time the user was created
* `updated` - Date and time the user was last updated
* `profile` - Permission profiles of the user (JSON array of objects with profile\_id and title)
Email addresses campaigns can be sent from, and their approval status.
**Key:** `sender_id` · **Sync:** full table
**Fields:**
* `sender_id` - Unique identifier of the sender
* `name` - Display name of the sender
* `email` - Email address the campaigns are sent from
* `status` - Approval status of the sender (active, moderation or rejected)
SMS senders (numbers or alphanumeric names) and their approval status.
**Key:** `sender_id` · **Sync:** full table
**Fields:**
* `sender_id` - Unique identifier of the sender
* `cellphone` - Sender shown on SMS messages (number or alphanumeric name)
* `type` - Type of SMS sender (alpha\_numeric or numeric)
* `status` - Approval status of the sender (active, moderation or rejected)
Emails, domains and numbers E-goi never sends to again (unsubscribes, bounces, manual blocks).
**Key:** `id` · **Sync:** full table
**Fields:**
* `id` - Unique identifier of the entry
* `value` - Suppressed value (email address, email domain, email user or number)
* `type` - Type of value (email, email\_domain, email\_user, cellphone or phone)
* `method` - How the value entered the suppression list (unsubscribe, bounce, manual, other or forgotten)
* `campaign_hash` - Hash of the campaign that caused the suppression, when there is one
* `created` - Date and time the value was added to the suppression list
# Eduzz as a data source
Source: https://docs.nekt.com/sources/eduzz
Bring data from Eduzz to your Lakehouse.
Eduzz is a Brazilian platform for selling digital products, covering checkout and payments, subscriptions, affiliate programs, financial settlement, and the Nutror course platform. Its public API exposes sales, subscribers, buyers, products, affiliates, financial statements, and course data.
## Configuring Eduzz as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Eduzz option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an access token from an Eduzz application. Sign in at the [Eduzz developer console](https://developers.eduzz.com) with the producer account that owns the data, create an application, enable the read scopes of the data you want to sync, and copy the application's access token. The token does not expire and only reaches the data of the account that generated it, so each source needs the token of its own producer account.
The following configurations are available:
* **Access Token**: The access token of your Eduzz application, used to authenticate every API call.
* **Initial sync date**: The earliest date from which records will be synced. When left empty, the connector goes back two years.
* **Date window (days)**: (Default: 30) Size of each date range requested from the endpoints that require a start and an end date: sales, chargebacks, financial statement, and fiscal documents. Larger windows mean fewer requests to the API.
* **Items per page**: (Default: 100) Number of records requested per page on the list endpoints.
* **Requests per minute**: (Default: 30) Maximum number of API calls per minute. Eduzz documents a limit of 30 requests per minute for the whole account, so lower this value if the same account is shared with other integrations.
The read scopes are granted per Eduzz module. If the application is missing the scope of a selected stream, the extraction fails with an authorization error; enable the scope in the developer console and run the source again.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
The `product_offers`, `product_campaigns`, and `product_partners` streams request one page per product, and `nutror_course_modules` requests one page per course. On accounts with many products or courses, deselect them if you don't need catalog detail: the extraction respects the 30 requests per minute limit, so they take proportionally longer.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Sales and subscriptions are read by their update date, so records edited after they were created (refunds, cancellations, status changes) are picked up again by the incremental sync.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Eduzz and their corresponding fields. Nested objects are flattened into columns such as `buyer_email` and `total_value`, and lists of objects are delivered as JSON strings.
Every sale (invoice) of the producer, including recurring charges of subscriptions. Synced incrementally by update date.
**Key Fields:**
* `id`: Unique identifier of the sale
* `contract_id`: Subscription contract the sale belongs to, when it is a recurring charge
* `status`: Current status of the sale (paid, open, canceled, refunded, chargeback, and others)
* `total_value` / `total_currency`: Total amount paid by the buyer
* `gross_gain_value`, `net_gain_value`: Gross and net amounts earned by the producer
* `fee_value`, `affiliates_gain_value`, `partners_gain_value`: Fee and commissions discounted from the sale
* `installments`, `total_interest_value`: Installment plan chosen by the buyer
* `paid_at`, `due_date`, `credit_date`: Payment, due, and credit dates
* `created_at`, `updated_at`: Creation and last update timestamps
* `payment_method`, `payment_detail`, `payment_link`: Payment method used in the checkout
* `product_id`, `product_name`, `product_type`, `product_billing_type`, `product_sku`: Product sold
* `offer_name`: Offer used in the checkout
* `buyer_id`, `buyer_name`, `buyer_email`, `buyer_document`, `buyer_phone`: Buyer data
* `affiliate_id`, `affiliate_name`: Affiliate credited for the sale
* `producer_id`, `producer_name`, `recipient_name`, `recipient_email`: Producer and recipient of the amount
* `refund_type`, `refund_value_value`, `refund_refunded_at`: Refund applied to the sale
* `utm_source`, `utm_campaign`, `utm_medium`, `utm_content`, `utm_term`: Campaign attribution
* `tracker`, `tracker2`, `tracker3`: Tracking codes sent by the checkout link
* `order_bump_has`, `order_bump_is_main_sale`, `order_bump_main_sale_id`: Order bump relationship
* `is_infinite_funnel`, `recovering_status`: Funnel and payment recovery flags
Chargebacks opened against the producer's sales. Synced incrementally by request date.
**Key Fields:**
* `id`: Identifier of the sale the chargeback was opened against
* `chargeback_status`: Status of the chargeback (new, won, lost, waitingDocuments, inAnalysis)
* `chargeback_requested_at`: Date the chargeback was requested
* `chargeback_limit_date`: Deadline to send the contesting documents
* `chargeback_documents_uploaded_at`: Date the documents were uploaded
* `total_value` / `total_currency`: Amount under chargeback
* `product_id`, `product_name`, `product_type`, `product_sku`: Product of the original sale
* `buyer_id`, `buyer_name`, `buyer_email`, `buyer_document`: Buyer who opened the chargeback
* `affiliate_id`, `affiliate_name`: Affiliate credited for the original sale
Subscription contracts of recurring products. Synced incrementally by update date.
**Key Fields:**
* `id`: Unique identifier of the subscription contract
* `status`: Current status of the subscription (active, late, canceled, and others)
* `created_at`, `updated_at`: Creation and last update timestamps
* `first_invoice_id`: First invoice (sale) of the subscription
* `is_finite`, `is_unlimited_installments`, `has_negotiation`: Contract flags
* `recurrence_starts_at`, `recurrence_next_due_date`, `recurrence_current_due_date`, `recurrence_final_due_date`, `recurrence_finishes_at`: Recurrence calendar
* `recurrence_frequency_type`, `recurrence_frequency_value`: Interval between charges
* `charges_total`, `charges_current`, `charges_paid`, `charges_pending`, `charges_negotiated`: Charge counters
* `payment_price_value`, `payment_method`, `payment_installments`: Amount and method of each charge
* `interruption_type`, `interruption_reason`, `interruption_created_at`, `interruption_remove_access_date`, `interruption_previous_status`, `interruption_responsible`, `interruption_origin`: Cancellation or suspension data
* `client_name`, `client_email`, `client_phone_number`: Subscriber data
* `products`: Products included in the subscription, with membership fee and delivery methods (JSON)
Buyers of the producer, with an overview of their purchase history.
**Key Fields:**
* `id`: Unique identifier of the buyer
* `name`: Full name of the buyer
* `email`: Email address used in the checkout
* `phone_country_code`, `phone_area_code`, `phone_number`: Phone number
* `purchase_history_overview_products_quantity`: Number of products purchased from the producer
* `purchase_history_overview_gross_value`, `purchase_history_overview_net_value`: Gross and net amounts generated by the buyer
Products registered by the producer.
**Key Fields:**
* `id`: Unique identifier of the product
* `name`, `description`, `author`: Product identification
* `type`: Type of the product (digital, physical, service)
* `status`: Current status of the product (active, inactive)
* `moderation`: Moderation state on Eduzz
* `producer_id`: Producer who owns the product
* `image_url`: Cover image of the product
* `payment_type`: Charge model (oneTime, subscription)
* `payment_price_value`, `payment_price_currency`: Default price
* `payment_methods_bankslip`, `payment_methods_pix`, `payment_methods_card`, `payment_methods_multiple_cards`: Accepted payment methods
* `created_at`, `updated_at`: Creation and last update timestamps
Offers (checkout links and prices) configured for each product.
**Key Fields:**
* `id`: Unique identifier of the offer
* `product_id`: Product the offer belongs to
* `name`: Name of the offer
* `is_default`: Whether this is the default offer of the product
* `is_visible_to_affiliates`: Whether affiliates can promote the offer
* `checkout_url`: Checkout URL of the offer
* `default_price_value`, `default_price_currency`: Price in the default currency
* `prices`: Prices per currency (JSON)
Affiliate campaigns created for each product.
**Key Fields:**
* `id`: Unique identifier of the campaign
* `product_id`: Product the campaign belongs to
* `key`: Key identifying the campaign in the checkout links
* `title`: Title of the campaign
* `url`: Destination URL of the campaign
* `result_per_click_affiliate`, `result_per_click_producer`: Result per click credited to each side
* `is_infinity_funnel`, `is_active`: Campaign flags
* `created_at`: Creation timestamp
Partners sharing the revenue of each product.
**Key Fields:**
* `id`: Unique identifier of the partner
* `product_id`: Product the partner is attached to
* `name`, `email`: Partner identification
* `commission_type`, `commission_value`, `commission_only_by_affiliate`: Commission rules
* `permission_can_manage_affiliates`, `permission_can_refund`, `permission_can_view_customer_data`: Permissions granted to the partner
Discount coupons created for the producer's products.
**Key Fields:**
* `code`: Code the buyer types in the checkout
* `description`: Description of the coupon
* `value`, `type`: Discount granted, as a fixed value or a percentage
* `quantity`: Number of times the coupon can still be redeemed
* `limit_date`: Date after which the coupon expires
* `products`: Products the coupon applies to, with discounted price and checkout link (JSON)
Affiliations between the producer's products and their affiliates.
**Key Fields:**
* `id`: Unique identifier of the affiliation
* `status`: Status of the affiliation request (approved, pending, refused)
* `requested_at`, `approved_at`, `created_at`, `updated_at`: Affiliation timeline
* `product_id`, `product_name`: Product the affiliate promotes
* `customer_id`, `customer_name`: Eduzz account affiliated to the product
Statement entries of the producer's Eduzz account. Synced incrementally by credit date.
**Key Fields:**
* `id`: Unique identifier of the statement entry
* `credit_date`: Date the amount was credited to the account
* `description`: Description shown in the statement
* `type`: Category of the entry (sale, refund, bonus, fee, chargeback, and others)
* `sale_id`: Sale that originated the entry
* `value_value`, `value_currency`: Amount of the entry
Withdrawals transferred from the Eduzz account to the producer's bank account.
**Key Fields:**
* `id`: Unique identifier of the transfer
* `value_value`, `value_currency`: Amount transferred
* `status`: Current status of the transfer
* `requested_at`, `completed_at`, `created_at`: Transfer timeline
* `details`: Details registered about the transfer
* `bank_account_type`, `bank_account_bank_code`, `bank_account_agency`, `bank_account_account`: Destination bank account
Fiscal documents issued to the producer's affiliates. Synced incrementally by emission date.
**Key Fields:**
* `document_id`: Unique identifier of the fiscal document
* `document_name`, `document_type`, `document_status`: Document identification and status
* `document_emission_date`, `document_reference_date`, `document_processing_date`: Document timeline
* `document_base_value_value`, `document_tax_value_value`, `document_tax_base`, `document_base_percent`: Calculation base and withheld tax
* `sale_id`, `sale_payment_date`, `sale_total_value`: Sale that originated the document
* `product_id`, `product_title`: Product of the originating sale
* `source`, `destination`, `client`: Issuer, recipient, and client data with addresses (JSON)
Fiscal documents issued to the producer's buyers. Synced incrementally by emission date. Same fields as the affiliate fiscal documents stream.
**Key Fields:**
* `document_id`: Unique identifier of the fiscal document
* `document_name`, `document_type`, `document_status`: Document identification and status
* `document_emission_date`, `document_reference_date`, `document_processing_date`: Document timeline
* `document_base_value_value`, `document_tax_value_value`: Calculation base and withheld tax
* `sale_id`, `sale_payment_date`, `sale_total_value`: Sale that originated the document
* `product_id`, `product_title`: Product of the originating sale
* `source`, `destination`, `client`: Issuer, recipient, and client data with addresses (JSON)
Courses published by the producer on Nutror, the Eduzz course platform.
**Key Fields:**
* `id`: Unique identifier of the course
* `title`: Title of the course
* `hash`: Hash identifying the course in the Nutror URLs
* `published`: Whether the course is published to the students
* `category_id`, `category_name`: Category of the course
* `my_eduzz_association_content_id`, `my_eduzz_association_title`: Product delivering the course
* `created_at`, `updated_at`: Creation and last update timestamps
Modules of each Nutror course.
**Key Fields:**
* `id`: Unique identifier of the module
* `course_id`: Course the module belongs to
* `title`: Title of the module
* `sequence`: Position of the module inside the course
* `is_draft`, `is_free`: Module flags
* `days_locked`, `available_days`, `release_date`: Release rules for the students
* `banner`: Banner image of the module
* `created_at`: Creation timestamp
Students enrolled in the producer's Nutror courses.
**Key Fields:**
* `id`: Unique identifier of the student
* `name`: Full name of the student
* `email`: Email address used to access Nutror
* `avatar`: Profile picture of the student
Comments left by students on the lessons of the Nutror courses.
**Key Fields:**
* `id`: Unique identifier of the comment
* `text`: Content of the comment
* `author`: Student who wrote the comment
* `type`: Type of the comment content
* `status`: Moderation status of the comment
* `lesson_id`: Lesson the comment was left on
* `is_read`, `is_answered`, `is_fixed`, `is_favorite`: Comment flags
* `created_at`: Creation timestamp
Nutror is a separate Eduzz product. If the account doesn't use it, the Nutror streams are skipped with a warning instead of failing the extraction.
# eKyte as a data source
Source: https://docs.nekt.com/sources/ekyte
Bring data from eKyte to your Lakehouse.
eKyte is a project, task, and help desk management platform for marketing teams and agencies. The connector extracts your company's tasks, projects, tickets, time tracking, and phase completions, the data-driven and social media metrics imported into eKyte, and reference data such as workspaces, squads, users, tags, task types, and knowledge base boards, so you can analyze your team's productivity and your clients' results in your Lakehouse.
The connector uses eKyte's BI API, which is available on the eKyte **Performance** plan.
## Configuring eKyte as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the eKyte option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
eKyte authenticates with a **BI access key** that belongs to your company. To get it, open **Minha Empresa** in eKyte and copy the key shown in the **BI** tab.
The key gives read access to all of your company's data in eKyte. If it is regenerated or revoked in eKyte, this connection stops working until you update it with the new key. If your company is not on the Performance plan, eKyte refuses the requests and the run log says so.
The following configurations are available:
* **API Key**: the BI access key of your company. Required.
* **Start Date**: the earliest date from which records will be synced: tasks, projects, tickets, and time entries created from this date on, and metrics dated from this date on. Leave it empty to extract the full history.
* **Include Comments**: also extract the comments of tasks and tickets. Syncs take longer when it is on.
* **Include History**: also extract the change history of tasks and the phase history of tickets. Syncs take longer when it is on.
* **Form IDs**: the IDs of the eKyte forms whose answers you want in the Form Answers stream. eKyte only returns answers form by form, so the stream is skipped when this list is empty.
* **Lookback Days**: how many days are re-read before the last sync point by the incremental streams, to pick up records eKyte changes after they first appear. The defaults are 7 days for time entries, 3 for phase completions, and 30 for metrics, which eKyte keeps updating as the ad platforms settle their numbers.
* **Requests Per Minute**: how fast the connector may call eKyte. The default is 60.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Time Trackings, Phase Completions, Insights, and Organic Insights support INCREMENTAL. The other streams are FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
eKyte's API does not tell when a task, project, or ticket last changed, so these streams are read in full (from the **Start Date**) on every sync. That is what keeps phase, executor, and status changes of older records up to date.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from eKyte and their corresponding fields. API reference: [eKyte BI API](https://developers.ekyte.com/api-reference/).
eKyte sends dates in Brasília time; the connector converts every timestamp to UTC. Lists and objects (phases, checklists, comments, history, notes, and form answers) are stored as JSON text.
Every task of your company, in every situation: active, paused, done, and cancelled (`GET /v1.2/tasks`). Phases and checklists always come along; comments and change history only when the corresponding settings are on.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the task. |
| `title` | String | Title of the task. |
| `description` | String | Description of the task. |
| `allocation_type` | Integer | How the effort of the task is controlled (allocation type code). |
| `workspace_id` | Integer | Identifier of the workspace (client) the task belongs to. |
| `workspace` | String | Name of the workspace. |
| `situation` | Integer | Situation of the task: 10 = active, 20 = paused, 30 = done, 40 = cancelled. |
| `priority` | Integer | Priority: 10 = not prioritized, 100 = low, 200 = medium, 300 = high, 400 = urgent. |
| `priority_group` | Integer | Priority group, from 0 to 100. |
| `estimated_time` | Number | Estimated effort, in minutes. |
| `actual_time` | Number | Effort tracked so far, in minutes. |
| `phase_id` | Integer | Identifier of the current phase of the task. |
| `phase` | String | Name of the current phase. |
| `co_phase_id` | Integer | Identifier of the current co-phase of the task. |
| `co_phase` | String | Name of the current co-phase. |
| `creation_date` | Datetime | When the task was created. |
| `original_due_date` | Datetime | Original due date of the task. |
| `current_due_date` | Datetime | Current due date of the task. |
| `phase_start_date` | Datetime | When the current phase started. |
| `phase_due_date` | Datetime | Due date of the current phase. |
| `resolved_date` | Datetime | When the task was completed (situation 30). |
| `created_by_id` | String | Identifier of the user who created the task. |
| `created_by` | String | Name of the user who created the task. |
| `executor_id` | String | Identifier of the current executor. |
| `executor` | String | Name of the current executor. |
| `co_executor_id` | String | Identifier of the current co-executor. |
| `co_executor` | String | Name of the current co-executor. |
| `project_id` | Integer | Identifier of the project the task belongs to. |
| `project` | String | Name of the project. |
| `project_alias` | String | Short identification (alias) of the project. |
| `ctc_task_type_id` | Integer | Identifier of the task type. |
| `ctc_task_type` | String | Name of the task type. |
| `ctc_task_predecessor_id` | Integer | Identifier of the predecessor task. |
| `ctc_task_predecessor` | String | Title of the predecessor task. |
| `channel_id` | Integer | Identifier of the channel. |
| `channel` | String | Name of the channel. |
| `media_id` | Integer | Identifier of the media. |
| `media` | String | Name of the media. |
| `quantity` | Number | Number of pieces the task delivers. |
| `phases` | String | Phases of the task's workflow with effort, dates and executor of each (JSON array of objects). |
| `checklists` | String | Checklist items of the task, with completion date and user (JSON array of objects). |
| `comments` | String | Comments of the task (JSON array of objects). Only filled when the 'Include Comments' setting is on. |
| `iterations` | String | Change history of the task: when, who, action, field, old and new value (JSON array of objects). Only filled when the 'Include History' setting is on. |
Projects of your company with planned and accomplished effort, budget, and expenses (`GET /v1.1/projects`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the project. |
| `name` | String | Name of the project. |
| `alias` | String | Short identification (alias) of the project. |
| `link` | String | Link to the project in eKyte. |
| `workspace_id` | Integer | Identifier of the workspace (client) the project belongs to. |
| `workspace` | String | Name of the workspace. |
| `start_date` | Date | Start date of the project. |
| `end_date` | Date | End date of the project. |
| `planned_tasks_count` | Integer | Number of tasks in the project. |
| `accomplished_tasks_count` | Integer | Number of completed tasks in the project. |
| `planned_tasks_time` | Number | Estimated effort of all the project's tasks, in minutes. |
| `accomplished_tasks_time` | Number | Effort tracked on the project's tasks, in minutes. |
| `planned_project_tasks_count` | Integer | Number of planned tasks. |
| `ctc_task_project_tasks_count` | Integer | Total number of tasks. |
| `unplanned_project_tasks_count` | Integer | Number of unplanned tasks. |
| `expected_effort` | Number | Expected effort of the project. |
| `trend` | Number | Trend indicator of the project. |
| `planned_manual_budget` | Number | Planned budget. |
| `hourly_budget` | Number | Hourly rate of the project. |
| `planned_total_estimated_hourly_rate` | Number | Planned value, estimated. |
| `planned_total_accomplished_hourly_rate` | Number | Planned value, accomplished. |
| `total_estimated_expense` | Number | Estimated expenses. |
| `total_realized_expense` | Number | Realized expenses. |
| `situation` | Integer | Situation of the project: 0 = inactive, 1 = active, 2 = archived. |
| `created_by_id` | String | Identifier of the user who created the project. |
| `created_by` | String | Name of the user who created the project. |
| `tags` | String | Tags of the project. |
| `created_in` | Datetime | When the project was created. |
| `planned` | Integer | Whether the project is planned: 0 = no, 1 = yes. |
Help desk tickets of your company, in every status (`GET /v1.1/tickets`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the ticket. |
| `subject` | String | Subject of the ticket. |
| `link` | String | Link to the ticket in eKyte. |
| `workspace_id` | Integer | Identifier of the workspace (client) the ticket belongs to. |
| `workspace` | String | Name of the workspace. |
| `status` | Integer | Status of the ticket: 1 = in progress, 2 = waiting for the requester, 3 = resolved, 9 = cancelled. |
| `created_in` | Datetime | When the ticket was created. |
| `expect_due_date` | Datetime | When the ticket is expected to be done. |
| `priority` | Integer | Priority: 10 = not prioritized, 100 = low, 200 = medium, 300 = high, 400 = urgent. |
| `priority_group` | Integer | Priority group, from 0 to 100. |
| `source` | Integer | Origin of the ticket: 1 = internal, 2 = external. |
| `type` | Integer | Type of the ticket: 1 = request, 2 = question, 3 = correction, 4 = quote, 5 = improvement. |
| `concluded_in` | Datetime | When the ticket was concluded. |
| `reopen_count` | Integer | How many times the ticket was reopened. |
| `created_by_id` | String | Identifier of the user who created the ticket. |
| `created_by` | String | Name of the user who created the ticket. |
| `requester_by_id` | String | Identifier of the requester. |
| `requester_by` | String | Name of the requester. |
| `executor_id` | String | Identifier of the current executor. |
| `executor` | String | Name of the current executor. |
| `executor_email` | String | Email of the current executor. |
| `phase_id` | Integer | Identifier of the current phase of the ticket. |
| `phase` | String | Name of the current phase. |
| `actual_time` | Number | Effort tracked on the ticket, in minutes. |
| `time_first_response` | Number | Minutes between the opening of the ticket and the first reply to the client. |
| `iterations` | Integer | Number of interactions with the client (external comments). |
| `ticket_iteration` | String | Phase and conclusion history of the ticket: field, from, to, when and action (JSON array of objects). Only filled when the 'Include History' setting is on. |
| `comments` | String | Comments of the ticket with author, date, content and origin (JSON array of objects). Only filled when the 'Include Comments' setting is on. |
Time entries tracked on tasks, tickets, plans, and ads (`GET /v1.0/time-trackings`).
Primary key: `id` · Replication key: `created_in`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the time entry. |
| `effort` | Number | Total time of the entry, in minutes. |
| `workspace_id` | Integer | Identifier of the workspace (client) of the entry. |
| `workspace` | String | Name of the workspace. |
| `status` | Integer | Status of the entry: 10 = pending (running timer), 20 = done, 30 = cancelled. |
| `created_in` | Datetime | When the entry was created, used as the incremental replication key. |
| `start_date` | Datetime | When the tracked time started. |
| `end_date` | Datetime | When the tracked time ended. |
| `comment` | String | Comment entered with the time. |
| `ctc_task_id` | Integer | Identifier of the task the time was tracked on. |
| `ctc_task` | String | Title of the task. |
| `ctc_task_type_id` | Integer | Identifier of the task type. |
| `ctc_task_type` | String | Name of the task type. |
| `executor_id` | String | Identifier of the user who tracked the time. |
| `executor` | String | Name of the user who tracked the time. |
| `accomplished_hourly_rate` | Number | Value of the tracked time at the user's hourly rate. |
| `phase_id` | Integer | Identifier of the phase. |
| `phase` | String | Name of the phase. |
| `planning_id` | Integer | Identifier of the plan. |
| `planning` | String | Name of the plan. |
| `ad_id` | Integer | Identifier of the ad. |
| `ad` | String | Name of the ad. |
| `ad_set_id` | Integer | Identifier of the ad set. |
| `ad_set` | String | Name of the ad set. |
| `ticket_id` | Integer | Identifier of the ticket the time was tracked on. |
| `ticket` | String | Subject of the ticket. |
One row per task phase concluded, with expected and actual dates and whether it was late (`GET /v1.2/activities/performance`).
Primary key: `id` · Replication key: `real_phase_due_date`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the record. |
| `task_id` | Integer | Identifier of the task. |
| `task` | String | Title of the task. |
| `workspace_id` | Integer | Identifier of the workspace (client) of the task. |
| `workspace` | String | Name of the workspace. |
| `executor` | String | Name of the executor of the phase. |
| `executor_email` | String | Email of the executor of the phase. |
| `phase` | String | Name of the phase. |
| `workflow` | String | Name of the workflow the phase belongs to. |
| `expected_phase_start_date` | Datetime | Expected start of the phase (only for tasks planned by phase). |
| `expected_phase_due_date` | Datetime | Expected conclusion of the phase (only for tasks planned by phase). |
| `real_phase_start_date` | Datetime | Actual start of the phase. |
| `real_phase_due_date` | Datetime | Actual conclusion of the phase, used as the incremental replication key. |
| `overdue` | Boolean | Whether the phase was delivered late. |
Answers submitted to the forms listed in **Form IDs** (`GET /v1.0/forms/answers`). Skipped when no form ID is configured.
Primary key: `form_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the answer. |
| `form_id` | Integer | Identifier of the form. |
| `form_name` | String | Name of the form. |
| `task_id` | Integer | Identifier of the task the answer is linked to. |
| `workspace_id` | Integer | Identifier of the workspace (client) of the answer. |
| `workspace` | String | Name of the workspace. |
| `date` | Datetime | When the answer was submitted. |
| `created_by` | String | Email of the user who filled in the form. |
| `answers` | String | Answers to the form's questions, keyed by question name (JSON object). |
Data-driven metrics imported into eKyte from paid media and analytics: cost, impressions, clicks, leads, revenue, and more (`GET /v1.0/insights`). Each row is one metric of one entity on one date; values are not aggregated and must be summed as needed.
Primary key: `row_key` · Replication key: `date`
| Field | Type | Description |
| :- | :- | :- |
| `row_key` | String | Surrogate key of the row: a hash of all its dimensions (every field but value). |
| `external_id` | String | External identifier of the row. |
| `value` | Number | Value of the metric (cost, leads, clicks...). Rows are not aggregated. |
| `workspace_id` | Integer | Identifier of the workspace (client). |
| `workspace` | String | Name of the workspace. |
| `date` | Date | Date the metric refers to, used as the incremental replication key. |
| `updated_in` | Datetime | When the row was last updated in eKyte. |
| `end_date` | Datetime | End of the period the metric refers to. |
| `metric` | Integer | Metric code: 10 = amount spent, 20 = impressions, 21 = reach, 30 = clicks, 31 = total traffic, 32 = sessions, 41 = Google Display remarketing base, 42 = Facebook remarketing base, 43 = email list base, 50 = leads, 51 = email base, 60 = presentation, 70 = proposal, 80 = trial, 90 = account, 100 = revenue, 110 = add to cart, 120 = balance (Meta Ads). |
| `metric_name` | String | Name of the metric. |
| `channel_id` | Integer | Identifier of the conversion channel. |
| `channel` | String | Name of the conversion channel. |
| `media_id` | Integer | Identifier of the conversion media. |
| `media` | String | Name of the conversion media. |
| `source_channel_id` | Integer | Identifier of the channel the data came from. |
| `source_channel` | String | Name of the channel the data came from. |
| `source_media_id` | Integer | Identifier of the media the data came from. |
| `source_media` | String | Name of the media the data came from. |
| `campaign_id` | String | Identifier of the campaign (Google Ads or Facebook Ads). |
| `campaign` | String | Name of the campaign. |
| `ad_set_id` | String | Identifier of the ad set / ad group (Google Ads or Facebook Ads). |
| `ad_set` | String | Name of the ad set / ad group. |
Social media monitoring metrics (followers, reach, engagement, reactions, comments, and shares), per profile or per post (`GET /v1.0/insights/organic`).
Primary key: `row_key` · Replication key: `date`
| Field | Type | Description |
| :- | :- | :- |
| `row_key` | String | Surrogate key of the row: a hash of all its dimensions (every field but value). |
| `value` | Number | Value of the metric. |
| `metric` | Integer | Metric code: 10 = followers, 20 = reach, 30 = engagement, 40 = reactions, 50 = comments, 60 = shares. |
| `metric_name` | String | Name of the metric. |
| `frequency` | Integer | Frequency code: 20 = daily, 50 = total. |
| `frequency_name` | String | Name of the frequency. |
| `workspace_id` | Integer | Identifier of the workspace (client). |
| `workspace` | String | Name of the workspace. |
| `channel_id` | Integer | Identifier of the channel (profile). |
| `channel` | String | Name of the channel (profile). |
| `media_id` | Integer | Identifier of the social network. |
| `media` | String | Name of the social network. |
| `post_id` | Integer | Identifier of the post (empty for profile metrics). |
| `post` | String | Text of the post, truncated by eKyte. |
| `external_created_at` | Datetime | When the post was published. |
| `date` | Datetime | Date the metric refers to, used as the incremental replication key. |
Workspaces, the clients or departments your team works for (`GET /v1.0/workspaces`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the workspace. |
| `name` | String | Name of the workspace. |
| `description` | String | Description of the workspace. |
| `squad_id` | Integer | Identifier of the squad serving the workspace. |
| `squad_name` | String | Name of the squad serving the workspace. |
| `active` | Integer | Status of the workspace. |
| `external_id` | String | External identifier of the workspace (usually the client's CNPJ). |
Squads, the teams of your company (`GET /v1.0/squads`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the squad. |
| `name` | String | Name of the squad. |
Users of your company with position and hourly rate (`GET /v1.0/users`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the user. |
| `email` | String | Email of the user. |
| `name` | String | Name of the user. |
| `username` | String | Nickname of the user. |
| `profile_company` | String | Access profile of the user in the company. |
| `position_id` | Integer | Identifier of the user's position. |
| `position` | String | Position (role) of the user. |
| `invited_in` | Datetime | When the user was invited. |
| `hourly_rate` | Number | Hourly rate of the user. |
Tags of your company (`GET /v1.0/tags`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the tag. |
| `name` | String | Name of the tag. |
| `type` | Integer | Type of the tag: 0 = shared default, 10 = data-driven, 20 = project. |
Task types with their workflow, category, tags, and phases (`GET /v1.0/task-types`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the task type. |
| `name` | String | Name of the task type. |
| `description` | String | Description of the task type. |
| `workflow_id` | Integer | Identifier of the workflow of the task type. |
| `workflow_name` | String | Name of the workflow of the task type. |
| `ctc_task_type_group_id` | Integer | Identifier of the category of the task type. |
| `ctc_task_type_group_name` | String | Name of the category of the task type. |
| `tags` | String | Tags of the task type with id, name and color (JSON array of objects). |
| `flow_phases` | String | Phases of the task type's workflow with their checklist (JSON array of objects). |
Knowledge base boards and their notes (`GET /v1.0/boards`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the board. |
| `title` | String | Title of the board. |
| `description` | String | Description of the board. |
| `created_at` | Datetime | When the board was created. |
| `active` | Boolean | Whether the board is active. |
| `workspace_id` | Integer | Identifier of the workspace of the board. |
| `workspace` | String | Name of the workspace of the board. |
| `created_by_id` | String | Identifier of the user who created the board. |
| `created_by_email` | String | Email of the user who created the board. |
| `notes` | String | Notes of the board with id, title, content and status (JSON array of objects). |
# Elasticsearch (Kibana) as a data source
Source: https://docs.nekt.com/sources/elasticsearch
Bring log data from Elasticsearch, accessed through Kibana, to Nekt.
Elasticsearch is a distributed search and analytics engine commonly used to store and query application logs, and Kibana is its visualization and management interface. This connector extracts log documents from your Elasticsearch indices, filtered by time period, service and free-text search, so you can analyze them alongside the rest of your data.
## Configuring Elasticsearch as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Elasticsearch option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **Kibana URL**: The base URL of your Kibana instance (e.g. `https://my-project.kb.us-central1.gcp.elastic.cloud`). Search requests are routed through Kibana, so this is the only endpoint you need.
* **API Key**: A base64-encoded Elasticsearch API key. You can create one in Kibana under **Stack Management > API Keys**. The same key authenticates both Kibana and Elasticsearch requests.
* **Log searches**: The list of searches to extract. Each search becomes one table in your catalog, with the following options:
* **Stream name**: The table name for this search (letters, numbers and underscores, e.g. `checkout_api_logs`).
* **Index pattern**: The Elasticsearch index pattern to search (e.g. `logs-*`). Check the `data_views` stream or your Kibana Discover page if you're unsure which patterns exist.
* **Service name** (optional): An exact filter on the `service.name` field (e.g. `Lastlink.Checkout.Api`).
* **Search query** (optional): A Lucene query applied to the `message` field by default. Use `field:value` syntax to target other fields (e.g. `event.category:antifraud`).
Optionally, you can define some advanced settings:
* **Elasticsearch URL**: When provided, log searches query the official Elasticsearch `_search` API directly instead of going through the Kibana console proxy. Recommended if your Kibana instance has the Dev Tools console disabled.
* **Start Date**: The earliest log timestamp (`@timestamp`) to extract on the first sync.
* **Page size**: Number of documents fetched per request (1000-5000 recommended).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Elasticsearch and their corresponding fields:
Log documents matching the search's index pattern and filters, synced incrementally by `@timestamp`.
| Field | Type | Description |
| - | - | - |
| `id` | string | Elasticsearch document `_id`. |
| `index` | string | Concrete index the document lives in. |
| `timestamp` | datetime | Document `@timestamp`, normalized to UTC. |
| `service_name` | string | Value of `service.name`, when present. |
| `message` | string | Log message, when present. |
| `source` | string | The full document `_source` as a JSON string, preserving every field. |
Data views registered in Kibana (formerly index patterns). Useful to discover which index patterns exist when configuring your log searches.
| Field | Type | Description |
| - | - | - |
| `id` | string | Data view unique identifier. |
| `name` | string | Display name of the data view. |
| `title` | string | Index pattern the data view targets. |
| `type` | string | Data view type, when present. |
| `time_field_name` | string | Default time field of the data view. |
| `namespaces` | array | Kibana spaces the data view belongs to. |
If there is any data you need that is not mapped here, just reach out to us through our [Slack](https://join.slack.com/t/nektcommunity/shared_invite/zt-2k3v2t1jq-cZG~ZWXzNZJF~Q1bAnyTsQ) channel and we'll evaluate adding it to the connector.
# Elephan.ai as a data source
Source: https://docs.nekt.com/sources/elephan-ai
Bring data from Elephan.ai to your Lakehouse.
Elephan.ai is a revenue intelligence platform for B2B teams. It records, transcribes and analyzes sales meetings (online or in person) and WhatsApp conversations, producing AI summaries, scorecards, detected objections and insights on top of them. The External API exposes those meetings, conversations, insights and scorecard aggregates so they can be analyzed alongside the rest of your data.
Elephan.ai (no trailing "t") is the revenue intelligence platform at [elephan.ai](https://www.elephan.ai/). It is a different product from Elephant.ai, an unrelated chatbot builder. This connector reads from the Elephan.ai External API.
## 1. Add your Elephan.ai access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Elephan.ai option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Key:** Your Elephan.ai API key, in the `elk_prod_...` format. Generate it inside Elephan.ai under **Configurações → API / Integrações → Gerar nova API Key**.
* **Start Date:** (Optional, advanced) The earliest record date to sync on the first run. Leave it empty to sync the full history of your team.
* **Requests per Minute:** (Optional, advanced) How many API requests per minute the connector is allowed to make. The default of 15 is deliberately conservative. Only raise it after confirming your account's real limit with Elephan.ai.
The API key grants read access to the data of the whole team it belongs to, including meeting transcriptions and summaries. Treat it as a secret.
3. Click **Next**.
## 2. Select your Elephan.ai streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Elephan.ai data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Elephan.ai data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## How the data is replicated
**Meetings, conversations and insights** replicate incrementally on `date_included`, the timestamp when the record was added to Elephan.ai. Each sync asks the API for everything included after the highest `date_included` already extracted.
Because the filter is on when a record was **added**, later edits to an older meeting (for example a re-run of the AI analysis, which updates `date_modified`) are not picked up by an incremental sync. Schedule an Additional Full Sync if you need those revisions reflected in your tables.
**Users, prompts and the scorecard aggregates** are extracted in full on every run, since they describe the current state of your team rather than a stream of events.
**The scorecard aggregates** (`scorecard_by_seller` and `scorecard_by_question`) are recomputed by Elephan.ai over a period rather than being an event log. The connector asks for the window that starts at your configured Start Date (or the last 365 days when none is set) and ends at the moment of the sync. Every row records the exact window it was computed over in `period_start` and `period_end`.
**Nested data is delivered as JSON strings.** Fields holding lists or objects, such as `answers`, `competitors`, `reasons`, `transcript_speakers`, `distribution` and `by_question`, arrive as JSON strings, so you can parse them in a Query when you need the individual keys. Simple lists of text, such as `keywords` and `tags`, remain regular arrays.
The API never returns the body of WhatsApp messages. The `conversation_messages` stream carries only metadata: timestamp, direction, author and message type. The AI-generated summary of a WhatsApp conversation is likewise not exposed by the listing endpoint.
## Rate limits
Elephan.ai does not publish a documented rate limit. The connector therefore throttles itself to 15 requests per minute by default and automatically waits and retries when the API asks it to. If Elephan.ai confirms a higher limit for your account, raise the **Requests per Minute** setting to speed up large extractions.
The `conversation_messages` stream makes one API request per conversation synced. On accounts with many WhatsApp conversations it is significantly slower than the other streams; select it only if you need the message-level timeline.
# Streams and Fields
Below you'll find all available data streams from Elephan.ai and their corresponding fields.
Meetings recorded, transcribed and analyzed by Elephan.ai. This is the core entity of the platform.
**Key Fields:**
* `id` - Unique identifier of the meeting
* `title` - Title of the meeting
* `date_included` - Timestamp when the meeting was added to Elephan.ai, used as the incremental replication key
* `date_modified` - Timestamp when the meeting record was last updated
* `status` - Processing status of the transcription, where D means done
* `duration` - Length of the recorded audio in seconds
* `user_id` - Identifier of the user who owns the meeting
* `user_name` - Name of the user who owns the meeting
* `user_email` - Email address of the user who owns the meeting
* `prompt_id` - Identifier of the meeting type applied to the analysis
* `prompt_name` - Name of the meeting type applied to the analysis
* `prompt_type` - Category of the meeting type, such as sales
* `deal_id` - Identifier of the CRM record linked to the meeting, as it exists in the CRM
* `deal_type` - Kind of CRM record linked to the meeting: deal, contact or company
* `deal_crm_url` - Direct link to the linked record in the CRM, when a CRM integration is configured
* `transcript_text` - Full text of the meeting transcription
* `transcript_speakers` - Per-speaker breakdown of the transcription, with the spoken text, its offsets in seconds and its sentiment, as a JSON string
* `answers` - Scorecard answers produced for the meeting, as a JSON string
* `keywords` - Keywords detected in the meeting
* `competitors` - Competitors mentioned during the meeting, with how often and the position taken, as a JSON string
* `reasons` - Reasons detected in the meeting, such as objections, as a JSON string
* `tags` - Tags applied to the meeting
* `important_points` - Key points the AI highlighted from the meeting
* `sentiment_total` - Overall sentiment of the meeting
* `sentiment_by_speaker` - Sentiment attributed to each speaker, as a JSON string
* `summary` - Plain-text summary of the meeting generated by the AI
WhatsApp conversations tracked by Elephan.ai, with metadata and scorecard statistics.
**Key Fields:**
* `id` - Unique identifier of the conversation
* `contact_name` - Name of the contact on the other side of the conversation
* `phone_number` - Phone number of the contact, in international format
* `picture_url` - URL of the contact's profile picture
* `user_id` - Identifier of the team member who owns the conversation
* `user_name` - Name of the team member who owns the conversation
* `user_email` - Email address of the team member who owns the conversation
* `message_count` - Number of messages exchanged in the conversation
* `last_message_at` - Timestamp of the most recent message in the conversation
* `scorecard_total_answers` - Number of scorecard answers evaluated for the conversation
* `scorecard_average` - Average scorecard score across the conversation's answers
* `date_included` - Timestamp when the conversation was added to Elephan.ai, used as the incremental replication key
Metadata of the messages exchanged in a WhatsApp conversation. Message bodies are never returned by the API.
**Key Fields:**
* `conversation_id` - Identifier of the conversation the message belongs to
* `id` - Identifier of the message. When the API does not provide one, a deterministic hash of the message metadata is used instead
* `date` - Timestamp when the message was sent
* `direction` - Direction of the message: in for messages received from the contact, out for messages sent by the team
* `from_me` - Whether the message was sent by the team member rather than the contact
* `author` - Phone number of whoever sent the message
* `type` - Kind of message, such as text
* `scorecard` - Per-message scorecard statistics, as a JSON string. Reserved by the API and currently always empty
Individual observations the AI extracted from meetings, such as objections, churn signals or feature requests.
**Key Fields:**
* `id` - Unique identifier of the insight
* `source` - Area the insight belongs to: product, sale or cs (customer success)
* `type` - Kind of insight, such as objection, churn\_signal, closing\_signal or feature\_request. The possible values depend on the source
* `description` - Short description of the insight
* `details` - Extended explanation of the insight
* `label` - Name of the cluster the insight was grouped into
* `cluster_id` - Identifier of the cluster the insight belongs to
* `category` - Additional categorization, present only on some objection types
* `date_included` - Timestamp when the insight was generated, used as the incremental replication key
* `transcribe_id` - Identifier of the meeting the insight was extracted from
* `transcribe_name` - Title of the meeting the insight was extracted from
* `deal_id` - Identifier of the CRM deal linked to the insight
* `deal_name` - Name of the CRM deal linked to the insight
Team insights aggregated by cluster, showing the recurring themes across meetings.
**Key Fields:**
* `cluster_key` - Primary key of the row: the cluster identifier as text, or unclustered for insights without one
* `cluster_id` - Identifier of the cluster
* `label` - Name given to the cluster, such as the recurring objection it captures
* `count` - Number of insights grouped under the cluster
* `types` - Breakdown of the insight types inside the cluster, each with its own count, as a JSON string
* `first_date` - Timestamp of the earliest insight in the cluster
* `last_date` - Timestamp of the most recent insight in the cluster
* `samples` - Up to three example descriptions taken from the cluster's insights
Average scorecard results per seller over the extracted period.
**Key Fields:**
* `user_id` - Identifier of the seller the scores are aggregated for
* `user_name` - Name of the seller
* `user_email` - Email address of the seller
* `meetings_count` - Number of meetings the seller had in the period
* `answers_count` - Number of scorecard answers evaluated for the seller in the period
* `average` - Average scorecard score of the seller across the period
* `by_question` - Per-question breakdown of the seller's scores, as a JSON string
* `period_start` - Start of the period the aggregate was computed over
* `period_end` - End of the period the aggregate was computed over
Average scorecard results per question over the extracted period.
**Key Fields:**
* `question` - Text of the scorecard question the scores are aggregated for
* `question_type` - Kind of question: range for numeric scores, yesno for boolean answers, text for free-form ones
* `average` - Average score of the question across the period
* `count` - Number of answers the average is based on
* `distribution` - How the answers were spread out, as a JSON string: counts per band for range questions, or per answer for yes/no questions
* `period_start` - Start of the period the aggregate was computed over
* `period_end` - End of the period the aggregate was computed over
Members of the Elephan.ai team the API key belongs to.
**Key Fields:**
* `id` - Unique identifier of the user
* `name` - Full name of the user as registered in Elephan.ai
* `email` - Email address the user signs in with
* `status` - Current account status of the user, such as active
* `type` - Permission level of the user inside the team: super\_admin, admin or user
* `role` - Business role assigned to the user, such as Vendedor
* `date_created` - Timestamp when the user was created
* `last_login` - Timestamp of the user's most recent sign-in
Meeting types available to the team, both built-in and custom. A prompt is the analysis template Elephan.ai applies to a meeting, driving its scorecard questions and summary.
**Key Fields:**
* `id` - Unique identifier of the meeting type
* `name` - Display name of the meeting type
* `type` - Category of the meeting type, such as sales
* `description` - Description of what the meeting type analyzes
* `active` - Whether the meeting type is currently available for new meetings
* `is_default` - Whether this is the meeting type applied when none is chosen
* `date_created` - Timestamp when the meeting type was created
Daily usage of the Elephan.ai API by your API key, over the last 30 days. Useful for keeping an eye on your account's request quota.
**Key Fields:**
* `date` - Day the usage figures refer to
* `count` - Total number of API calls made with the API key that day
* `avg_response_time` - Average response time of those calls, in milliseconds
* `success_count` - Number of calls that succeeded that day
* `error_count` - Number of calls that failed that day
# Elofy as a data source
Source: https://docs.nekt.com/sources/elofy
Bring data from Elofy to your Lakehouse.
Elofy is a Brazilian people management platform covering performance reviews, OKRs and goals, individual development plans (IDPs), 1-on-1 meetings, feedback, praise and engagement surveys. Nekt extracts this data through the Elofy open data API.
## Configuring Elofy as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Elofy option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The token comes from your own Elofy account. In Elofy, go to **Configurações > Integrações e API** and copy the value of **"Token API Pública - Dados dos módulos"**.
The token must belong to a user whose **only** profile is administrator. Elofy rejects API requests from users with more than one profile attached (for example manager and administrator).
The following configuration is required:
* **API Token**: The public API token copied from Configurações > Integrações e API.
Optionally, under advanced settings:
* **Start Date**: The earliest date used by the streams that require an explicit date range (Individual Development Plans). When omitted, the full history is fetched.
* **Updated Since**: Only fetches records updated on or after this date, using the API's `updatedAt` filter. Useful to bound the volume of large streams. When omitted, every stream reads its full history.
* **Page Size**: Number of records requested per page (defaults to 500). Lower it if Elofy times out on large streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Elofy exposes data per module, and the API only returns what your profile has permission to access. If your company does not use a given module (succession, surveys or nine-box, for example), its streams return no data, and the extraction continues without failing.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
The Elofy API does not return a last-modified timestamp on its records, so every stream is extracted as a full table. Use the **Updated Since** setting if you need to limit how far back each extraction reads.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Elofy and their corresponding fields. All streams are extracted as full tables. Column names are normalized to snake\_case, since the Elofy API returns field names in mixed formats.
Some streams depend on another one to know what to request: the performance review and goal streams are extracted once per period, the survey answer streams once per survey, and the detailed review stream once per configured review cycle. Nekt resolves those automatically.
## Organization
Company registered in Elofy, with its economic group.
Table: `company`
**Key Fields:**
* `id_empresa` - Identifier of the company the record belongs to.
* `nome_da_empresa` - Name of the company the record belongs to.
* `id_grupo_economico` - Identifier of the economic group the company belongs to.
Employees registered in Elofy, with team, job position, manager and access profiles.
Table: `users`
**Key Fields:**
* `id_usuario` - Identifier of the user.
* `nome_usuario` - Name of the user.
* `matricula` - Employee registration number (matrícula) of the user.
* `status` - Current status of the record.
* `data_de_admissao` - Date the employee was hired.
* `data_de_deligamento` - Date the employee was offboarded, when applicable.
* `id_gestor` - Identifier of the manager.
* `gestor` - Name of the manager.
* `email` - Email address of the user.
* `login` - Login used by the user to access Elofy.
* `tipo_cargo` - Type of the job position (for example leadership or individual contributor).
* `id_cargo` - Identifier of the job position.
Teams of the organizational structure, including the parent team hierarchy.
Table: `teams`
**Key Fields:**
* `id_time` - Identifier of the team.
* `time` - Name of the team the employee belongs to.
* `status` - Current status of the record.
* `codigo_time_sistema_origem` - Team code in the HR system that feeds Elofy.
* `id_usuario_responsavel` - Identifier of the user responsible for the team.
* `usuarios_responsavel` - Users responsible for the team.
* `id_time_pai` - Identifier of the parent team.
* `time_pai` - Name of the parent team in the organizational hierarchy.
* `ids_tags` - Comma-separated identifiers of the tags linked to the record.
* `tags` - Tags linked to the record.
Job positions, with replacement difficulty, business impact and succession mapping.
Table: `positions`
**Key Fields:**
* `id_cargo` - Identifier of the job position.
* `cargo` - Name of the job position held by the employee.
* `status` - Current status of the record.
* `codigo_cargo_sistema_origem` - Job position code in the HR system that feeds Elofy.
* `descricao` - Free-text description of the record.
* `dificuldade` - How hard the job position is to replace.
* `impacto` - Business impact of the job position.
* `mapeado_para_sucessao` - Whether the job position is mapped for succession planning.
* `regua` - Rating scale (régua) applied to the record.
Competency library, with expected behaviours and the teams and positions it applies to.
Table: `competencies`
**Key Fields:**
* `id_competencia` - Identifier of the competency.
* `competencia` - Name of the competency.
* `categoria_competencia` - Category the competency is grouped under.
* `descricao` - Free-text description of the record.
* `ids_times` - Comma-separated identifiers of the teams the competency applies to.
* `nomes_times` - Names of the teams the competency applies to.
* `ids_cargos` - Comma-separated identifiers of the job positions the competency applies to.
* `nomes_cargos` - Names of the job positions the competency applies to.
* `esperado` - Behaviour expected from employees on this competency.
* `o_que_queremos` - Behaviours the company wants to see on this competency.
* `o_que_nao_queremos` - Behaviours the company does not want to see on this competency.
* `comportamentos` - Observable behaviours described for the competency.
## Periods and cycles
Periods (planning windows) in Elofy. Their ids drive the period-filtered streams.
Table: `periods`
**Key Fields:**
* `id_periodo` - Identifier of the period.
* `periodo` - Name of the period the record belongs to.
* `data_de_inicio` - Date the period or cycle starts.
* `data_de_fim` - Date the period or cycle ends.
* `situacao` - Current state of the record as shown in Elofy.
* `escala_de_peso_de_objetivos` - Weight scale applied to objectives in the period.
* `nome_padrao_de_objetivo` - Default label used for objectives in the period.
* `escala_de_peso_de_resultados_chave` - Weight scale applied to key results in the period.
* `periodicidade_padrao_resultados_chave` - Default measurement frequency for key results in the period.
* `modulo` - Elofy module the record belongs to.
Periods with the full module configuration and volume counters of each one.
Table: `periods_detailed`
**Key Fields:**
* `id_periodo` - Identifier of the period.
* `nome_empresa` - Name of the company the record belongs to.
* `periodo` - Name of the period the record belongs to.
* `data_inicial_periodo` - Date the period starts.
* `data_final_periodo` - Date the period ends.
* `escala_pesos_kr` - Weight scale applied to key results in the period.
* `escala_pesos_meta` - Weight scale applied to goals in the period.
* `possui_ciclo_competencias` - Whether the period includes a competency cycle.
* `ciclos_com_competencias` - Cycles of the period that include a competency evaluation.
* `amplitude_competencia` - Score range used by the competency cycle.
* `possui_ciclo_potencial` - Whether the period includes a potential cycle.
* `ciclos_com_potencial` - Cycles of the period that include a potential evaluation.
Cycles inside each period, with their start and end dates and status.
Table: `cycles`
**Key Fields:**
* `id_ciclo` - Identifier of the cycle.
* `ciclo` - Name of the cycle.
* `id_periodo` - Identifier of the period.
* `data_de_inicio` - Date the period or cycle starts.
* `data_de_fim` - Date the period or cycle ends.
* `situacao` - Current state of the record as shown in Elofy.
* `status` - Current status of the record.
## Goals and OKRs
Objectives of each period, with owner, driver, parent objective, weight and progress. Extracted once per period registered in Elofy.
Table: `objectives`
**Key Fields:**
* `id_objetivo` - Identifier of the objective.
* `id_periodo` - Identifier of the period.
* `periodo` - Name of the period the record belongs to.
* `id_ciclo` - Identifier of the cycle.
* `ciclo` - Name of the cycle.
* `id_direcionador` - Identifier of the strategic driver.
* `direcionador` - Name of the strategic driver.
* `id_objetivo_pai` - Identifier of the parent objective.
* `objetivo_pai` - Name of the parent objective.
* `objetivo` - Name of the objective.
* `objetivo_estrategico` - Whether the objective is a strategic one.
* `descricao` - Free-text description of the record.
Key results of the objectives, with baseline, target, latest measurement and progress. Extracted once per period registered in Elofy.
Table: `key_results`
**Key Fields:**
* `id_resultado_chave` - Identifier of the key result.
* `id_objetivo` - Identifier of the objective.
* `resultado_chave` - Name of the key result.
* `descricao` - Free-text description of the record.
* `workflow` - Approval workflow stage of the record.
* `id_responsavel` - Identifier of the owner.
* `responsavel` - Name of the owner of the record.
* `id_corresponsavel` - Identifier of the co-owner.
* `corresponsavel` - Name of the co-owner of the record.
* `peso` - Relative weight of the record in the score calculation.
* `unidade_de_medida` - Unit of measurement of the key result.
* `meta` - Target value defined for the record.
Objectives in the V2 payload, carrying their key results and co-owners inline. Extracted once per period registered in Elofy.
Table: `okrs_v2`
**Key Fields:**
* `id_objetivo` - Identifier of the objective.
* `nome_empresa` - Name of the company the record belongs to.
* `nome_objetivo` - Name of the objective.
* `ativo` - Whether the record is currently active.
* `workflow` - Approval workflow stage of the record.
* `id_responsavel` - Identifier of the owner.
* `nome_responsavel` - Name of the owner.
* `matricula_responsavel` - Employee registration number (matrícula) of the owner.
* `id_cargo` - Identifier of the job position.
* `cargo` - Name of the job position held by the employee.
* `nivel_responsabilidade` - Responsibility level assigned to the job position.
* `id_time_responsavel` - Identifier of the owner's team.
Goal contracts (contrato de metas) in the V2 payload, carrying their goals inline. Extracted once per period registered in Elofy.
Table: `goals_contract_v2`
**Key Fields:**
* `id_objetivo` - Identifier of the objective.
* `nome_empresa` - Name of the company the record belongs to.
* `nome_objetivo` - Name of the objective.
* `ativo` - Whether the record is currently active.
* `workflow` - Approval workflow stage of the record.
* `id_responsavel` - Identifier of the owner.
* `nome_responsavel` - Name of the owner.
* `matricula_responsavel` - Employee registration number (matrícula) of the owner.
* `id_cargo` - Identifier of the job position.
* `cargo` - Name of the job position held by the employee.
* `nivel_responsabilidade` - Responsibility level assigned to the job position.
* `id_time_responsavel` - Identifier of the owner's team.
Strategic drivers, with owner, progress and the objectives and goals linked to them.
Table: `drivers`
**Key Fields:**
* `id_direcionador` - Identifier of the strategic driver.
* `nome_empresa` - Name of the company the record belongs to.
* `id_periodo` - Identifier of the period.
* `nome_periodo` - Name of the period the record belongs to.
* `nome_direcionador` - Name of the strategic driver.
* `progresso_direcionador` - Current progress of the strategic driver, as a percentage.
* `descricao_direcionador` - Description of the strategic driver.
* `responsavel` - Name of the owner of the record.
* `nome_responsavel` - Name of the owner.
* `matricula_responsavel` - Employee registration number (matrícula) of the owner.
* `id_cargo` - Identifier of the job position.
* `cargo` - Name of the job position held by the employee.
Indicator library, with the monthly goal and check-in values of each indicator.
Table: `base_indicators`
**Key Fields:**
* `id_indicador` - Identifier of the indicator.
* `nome_empresa` - Name of the company the record belongs to.
* `nome_indicador` - Name of the indicator.
* `descricao` - Free-text description of the record.
* `ativo` - Whether the record is currently active.
* `dimensao` - Dimension the indicator is classified under.
* `origem` - Where the record originated (for example manual creation or an integration).
* `peso` - Relative weight of the record in the score calculation.
* `tipo_informacao` - Kind of value tracked by the indicator (for example number or percentage).
* `forma_calculo` - Formula used to calculate the indicator progress.
* `tipo_score` - Score calculation type applied to the record.
* `id_periodo` - Identifier of the period.
## Performance review
Performance review setup, one row per review phase, with the rules enabled for it.
Table: `cycle_review_config`
**Key Fields:**
* `nome_empresa` - Name of the company the record belongs to.
* `id_revisao` - Identifier of the cycle review.
* `nome_ciclo` - Name of the cycle.
* `id_ciclo` - Identifier of the cycle.
* `nome_revisao_de_ciclo` - Name of the cycle review.
* `status_revisao_de_ciclo` - Current status of the cycle review.
* `liberacao_de_notas` - Whether scores have been released to employees.
* `calibracao_de_notas` - Whether score calibration is enabled for the cycle review.
* `responder_avaliacoes` - Whether participants can still answer evaluations.
* `ano` - Year the cycle review refers to.
* `janela` - Evaluation window the cycle review belongs to.
* `trimestre_inicio` - Quarter the cycle review starts in.
Detailed performance review, one row per evaluation, with phases and scored items inline. Extracted once per configured cycle review.
Table: `cycle_review_detailed`
**Key Fields:**
* `id_revisao_ciclo` - Identifier of the cycle review.
* `ano` - Year the cycle review refers to.
* `nome_ciclo` - Name of the cycle.
* `id_revisao_ciclos_avaliacoes` - Identifier of the cycle review evaluation.
* `id_avaliado` - Identifier of the evaluated employee.
* `matricula_avaliado` - Employee registration number (matrícula) of the evaluated employee.
* `nome_avaliado` - Name of the evaluated employee.
* `id_time_avaliado` - Identifier of the evaluated employee's team.
* `nome_time_avaliado` - Name of the evaluated employee's team.
* `tipo_cargo_avaliado` - Type of the evaluated employee's job position.
* `id_tipo_cargo_avaliado` - Identifier of the evaluated employee's job position type.
* `id_cargo_avaliado` - Identifier of the evaluated employee's job position.
Performance review audience, with the phase averages of each evaluated employee. Extracted once per period registered in Elofy.
Table: `cycle_review_public`
**Key Fields:**
* `id_revisao_de_ciclo_score` - Identifier of the cycle review score.
* `id_revisao_de_ciclo` - Identifier of the cycle review.
* `nome_revisao_de_ciclo` - Name of the cycle review.
* `id_usuario_avaliado` - Identifier of the evaluated employee.
* `matricula_usuario_avaliado` - Employee registration number (matrícula) of the evaluated employee.
* `nome_usuario_avaliado` - Name of the evaluated employee.
* `status_usuario_avaliado` - Status of the evaluated employee in the cycle.
* `media_final` - Final average score of the evaluated employee.
* `fase` - Evaluation phase the row refers to.
* `media_fase` - Average score of the evaluation phase.
* `media_fase_auto` - Average of the evaluation phase coming from the self-evaluation.
* `media_fase_gestor` - Average of the evaluation phase coming from the manager's evaluation.
Performance review scores per phase, broken down by evaluation source and rating label. Extracted once per period registered in Elofy.
Table: `cycle_review_public_note_step`
**Key Fields:**
* `id_revisao_ciclo_score` - Identifier of the cycle review score.
* `id_revisao_ciclo` - Identifier of the cycle review.
* `nome_revisao_ciclo` - Name of the cycle review.
* `id_usuario_avaliado` - Identifier of the evaluated employee.
* `nome_usuario` - Name of the user.
* `status_usuario_avaliado` - Status of the evaluated employee in the cycle.
* `id_gestor` - Identifier of the manager.
* `nome_gestor` - Name of the manager.
* `media_final_ciclo` - Final average score of the employee in the cycle.
* `conceito_final_ciclo` - Final rating label of the employee in the cycle.
* `conceito_final_ciclo_calibrado` - Final rating label of the employee in the cycle after calibration.
* `nine_box` - Nine-box matrix the employee was placed in.
Competency phase averages of the performance review, per evaluated employee. Extracted once per period registered in Elofy.
Table: `cycle_review_average_step_competencies`
**Key Fields:**
* `id_revisao_ciclo_score` - Identifier of the cycle review score.
* `id_revisao_ciclo` - Identifier of the cycle review.
* `nome_revisao_ciclo` - Name of the cycle review.
* `periodo` - Name of the period the record belongs to.
* `id_usuario_avaliado` - Identifier of the evaluated employee.
* `nome_usuario_avaliado` - Name of the evaluated employee.
* `id_gestor` - Identifier of the manager.
* `nome_gestor` - Name of the manager.
* `conceito_final_ciclo` - Final rating label of the employee in the cycle.
* `media_final_ciclo_calibrado` - Final average score of the employee in the cycle after calibration.
* `conceito_final_ciclo_calibrado` - Final rating label of the employee in the cycle after calibration.
* `media_competencia` - Average competency score of the employee.
Results phase averages of the performance review, per evaluated employee and item. Extracted once per period registered in Elofy.
Table: `cycle_review_average_step_results`
**Key Fields:**
* `id_revisao_ciclo_avaliacoes_fase` - Identifier of the cycle review phase evaluation.
* `id_revisao_ciclo` - Identifier of the cycle review.
* `nome_revisao_ciclo` - Name of the cycle review.
* `id_usuario_avaliado` - Identifier of the evaluated employee.
* `nome_usuario_avaliado` - Name of the evaluated employee.
* `id_gestor` - Identifier of the manager.
* `nome_gestor` - Name of the manager.
* `conceito_final_ciclo` - Final rating label of the employee in the cycle.
* `media_final_ciclo_calibrado` - Final average score of the employee in the cycle after calibration.
* `conceito_final_ciclo_calibrado` - Final rating label of the employee in the cycle after calibration.
* `media_final_fase_resultados` - Final average score of the results phase.
* `item_avaliado` - Item that was evaluated in the phase.
Nine-box placement of each evaluated employee, before and after calibration. Extracted once per period registered in Elofy.
Table: `cycle_review_ninebox`
**Key Fields:**
* `id_revisao_ciclo_score` - Identifier of the cycle review score.
* `id_revisao_ciclo` - Identifier of the cycle review.
* `periodo` - Name of the period the record belongs to.
* `nome_revisao_ciclo` - Name of the cycle review.
* `data_inicio` - Start date of the record.
* `data_fim` - End date of the record.
* `id_usuario_avaliado` - Identifier of the evaluated employee.
* `nome_usuario_avaliado` - Name of the evaluated employee.
* `status_usuario_avaliado` - Status of the evaluated employee in the cycle.
* `id_gestor` - Identifier of the manager.
* `nome_gestor` - Name of the manager.
* `media_final_ciclo` - Final average score of the employee in the cycle.
## Development
IDPs (individual development plans), one row per plan initiative. Extracted over the date window defined by the Start Date setting.
Table: `pdi`
**Key Fields:**
* `id_responsavel` - Identifier of the owner.
* `matricula` - Employee registration number (matrícula) of the user.
* `responsavel` - Name of the owner of the record.
* `id_pdi` - Identifier of the IDP (individual development plan).
* `titulo_pdi` - Title of the IDP (individual development plan).
* `cargo_responsavel` - Job position of the owner.
* `time_responsavel` - Team of the owner.
* `cargos_vinculados` - Job positions linked to the IDP.
* `competencias_vinculadas` - Competencies linked to the IDP.
* `prazo` - Due date of the record.
* `status_pdi` - Current status of the IDP.
* `progresso_pdi` - Current progress of the IDP, as a percentage.
Initiatives, with owner, hierarchy, progress and the key result each one supports.
Table: `initiatives`
**Key Fields:**
* `id_iniciativa` - Identifier of the initiative.
* `nome_empresa` - Name of the company the record belongs to.
* `iniciativa` - Name of the initiative.
* `descricao` - Free-text description of the record.
* `nro_sub_iniciativas` - Number of sub-initiatives under the initiative.
* `id_iniciativa_pai` - Identifier of the parent initiative.
* `nome_iniciativa_pai` - Name of the parent initiative.
* `status_iniciativa` - Current status of the initiative.
* `data_inicio` - Start date of the record.
* `data_final` - Planned end date of the record.
* `situacao_iniciativa` - Current state of the initiative (for example on time or late).
* `progresso_iniciativa` - Current progress of the initiative, as a percentage.
Initiatives created inside IDPs (individual development plans).
Table: `initiatives_pdi`
**Key Fields:**
* `id_iniciativa` - Identifier of the initiative.
* `nome_empresa` - Name of the company the record belongs to.
* `iniciativa` - Name of the initiative.
* `descricao` - Free-text description of the record.
* `status_iniciativa` - Current status of the initiative.
* `data_inicio` - Start date of the record.
* `data_final` - Planned end date of the record.
* `situacao_iniciativa` - Current state of the initiative (for example on time or late).
* `data_conclusao` - Date the initiative was completed.
* `progresso_iniciativa` - Current progress of the initiative, as a percentage.
* `disponivel_avaliacao` - Whether the initiative is available for evaluation.
* `origem` - Where the record originated (for example manual creation or an integration).
Initiatives created inside 1-on-1 meetings.
Table: `initiatives_one_one`
**Key Fields:**
* `id_iniciativa` - Identifier of the initiative.
* `nome_empresa` - Name of the company the record belongs to.
* `iniciativa` - Name of the initiative.
* `status_iniciativa` - Current status of the initiative.
* `data_inicio` - Start date of the record.
* `data_final` - Planned end date of the record.
* `situacao_iniciativa` - Current state of the initiative (for example on time or late).
* `data_conclusao` - Date the initiative was completed.
* `progresso_iniciativa` - Current progress of the initiative, as a percentage.
* `disponivel_avaliacao` - Whether the initiative is available for evaluation.
* `origem` - Where the record originated (for example manual creation or an integration).
* `id_responsavel` - Identifier of the owner.
1-on-1 meetings, with participants, talking points and linked initiatives.
Table: `one_one`
**Key Fields:**
* `id_one_one` - Identifier of the 1-on-1 meeting.
* `nome_empresa` - Name of the company the record belongs to.
* `template` - Template used to create the record.
* `status_one_one` - Current status of the 1-on-1 meeting.
* `id_usuario_remetente` - Identifier of the sender.
* `nome_usuario_remetente` - Name of the user who sent the record.
* `status_remetente` - Status of the employee who requested the 1-on-1 meeting.
* `id_gestor_remetente` - Identifier of the sender's manager.
* `nome_gestor_remetente` - Name of the sender's manager.
* `data` - Date the record refers to.
* `horario` - Scheduled time of the meeting.
* `duracao` - Duration of the meeting, in minutes.
Succession mapping per job position, with current holders and mapped candidates.
Table: `succession`
**Key Fields:**
* `id_cargo` - Identifier of the job position.
* `nome_empresa` - Name of the company the record belongs to.
* `nome_cargo` - Name of the job position.
* `descricao` - Free-text description of the record.
* `dificuldade` - How hard the job position is to replace.
* `impacto` - Business impact of the job position.
* `situacao_cargo` - Current state of the job position.
* `classificacao` - Classification assigned to the record.
* `requisitos_posicao` - Requirements defined for the job position.
* `usuarios_atuais` - Employees currently holding the job position (JSON array of objects).
* `sucessao` - Succession candidates mapped for the job position (JSON array of objects).
## Engagement
Feedback exchanged between employees, with sender, recipient, question and answer.
Table: `feedbacks`
**Key Fields:**
* `id_feedback` - Identifier of the feedback.
* `acao` - Action recorded on the feedback (for example sent or requested).
* `data_feedback` - Date the feedback was registered.
* `prazo_esperado` - Date by which the feedback request is expected to be answered.
* `id_usuario_remetente` - Identifier of the sender.
* `matricula_usuario_remetente` - Employee registration number (matrícula) of the sender.
* `usuario_remetente` - Name of the user who sent the record.
* `cargo_usuario_remetente` - Job position of the sender.
* `time_usuario_remetente` - Team of the sender.
* `id_usuario_destinatario` - Identifier of the recipient.
* `matricula_usuario_destinatario` - Employee registration number (matrícula) of the recipient.
* `usuario_destinatario` - Name of the user who received the record.
Praise wall entries, with the badge and tags attached to each praise.
Table: `praises_board`
**Key Fields:**
* `id_elogio` - Identifier of the praise.
* `elogio` - Text of the praise.
* `data` - Date the record refers to.
* `hora` - Time of day the record was created.
* `id_usuario_remetente` - Identifier of the sender.
* `usuario_remetente` - Name of the user who sent the record.
* `badge` - Badge attached to the praise.
* `tags` - Tags linked to the record.
* `id_usuario_elogiado` - Identifier of the praised employee.
* `usuario_elogiado` - Name of the praised employee.
Mood thermometer entries, with the reported mood, its reason and work categories.
Table: `feelings`
**Key Fields:**
* `id_sentimento` - Identifier of the mood entry.
* `id_usuario` - Identifier of the user.
* `usuario` - Name of the user.
* `id_cargo` - Identifier of the job position.
* `cargo` - Name of the job position held by the employee.
* `id_time` - Identifier of the team.
* `time` - Name of the team the employee belongs to.
* `id_gestor` - Identifier of the manager.
* `gestor` - Name of the manager.
* `id_tipo_sentimento` - Identifier of the mood type.
* `tipo_sentimento` - Type of mood reported by the employee.
* `id_motivo` - Identifier of the reason.
## Surveys
Surveys and their participation summary. Their ids drive the answer streams.
Table: `surveys`
**Key Fields:**
* `id_pesquisa` - Identifier of the survey.
* `nome_pesquisa` - Name of the survey.
* `id_tipo_pesquisa` - Identifier of the survey type.
* `tipo_pesquisa` - Type of the survey.
* `id_questionario` - Identifier of the questionnaire.
* `questionario` - Name of the questionnaire used by the survey.
* `data_inicio` - Start date of the record.
* `data_fim` - End date of the record.
* `situacao` - Current state of the record as shown in Elofy.
* `pesquisa_anonima` - Whether answers are collected anonymously.
* `numero_pendencias_enviadas` - Number of survey invitations sent.
* `numero_respondentes` - Number of employees who answered the survey.
Answers to standard surveys and polls, one row per respondent and question. Extracted per survey, in batches of survey ids.
Table: `survey_standard`
**Key Fields:**
* `id_pesquisa` - Identifier of the survey.
* `nome_pesquisa` - Name of the survey.
* `situacao_pesquisa` - Current state of the survey.
* `data_envio_pesquisa` - Date the survey was sent.
* `id_usuario` - Identifier of the user.
* `usuario` - Name of the user.
* `id_cargo` - Identifier of the job position.
* `cargo` - Name of the job position held by the employee.
* `id_time` - Identifier of the team.
* `time` - Name of the team the employee belongs to.
* `id_gestor` - Identifier of the manager.
* `nome_gestor` - Name of the manager.
Answers to pulse surveys, one row per respondent and question. Extracted per survey, in batches of survey ids.
Table: `survey_pulse`
**Key Fields:**
* `id_pesquisa` - Identifier of the survey.
* `nome_pesquisa` - Name of the survey.
* `situacao_pesquisa` - Current state of the survey.
* `data_pulso` - Date the pulse survey round refers to.
* `data_resposta` - Date the answer was submitted.
* `id_usuario` - Identifier of the user.
* `usuario` - Name of the user.
* `id_time` - Identifier of the team.
* `time` - Name of the team the employee belongs to.
* `id_gestor` - Identifier of the manager.
* `gestor` - Name of the manager.
* `id_categoria_pergunta` - Identifier of the question category.
Answers to offboarding surveys, one row per respondent and question. Extracted per survey, in batches of survey ids.
Table: `survey_dismiss`
**Key Fields:**
* `id_pesquisa` - Identifier of the survey.
* `nome_pesquisa` - Name of the survey.
* `id_publico_pesquisa` - Identifier of the survey audience.
* `id_usuario_desligado` - Identifier of the offboarded employee.
* `matricula_usuario_desligado` - Employee registration number (matrícula) of the offboarded employee.
* `usuario_desligado` - Name of the offboarded employee.
* `id_cargo_usuario_desligado` - Identifier of the offboarded employee's job position.
* `cargo_usuario_desligado` - Job position of the offboarded employee.
* `id_time_usuario_desligado` - Identifier of the offboarded employee's team.
* `time_usuario_desligado` - Team of the offboarded employee.
* `id_gestor_desligado` - Identifier of the offboarded employee's manager.
* `matricula_gestor_desligado` - Employee registration number (matrícula) of the offboarded employee's manager.
Answers to temporal surveys (follow-ups), one row per respondent and question. Extracted per survey, in batches of survey ids.
Table: `survey_temporal`
**Key Fields:**
* `id_pesquisa` - Identifier of the survey.
* `nome_pesquisa` - Name of the survey.
* `situacao_pesquisa` - Current state of the survey.
* `data_envio_pesquisa` - Date the survey was sent.
* `id_tipo_respondente` - Identifier of the respondent type.
* `tipo_respondente` - Type of the respondent in the temporal survey.
* `id_usuario_respondente` - Identifier of the respondent.
* `usuario_respondente` - Name of the respondent.
* `id_cargo_usuario_respondente` - Identifier of the respondent's job position.
* `cargo_usuario_respondente` - Job position of the respondent.
* `id_time_usuario_respondente` - Identifier of the respondent's team.
* `time_usuario_respondente` - Team of the respondent.
## Integrations
Per-record result of the HR system integration batches that feed Elofy.
Table: `integration_situation`
**Key Fields:**
* `id_lote` - Identifier of the integration batch.
* `codigomovimento` - Code of the movement processed in the integration batch.
* `codigoempresa` - Company code sent by the source HR system.
* `matricula` - Employee registration number (matrícula) of the user.
* `cpf` - CPF (Brazilian taxpayer registration) of the user.
* `email_usuario` - Email address of the user in the source HR system.
* `login` - Login used by the user to access Elofy.
* `tipousuario` - User type sent by the source HR system.
* `nomecompleto` - Full name of the employee sent by the source HR system.
* `dataadmissao` - Hiring date sent by the source HR system.
* `datadesligamento` - Termination date sent by the source HR system.
* `datanascimento` - Date of birth sent by the source HR system.
HR system integration batches, with their window and processed record count.
Table: `integration_batch`
**Key Fields:**
* `id_lote` - Identifier of the integration batch.
* `codigo_grupo_economico` - Code of the economic group in the source HR system.
* `data_inicio` - Start date of the record.
* `data_fim` - End date of the record.
* `qtd_registros_integrados` - Number of records integrated in the batch.
* `situacao_lote` - Processing state of the integration batch.
# Everest as a data source
Source: https://docs.nekt.com/sources/everest
Bring data from Everest to Nekt.
Everest is a comprehensive ERP (Enterprise Resource Planning) platform designed to manage and integrate a company's financials, supply chain, operations, commerce, reporting, manufacturing, and human resource activities.
## Configuring Everest as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Everest option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Everest `Username`, `Password` and `Entity` for this connection. Once you have it, add the account access and the connector configuration.
The following configurations are available:
* **Username**: Username for Basic Auth authentication.
* **Password**: Password for Basic Auth authentication.
* **Entity**: Entity code used in SIS endpoints.
* **Company Code**: Company code used in SIS endpoints. Default: 1
* **Start Date**: The earliest date from which records will be synced. This should be in YYYY-MM-DDTHH:mm:ssZ format.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Everest and their corresponding fields:
**Primary Keys:** `cd_origemcompra`
**Fields:**
* `cd_origemcompra` - Código da origem de compra.
* `ds_origemcompra` - Descrição da origem de compra.
* `at_situacao` - Situação da origem de compra.
**Primary Keys:** `id_pedidocompra`
**Fields:**
* `id_pedidocompra` - Identificador do pedido de compra.
* `cd_empresa` - Código da empresa.
* `cd_fornecedor` - Código do fornecedor.
* `cd_pedidocompratipo` - Código do tipo de pedido de compra.
* `dh_emissao` - Data e hora de emissão.
* `vl_total_pedido` - Valor total do pedido.
* `observacao` - Observação.
* `cd_comprador` - Código do comprador.
* `at_situacao` - Situação do pedido.
* `itens` - Itens do pedido de compra.
* `id_item` - Identificador do item.
* `qt_item` - Quantidade do item.
* `vl_unitario` - Valor unitário.
* `vl_desconto` - Valor do desconto.
* `vl_total_item` - Valor total do item.
**Primary Keys:** `cd_pedidocompratipo`
**Fields:**
* `cd_pedidocompratipo` - Código do tipo de pedido de compra.
* `ds_pedidocompratipo` - Descrição do tipo de pedido de compra.
* `at_situacao` - Situação do tipo de pedido.
**Primary Keys:** `id_solicitacaocompra`
**Fields:**
* `id_solicitacaocompra` - Identificador da solicitação de compra.
* `cd_empresa` - Código da empresa.
* `dh_emissao` - Data e hora de emissão.
* `cd_solicitante` - Código do solicitante.
* `observacao` - Observação.
* `at_situacao` - Situação da solicitação.
* `itens` - Itens da solicitação de compra.
* `id_item` - Identificador do item.
* `qt_solicitada` - Quantidade solicitada.
* `qt_aprovada` - Quantidade aprovada.
* `vl_unitario` - Valor unitário.
* `vl_total` - Valor total.
**Primary Keys:** `cd_solicitacaocompratipo`
**Fields:**
* `cd_solicitacaocompratipo` - Código do tipo de solicitação de compra.
* `ds_solicitacaocompratipo` - Descrição do tipo de solicitação de compra.
* `at_situacao` - Situação do tipo.
**Primary Keys:** `cd_tabelaprecocompra`
**Parent Stream:** `FornecedorStream`
**Fields:**
* `cd_tabelaprecocompra` - Código da tabela de preço de compra.
* `cd_fornecedor` - Código do fornecedor.
* `ds_tabelaprecocompra` - Descrição da tabela de preço.
* `dt_inicio_vigencia` - Data de início da vigência.
* `dt_fim_vigencia` - Data de fim da vigência.
* `at_situacao` - Situação da tabela.
**Primary Keys:** `cd_tabelaprecocompra`, `cd_item`
**Parent Stream:** `AdcTabelaPrecoCompraFornecedorStream`
**Fields:**
* `cd_tabelaprecocompra` - Código da tabela de preço de compra.
* `cd_item` - Código do item.
* `vl_unitario` - Valor unitário.
* `dh_alteracao` - Data e hora da alteração.
**Primary Keys:** `cd_embalagem`
**Fields:**
* `cd_embalagem` - Código da embalagem.
* `ds_embalagem` - Descrição da embalagem.
* `qt_embalagem` - Quantidade da embalagem.
* `at_situacao` - Situação da embalagem.
**Primary Keys:** `cd_empresa`, `cd_deposito`, `cd_item`
**Parent Stream:** `DepositoStream`
**Fields:**
* `cd_empresa` - Código da empresa.
* `cd_deposito` - Código do depósito.
* `cd_item` - Código do item.
* `qt_estoque` - Quantidade em estoque.
* `qt_estoque_minimo` - Quantidade mínima de estoque.
* `qt_estoque_maximo` - Quantidade máxima de estoque.
* `qt_estoque_reposicao` - Quantidade de reposição de estoque.
* `qt_estoque_seguranca` - Quantidade de segurança de estoque.
* `qt_estoque_condicional` - Quantidade de estoque condicional.
* `qt_estoque_terceiros` - Quantidade de estoque de terceiros.
* `qt_estoque_avariado` - Quantidade de estoque avariado.
* `qt_estoque_empenhado` - Quantidade de estoque empenhado.
* `qt_estoque_disponivel` - Quantidade de estoque disponível.
* `dt_ultimo_inventario` - Data do último inventário.
* `dt_ultima_compra` - Data da última compra.
* `dt_ultima_venda` - Data da última venda.
* `vl_ultima_compra` - Valor da última compra.
* `vl_custo_medio` - Valor do custo médio.
* `dt_custo_medio` - Data do custo médio.
* `vl_custo_reposicao` - Valor do custo de reposição.
* `dt_custo_reposicao` - Data do custo de reposição.
* `localizacao` - Localização no estoque.
**Primary Keys:** `cd_fabricante`
**Fields:**
* `cd_fabricante` - Código do fabricante.
* `ds_fabricante` - Descrição do fabricante.
* `at_situacao` - Situação do fabricante.
**Primary Keys:** `cd_familia`
**Fields:**
* `cd_familia` - Código da família.
* `ds_familia` - Descrição da família.
* `at_situacao` - Situação da família.
**Primary Keys:** `id_fichatecnica`
**Fields:**
* `id_fichatecnica` - Identificador da ficha técnica.
* `ds_fichatecnica` - Descrição da ficha técnica.
* `dt_revisao` - Data da revisão.
* `at_situacao` - Situação da ficha técnica.
**Primary Keys:** `cd_grandegrupo`
**Fields:**
* `cd_grandegrupo` - Código do grande grupo.
* `ds_grandegrupo` - Descrição do grande grupo.
* `at_situacao` - Situação do grande grupo.
**Primary Keys:** `cd_grupo`
**Fields:**
* `cd_grupo` - Código do grupo.
* `ds_grupo` - Descrição do grupo.
* `at_situacao` - Situação do grupo.
**Primary Keys:** `id_inventario`
**Fields:**
* `id_inventario` - Identificador do inventário.
* `cd_empresa` - Código da empresa.
* `cd_deposito` - Código do depósito.
* `dt_inventario` - Data do inventário.
* `at_situacao` - Situação do inventário.
**Primary Keys:** `cd_item`
**Fields:**
* `cd_item` - Código do item.
* `ds_item` - Descrição do item.
* `cd_unidade_medida` - Código da unidade de medida.
* `cd_familia` - Código da família.
* `cd_grupo` - Código do grupo.
* `cd_subgrupo` - Código do subgrupo.
* `cd_grandegrupo` - Código do grande grupo.
* `cd_fabricante` - Código do fabricante.
* `cd_ncm` - Código NCM.
* `cd_itemorigem` - Código de origem do item.
* `cd_itemtipo` - Código do tipo de item.
* `vl_custo_reposicao` - Valor do custo de reposição.
* `at_situacao` - Situação do item.
**Primary Keys:** `id_itemdescricao`
**Fields:**
* `id_itemdescricao` - Identificador da descrição do item.
* `cd_item` - Código do item.
* `ds_item` - Descrição do item.
* `id_itemdescricaotipo` - Identificador do tipo de descrição do item.
**Primary Keys:** `id_itemdescricaotipo`
**Fields:**
* `id_itemdescricaotipo` - Identificador do tipo de descrição do item.
* `ds_itemdescricaotipo` - Descrição do tipo de descrição do item.
* `at_situacao` - Situação do tipo de descrição.
**Primary Keys:** `cd_item`, `cd_empresa`
**Fields:**
* `cd_item` - Código do item.
* `cd_empresa` - Código da empresa.
* `vl_custo_medio` - Valor do custo médio.
* `dt_custo_medio` - Data do custo médio.
* `cd_planocontabil` - Código do plano contábil.
**Primary Keys:** `cd_item`, `cd_fabricante`
**Fields:**
* `cd_item` - Código do item.
* `cd_fabricante` - Código do fabricante.
* `cd_item_fabricante` - Código do item no fabricante.
**Primary Keys:** `cd_item`
**Fields:**
* `cd_item` - Código do item.
* `peso_liquido` - Peso líquido.
* `peso_bruto` - Peso bruto.
* `altura` - Altura.
* `largura` - Largura.
* `comprimento` - Comprimento.
* `qt_volume` - Quantidade de volume.
* `qt_lastro` - Quantidade de lastro.
* `qt_camada` - Quantidade de camada.
**Primary Keys:** `cd_itemorigem`
**Fields:**
* `cd_itemorigem` - Código da origem do item.
* `ds_itemorigem` - Descrição da origem do item.
**Primary Keys:** `cd_itemtipo`
**Fields:**
* `cd_itemtipo` - Código do tipo de item.
* `ds_itemtipo` - Descrição do tipo de item.
* `at_situacao` - Situação do tipo de item.
**Primary Keys:** `id_movimentoestoque`
**Fields:**
* `id_movimentoestoque` - Identificador do movimento de estoque.
* `cd_empresa` - Código da empresa.
* `cd_deposito` - Código do depósito.
* `cd_item` - Código do item.
* `cd_movimentoestoquetipo` - Código do tipo de movimento de estoque.
* `dh_movimento` - Data e hora do movimento.
* `qt_movimento` - Quantidade do movimento.
* `vl_movimento` - Valor do movimento.
* `ds_observacao` - Observação.
**Primary Keys:** `cd_movimentoestoquetipo`
**Fields:**
* `cd_movimentoestoquetipo` - Código do tipo de movimento de estoque.
* `ds_movimentoestoquetipo` - Descrição do tipo de movimento.
* `at_tipo` - Tipo (entrada/saída).
**Primary Keys:** `cd_ncm`
**Fields:**
* `cd_ncm` - Código NCM.
* `ds_ncm` - Descrição NCM.
* `ali_ipi` - Alíquota de IPI.
**Primary Keys:** `cd_ncm`, `cd_cest`
**Fields:**
* `cd_ncm` - Código NCM.
* `cd_cest` - Código CEST.
* `ds_cest` - Descrição CEST.
**Primary Keys:** `cd_cest`
**Fields:**
* `cd_cest` - Código CEST.
* `ds_segmento` - Descrição do segmento.
**Primary Keys:** `id_rmi`
**Fields:**
* `id_rmi` - Identificador da RMI.
* `cd_empresa` - Código da empresa.
* `dh_emissao` - Data e hora de emissão.
* `cd_solicitante` - Código do solicitante.
* `at_situacao` - Situação da RMI.
**Primary Keys:** `id_rmi`, `id_rmiexpedicao`
**Fields:**
* `id_rmi` - Identificador da RMI.
* `id_rmiexpedicao` - Identificador da expedição.
* `dh_expedicao` - Data e hora da expedição.
* `qt_expedida` - Quantidade expedida.
**Primary Keys:** `id_rminaoconformidade`
**Fields:**
* `id_rminaoconformidade` - Identificador da não conformidade.
* `id_rmi` - Identificador da RMI.
* `ds_naoconformidade` - Descrição da não conformidade.
**Primary Keys:** `cd_subgrupo`
**Fields:**
* `cd_subgrupo` - Código do subgrupo.
* `ds_subgrupo` - Descrição do subgrupo.
* `cd_grupo` - Código do grupo.
**Primary Keys:** `cd_unidade_medida`
**Fields:**
* `cd_unidade_medida` - Código da unidade de medida.
* `ds_unidade_medida` - Descrição da unidade de medida.
* `sg_unidade_medida` - Sigla da unidade de medida.
**Primary Keys:** `id_naturezafiscal`
**Fields:**
* `id_naturezafiscal` - Identificador da natureza fiscal.
* `ds_naturezafiscal` - Descrição da natureza fiscal.
* `cd_cfop` - Código CFOP.
**Primary Keys:** `id_pedidovenda`
**Fields:**
* `id_pedidovenda` - Identificador do pedido de venda.
* `cd_empresa` - Código da empresa.
* `cnpj_empresa` - CNPJ da empresa.
* `cd_cadastro` - Código do cadastro do cliente.
* `cnpj_cadastro` - CNPJ do cadastro do cliente.
* `id_naturezafiscal` - Identificador da natureza fiscal.
* `cd_presenca_comprador` - Código de presença do comprador.
* `at_destinacao` - Tipo de destinação.
* `cd_tipo_frete` - Código do tipo de frete.
* `cd_comissionado` - Código do comissionado.
* `cd_condicaocompravenda` - Código da condição de compra/venda.
* `cd_portador` - Código do portador.
* `dh_emissao` - Data e hora de emissão do pedido.
* `dt_entrega` - Data de entrega.
* `dt_previsao_entrega` - Data de previsão de entrega.
* `qt_parcelas` - Quantidade de parcelas.
* `dt_primeiro_vencimento` - Data do primeiro vencimento.
* `qt_dias_vencimento` - Quantidade de dias entre vencimentos.
* `documento` - Documento de referência.
* `pr_desconto` - Percentual de desconto.
* `vl_desconto` - Valor de desconto.
* `vl_frete` - Valor do frete.
* `vl_seguro` - Valor do seguro.
* `id_canalvenda` - Identificador do canal de venda.
* `observacao_prenota` - Observação da pré-nota.
* `observacao_nota` - Observação da nota.
* `cd_transportador` - Código do transportador.
* `cd_deposito` - Código do depósito.
* `cd_tabelaprecovenda` - Código da tabela de preço de venda.
* `cd_indicador_intermediador` - Código do indicador de intermediador.
* `cd_intermediador` - Código do intermediador.
* `cd_meiopagamento` - Código do meio de pagamento.
* `cd_intermediador_financeiro` - Código do intermediador financeiro.
* `id_prenota` - Identificador da pré-nota.
* `itens` - Itens do pedido de venda.
* `id_item` - Identificador do item.
* `id_embalagem` - Identificador da embalagem.
* `qt_embalagem` - Quantidade de itens na embalagem.
* `vl_unitario` - Valor unitário do item.
* `vl_desconto` - Valor de desconto no item.
* `vl_total` - Valor total do item.
* `parcelas` - Parcelas e pagamentos associados ao pedido.
* `cd_meiopagamento` - Código do meio de pagamento da parcela.
* `cd_intermediador_financeiro` - Código do intermediador financeiro da parcela.
* `pagamentos` - Informações de pagamento das parcelas.
* `nr_parcela` - Número da parcela.
* `dt_vencimento` - Data de vencimento da parcela.
* `vl_parcela` - Valor da parcela.
* `vl_pagamento` - Valor pago.
* `dt_pagamento` - Data de pagamento.
**Primary Keys:** `id_prenota`
**Fields:**
* `id_prenota` - Identificador da pré-nota.
* `cd_empresa` - Código da empresa.
* `dh_emissao` - Data e hora de emissão.
* `vl_total` - Valor total.
**Primary Keys:** `cd_presenca_comprador`
**Fields:**
* `cd_presenca_comprador` - Código de presença do comprador.
* `ds_presenca_comprador` - Descrição da presença do comprador.
**Primary Keys:** `cd_tabelaprecovenda`
**Fields:**
* `cd_tabelaprecovenda` - Código da tabela de preço de venda.
* `ds_tabelaprecovenda` - Descrição da tabela.
* `dt_inicio_vigencia` - Data de início da vigência.
**Primary Keys:** `cd_tabelaprecovenda`, `cd_cliente`
**Parent Stream:** `AdvTabelaPrecoVendaStream`
**Fields:**
* `cd_tabelaprecovenda` - Código da tabela de preço de venda.
* `cd_cliente` - Código do cliente.
**Primary Keys:** `cd_tabelaprecovenda`, `cd_item`
**Parent Stream:** `AdvTabelaPrecoVendaStream`
**Fields:**
* `cd_tabelaprecovenda` - Código da tabela de preço de venda.
* `cd_item` - Código do item.
* `vl_unitario` - Valor unitário.
**Primary Keys:** `id_conciliacaobancoarquivo`
**Fields:**
* `id_conciliacaobancoarquivo` - Identificador da conciliação.
* `cd_empresa` - Código da empresa.
* `cd_contabanco` - Código da conta bancária.
* `dt_arquivo` - Data do arquivo.
**Primary Keys:** `id_fornecedortransacaocontacorrente`
**Fields:**
* `id_fornecedortransacaocontacorrente` - Identificador da transação.
* `cd_fornecedor` - Código do fornecedor.
* `vl_transacao` - Valor da transação.
**Primary Keys:** `id_provisao`
**Fields:**
* `id_provisao` - Identificador da provisão.
* `cd_empresa` - Código da empresa.
* `vl_provisao` - Valor da provisão.
**Primary Keys:** `id_titulofornecedor`
**Fields:**
* `id_titulofornecedor` - Identificador do título do fornecedor.
* `cd_empresa` - Código da empresa.
* `cd_fornecedor` - Código do fornecedor.
* `vl_titulo` - Valor do título.
**Primary Keys:** `id_clientetransacaocontacorrente`
**Fields:**
* `id_clientetransacaocontacorrente` - Identificador da transação.
* `cd_cliente` - Código do cliente.
* `vl_transacao` - Valor da transação.
**Primary Keys:** `id_conciliacaofinanceira`
**Fields:**
* `id_conciliacaofinanceira` - Identificador da conciliação.
* `cd_empresa` - Código da empresa.
* `dt_conciliacao` - Data da conciliação.
**Primary Keys:** `id_titulocliente`
**Fields:**
* `id_titulocliente` - Identificador do título do cliente.
* `cd_empresa` - Código da empresa.
* `cd_cliente` - Código do cliente.
* `vl_titulo` - Valor do título.
**Primary Keys:** `cd_banco`
**Fields:**
* `cd_banco` - Código do banco.
* `ds_banco` - Nome do banco.
**Primary Keys:** `cd_banco`, `cd_agencia`
**Fields:**
* `cd_banco` - Código do banco.
* `cd_agencia` - Código da agência.
* `ds_agencia` - Nome da agência.
**Primary Keys:** `cd_bandeira`
**Fields:**
* `cd_bandeira` - Código da bandeira.
* `ds_bandeira` - Descrição da bandeira.
**Primary Keys:** `id_canalvenda`
**Fields:**
* `id_canalvenda` - Identificador do canal de venda.
* `ds_canalvenda` - Descrição do canal de venda.
**Primary Keys:** `cd_cartaocredito`
**Fields:**
* `cd_cartaocredito` - Código do cartão de crédito.
* `ds_cartaocredito` - Descrição do cartão.
**Primary Keys:** `cd_cartaocredito`, `dt_reembolso`
**Parent Stream:** `CartaoCreditoStream`
**Fields:**
* `cd_cartaocredito` - Código do cartão de crédito.
* `dt_reembolso` - Data do reembolso.
* `vl_reembolso` - Valor do reembolso.
**Primary Keys:** `cd_centrocusto`
**Fields:**
* `cd_centrocusto` - Código do centro de custo.
* `ds_centrocusto` - Descrição do centro de custo.
**Primary Keys:** `cd_centrocusto`, `cd_centrocustoconta`
**Parent Stream:** `CentroCustoStream`
**Fields:**
* `cd_centrocusto` - Código do centro de custo.
* `cd_centrocustoconta` - Código da conta do centro de custo.
* `ds_centrocustoconta` - Descrição da conta.
**Primary Keys:** `cd_cfop`
**Fields:**
* `cd_cfop` - Código CFOP.
* `ds_cfop` - Descrição CFOP.
**Primary Keys:** `cd_cliente`
**Fields:**
* `cd_cliente` - Código identificador do cliente.
* `razao` - Razão social do cliente.
* `fantasia` - Nome fantasia do cliente.
* `cpf_cnpj` - CPF ou CNPJ do cliente.
* `inscricao_estadual` - Inscrição estadual do cliente.
* `cep` - CEP do endereço.
* `endereco` - Endereço do cliente.
* `complemento` - Complemento do endereço.
* `bairro` - Bairro do cliente.
* `cd_cidade` - Código da cidade.
* `nr_fone` - Número de telefone.
* `email` - Email do cliente.
* `cd_contabil` - Código contábil associado.
* `inscricao_municipal` - Inscrição municipal do cliente.
* `site` - Site do cliente.
* `nr_endereco` - Número do endereço.
* `nr_fax` - Número de fax.
* `nr_celular` - Número de celular.
* `cd_municipio_ibge` - Código do município IBGE.
* `at_situacao_fiscal` - Situação fiscal do cliente.
* `dt_situacao_fiscal` - Data da situação fiscal.
* `cd_naturezajuridica` - Código da natureza jurídica.
* `id_cadastronota` - Identificador do cadastro de nota.
* `nr_telefone_sefaz` - Número de telefone SEFAZ.
* `fl_sujeito_protesto` - Flag que indica se é sujeito a protesto.
* `fl_consumidor_final` - Flag que indica se é consumidor final.
* `fl_usa_ipi` - Flag que indica se usa IPI.
* `fl_usa_icms` - Flag que indica se usa ICMS.
* `fl_usa_icms_substituicao` - Flag que indica se usa ICMS substituição tributária.
* `cd_condicaovendacompra` - Código da condição de venda/compra.
* `fl_usa_base_reduzida` - Flag que indica se usa base reduzida.
* `at_orgao_publico` - Indicador de órgão público.
* `nr_regime_icms` - Número do regime ICMS.
* `dt_validade_regime` - Data de validade do regime.
* `dt_registro_regime` - Data de registro do regime.
* `fl_simples_federal` - Flag que indica se é Simples Federal.
* `fl_retencao_iss` - Flag que indica se há retenção de ISS.
* `fl_produtor_rural` - Flag que indica se é produtor rural.
* `cd_carimbo` - Código do carimbo.
* `fl_autarquia` - Flag que indica se é autarquia.
* `dt_fundacao` - Data de fundação.
* `at_regime_cliente` - Regime do cliente.
* `fl_calcula_fecp` - Flag que indica se calcula FECP.
* `id_clientetipo` - Identificador do tipo de cliente.
* `cd_portador` - Código do portador.
* `at_contribuinte_icms` - Indicador de contribuinte ICMS.
* `fl_suframa` - Flag que indica se é SUFRAMA.
* `at_situacao` - Situação do cliente.
* `cd_empresa` - Lista de códigos de empresas associadas ao cliente (retornado como string serializada).
* `inscricao_atividade` - Inscrição de atividade do cliente.
* `nit` - Número de Identificação do Trabalhador (NIT).
* `nr_documento_estrangeiro` - Número do documento estrangeiro.
* `tp_documento_estrangeiro` - Tipo do documento estrangeiro.
* `tp_indicativo_comercializacao` - Tipo indicativo de comercialização.
* `cnpj_autarquia` - CNPJ da autarquia.
* `cd_endereco_cobranca` - Código do endereço de cobrança.
**Primary Keys:** `id_clientebloqueio`
**Fields:**
* `id_clientebloqueio` - Identificador do bloqueio.
* `cd_cliente` - Código do cliente.
* `dt_bloqueio` - Data do bloqueio.
* `ds_motivo_bloqueio` - Motivo do bloqueio.
**Primary Keys:** `id_clientecontacorrente`
**Fields:**
* `id_clientecontacorrente` - Identificador da conta corrente.
* `cd_cliente` - Código do cliente.
* `ds_contacorrente` - Descrição da conta corrente.
**Primary Keys:** `id_clienteemail`
**Fields:**
* `id_clienteemail` - Identificador do email.
* `cd_cliente` - Código do cliente.
* `ds_email` - Endereço de email.
**Primary Keys:** `cd_clientesegmento`
**Fields:**
* `cd_clientesegmento` - Código do segmento.
* `ds_clientesegmento` - Descrição do segmento.
**Primary Keys:** `id_clientetipo`
**Fields:**
* `id_clientetipo` - Identificador do tipo de cliente.
* `ds_clientetipo` - Descrição do tipo de cliente.
**Primary Keys:** `cd_comissionado`
**Fields:**
* `cd_comissionado` - Código do comissionado.
* `nm_comissionado` - Nome do comissionado.
**Primary Keys:** `cd_comissionado`, `cd_cliente`
**Fields:**
* `cd_comissionado` - Código do comissionado.
* `cd_cliente` - Código do cliente.
**Primary Keys:** `cd_condicaovendacompra`
**Fields:**
* `cd_condicaovendacompra` - Código da condição.
* `ds_condicaovendacompra` - Descrição da condição.
**Primary Keys:** `id_configuracao`
**Fields:**
* `id_configuracao` - Identificador da configuração.
* `ds_configuracao` - Descrição da configuração.
* `valor` - Valor da configuração.
**Primary Keys:** `id_configuracaormi`
**Fields:**
* `id_configuracaormi` - Identificador da configuração RMI.
* `ds_configuracaormi` - Descrição da configuração RMI.
**Primary Keys:** `cd_contabanco`
**Fields:**
* `cd_contabanco` - Código da conta bancária.
* `ds_contabanco` - Descrição da conta.
* `cd_banco` - Código do banco.
**Primary Keys:** `cd_contacaixa`
**Fields:**
* `cd_contacaixa` - Código da conta caixa.
* `ds_contacaixa` - Descrição da conta caixa.
**Primary Keys:** `cd_empresa`, `cd_deposito`
**Fields:**
* `cd_deposito` - Código identificador do depósito.
* `ds_deposito` - Descrição do depósito.
* `nm_responsavel` - Nome do responsável pelo depósito.
* `at_tipo_deposito` - Tipo do depósito.
* `cd_fornecedor` - Código do fornecedor associado.
* `cep` - CEP do depósito.
* `endereco` - Endereço do depósito.
* `nr_endereco` - Número do endereço.
* `complemento` - Complemento do endereço.
* `bairro` - Bairro do depósito.
* `cd_cidade` - Código da cidade.
* `nr_fone` - Telefone de contato.
* `qt_capacidade_m3` - Capacidade em metros cúbicos.
* `qt_capacidade_tonelada` - Capacidade em toneladas.
* `fl_calcula_custo_medio` - Indicador se calcula custo médio.
* `fl_saldo_planejamento` - Indicador de saldo para planejamento.
* `at_situacao` - Situação do depósito.
* `cd_empresa` - Empresas associadas ao depósito (retornado como string serializada).
**Primary Keys:** `cd_empresa`
**Fields:**
* `cd_empresa` - Código identificador da empresa.
* `razao` - Razão social da empresa.
* `fantasia` - Nome fantasia da empresa.
* `cpf_cnpj` - CPF ou CNPJ da empresa.
* `inscricao_estadual` - Inscrição estadual da empresa.
* `cep` - CEP do endereço da empresa.
* `endereco` - Endereço da empresa.
* `complemento` - Complemento do endereço.
* `bairro` - Bairro do endereço.
* `cd_cidade` - Código da cidade.
* `nr_fone` - Número de telefone.
* `email` - Endereço de e-mail.
* `inscricao_atividade` - Inscrição de atividade econômica.
* `cd_contabil` - Código contábil.
* `inscricao_municipal` - Inscrição municipal.
* `site` - Site da empresa.
* `nit` - Número de Identificação do Trabalhador.
* `nr_endereco` - Número do endereço.
* `nr_fax` - Número de fax.
* `nr_celular` - Número de celular.
* `nr_documento_estrangeiro` - Número de documento estrangeiro.
* `tp_documento_estrangeiro` - Tipo de documento estrangeiro.
* `cd_municipio_ibge` - Código IBGE do município.
* `at_situacao_fiscal` - Situação fiscal.
* `dt_situacao_fiscal` - Data da situação fiscal.
* `cd_naturezajuridica` - Código da natureza jurídica.
* `id_cadastronota` - Identificador de cadastro de nota.
* `tp_indicativo_comercializacao` - Indicativo de comercialização.
* `nr_telefone_sefaz` - Telefone registrado na SEFAZ.
* `registro` - Registro da empresa.
* `dt_registro` - Data do registro.
* `uf_registro` - UF do registro.
* `pg_entrada_eef` - Página de entrada EEF.
* `pg_saida_eef` - Página de saída EEF.
* `pg_ap_icms_eef` - Página AP ICMS EEF.
* `pg_ipi_eef` - Página IPI EEF.
* `pg_iss_eef` - Página ISS EEF.
* `nr_entrada_eef` - Número de entrada EEF.
* `nr_saida_eef` - Número de saída EEF.
* `nr_ap_icms_eef` - Número AP ICMS EEF.
* `nr_ipi_eef` - Número IPI EEF.
* `nr_iss_eef` - Número ISS EEF.
* `cd_empresa_matriz` - Código da empresa matriz.
* `nr_inventario_eef` - Número de inventário EEF.
* `razao_fiscal` - Razão fiscal da empresa.
* `cd_planocontabil` - Código do plano contábil.
* `cd_planogerencial` - Código do plano gerencial.
* `url_web_esupri` - URL do e-Supri.
* `usuario_esupri` - Usuário do e-Supri.
* `senha_esupri` - Senha do e-Supri.
* `cd_carimbo_diretor` - Código do carimbo do diretor.
* `cd_carimbo_contador` - Código do carimbo do contador.
* `at_tipo_local_registro` - Tipo do local de registro.
* `pg_rcpe` - Página RCPE.
* `nr_livro_diario_apagar` - Número do livro diário a pagar.
* `nr_livro_razao_apagar` - Número do livro razão a pagar.
* `nr_livro_diario_receber` - Número do livro diário a receber.
* `nr_livro_razao_receber` - Número do livro razão a receber.
* `nr_livro_diario_esc` - Número do livro diário ESC.
* `nr_livro_razao_esc` - Número do livro razão ESC.
* `nr_reg_producao` - Número de registro de produção.
* `cd_centrocusto` - Código do centro de custo padrão.
* `dt_pre_operacional_inicial` - Data inicial pré-operacional.
* `dt_pre_operacional_final` - Data final pré-operacional.
* `pg_diario_cliente` - Página do diário do cliente.
* `pg_razao_cliente` - Página do razão do cliente.
* `pg_diario_fornecedor` - Página do diário do fornecedor.
* `pg_razao_fornecedor` - Página do razão do fornecedor.
* `pg_diario_esc` - Página do diário ESC.
* `pg_razao_esc` - Página do razão ESC.
* `pg_diario_eef` - Página do diário EEF.
* `pg_razao_eef` - Página do razão EEF.
* `nr_livro_iss_modelo56` - Número do livro ISS modelo 56.
* `pg_livro_iss_modelo56` - Página do livro ISS modelo 56.
* `pg_nota_explicativa_esc` - Página de nota explicativa ESC.
* `cd_vertical` - Código da vertical da empresa.
* `cd_centrocustoconta_rateio` - Conta de rateio do centro de custo.
* `cd_centrocustoconta_pre_ope` - Conta pré-operacional do centro de custo.
* `cd_grupocoligada` - Código do grupo coligado.
* `at_situacao` - Situação da empresa.
* `pg_inventario_eef` - Página do inventário EEF.
* `cd_deposito_entrada` - Código do depósito de entrada.
* `fl_danfe_entrada_plo` - Indicador de DANFE de entrada PLO.
* `cd_atividade_economica` - Código da atividade econômica.
* `cd_tabelaprecovenda_transferencia` - Código da tabela de preço de transferência.
* `id_cmvformula` - Identificador da fórmula CMV.
* `pg_ap_ipi_eef` - Página AP IPI EEF.
* `fl_utilizar_ccusto_entrada_danfe` - Indicador de uso de centro de custo na entrada DANFE.
* `cd_deposito_estoque_virtual` - Código do depósito de estoque virtual.
* `cd_empresa_estoque_virtual` - Código da empresa de estoque virtual.
* `cd_deposito_devolucao` - Código do depósito de devolução.
* `fl_nfse_recebida_plo` - Indicador de NFSe recebida PLO.
* `fl_valida_dias_vencimento_titulo` - Indicador de validação de dias para vencimento.
* `nr_dias_minimo_danfe` - Número mínimo de dias para DANFE.
* `nr_dias_minimo_nota_servico` - Número mínimo de dias para nota de serviço.
* `at_titulo_vencer_danfe` - Indicador de título a vencer para DANFE.
* `at_titulo_vencer_notaservico` - Indicador de título a vencer para nota de serviço.
* `fl_utilizar_localizador` - Indicador de uso de localizador.
**Primary Keys:** `cd_empresa`, `cd_cadastro`
**Fields:**
* `cd_cadastro` - Código do cadastro do fornecedor.
* `razao` - Razão social do fornecedor.
* `fantasia` - Nome fantasia do fornecedor.
* `cpf_cnpj` - CPF ou CNPJ do fornecedor.
* `cep` - CEP do endereço.
* `endereco` - Endereço do fornecedor.
* `complemento` - Complemento do endereço.
* `bairro` - Bairro do fornecedor.
* `cd_cidade` - Código da cidade.
* `cd_contabil` - Código contábil associado.
* `nr_endereco` - Número do endereço.
* `ds_fornecedortipo` - Descrição do tipo de fornecedor.
* `fl_simples_federal` - Indicador de Simples Federal.
* `fl_me_epp_icms` - Indicador ME/EPP para ICMS.
* `fl_substituto_tributario_pis_cofins` - Indicador de substituto tributário PIS/COFINS.
* `fl_substituto_tributario_icms` - Indicador de substituto tributário ICMS.
* `fl_revendedor` - Indicador de revendedor.
* `fl_produtor_rural` - Indicador de produtor rural.
* `id_fornecedortipo` - Identificador do tipo de fornecedor.
* `dt_fundacao_nascimento` - Data de fundação ou nascimento.
* `fl_transportadora` - Indicador de transportadora.
* `cd_antt` - Código ANTT.
* `cd_tributacao` - Código de tributação.
* `tp_indicadorcprbreinf` - Indicador CPRB para REINF.
* `at_contribuinte_icms` - Indicador de contribuinte ICMS.
* `dh_alteracao` - Data e hora da última alteração.
* `tp_indOpcaoContribPrev` - Indicador de opção de contribuição previdenciária.
* `fl_retencao_csrf` - Indicador de retenção CSRF.
* `fl_retencao_pis` - Indicador de retenção PIS.
* `fl_retencao_cofins` - Indicador de retenção COFINS.
* `fl_retencao_csll` - Indicador de retenção CSLL.
* `fl_retencao_inss` - Indicador de retenção INSS.
* `fl_retencao_irrf` - Indicador de retenção IRRF.
* `fl_retencao_iss` - Indicador de retenção ISS.
* `at_situacao` - Situação do fornecedor.
* `nr_fone` - Número de telefone.
* `nr_fax` - Número de fax.
* `nr_celular` - Número de celular.
* `email` - Email do fornecedor.
* `dh_cadastro` - Data e hora do cadastro.
* `cd_fornecedor` - Código do fornecedor.
**Primary Keys:** `id_cadastrobloqueio`
**Fields:**
* `id_cadastrobloqueio` - Identificador do bloqueio.
* `cd_cadastro` - Código do cadastro do fornecedor.
* `dt_bloqueio` - Data do bloqueio.
* `ds_motivo_bloqueio` - Motivo do bloqueio.
**Primary Keys:** `id_cadastrocontacorrente`
**Fields:**
* `id_cadastrocontacorrente` - Identificador da conta corrente.
* `cd_fornecedor` - Código do fornecedor.
* `ds_contacorrente` - Descrição da conta.
**Primary Keys:** `cd_fornecedor`, `cd_item`
**Fields:**
* `cd_fornecedor` - Código do fornecedor.
* `cd_item` - Código do item.
**Primary Keys:** `cd_fornecedor_perfil`
**Parent Stream:** `FornecedorStream`
**Fields:**
* `cd_fornecedor_perfil` - Código do perfil.
* `ds_fornecedor_perfil` - Descrição do perfil.
**Primary Keys:** `cd_fornecedor_perfil`, `cd_plano`
**Parent Stream:** `FornecedorStream`
**Fields:**
* `cd_fornecedor_perfil` - Código do perfil.
* `cd_plano` - Código do plano.
**Primary Keys:** `cd_grupoinventario`
**Fields:**
* `cd_grupoinventario` - Código do grupo.
* `ds_grupoinventario` - Descrição do grupo.
**Primary Keys:** `cd_grupoinventario`, `cd_empresa`
**Parent Stream:** `GrupoInventarioStream`
**Fields:**
* `cd_grupoinventario` - Código do grupo.
* `cd_empresa` - Código da empresa.
**Primary Keys:** `cd_grupoinventario`, `cd_item`
**Parent Stream:** `GrupoInventarioStream`
**Fields:**
* `cd_grupoinventario` - Código do grupo.
* `cd_item` - Código do item.
**Primary Keys:** `cd_gruposolicitacao`
**Fields:**
* `cd_gruposolicitacao` - Código do grupo.
* `ds_gruposolicitacao` - Descrição do grupo.
**Primary Keys:** `cd_gruposolicitacao`, `cd_empresa`
**Parent Stream:** `GrupoSolicitacaoStream`
**Fields:**
* `cd_gruposolicitacao` - Código do grupo.
* `cd_empresa` - Código da empresa.
**Primary Keys:** `cd_gruposolicitacao`, `cd_item`
**Parent Stream:** `GrupoSolicitacaoStream`
**Fields:**
* `cd_gruposolicitacao` - Código do grupo.
* `cd_item` - Código do item.
**Primary Keys:** `cd_meiopagamento`
**Fields:**
* `cd_meiopagamento` - Código do meio de pagamento.
* `ds_meiopagamento` - Descrição do meio de pagamento.
**Primary Keys:** `cd_moeda`
**Fields:**
* `cd_moeda` - Código da moeda.
* `ds_moeda` - Descrição da moeda.
* `sg_moeda` - Símbolo da moeda.
**Primary Keys:** `cd_naturezajuridica`
**Fields:**
* `cd_naturezajuridica` - Código da natureza jurídica.
* `ds_naturezajuridica` - Descrição da natureza jurídica.
**Primary Keys:** `cd_naturezatipocredito`
**Fields:**
* `cd_naturezatipocredito` - Código do tipo de crédito.
* `ds_naturezatipocredito` - Descrição do tipo de crédito.
**Primary Keys:** `cd_operacaofinanceira`
**Fields:**
* `cd_operacaofinanceira` - Código da operação financeira.
* `ds_operacaofinanceira` - Descrição da operação financeira.
**Primary Keys:** `cd_origemnota`
**Fields:**
* `cd_origemnota` - Código da origem da nota.
* `ds_origemnota` - Descrição da origem da nota.
**Primary Keys:** `cd_planogerencial`
**Fields:**
* `cd_planogerencial` - Código do plano gerencial.
* `ds_planogerencial` - Descrição do plano gerencial.
**Primary Keys:** `cd_planogerencial`, `cd_planogerencialconta`
**Parent Stream:** `PlanoGerencialStream`
**Fields:**
* `cd_planogerencial` - Código do plano gerencial.
* `cd_planogerencialconta` - Código da conta.
* `ds_planogerencialconta` - Descrição da conta.
**Primary Keys:** `cd_portador`
**Fields:**
* `cd_portador` - Código do portador.
* `ds_portador` - Descrição do portador.
**Primary Keys:** `cd_ramoatividade`
**Fields:**
* `cd_ramoatividade` - Código do ramo de atividade.
* `ds_ramoatividade` - Descrição do ramo de atividade.
**Primary Keys:** `cd_retencao`
**Fields:**
* `cd_retencao` - Código da retenção.
* `ds_retencao` - Descrição da retenção.
* `pr_retencao` - Percentual de retenção.
**Primary Keys:** `cd_servicocomunicacao`
**Fields:**
* `cd_servicocomunicacao` - Código do serviço de comunicação.
* `ds_servicocomunicacao` - Descrição do serviço.
**Primary Keys:** `cd_simplesnacional`
**Fields:**
* `cd_simplesnacional` - Código do Simples Nacional.
* `ds_simplesnacional` - Descrição do Simples Nacional.
**Primary Keys:** `id_tipodocumento`
**Fields:**
* `id_tipodocumento` - Identificador do tipo de documento.
* `ds_tipodocumento` - Descrição do tipo de documento.
**Primary Keys:** `id_transacaocaixa`
**Fields:**
* `id_transacaocaixa` - Identificador da transação de caixa.
* `vl_lancamento` - Valor do lançamento.
* `dt_lancamento` - Data do lançamento.
**Primary Keys:** `id_transacaocliente`
**Fields:**
* `id_transacaocliente` - Identificador da transação do cliente.
* `cd_cliente` - Código do cliente.
* `vl_lancamento` - Valor do lançamento.
**Primary Keys:** `id_transacaofornecedor`
**Fields:**
* `id_transacaofornecedor` - Identificador da transação do fornecedor.
* `cd_fornecedor` - Código do fornecedor.
* `vl_lancamento` - Valor do lançamento.
**Primary Keys:** `cd_tributacao`
**Fields:**
* `cd_tributacao` - Código da tributação.
* `ds_tributacao` - Descrição da tributação.
**Primary Keys:** `cd_tributo`
**Fields:**
* `cd_tributo` - Código do tributo.
* `ds_tributo` - Descrição do tributo.
* `sg_tributo` - Sigla do tributo.
**Primary Keys:** `cd_tributotipo`
**Fields:**
* `cd_tributotipo` - Código do tipo de tributo.
* `ds_tributotipo` - Descrição do tipo de tributo.
**Primary Keys:** `id_configuracaormi`, `cd_usuario`
**Fields:**
* `id_configuracaormi` - Identificador da configuração RMI.
* `cd_usuario` - Código do usuário.
* `fg_permitir_lancar` - Flag para permitir lançamento.
## Data Model
The following diagram illustrates the relationships between some of the core data streams in Everest.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Empresa("Empresa (sis_empresa)");
Cliente("Cliente (sis_cliente)");
Fornecedor("Fornecedor (sis_fornecedor)");
end
subgraph "Sales"
PedidoVenda("Pedido de Venda (adv_pedido_venda)");
end
Cliente -- "cd_empresa" --> Empresa;
Fornecedor -- "cd_empresa" --> Empresa;
PedidoVenda -- "cd_empresa" --> Empresa;
PedidoVenda -- "cd_cadastro (cd_cliente)" --> Cliente;
```
## Use Cases for Data Analysis
This section will be updated with valuable business intelligence use cases and SQL queries for Everest data.
## Implementation Notes
* Ensure that the user credentials have the necessary permissions to access the desired endpoints in Everest.
* The amount of data can be large, especially for streams with a long history. Use the `start_date` parameter to limit the initial sync period.
* Some streams are dependent on others (parent-child relationships). Make sure to select parent streams if you need to sync child stream data.
## Skills for agents
Everest connector documentation as plain markdown, for use in AI agent contexts.
# Exact Sales as a data source
Source: https://docs.nekt.com/sources/exact-sales
Bring data from Exact Sales (Exact Spotter) to Nekt.
Exact Sales is a Brazilian sales engagement and pre-sales (SDR) platform. Its product, Exact Spotter, structures the pre-sales process with lead qualification scripts, call tracking, and meeting scheduling between SDRs and sales reps. This connector extracts leads, meetings, calls, qualifications, sales, and configuration data from the Exact Spotter API.
The connector supports both generations of the product. Your account belongs to exactly one of them, and its token only works there:
* **Spotter V2 (current)**: the default option, covering the modern Exact Spotter platform.
* **Spotter V1 (API v3, legacy)**: for accounts still on the previous generation.
Each generation exposes its own set of streams, listed below.
## Configuring Exact Sales as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Exact Sales option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Exact Spotter API token for this connection. In Exact Spotter, go to **Settings > Integrations** and copy the **Token Exact API** value. The token is account-level (one per company) and is only visible to manager users.
Generating a new token invalidates the previous one, which breaks any integration still using it. Reuse the existing token whenever possible.
The following configurations are available:
* **Token Exact API**: The company token used to authenticate against the Exact Spotter API.
* **API Generation**: Select **Spotter V2 (current)** (the default) or **Spotter V1 (API v3, legacy)**, matching the generation of your account. If unsure, your Exact Sales account manager can confirm it.
* **Initial sync date**: The earliest date from which records will be synced. Records created or updated after this date will be extracted from the source.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync; you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Streams: Spotter V2 (default)
The table below lists every Spotter V2 stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked incremental sync based on the date field in parentheses.
| Stream | Slug | Description |
| - | - | - |
| Leads | `leads` | Leads with company data and funnel stage. Incremental (`updateDate`). |
| Leads and Persons | `leads_and_persons` | Leads together with their contacts. Incremental (`updateDate`). |
| Meetings | `meetings` | Meeting schedulings and their statuses. Incremental (`registerDate`). |
| Pending Feedbacks | `pending_feedbacks` | Meetings with pending feedback. |
| Boxes | `boxes` | Sales reps' availability slots for scheduling. |
| Meeting Settings | `meeting_settings` | Company meeting scheduling settings (single record). |
| Calls History | `calls_history` | History of calls. Incremental (`startDate`). |
| Calls Results | `calls_results` | Types of call results. |
| Transcriptions | `transcriptions` | Call transcriptions with summaries. Incremental (`createdAt`). |
| Qualification Histories | `qualification_histories` | Filter applications with questions and answers. Incremental (`qualificationDate`). |
| Answers | `answers` | Qualification questionnaires per funnel stage. |
| Scores | `scores` | Score entries received by pre-sales reps. Incremental (`scoreDate`). |
| Qualification Temperatures | `qualification_temperatures` | Qualification temperatures used in evaluations. |
| Leads Sold | `leads_sold` | Sales with values and products. Incremental (`saleDate`). |
| Losts | `losts` | Discarded leads with stage, reason, and date. Incremental (`date`). |
| Transfer History | `transfer_history` | Lead transfers between pre-sales reps. Incremental (`createdAt`). |
| Bonus | `bonus` | Bonuses received by pre-sales reps. Incremental (`bonusDate`). |
| Goals | `goals` | Goals of pre-sales and sales reps per period. |
| Custom Fields | `custom_fields` | Definitions of lead custom fields. |
| Custom Fields Organizations | `custom_fields_organizations` | Definitions of organization custom fields. |
| Leads Custom Fields | `leads_custom_fields` | Custom field values filled in on leads. |
| Organizations Custom Fields | `organizations_custom_fields` | Custom field values filled in on organizations. |
| Organizations | `organizations` | Organizations (companies) registered. Incremental (`updateDate`). |
| Stages | `stages` | Funnel stages configured for the company. |
| Funnels | `funnels` | Sales funnels of the company. |
| Groups | `groups` | User groups with their members. |
| Sources | `sources` | Lead sources and sub-sources. |
| Industries | `industries` | Industries (markets) configured. |
| Cities | `cities` | Locations (city, state, country) in the base. |
| Discard Reasons | `discard_reasons` | Discard reasons configured. |
| Products | `products` | Active products of the company. |
| Recommended Products | `recommended_products` | Products suggested to leads. |
| Pain Points | `pain_points` | Pain points registered for leads. |
| SDRs | `sdrs` | Pre-sales reps registered in the system. |
| Sellers | `sellers` | Sales reps of the company. |
| Tasks | `tasks` | Activities registered. Incremental (`updatedAt`). |
| Tasks Types | `tasks_types` | Activity types available. |
Spotter V2 has no company-wide contacts endpoint, so contacts are extracted through the Leads and Persons stream. The connector automatically respects the Spotter V2 rate limit of 30 requests per 20 seconds.
### Fields by stream (Spotter V2)
Leads registered in Exact Spotter.
* `id` - Unique identifier for the lead
* `lead` - Company name of the lead
* `registerDate` / `updateDate` - Creation and last update timestamps (incremental replication key)
* `stage` - Current funnel stage
* `funnelId` - Funnel the lead belongs to
* `cnpj`, `website`, `phone1`, `phone2`, `telephones` - Company identification and contact data
* `street`, `cep`, `number`, `complement`, `district`, `city`, `state`, `country` - Address
* `leadProduct`, `description`, `mktLink`, `publicLink` - Sales context
* `closingForecastDate` - Forecast closing date of the deal
* `integrationId` - External CRM identifier
* `source`, `subSource`, `industry`, `sdr`, `salesRep` - Related entities (JSON objects)
Leads together with their contacts: the way to extract all contacts on Spotter V2.
* Same lead fields as the Leads stream, plus `ddiPhone1`/`ddiPhone2`
* `persons` - Contacts of the lead (JSON array of objects with id, email, name, jobTitle, phones, messaging platform, and main contact flag)
Meeting schedulings and their statuses.
* `id` - Unique identifier for the meeting record
* `type` - Status: Vigente, Reagendada, Cancelada, Concluído, or NaoOcorreu
* `registerDate` - When the record was created (incremental replication key)
* `meetingType` - Presencial, Online, or Telefone
* `meetingDate`, `startTime`, `finalTime` - Meeting schedule
* `managerDescription`, `sdrDescription`, `reference` - Notes
* `meetingFeedbackUrl`, `score`, `qualificationDate` - Meeting feedback data
* `lead`, `salesRep`, `user` - Related entities (JSON objects)
Meetings with pending feedback.
* `meetingId` - Identifier of the meeting awaiting feedback
* `leadId` / `leadName` - Lead of the meeting
* `stageId` / `stageName` - Current funnel stage of the lead
* `sellerId`, `sellerName`, `sellerLastName` - Sales rep of the meeting
* `date` - Date of the meeting
Sales reps' availability slots (boxes) for meeting scheduling.
* `id` - Unique identifier for the box
* `start` / `end` - Slot window
* `salesRep` / `salesRepEmail` - Sales rep the slot belongs to
* `status` - available or busy
* `typeMeeting` - on site, web, or call
* `description`, `address` - Slot details
* `leadId` - Lead occupying the slot, when busy
Company meeting scheduling settings (single record).
* `meetingBoxInterval` - Display interval of meeting boxes, in minutes
* `businessHoursEnabled` - Whether the company uses business hours
* `companyBusinessHours` - Company business hours per weekday (JSON array)
* `userBusinessHours` - Business hours of each sales rep (JSON array)
History of calls made through Exact Spotter.
* `id` - Unique identifier for the call
* `leadId` / `leadName` - Lead called
* `stage`, `funnelId` - Lead position at the time of the call
* `ddi`, `caller`, `called` - Phone numbers
* `startDate` (incremental replication key) / `endDate` - Call window
* `price`, `talkingDurationSeconds`, `result`, `callAnswered`, `manualSet` - Call outcome
* `description`, `audio` - Notes and recording URL
* `userId`, `userEmail`, `userName`, `userLastName` - User who made the call
Types of call results.
* `id` - Unique identifier for the call result type
* `value` - Name of the call result type
Call transcriptions with summaries and key points.
* `id` - Unique identifier for the transcription
* `leadId` - Lead the transcription belongs to
* `callId` / `conferenceId` - Transcribed call or conference
* `createdAt` - When the transcription was created (incremental replication key)
* `transcription` - Transcription payload (JSON string with summary, timestamped speech segments, and key points)
Filter (qualification) applications with questions and answers.
* `leadId`, `stageId`, `stage`, `funnelId` - Lead and stage of the qualification
* `cycle` - Lead cycle of the qualification
* `score` - Qualification obtained
* `qualificationDate` - When it was registered (incremental replication key)
* `meetingDate` - Related meeting date, when any
* `questionAnswers` - Questions and answers given (JSON array)
Qualification questionnaires per funnel stage.
* `order` - Position of the stage in the funnel
* `stage` - Name of the funnel stage
* `questions` - Questions with possible answers (JSON array)
Score entries received by pre-sales reps.
* `id` - Unique identifier for the score entry
* `leadId` - Lead the score refers to
* `scoreDate` - When the score was received (incremental replication key)
* `score` - Score value
* `stage`, `funnelId`, `inbox` - Context of the score
* `sdr` - Pre-sales rep who received it (JSON object)
Qualification temperatures used in evaluations.
* `id` - Unique identifier for the temperature
* `order` - Ranking of the temperature
* `description` - Description (e.g. Quente, Fria)
Sales with values and products.
* `id` - Unique identifier for the sale
* `leadId` - Sold lead
* `saleDate` - When the sale happened (incremental replication key)
* `saleStage`, `cycle` - Lead position when sold
* `totalDealValue` - Total value of the sale
* `salesRep` - Sales rep who closed (JSON object)
* `products` - Products sold with quantities, discounts, and values (JSON array)
Discarded leads with the discard stage, reason, and date.
* `leadId` - Discarded lead
* `stage`, `funnelId` - Position at the discard
* `reason` - Discard reason
* `date` - When the discard happened (incremental replication key)
Lead transfers between pre-sales reps.
* `leadId` - Transferred lead
* `stageId`, `funnelId` - Position at the transfer
* `originUserId` / `destinationUserId` - Reps involved
* `actionUserId` - User who performed the transfer
* `createdAt` - When it happened (incremental replication key)
Bonuses received by pre-sales reps.
* `id` - Unique identifier for the bonus
* `bonusDate` - When it was received (incremental replication key)
* `bonus` - Amount received
* `metricType` - 4 sale, 5 goal, 6 super-goal
* `saleId` - Related sale, when bonus by sale
* `sdr` - Pre-sales rep (JSON object)
Goals of pre-sales reps and sales reps per period.
* `period` - Day, Month, or Commercial Month
* `start` / `end` - Goal period window
* `listSdrs` / `listSellers` - Goals and achievements per rep: schedules, calls, minutes, sales, score (JSON arrays)
Definitions of the custom fields registered for leads and organizations.
* `id` - Unique identifier for the custom field
* `name` / `key` - Name and slug of the field
* `type` - Texto, TextoLongo, Numero, EscolhaUnica, EscolhaMultipla, or Link
* `required`, `uniqueValue`, `maxLength`, `format` - Validation rules
* `onlyApi` - Whether the field can only be filled through the API (leads only)
* `options` - Available answers for choice fields (JSON array)
Custom field values filled in on leads and organizations.
* `id` - Identifier of the custom field (parent field for choice types)
* `key` - Slug of the custom field
* `value` - Answer for text, long text, and number fields
* `leadId` / `organizationId` - Record the value belongs to
* `registerDate` / `updateDate` - When the value was filled in and last edited
* `type` - Type of the custom field
* `options` - Selected answers for choice fields (JSON array)
Organizations (companies) registered in Exact Spotter.
* `id` - Unique identifier for the organization
* `name` - Organization name
* `registerDate` / `updateDate` - Creation and last update timestamps (incremental replication key)
* `cpfCnpj`, `website`, `publicLink` - Identification
* `street`, `number`, `complement`, `neighborhood`, `zipCode`, `city`, `state`, `country` - Address
* `registrationUserId` - User who registered it
* `phones` - Phone numbers (JSON array)
Funnel stages configured for the company.
* `id` - Unique identifier for the stage
* `value` - Name of the stage
* `position` - Position in the funnel
* `active` - Whether the stage is active
* `funnelId` - Funnel the stage belongs to
* `gateType` - Exit type: 1 Manual, 2 Filter, 3 Scheduling, 4 Sale
Sales funnels of the company.
* `id` - Unique identifier for the funnel
* `value` - Name of the funnel
* `active` - Whether the funnel is active
User groups of the company.
* `id` - Unique identifier for the group
* `value` - Name of the group
* `listSeller` / `listSdr` - Members with email, userId, and createdAt (JSON arrays)
Lead sources and sub-sources configured for the company.
* `id` - Unique identifier for the source
* `value` - Name of the source
* `type` - Type of the source
* `default` - Whether it is the default source for integrations
* `active` - Whether the source is active
* `subSources` - Sub-sources with id, value, and active (JSON)
Industries (markets) configured for the company.
* `id` - Unique identifier for the industry
* `value` - Name of the industry
* `active` - Whether the industry is active
Locations present in the company's base.
* `city` - Name of the city
* `state` - Name of the state
* `country` - Name of the country
Discard reasons configured for the company.
* `id` - Unique identifier for the discard reason
* `value` - Name of the discard reason
* `active` - Whether it is active
* `createdAt` / `updatedAt` - Creation and last edit timestamps
Active products of the company.
* `id` - Unique identifier for the product
* `value` - Price of the product
* `description` - Name of the product
Products recommended (suggested) to leads.
* `leadId` - Lead the product is suggested to
* `productId` - Identifier of the product
* `quantity`, `labelValue`, `discountType`, `discountValue`, `amount` - Quantities and values
Pain points registered for the company's leads.
* `id` - Unique identifier for the pain point
* `leadId` - Lead the pain point belongs to
* `value` - Name of the pain point
Pre-sales reps and sales reps of the company.
* `id` - Unique identifier for the user
* `name` / `lastName` - Name of the user
* `email` - Email of the user
* `ddi`, `phone`, `phone2` - Contact phones
* `active` - Whether the user is active
Activities registered for the company.
* `id` - Unique identifier for the activity
* `leadId` - Lead the activity belongs to
* `title`, `note` - Description and notes
* `dueDate`, `closedDate` - Due and completion dates
* `activityTypeId` - Type of the activity
* `timeNotification` - Minutes before the due date to notify
* `userEmail`, `status` - Assignment and status
* `createdAt` / `updatedAt` - Creation and last update timestamps (incremental replication key)
* `contacts` - Linked contacts (JSON array)
* `sdr` - Pre-sales rep (JSON object)
Activity types available for the company.
* `id` - Unique identifier for the activity type
* `value` - Name of the activity type
* `isCustom` - Whether it was created by the company or is a Spotter default
## Streams: Spotter V1 (API v3, legacy)
| Stream | Slug | Description |
| - | - | - |
| Leads | `leads` | Leads with company data and funnel stage. Incremental (`updateDate`). |
| Meetings | `meetings` | Meeting schedulings, reschedulings, and cancellations. Incremental (`registerDate`). |
| Calls History | `calls_history` | History of calls. Incremental (`startDate`). |
| Qualification Histories | `qualification_histories` | Lead qualification records with scores and answers. Incremental (`qualificationDate`). |
| Losts | `losts` | Discarded leads with stage, reason, and date. Incremental (`date`). |
| Persons | `persons` | Contacts of every lead in the company's base. |
| Pain Points | `pain_points` | Pain points registered for leads. |
| Custom Fields | `custom_fields` | Definitions of the custom fields. |
| Leads Custom Fields | `leads_custom_fields` | Custom field values filled in on leads. |
| Stages | `stages` | Funnel stages with their qualification questions. |
| Calls Results | `calls_results` | Types of call results. |
| Tasks | `tasks` | Activities registered for the company. |
| Tasks Types | `tasks_types` | Activity types available. |
| Boxes | `boxes` | Sales reps' availability slots. |
On Spotter V1, the Qualification Histories stream returns the qualification of every lead in the funnel stages, but only for each lead's current cycle; previous cycles are not returned by the API. The field composition of the V1 streams mirrors the V2 streams of the same name, with small differences documented in the connector.
## Skills for agents
Exact Sales connector documentation as plain markdown, for use in AI agent contexts.
# Ezypay as a data source
Source: https://docs.nekt.com/sources/ezypay
Bring data from Ezypay to Nekt.
Ezypay is a subscription billing and payment platform that helps businesses automate recurring payments and manage customer subscriptions. It provides tools for invoicing, payment collection, and financial reporting to streamline revenue operations.
![]()
## Configuring Ezypay as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Ezypay** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Ezypay credentials to authorize Nekt to access your data. Check the [linked documentation](https://developer.ezypay.com/reference/about-us) if you need help finding any of the credentials.
The following configurations are available:
* **Client ID**: Your Ezypay application Client ID.
* **Client Secret**: Your Ezypay application Client Secret.
* **Merchant ID**: Your unique Ezypay Merchant ID.
* **Username**: The username for your Ezypay account.
* **Password**: The password for your Ezypay account.
* **Start Date**: The date of the first record you want to sync with. We will extract every record created or updated after this date. Format: `YYYY-MM-DD`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. A stream is an entity and you can select the whole entity (meaning all available attributes will come in each table) or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables once the extraction runs successfully.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once to speed up configuration.
* **Sync Type**: You can choose between **INCREMENTAL** and **FULL\_TABLE**.
* **Incremental**: Each run fetches only new or updated data since the last replication.
* **Full table**: Each run gets the current state of the entire dataset.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* **Delta Log Retention**: Determine how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* **Additional Full Sync**: Determine when to execute an [Additional Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
When you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once the run completes successfully, your data will appear in your Catalog.
You need at least one successful source run to see the table in your [Catalog](https://app.nekt.ai/catalog).
# Streams and Fields
Below you'll find the available data streams from Ezypay.
*(Note: Data is automatically formatted so that all field names are converted to snake\_case.)*
Stream containing details about your Ezypay merchants, including details for any supplementary merchants defined in your connection settings.
Stream containing summary records and overviews of your settlements.
Stream containing highly detailed settlement information, dynamically fetched and parsed from Ezypay CSV settlement reports.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `document_id` | String | Unique identifier for the settlement document. |
| `settlement_date` | String | The relevant date associated with the settlement. |
# Implementation Notes
### Incremental Sync and Settlement Reports Lookback
EzyPay generates settlement report files asynchronously. This means a settlement that recently appeared in the API might not have its detailed report ready to download yet, which would ordinarily cause the connector to silently skip those records.
To prevent missing data, the Ezypay connector is designed with a **7-day lookback window** during incremental syncs (specifically for settlement-related endpoints). On every incremental run, the connector will automatically subtract 7 days from your starting timestamp to re-request recent settlements. This ensures that any recent settlements are retried and successfully fetched once EzyPay has finished generating their reports.
* **Data Integrity and Error Handling**: When fetching data for the `Settlement Report Details` stream, any row missing a `document_id` in the underlying CSV report is automatically skipped. This prevents the sync from crashing and ensures data integrity.
## Skills for agents
Ezypay connector documentation as plain markdown, for use in AI agent contexts.
# F360 as a data source
Source: https://docs.nekt.com/sources/f360
Bring data from F360 to Nekt.
F360 is a financial management platform that provides tools for accounting, invoicing, bank reconciliation, cash flow management, and integration with banks and accountants. It offers APIs for extracting data on bank accounts, transactions, invoices, people (clients and suppliers), cost centers, chart of accounts, and more.
![]()
## Configuring F360 as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the F360 option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your F360 API credentials to authorize Nekt to access your financial data.
The following configurations are available:
* **Token Login**: The API token for F360. Obtain this from the F360 platform when creating a public API webservice. The JWT token will be obtained automatically via login.
* **Start Date**: (Required for some streams) The earliest date from which records will be synced (YYYY-MM-DD). Required by `notas`, `parcelas_titulos`, `parcelas_titulos_despesa`, `parcelas_titulos_receita`, `parcelas_titulos_despesa_aberto`, `parcelas_titulos_receita_aberto`, `parcelas_cartoes`, and `extrato_bancario` streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from F360 and their corresponding fields:
Stream for Chart of Accounts (Plano de Contas). Contains the accounting structure used for classifying revenues and expenses.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `PlanoDeContasId` | String | Unique identifier for the chart of accounts entry |
| `Nome` | String | Name of the chart of accounts entry |
| `Tipo` | String | Type of the entry: 'A receber' or 'A pagar' |
Stream for Cost Centers (Centros de Custo). Used for allocating expenses and revenues by department or project.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `CentroDeCustoId` | String | Unique identifier for the cost center |
| `Nome` | String | Cost center name |
Stream for Product Classifications. Maps NCM and CFOP codes to chart of accounts entries.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `NCM` | String | NCM (Nomenclatura Comum do Mercosul) code for product classification |
| `CFOP` | String | CFOP (Código Fiscal de Operações e Prestações) code |
| `PlanoDeContasId` | String | Unique identifier for the chart of accounts entry |
Stream for Bank Accounts. Contains all bank accounts configured in F360. This stream is the parent of Extrato Bancário.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `Id` | String | Unique identifier for the bank account |
| `Nome` | String | Bank account name |
| `TipoDeConta` | String | Account type (e.g., Conta Corrente, Conta Dinheiro) |
| `Carteira` | String | Wallet number |
| `Agencia` | String | Branch/agency number |
| `Conta` | String | Account number |
| `DigitoConta` | String | Account check digit |
| `NumeroBanco` | String | Bank number (e.g., 33 for Santander) |
Stream for Bank Statement (Extrato Bancário). Child of Contas Bancárias: contains transaction-level data for each bank account. Uses the synthetic model (modelo Sintético).
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `Data` | String | Transaction date (YYYY-MM-DD) |
| `Conta` | String | Account name |
| `Valor` | Number | Transaction value |
| `Historico` | String | Transaction history/description |
| `Complemento` | String | Additional details |
| `Documento` | String | Document number |
| `Empresa` | String | Company name |
| `Origem` | String | Data origin (e.g., OFX, Extrato pluggy) |
| `Status` | String | Reconciliation status (Conciliado, etc.) |
| `TipoDeConciliacao` | String | Reconciliation type (e.g., Parcelas de Títulos, Transferência entre Contas) |
| `DetalhesConciliacao` | String | Reconciliation details (when available) |
Stream for People (Clients and Suppliers). Contains contact and company information for customers and vendors.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `Id` | String | Unique identifier for the person |
| `Nome` | String | Name of the person |
| `RazaoSocial` | String | Legal name (for companies) |
| `Telefone` | String | Phone number |
| `Email` | String | Email address |
| `CpfCnpj` | String | CPF or CNPJ document number |
| `Tipo` | String | Type: 'Pessoa física' or 'Pessoa Jurídica' |
| `Definicao` | String | Definition: 'Cliente', 'Fornecedor', or 'Ambos' |
| `Endereco` | Object | Address information (Logradouro, Complemento, Numero, Bairro, Cidade, UF, CEP) |
| `IndicadorIE` | String | Tax registration field |
| `InscricaoEstadual` | String | Tax registration field |
| `InscricaoMunicipal` | String | Tax registration field |
Stream for Invoices (Notas Fiscais). Contains NFe, NFSe, and Nota de Débito documents. Incremental by emission date.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `Id` | String | Unique identifier for the invoice |
| `Registro` | String | Type: NFe, NFSe, or NotaDeDebito |
| `Tipo` | String | Type of operation (e.g., 'A Pagar', 'Emissão') |
| `Numero` | String | Invoice/document number |
| `Emissao` | String | Emission date (ISO 8601) |
| `Valor` | Number | Monetary value of the invoice |
| `Empresa` | Object | Company information (Nome, CpfCnpj) |
| `ClienteFornecedor` | Object | Client or supplier information (Nome, CpfCnpj) |
Stream for Title Installments (Parcelas de Títulos). Contains installments of receivables and payables. Incremental by DataDeAtualizacao.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ParcelaId` | String | Unique identifier for the installment |
| `Registro` | String | Record type (e.g., 'Parcela de título') |
| `Tipo` | String | Type: Despesa, Receita, or Ambos |
| `Numero` | String | Installment number |
| `Vencimento` | String | Due date (YYYY-MM-DD) |
| `Liquidacao` | String | Liquidation date |
| `DataDeAtualizacao` | String | Last update date |
| `ValorBruto` | Number | Gross value |
| `ValorLiquido` | Number | Net value |
| `Conta` | String | Account name |
| `MeioDePagamento` | String | Payment method |
| `Status` | String | Installment status |
| `ConciliadaComExtrato` | Boolean | Whether reconciled with bank statement |
| `DadosDoTitulo` | Object | Parent title data (NumeroDoTitulo, TituloId, Emissao, Empresa, ClienteFornecedor) |
| `Rateio` | Object | Allocation breakdown by cost center and chart of accounts |
Stream for Expense Title Installments (Parcelas de Títulos: Despesa). Contains installments of payables specifically. Incremental by DataDeAtualizacao.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ParcelaId` | String | Unique identifier for the installment |
| `Registro` | String | Record type (e.g., 'Parcela de título') |
| `Tipo` | String | Type: Despesa |
| `Numero` | String | Installment number |
| `Vencimento` | String | Due date (YYYY-MM-DD) |
| `Liquidacao` | String | Liquidation date |
| `DataDeAtualizacao` | String | Last update date |
| `ValorBruto` | Number | Gross value |
| `ValorLiquido` | Number | Net value |
| `Conta` | String | Account name |
| `MeioDePagamento` | String | Payment method |
| `Status` | String | Installment status |
| `ConciliadaComExtrato` | Boolean | Whether reconciled with bank statement |
| `DadosDoTitulo` | Object | Parent title data (NumeroDoTitulo, TituloId, Emissao, Empresa, ClienteFornecedor) |
| `Rateio` | Object | Allocation breakdown by cost center and chart of accounts |
Stream for Revenue Title Installments (Parcelas de Títulos: Receita). Contains installments of receivables specifically. Incremental by DataDeAtualizacao.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ParcelaId` | String | Unique identifier for the installment |
| `Registro` | String | Record type (e.g., 'Parcela de título') |
| `Tipo` | String | Type: Receita |
| `Numero` | String | Installment number |
| `Vencimento` | String | Due date (YYYY-MM-DD) |
| `Liquidacao` | String | Liquidation date |
| `DataDeAtualizacao` | String | Last update date |
| `ValorBruto` | Number | Gross value |
| `ValorLiquido` | Number | Net value |
| `Conta` | String | Account name |
| `MeioDePagamento` | String | Payment method |
| `Status` | String | Installment status |
| `ConciliadaComExtrato` | Boolean | Whether reconciled with bank statement |
| `DadosDoTitulo` | Object | Parent title data (NumeroDoTitulo, TituloId, Emissao, Empresa, ClienteFornecedor) |
| `Rateio` | Object | Allocation breakdown by cost center and chart of accounts |
Stream for Open/Unpaid Expense Title Installments (Parcelas de Títulos: Despesa em Aberto). Contains pending installments of payables specifically, filtered by due date (Vencimento). This stream fetches installments with due dates up to 2 years in the future.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ParcelaId` | String | Unique identifier for the installment |
| `Registro` | String | Record type (e.g., 'Parcela de título') |
| `Tipo` | String | Type: Despesa |
| `Numero` | String | Installment number |
| `Vencimento` | String | Due date (YYYY-MM-DD) |
| `Liquidacao` | String | Liquidation date |
| `DataDeAtualizacao` | String | Last update date |
| `ValorBruto` | Number | Gross value |
| `ValorLiquido` | Number | Net value |
| `Conta` | String | Account name |
| `MeioDePagamento` | String | Payment method |
| `Status` | String | Installment status |
| `ConciliadaComExtrato` | Boolean | Whether reconciled with bank statement |
| `DadosDoTitulo` | Object | Parent title data (NumeroDoTitulo, TituloId, Emissao, Empresa, ClienteFornecedor) |
| `Rateio` | Object | Allocation breakdown by cost center and chart of accounts |
Stream for Open/Unpaid Revenue Title Installments (Parcelas de Títulos: Receita em Aberto). Contains pending installments of receivables specifically, filtered by due date (Vencimento). This stream fetches installments with due dates up to 2 years in the future.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ParcelaId` | String | Unique identifier for the installment |
| `Registro` | String | Record type (e.g., 'Parcela de título') |
| `Tipo` | String | Type: Receita |
| `Numero` | String | Installment number |
| `Vencimento` | String | Due date (YYYY-MM-DD) |
| `Liquidacao` | String | Liquidation date |
| `DataDeAtualizacao` | String | Last update date |
| `ValorBruto` | Number | Gross value |
| `ValorLiquido` | Number | Net value |
| `Conta` | String | Account name |
| `MeioDePagamento` | String | Payment method |
| `Status` | String | Installment status |
| `ConciliadaComExtrato` | Boolean | Whether reconciled with bank statement |
| `DadosDoTitulo` | Object | Parent title data (NumeroDoTitulo, TituloId, Emissao, Empresa, ClienteFornecedor) |
| `Rateio` | Object | Allocation breakdown by cost center and chart of accounts |
Stream for Card Installments (Parcelas de Cartões). Contains credit/debit card transaction installments. Incremental by DataDeAtualizacao.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ParcelaId` | String | Unique identifier for the card installment |
| `Registro` | String | Record type (e.g., 'Parcela de Cartão') |
| `Tipo` | String | Type: Despesa, Receita, or Ambos |
| `Numero` | String | Installment number |
| `Vencimento` | String | Due date (YYYY-MM-DD) |
| `Liquidacao` | String | Liquidation date |
| `DataDeAtualizacao` | String | Last update date |
| `ValorBruto` | Number | Gross value |
| `ValorLiquido` | Number | Net value |
| `Taxa` | Number | Fee rate applied |
| `Conta` | String | Account name |
| `Modalidade` | String | Payment modality (e.g., PIX, credit card) |
| `Status` | String | Installment status |
| `DadosDoCartao` | Object | Card transaction data (CartaoId, DataDaVenda, Adquirente, Bandeira, Empresa) |
| `Rateio` | Object | Allocation breakdown by cost center and chart of accounts |
# Data Model
The following diagram illustrates the relationships between the core data streams in F360. The arrows indicate the parent-child and join relationships.
```mermaid theme={null}
graph TD;
subgraph "Master Data"
PlanosContas("Planos de Contas");
CentrosDeCusto("Centros de Custo");
ClassificacaoProdutos("Classificação de Produtos");
Pessoas("Pessoas");
end
subgraph "Banking"
ContasBancarias("Contas Bancárias");
ExtratoBancario("Extrato Bancário");
end
subgraph "Transactions"
Notas("Notas");
ParcelasTitulos("Parcelas de Títulos");
ParcelasCartoes("Parcelas de Cartões");
end
ExtratoBancario -- "parent" --> ContasBancarias;
ParcelasTitulos -- "Rateio.CentroDeCusto" --> CentrosDeCusto;
ParcelasTitulos -- "Rateio.PlanoDeContas" --> PlanosContas;
ParcelasCartoes -- "Rateio.CentroDeCusto" --> CentrosDeCusto;
ParcelasCartoes -- "Rateio.PlanoDeContas" --> PlanosContas;
Notas -- "ClienteFornecedor" --> Pessoas;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating F360 data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Bank Reconciliation Overview
Track bank account balances and reconcile transactions with titles and card installments.
**Business Value:**
* Identify unreconciled transactions
* Monitor cash flow by account
* Verify bank statement accuracy
```sql theme={null}
SELECT
conta,
data,
COUNT(*) AS transaction_count,
SUM(valor) AS total_value,
SUM(CASE WHEN valor > 0 THEN valor ELSE 0 END) AS credits,
SUM(CASE WHEN valor < 0 THEN valor ELSE 0 END) AS debits
FROM
nekt_raw.f360_extrato_bancario
WHERE
data >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
conta,
data
ORDER BY
conta,
data DESC
```
```sql theme={null}
SELECT
conta,
data,
COUNT(*) AS transaction_count,
SUM(valor) AS total_value,
SUM(CASE WHEN valor > 0 THEN valor ELSE 0 END) AS credits,
SUM(CASE WHEN valor < 0 THEN valor ELSE 0 END) AS debits
FROM
`nekt_raw.f360_extrato_bancario`
WHERE
DATE(data) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
conta,
data
ORDER BY
conta,
data DESC
```
### 2. Revenue and Expense Summary by Type
Analyze financial performance by transaction type (Receita vs Despesa) from titles and card installments.
**Business Value:**
* Understand revenue vs expense breakdown
* Track cash flow by transaction type
* Support budget vs. actual analysis
```sql theme={null}
SELECT
tipo,
LEFT(liquidacao, 7) AS month,
COUNT(*) AS transaction_count,
SUM(valor_liquido) AS total_value
FROM
nekt_raw.f360_parcelas_titulos
WHERE
liquidacao IS NOT NULL
AND liquidacao != ''
AND cancelada = false
GROUP BY
tipo,
LEFT(liquidacao, 7)
ORDER BY
month DESC,
total_value DESC
```
```sql theme={null}
SELECT
tipo,
SUBSTR(liquidacao, 1, 7) AS month,
COUNT(*) AS transaction_count,
SUM(valor_liquido) AS total_value
FROM
`nekt_raw.f360_parcelas_titulos`
WHERE
liquidacao IS NOT NULL
AND liquidacao != ''
AND cancelada = false
GROUP BY
tipo,
SUBSTR(liquidacao, 1, 7)
ORDER BY
month DESC,
total_value DESC
```
## Implementation Notes
### Data Quality Considerations
* The **Extrato Bancário** stream automatically deduplicates records; the API may return the same transaction across pagination boundaries.
* **Start Date** is required for streams that use date windows: `notas`, `parcelas_titulos`, `parcelas_titulos_despesa`, `parcelas_titulos_receita`, `parcelas_titulos_despesa_aberto`, `parcelas_titulos_receita_aberto`, `parcelas_cartoes`, and `extrato_bancario`.
* The open installment streams (**Parcelas de Títulos: Despesa em Aberto** and **Parcelas de Títulos: Receita em Aberto**) fetch data up to 2 years in the future to capture upcoming receivables and payables.
* **Extrato Bancário** is a child stream of **Contas Bancárias**; it fetches transaction data per bank account.
### API Limits & Performance
* For faster extractions, select only the streams necessary for your analysis.
* Streams with incremental sync or date-based fetching (`notas`, `parcelas_titulos`, `parcelas_titulos_despesa`, `parcelas_titulos_receita`, `parcelas_titulos_despesa_aberto`, `parcelas_titulos_receita_aberto`, `parcelas_cartoes`) use `start_date` to filter data.
* The `extrato_bancario` stream uses 30-day date windows for efficient API requests.
## Skills for agents
F360 connector documentation as plain markdown, for use in AI agent contexts.
# Facebook Ads as a data source
Source: https://docs.nekt.com/sources/facebook-ads
Bring data from Facebook Ads to Nekt.
Facebook Ads (also known as Meta Ads) is Meta's advertising platform that allows businesses to create and manage digital advertising campaigns across Facebook, Instagram, and other Meta-owned platforms. It provides tools for targeting audiences, tracking performance, and optimizing ad spend for maximum return on investment.
![]()
## Configuring Facebook Ads as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Facebook Ads option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Facebook Ads data. Click on the `Facebook Authorization` button and log in with your Facebook account. Grant the necessary permissions for the ad accounts you want to extract data from. After authentication, select the specific Ad Account for this source and define a start date for data retrieval. Facebook's API allows fetching reports up to 36 months in the past.
The following configurations are available:
* **Start Date**: The earliest date from which records will be synced. This should be in DD-MM-YYYY format.
* **Lookback Window**: (Default: 28 days) Facebook "freezes" insight data 28 days after it's generated. This means data from the past 28 days can change. Nekt uses a lookback window to re-fetch data within this period to ensure your data is up-to-date.
* **Enable Advanced Reports**: (Default: `false`) When enabled, this option unlocks additional, more granular `AdInsights` reports, broken down by dimensions like age and gender, country, device, and region. Be aware that enabling this can significantly increase extraction times due to the volume of data being processed.
* **Creative Fields Mode**: (Default: `basic`) This setting controls the level of detail extracted for ad creatives.
* `basic`: Extracts essential fields for faster processing and lower API usage.
* `advanced`: Includes all media and configuration details, which requires more processing time by Facebook's API.
* **Ad Accounts Fields Mode**: (Default: `basic`) Controls which fields to extract from the Ad Accounts stream.
* `basic`: Core account fields that work with standard permissions. Use this if you experience permission errors.
* `extended`: Includes additional fields like owner info, tax details, agency declarations, business manager info, and funding source details. Requires elevated (admin/finance) permissions on all connected ad accounts.
You can also customize the behavior of the `AdInsights` stream using the advanced settings:
* **Attribution Windows**: Define the attribution windows for view and click conversions (e.g., `1d_view`, `7d_click`). This determines how conversions are credited to your ads based on when they were seen or clicked.
* **Action Report Time**: Determines how the time of an action is recorded. It can be based on `impression` time, `conversion` time, or `mixed`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked *optional* are only discovered when the corresponding setting is enabled.
| Stream | Slug | Description |
| - | - | - |
| Ad Accounts | `adaccounts` | Ad account information (settings, spending limits, business info, funding). |
| Campaigns | `campaigns` | Ad campaigns and their settings. |
| Ad Sets | `adsets` | Ad sets (targeting, budget, schedule) within campaigns. |
| Ads | `ads` | Individual ads and their configuration. |
| Ad Creatives | `creatives` | Creative assets (copy, images, links) used by ads. |
| Ad Labels | `adlabels` | Labels used to organize ads and campaigns. |
| Ad Images | `adimages` | Image assets in the ad account. |
| Ad Videos | `advideos` | Video assets in the ad account. |
| Custom Audiences | `customaudiences` | Custom audiences defined in the account. |
| Custom Conversions | `customconversions` | Custom conversion definitions. |
| Activities | `activities` | Change / activity log for the ad account. |
| Ads Insights | `adsinsights` | Performance metrics (impressions, clicks, spend, conversions) at the ad level. |
| Campaign Insights | `campaign_insights` | *Optional.* Insights aggregated at the campaign level (requires the campaign insights setting). |
| Ads Insights by Age & Gender | `adsinsights_by_age_and_gender` | *Optional (advanced reports).* Ads Insights broken down by age and gender. |
| Ads Insights by Country | `adsinsights_by_country` | *Optional (advanced reports).* Ads Insights broken down by country. |
| Ads Insights by Device Platform | `adsinsights_by_device_platform` | *Optional (advanced reports).* Ads Insights broken down by device platform. |
| Ads Insights by Region | `adsinsights_by_region` | *Optional (advanced reports).* Ads Insights broken down by region. |
| Ads Insights Hourly (Advertiser Timezone) | `adsinsights_hourly_advertiser_timezone` | *Optional (advanced reports).* Ads Insights segmented by hour in the advertiser's timezone. |
## Fields by stream
Below you'll find all available data streams from Facebook Ads and their corresponding fields:
Stream containing information about your Facebook Ad Accounts, including account settings, spending limits, business information, and funding details.
> **Field Modes**: You can configure the extraction level using the **Ad Accounts Fields Mode** setting:
>
> * `Basic` (default): Core account fields that work with standard permissions
> * `Extended`: Includes owner, tax, agency, business manager, and funding source details (requires admin/finance permissions)
**Basic Fields (always included):**
*Account Identifiers:*
* `id` - Unique identifier for the ad account
* `account_id` - The ad account ID (primary key)
* `name` - Name of the ad account
*Account Status & Settings:*
* `account_status` - Status of the account (1=Active, 2=Disabled, etc.)
* `age` - Age of the account in days
* `created_time` - When the account was created
* `currency` - Currency used for billing
* `disable_reason` - Reason code if account is disabled
* `is_personal` - Whether this is a personal account
* `is_prepay_account` - Whether this is a prepay account
*Spending & Budget:*
* `amount_spent` - Total amount spent on the account (in cents)
* `balance` - Current account balance (in cents)
* `spend_cap` - Maximum spending limit for the account
* `min_daily_budget` - Minimum daily budget allowed
* `min_campaign_group_spend_cap` - Minimum campaign group spend cap
*Business Information:*
* `business_name` - Name of the business
* `business_street`, `business_street2` - Business address
* `business_city`, `business_state`, `business_zip` - Business location
* `business_country_code` - Country code for the business
*Timezone:*
* `timezone_id` - Timezone identifier
* `timezone_name` - Name of the timezone
* `timezone_offset_hours_utc` - UTC offset in hours
*Capabilities & Features:*
* `capabilities` - Array of enabled capabilities for the account
* `can_create_brand_lift_study` - Whether brand lift studies are available
* `is_direct_deals_enabled` - Whether direct deals are enabled
* `is_notifications_enabled` - Whether notifications are enabled
* `offsite_pixels_tos_accepted` - Whether offsite pixels ToS was accepted
**Extended Fields (only in extended mode):**
*Owner & Tax Information:*
* `owner` - Owner of the ad account
* `tax_id` - Tax identification number
* `tax_id_status` - Status of tax ID verification
* `tax_id_type` - Type of tax ID
*Agency Client Declaration:*
* `agency_client_declaration_*` - Fields related to agency-client relationships, including client name, location, and mandate information
*Business Manager:*
* `business_manager_name` - Name of the Business Manager
* `business_manager_id` - Business Manager ID
* `business_manager_created_time` - When Business Manager was created
* `business_manager_verification_status` - Verification status
* `business_manager_vertical` - Business vertical category
*Funding & Invoicing:*
* `funding_source_details` - Object containing funding source information:
* `id` - Funding source ID
* `type` - Type of funding source
* `display_string` - Display name
* `coupons[]` - Array of coupon objects with amount, currency, and expiration
* `extended_credit_invoice_group_*` - Fields related to invoice groups
* `io_number` - Insertion order number
* `media_agency` - Media agency information
* `partner` - Partner information
Stream for retrieving ad account activities, including billing events such as charges, refunds, and failed transactions.
> **Note**: This stream retrieves activities from the **last 7 days** only. It provides a historical record of account-level events, with a focus on billing-related activities.
**Key Fields:**
* `event_time` - The time when the event occurred (ISO 8601 format), used for incremental sync
* `event_type` - The type of event (see billing events below)
* `actor_id` - ID of the user/entity who triggered the activity
* `actor_name` - Name of the user/entity who triggered the activity
* `extra_data` - Additional data about the event in JSON format (e.g., transaction details)
**Computed Billing Fields:**
* `charge_amount` - Extracted charge amount from extra\_data (if available and event is billing-related)
* `currency` - Currency code extracted from extra\_data (if available)
* `is_billing_event` - Boolean flag indicating if this is a billing-related event
**Billing Event Types:**
* `ad_account_billing_charge` - Charges made to credit card
* `ad_account_billing_charge_failed` - Failed billing attempts
* `ad_account_billing_refund` - Refunds processed
Other event types include account changes, user access modifications, and campaign/ad set/ad updates.
Stream for managing ad campaigns and their settings.
**Key Fields:**
* `id` - Unique identifier for the campaign
* `name` - Name of the campaign
* `objective` - The objective of the campaign (e.g., brand awareness, conversions)
* `status` - The status of the campaign (e.g., active, paused)
* `daily_budget` - The daily budget for the campaign
* `lifetime_budget` - The lifetime budget for the campaign
* `start_time` - The start time of the campaign
* `stop_time` - The stop time of the campaign
* `created_time` - When the campaign was created
* `updated_time` - When the campaign was last updated
**Budget & Spending:**
* `budget_remaining` - The remaining budget for the campaign
* `spend_cap` - Maximum amount that can be spent on the campaign
* `budget_rebalance_flag` - Whether budget rebalancing is enabled
* `last_budget_toggling_time` - Last time the budget was modified
* `bid_strategy` - The bidding strategy used for the campaign
**Campaign Settings:**
* `account_id` - The ID of the ad account
* `buying_type` - The buying type of the campaign (e.g., auction, fixed price)
* `configured_status` - The configured status of the campaign
* `effective_status` - The effective status of the campaign
* `pacing_type` - Array of pacing types for the campaign
* `primary_attribution` - Primary attribution setting
* `source_campaign_id` - ID of the source campaign if this is a copy
* `boosted_object_id` - ID of the boosted object
* `topline_id` - ID of the topline
**Special Ad Settings:**
* `special_ad_category` - Category for special ads (e.g., housing, employment)
* `special_ad_categories` - Array of special ad categories
* `special_ad_category_country` - Array of countries for special ad targeting
**Advanced Features:**
* `can_create_brand_lift_study` - Whether brand lift studies are available
* `can_use_spend_cap` - Whether spend caps can be used
* `has_secondary_skadnetwork_reporting` - Whether secondary SKAdNetwork reporting is enabled
* `is_skadnetwork_attribution` - Whether SKAdNetwork attribution is enabled
* `smart_promotion_type` - Type of smart promotion
* `ad_strategy_group_id` - ID of the ad strategy group
* `ad_strategy_id` - ID of the ad strategy
**Labels:**
* `adlabels` - Array of labels attached to the campaign, each containing:
* `id` - Label ID
* `name` - Label name
* `created_time` - When the label was created
* `updated_time` - When the label was last updated
Stream for managing ad sets, which control targeting, budget, and scheduling for a group of ads.
**Key Fields:**
* `id` - Unique identifier for the ad set
* `name` - Name of the ad set
* `campaign_id` - The ID of the campaign this ad set belongs to
* `status` - The status of the ad set
* `daily_budget` - The daily budget for the ad set
* `lifetime_budget` - The lifetime budget for the ad set
* `start_time` - The start time of the ad set
* `end_time` - The end time of the ad set
* `created_time` - When the ad set was created
* `updated_time` - When the ad set was last updated
**Budget & Bidding:**
* `account_id` - The ID of the ad account
* `bid_strategy` - The bid strategy for the ad set
* `bid_amount` - The bid amount
* `bid_info` - Detailed bid information including:
* `CLICKS` - Bid for clicks
* `ACTIONS` - Bid for actions
* `REACH` - Bid for reach
* `IMPRESSIONS` - Bid for impressions
* `SOCIAL` - Bid for social impressions
* `budget_remaining` - Remaining budget
* `daily_min_spend_target` - Minimum daily spend target
* `lifetime_min_spend_target` - Minimum lifetime spend target
* `lifetime_spend_cap` - Maximum lifetime spend
**Optimization & Delivery:**
* `billing_event` - The billing event type
* `optimization_goal` - The optimization goal
* `optimization_sub_event` - Sub-event for optimization
* `pacing_type` - Array of pacing types
* `destination_type` - Type of destination
* `is_dynamic_creative` - Whether dynamic creative is enabled
* `source_adset_id` - ID of source ad set if this is a copy
**Targeting:**
* `targeting` - Complex targeting object containing:
* `age_max` - Maximum age
* `age_min` - Minimum age
* `genders` - Array of targeted genders
* `geo_locations` - Geographic targeting settings
* `interests` - Targeted interests
* `behaviors` - Targeted behaviors
* `custom_audiences` - Custom audience targeting
* `excluded_custom_audiences` - Excluded custom audiences
* `device_platforms` - Targeted devices
* `publisher_platforms` - Targeted publishing platforms
* `facebook_positions` - Ad positions on Facebook
* `instagram_positions` - Ad positions on Instagram
* `excluded_publisher_categories` - Excluded publisher categories
* And many more targeting options...
**Performance & Learning:**
* `learning_stage_info` - Information about the learning stage:
* `status` - Current learning status
* `conversions` - Number of conversions
* `last_sig_edit_ts` - Last significant edit timestamp
* `attribution_windows` - Array of attribution windows
**Labels & Metadata:**
* `adlabels` - Array of labels attached to the ad set
* `attribution_spec` - Attribution specifications
* `review_feedback` - Review feedback
* `rf_prediction_id` - RF prediction ID
Stream for managing individual ads within an ad set.
**Key Fields:**
* `id` - Unique identifier for the ad
* `name` - Name of the ad
* `adset_id` - The ID of the ad set this ad belongs to
* `campaign_id` - The ID of the campaign this ad belongs to
* `status` - The status of the ad
* `created_time` - When the ad was created
* `updated_time` - When the ad was last updated
**Creative & Content:**
* `creative` - The creative object containing:
* `id` - Creative ID
* `creative_id` - Alternative creative ID reference
* `tracking_specs` - Array of tracking specifications:
* `application` - Application tracking
* `post` - Post tracking
* `conversion_id` - Conversion tracking
* `action_type` - Types of actions to track
* `fb_pixel` - Facebook pixel tracking
* And many more tracking options...
* `conversion_specs` - Array of conversion specifications
* `conversion_domain` - Domain for conversions
**Bidding & Budget:**
* `bid_type` - Type of bidding
* `bid_amount` - Bid amount
* `bid_info` - Detailed bidding information
**Status & Configuration:**
* `account_id` - The ID of the ad account
* `effective_status` - The effective status of the ad
* `configured_status` - The configured status
* `last_updated_by_app_id` - ID of the app that last updated the ad
* `source_ad_id` - ID of the source ad if this is a copy
**Additional Fields:**
* `recommendations` - Array of recommendations containing:
* `blame_field` - Field causing the issue
* `code` - Recommendation code
* `confidence` - Confidence level
* `importance` - Importance level
* `message` - Recommendation message
* `title` - Recommendation title
* `adlabels` - Array of labels attached to the ad
Stream for managing ad creatives, which are the visual components of an ad. This stream extracts creative data from the ads stream context, providing incremental sync based on ad updates.
> **Field Modes**: You can configure the extraction level using:
>
> * `Basic` (default): Essential creative fields for faster processing
> * `Advanced`: Includes all media and configuration details
**Basic Fields (always included):**
* `id` - Unique identifier for the creative
* `account_id` - The ID of the ad account
* `name` - Name of the creative
* `title` - The title of the creative
* `body` - The body text of the creative
* `status` - Status of the creative
* `actor_id` - The ID of the actor (e.g., page) associated with the creative
* `authorization_category` - Category for authorization
* `call_to_action_type` - Type of call to action button
* `enable_direct_install` - Whether direct install is enabled
* `link_url` - The URL the ad links to
* `object_id` - ID of the object
* `object_type` - Type of object
* `object_url` - URL of the object
* `use_page_actor_override` - Whether to override the page actor
**Advanced Fields (included only in advanced mode):**
*Media & Assets:*
* `image_hash` - Hash of the image used
* `image_url` - The URL of the image used in the creative
* `video_id` - The ID of the video used in the creative
* `thumbnail_id` - ID of the thumbnail
* `thumbnail_url` - URL of the thumbnail
* `playable_asset_id` - ID of the playable asset
*Links & URLs:*
* `link_destination_display_url` - Display URL for the link
* `link_og_id` - Open Graph ID for the link
* `object_store_url` - Store URL for the object
* `template_url` - Template URL
* `url_tags` - URL parameters for tracking
*Instagram Integration:*
* `instagram_user_id` - ID of the Instagram user
* `instagram_permalink_url` - Permalink URL for Instagram
* `instagram_story_id` - ID of the Instagram story
* `effective_instagram_story_id` - Effective ID of the Instagram story
* `effective_instagram_media_id` - Effective ID of the Instagram media
* `source_instagram_media_id` - Source Instagram media ID
*Advanced Configuration:*
* `applink_treatment` - Treatment for app links
* `branded_content_sponsor_page_id` - ID of the sponsor page for branded content
* `bundle_folder_id` - ID of the bundle folder
* `categorization_criteria` - Criteria for categorization
* `category_media_source` - Source of category media
* `degrees_of_freedom_spec` - Degrees of freedom specification
* `destination_set_id` - ID of the destination set
* `dynamic_ad_voice` - Voice setting for dynamic ads
* `effective_authorization_category` - Effective authorization category
* `effective_object_story_id` - Effective ID of the object story
* `object_story_id` - ID of the object story
* `place_page_set_id` - ID of the place page set
**Ad Context Fields (always included):**
* `ad_id` - ID of the parent ad
* `ad_updated_time` - Last update time of the parent ad (used for incremental sync)
Stream for ad labels used to organize and filter campaigns, ad sets, and ads. Supports incremental sync by `updated_time`.
**Key Fields:**
* `id` - Unique ID of the ad label (primary key)
* `account` - Ad account this label belongs to (object with account\_id, id)
* `created_time` - When the label was created
* `updated_time` - When the label was last updated (replication key)
* `name` - Name of the ad label
Stream for custom conversions defined at the ad account level. Custom conversions let you track specific URL or event-based actions. Supports incremental sync by `creation_time`.
**Key Fields:**
* `id` - Unique ID of the custom conversion (primary key)
* `account_id` - ID of the ad account that owns the custom conversion
* `name` - Name of the custom conversion
* `creation_time` - When the custom conversion was created (replication key)
* `business` - Business (Business Manager) that owns this custom conversion (object with id, name, etc.)
* `is_archived` - Whether the custom conversion is archived
* `is_unavailable` - Whether the custom conversion is unavailable
* `last_fired_time` - When the custom conversion last fired
Stream for custom audiences (customer lists, website visitors, engagement audiences, lookalikes, etc.) associated with the ad account.
**Key Fields:**
* `id` - Unique ID of the custom audience (primary key)
* `account_id` - ID of the ad account that owns the custom audience
* `name` - Name of the custom audience
* `approximate_count_lower_bound`, `approximate_count_upper_bound` - Approximate audience size range
* `time_created`, `time_updated` - Unix timestamps for creation and last update
* `customer_file_source` - Source of the customer file (e.g. USER\_PROVIDED\_ONLY, PARTNER\_PROVIDED\_ONLY)
* `data_source` - Data source type and sub-type (object)
* `delivery_status` - Delivery status code and description (object)
* `description` - Description of the custom audience
* `lookalike_spec` - Specification for lookalike audience (country, ratio, origin) if applicable
* `is_value_based` - Whether this is a value-based custom audience
* `operation_status` - Operation status code and description
* `pixel_id` - Facebook Pixel ID associated with the audience
* `retention_days` - Number of days to retain the audience
* `subtype` - Subtype of the custom audience
* `rule_aggregation` - How rules are aggregated (AND or OR)
* `opt_out_link` - Opt-out link for the audience
Stream for images uploaded to the ad account that can be used in ad creatives. Supports incremental sync by `id`.
**Key Fields:**
* `id` - The ID of the image (primary key, replication key)
* `account_id` - The ad account that owns the image
* `created_time`, `updated_time` - When the image was created and last updated
* `creatives` - List of ad creative IDs that use this image (array)
* `hash` - Hash which uniquely identifies the image
* `height`, `width` - Dimensions of the image in pixels
* `original_height`, `original_width` - Dimensions as originally uploaded
* `name` - The filename of the image (max 100 characters)
* `permalink_url` - Permanent URL of the image for use in story creatives
* `status` - Status of the image (ACTIVE, INTERNAL, DELETED)
* `url` - Temporary URL to retrieve the image
* `url_128` - Temporary URL for version resized to fit within 128x128 pixels
* `is_associated_creatives_in_adgroups` - Whether this image is associated with creatives in ad groups
Stream for videos uploaded to the ad account that can be used in ad creatives. Supports incremental sync by `id`.
**Key Fields:**
* `id` - The ID of the video (primary key, replication key)
* `account_id` - The ad account that owns the video
* `created_time`, `updated_time` - When the video was created and last updated
* `title` - Title of the video
* `description` - Description of the video
* `length` - Length of the video in seconds
* `source` - Source URL of the video file
* `permalink_url` - Permanent URL to the video
* `status_value` - Processing status of the video
* `status_processing_progress` - Video processing progress percentage
* `views` - Number of views
* `published` - Whether the video is published
* `format` - Available formats (dimensions, thumbnail) for the video (array of objects)
* `embeddable` - Whether the video can be embedded
* `content_category` - Content category of the video
* `is_crosspost_video`, `is_crossposting_eligible` - Crossposting flags
* `is_instagram_eligible` - Whether the video is eligible for Instagram
* `from_object` - Object (page, user) that owns or uploaded the video
* `post_views` - Number of post views
* `scheduled_publish_time` - Scheduled publish time for the video
* `live_status` - Live status if the video is a live broadcast
* `universal_video_id` - Universal video ID across Facebook properties
Stream for retrieving performance data for ads, ad sets, and campaigns.
> **How it works**: The Ad Insights stream uses Facebook's asynchronous API. This means that when an extraction runs, Nekt requests a report from Facebook, which gets queued and processed. Once the report is ready, Nekt downloads the data. This process is highly efficient for large data volumes but can introduce latency depending on Facebook's processing times.
**Identifiers & Time:**
* `ad_id` - The ID of the ad
* `adset_id` - The ID of the ad set
* `campaign_id` - The ID of the campaign
* `account_id` - The ID of the ad account
* `date_start` - The start date of the data
* `date_stop` - The stop date of the data
**Basic Metrics:**
* `impressions` - Number of times the ad was shown
* `reach` - Number of unique people who saw the ad
* `frequency` - Average number of times each person saw the ad
* `clicks` - Number of clicks on the ad
* `unique_clicks` - Number of unique clicks
* `ctr` - Click-through rate
**Cost Metrics:**
* `spend` - Amount spent on the ad
* `cpc` - Cost per click
* `cpm` - Cost per thousand impressions
* `cpp` - Cost per 1,000 people reached
* `cost_per_unique_click` - Cost per unique click
* `cost_per_action_type` - Cost per action type
**Conversion Metrics:**
* `conversions` - Number of conversions
* `cost_per_conversion` - Cost per conversion
* `conversion_rate_ranking` - Ranking of conversion rate
* `conversion_values` - Values of conversions
* `cost_per_conversion` - Cost per conversion
* `website_purchase_roas` - Return on ad spend for website purchases
* `purchase_roas` - Overall return on ad spend
**Engagement Metrics:**
* `social_spend` - Spend on social impressions
* `social_impressions` - Number of social impressions
* `actions` - Detailed breakdown of different types of actions
* `video_p25_watched_actions` - Video views at 25%
* `video_p50_watched_actions` - Video views at 50%
* `video_p75_watched_actions` - Video views at 75%
* `video_p95_watched_actions` - Video views at 95%
* `video_p100_watched_actions` - Complete video views
* `video_avg_time_watched_actions` - Average video watch time
* `video_play_actions` - Number of video plays
**Quality & Relevance:**
* `quality_ranking` - Quality ranking of the ad
* `engagement_rate_ranking` - Ranking of engagement rate
* `quality_score_organic` - Organic quality score
* `quality_score_ectr` - Expected click-through rate score
* `quality_score_ecvr` - Expected conversion rate score
#### Advanced Insight Reports (Breakdowns)
When you enable the "Advanced Reports" option in the source configuration, a set of additional streams become available. These streams provide the `AdInsights` data broken down by different dimensions, allowing for more granular analysis.
* **`adsinsights_by_age_and_gender`**: Breaks down performance data by age group and gender.
* **`adsinsights_by_country`**: Segments data by country.
* **`adsinsights_by_device_platform`**: Provides insights into performance across different devices (e.g., mobile, desktop), publisher platforms (e.g., Facebook, Instagram), and placements (e.g., feed, stories).
* **`adsinsights_by_region`**: Breaks down data by geographical region (e.g., state, province).
* **`adsinsights_hourly_advertiser_timezone`**: Aggregates data on an hourly basis, based on the ad account's timezone.
# Data Model
The following diagram illustrates the relationships between the core data streams in Facebook Ads. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Account Level"
AdAccounts("Ad Accounts");
end
subgraph "Core Entities"
Campaigns("Campaigns");
AdSets("Ad Sets");
Ads("Ads");
Creatives("Creatives");
end
subgraph "Account-Level Streams"
AdLabels("Ad Labels");
CustomConversions("Custom Conversions");
CustomAudiences("Custom Audiences");
AdImages("Ad Images");
AdVideos("Ad Videos");
end
subgraph "Activity & Performance"
Activities("Activities");
AdInsights("Ad Insights");
end
Campaigns -- "account_id" --> AdAccounts;
AdSets -- "account_id" --> AdAccounts;
AdSets -- "campaign_id" --> Campaigns;
Ads -- "account_id" --> AdAccounts;
Ads -- "adset_id" --> AdSets;
Ads -- "campaign_id" --> Campaigns;
Ads -- "creative.id" --> Creatives;
AdLabels -- "account" --> AdAccounts;
CustomConversions -- "account_id" --> AdAccounts;
CustomAudiences -- "account_id" --> AdAccounts;
AdImages -- "account_id" --> AdAccounts;
AdVideos -- "account_id" --> AdAccounts;
Activities -- "account_id" --> AdAccounts;
AdInsights -- "account_id" --> AdAccounts;
AdInsights -- "ad_id" --> Ads;
AdInsights -- "adset_id" --> AdSets;
AdInsights -- "campaign_id" --> Campaigns;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Facebook Ads data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
## Campaign Performance Analysis
### 1. Campaign Performance Metrics
Track the overall performance of your campaigns, including detailed conversion metrics.
**Business Value:**
* Identify which campaigns deliver the best value for money
* Understand the relationship between quality scores and performance
* Optimize budget allocation based on performance metrics
```sql theme={null}
WITH
campaign_metrics AS (
SELECT
c."name" AS "campaign_name",
c."objective",
c."status",
c."buying_type",
SUM(TRY_CAST (ai."impressions" AS BIGINT)) AS "total_impressions",
SUM(TRY_CAST (ai."reach" AS BIGINT)) AS "total_reach",
SUM(TRY_CAST (ai."clicks" AS BIGINT)) AS "total_clicks",
SUM(TRY_CAST (ai."spend" AS DOUBLE)) AS "total_spend",
AVG(TRY_CAST (ai."frequency" AS DOUBLE)) AS "avg_frequency",
SUM(TRY_CAST (ai."clicks" AS DOUBLE)) * 100.0 / NULLIF(SUM(TRY_CAST (ai."impressions" AS DOUBLE)), 0) AS "ctr",
SUM(TRY_CAST (ai."spend" AS DOUBLE)) / NULLIF(SUM(TRY_CAST (ai."clicks" AS DOUBLE)), 0) AS "cpc",
SUM(TRY_CAST (ai."spend" AS DOUBLE)) * 1000.0 / NULLIF(SUM(TRY_CAST (ai."impressions" AS DOUBLE)), 0) AS "cpm"
FROM
"nekt_raw"."facebook_ads_campaigns" c
LEFT JOIN "nekt_raw"."facebook_ads_adsinsights" ai ON c."id" = ai."campaign_id"
WHERE
c."effective_status" = 'ACTIVE'
AND date_parse (ai."date_start", '%Y-%m-%d') >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
c."name",
c."objective",
c."status",
c."buying_type"
)
SELECT
"campaign_name",
"objective",
"total_spend",
"total_impressions",
"total_reach",
"total_clicks",
"avg_frequency",
"ctr",
"cpc",
"cpm"
FROM
campaign_metrics
```
```sql theme={null}
WITH
campaign_metrics AS (
SELECT
c.name AS campaign_name,
c.objective,
c.status,
c.buying_type,
SUM(SAFE_CAST(ai.impressions AS INT64)) AS total_impressions,
SUM(SAFE_CAST(ai.reach AS INT64)) AS total_reach,
SUM(SAFE_CAST(ai.clicks AS INT64)) AS total_clicks,
SUM(SAFE_CAST(ai.spend AS FLOAT64)) AS total_spend,
AVG(SAFE_CAST(ai.frequency AS FLOAT64)) AS avg_frequency,
SAFE_DIVIDE(SUM(SAFE_CAST(ai.clicks AS FLOAT64)) * 100.0, SUM(SAFE_CAST(ai.impressions AS FLOAT64))) AS ctr,
SAFE_DIVIDE(SUM(SAFE_CAST(ai.spend AS FLOAT64)), SUM(SAFE_CAST(ai.clicks AS FLOAT64))) AS cpc,
SAFE_DIVIDE(SUM(SAFE_CAST(ai.spend AS FLOAT64)) * 1000.0, SUM(SAFE_CAST(ai.impressions AS FLOAT64))) AS cpm
FROM
`nekt_raw.facebook_ads_campaigns` c
LEFT JOIN `nekt_raw.facebook_ads_adsinsights` ai ON c.id = ai.campaign_id
WHERE
c.effective_status = 'ACTIVE'
AND PARSE_DATE('%Y-%m-%d', ai.date_start) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
c.name,
c.objective,
c.status,
c.buying_type
)
SELECT
campaign_name,
objective,
total_spend,
total_impressions,
total_reach,
total_clicks,
avg_frequency,
ctr,
cpc,
cpm
FROM
campaign_metrics
```
| campaign\_name | objective | total\_spend | total\_impressions | total\_reach | total\_clicks | avg\_frequency | ctr | cpc | cpm |
| - | - | - | - | - | - | - | - | - | - |
| Summer Sale 2024 | CONVERSIONS | 8,542.30 | 1,245,890 | 892,340 | 42,156 | 1.40 | 3.38 | 0.20 | 6.85 |
| Brand Awareness Q4 | REACH | 5,686.00 | 3,892,450 | 2,145,670 | 28,430 | 1.81 | 0.73 | 0.20 | 1.46 |
| Product Launch | TRAFFIC | 4,998.50 | 892,340 | 567,230 | 31,245 | 1.57 | 3.50 | 0.16 | 5.60 |
| Retargeting Campaign | CONVERSIONS | 3,784.00 | 2,145,670 | 456,890 | 18,920 | 4.70 | 0.88 | 0.20 | 1.76 |
| Lead Gen Campaign | LEAD\_GENERATION | 2,890.00 | 567,230 | 312,450 | 12,340 | 1.82 | 2.18 | 0.23 | 5.09 |
### 2. Creative Performance and Content Analysis
Analyze how different creative elements and content types perform across your campaigns, helping optimize your creative strategy.
**Business Value:**
* Identify which creative formats drive the best engagement
* Understand the impact of different call-to-action types
* Optimize creative elements based on performance data
* Guide future creative development with data-driven insights
```sql theme={null}
WITH
creative_metrics AS (
SELECT
cr.name AS creative_name,
cr.title,
cr.body,
cr.call_to_action_type,
cr.object_type,
COALESCE(cr.video_id, 'image') AS content_type,
SUM(TRY_CAST (ai.impressions AS BIGINT)) AS total_impressions,
SUM(TRY_CAST (ai.reach AS BIGINT)) AS total_reach,
SUM(TRY_CAST (ai.clicks AS BIGINT)) AS total_clicks,
SUM(TRY_CAST (ai.spend AS DECIMAL(20, 2))) AS total_spend,
AVG(TRY_CAST (ai.frequency AS DECIMAL(10, 2))) AS avg_frequency,
CAST(
SUM(TRY_CAST (ai.clicks AS DOUBLE)) * 100.0 / NULLIF(SUM(TRY_CAST (ai.impressions AS DOUBLE)), 0) AS DECIMAL(10, 2)
) AS ctr,
SUM(
CASE
WHEN action.action_type = 'video_view' THEN TRY_CAST (action.value AS DECIMAL(20, 2))
ELSE 0
END
) AS total_video_views,
AVG(
CASE
WHEN action.action_type = 'video_view' THEN TRY_CAST (action.value AS DECIMAL(20, 2))
ELSE NULL
END
) AS avg_video_completion_rate
FROM
nekt_raw.facebook_ads_creatives cr
LEFT JOIN nekt_raw.facebook_ads_ads a ON cr.id = a.creative.id
LEFT JOIN nekt_raw.facebook_ads_adsinsights ai ON a.id = ai.ad_id
CROSS JOIN UNNEST (ai.actions) AS t ("action")
WHERE
date_parse (ai.date_start, '%Y-%m-%d') >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
cr.name,
cr.title,
cr.body,
cr.call_to_action_type,
cr.object_type,
cr.video_id
)
SELECT
creative_name,
content_type,
call_to_action_type,
object_type,
total_impressions,
total_reach,
total_clicks,
total_spend,
ctr,
CAST(
total_spend / NULLIF(total_clicks, 0) AS DECIMAL(10, 2)
) AS cost_per_click,
CASE
WHEN content_type != 'image' THEN total_video_views
ELSE NULL
END AS video_views,
CASE
WHEN content_type != 'image' THEN avg_video_completion_rate
ELSE NULL
END AS video_completion_rate
FROM
creative_metrics
ORDER BY
total_clicks DESC
```
```sql theme={null}
WITH
creative_metrics AS (
SELECT
cr.name AS creative_name,
cr.title,
cr.body,
cr.call_to_action_type,
cr.object_type,
COALESCE(cr.video_id, 'image') AS content_type,
SUM(SAFE_CAST (ai.impressions AS INT64)) AS total_impressions,
SUM(SAFE_CAST (ai.reach AS INT64)) AS total_reach,
SUM(SAFE_CAST (ai.clicks AS INT64)) AS total_clicks,
SUM(SAFE_CAST (ai.spend AS BIGNUMERIC)) AS total_spend, -- Changed NUMERIC to BIGNUMERIC for consistency/precision
AVG(SAFE_CAST (ai.frequency AS BIGNUMERIC)) AS avg_frequency, -- Changed NUMERIC to BIGNUMERIC
ROUND(
SAFE_DIVIDE (
SUM(SAFE_CAST (ai.clicks AS FLOAT64)) * 100.0,
SUM(SAFE_CAST (ai.impressions AS FLOAT64))
),
2
) AS ctr,
SUM(
CASE
WHEN action.action_type = 'video_view' THEN SAFE_CAST (action.value AS BIGNUMERIC) -- Changed NUMERIC to BIGNUMERIC
ELSE 0
END
) AS total_video_views,
AVG(
CASE
WHEN action.action_type = 'video_view' THEN SAFE_CAST (action.value AS BIGNUMERIC) -- Changed NUMERIC to BIGNUMERIC
ELSE NULL
END
) AS avg_video_completion_rate
FROM
`nekt_raw.facebook_ads_creatives` cr
LEFT JOIN `nekt_raw.facebook_ads_ads` a ON cr.id = a.creative.id
LEFT JOIN `nekt_raw.facebook_ads_adsinsights` ai ON a.id = ai.ad_id
-- Using comma syntax for UNNEST is BigQuery standard
LEFT JOIN UNNEST (ai.actions) AS action ON TRUE
WHERE
PARSE_DATE ('%Y-%m-%d', ai.date_start) >= DATE_SUB (CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
cr.name,
cr.title,
cr.body,
cr.call_to_action_type,
cr.object_type,
cr.video_id
)
SELECT
creative_name,
content_type,
call_to_action_type,
object_type,
total_impressions,
total_reach,
total_clicks,
FORMAT ("%.2f", total_spend) AS total_spend,
FORMAT ("%.2f", ctr) AS ctr,
FORMAT ("%2f", SAFE_DIVIDE (total_spend, total_clicks)) AS cost_per_click, -- CPC calculated in the final select.
CASE
WHEN content_type != 'image' THEN total_video_views
ELSE NULL
END AS video_views,
CASE
WHEN content_type != 'image' THEN FORMAT ("%.2f", avg_video_completion_rate) -- Format this rate as well
ELSE NULL
END AS video_completion_rate
FROM
creative_metrics
ORDER BY
total_clicks DESC
```
| creative\_name | content\_type | call\_to\_action\_type | object\_type | total\_impressions | total\_reach | total\_clicks | total\_spend | ctr | cost\_per\_click | video\_views | video\_completion\_rate |
| - | - | -: | - | - | - | - | - | - | - | - | - |
| Summer Video Ad | video | SHOP\_NOW | VIDEO | 456,780 | 312,450 | 12,340 | 2,468.00 | 2.70 | 0.20 | 89,450 | 45.20 |
| Product Carousel | image | LEARN\_MORE | SHARE | 234,560 | 156,780 | 8,920 | 1,784.00 | 3.80 | 0.20 | NULL | NULL |
| Brand Story Video | video | WATCH\_MORE | VIDEO | 189,340 | 123,450 | 5,670 | 1,134.00 | 2.99 | 0.20 | 56,780 | 38.90 |
| Sale Banner | image | SHOP\_NOW | SHARE | 345,670 | 234,560 | 15,890 | 3,178.00 | 4.60 | 0.20 | NULL | NULL |
| Testimonial Video | video | SIGN\_UP | VIDEO | 123,450 | 89,670 | 4,230 | 846.00 | 3.43 | 0.20 | 34,560 | 52.30 |
### 3. Time-Based Performance Analysis
Track campaign performance trends over time to identify patterns and optimize campaign timing.
**Business Value:**
* Understand daily and weekly performance patterns
* Identify best-performing days and times
* Optimize campaign scheduling and budget pacing
* Track performance trends for better planning
```sql theme={null}
WITH
daily_metrics AS (
SELECT
date_parse (ai.date_start, '%Y-%m-%d') AS date,
date_format (date_parse (ai.date_start, '%Y-%m-%d'), '%W') AS day_of_week,
SUM(TRY_CAST (ai.impressions AS BIGINT)) AS total_impressions,
SUM(TRY_CAST (ai.reach AS BIGINT)) AS total_reach,
SUM(TRY_CAST (ai.clicks AS BIGINT)) AS total_clicks,
SUM(TRY_CAST (ai.spend AS DECIMAL(20, 2))) AS total_spend,
SUM(TRY_CAST (ai.clicks AS DOUBLE)) * 100.0 / NULLIF(SUM(TRY_CAST (ai.impressions AS DOUBLE)), 0) AS ctr,
SUM(TRY_CAST (ai.spend AS DOUBLE)) / NULLIF(SUM(TRY_CAST (ai.clicks AS DOUBLE)), 0) AS cpc,
COUNT(DISTINCT ai.campaign_id) AS active_campaigns,
SUM(
CASE
WHEN action.action_type = 'purchase' THEN TRY_CAST (action.value AS DECIMAL(20, 2))
ELSE 0
END
) AS total_conversions
FROM
nekt_raw.facebook_ads_adsinsights ai
CROSS JOIN UNNEST (ai.actions) AS t (action)
WHERE
date_parse (ai.date_start, '%Y-%m-%d') >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
ai.date_start
),
weekly_summary AS (
SELECT
day_of_week,
AVG(total_impressions) AS avg_daily_impressions,
AVG(total_reach) AS avg_daily_reach,
AVG(total_clicks) AS avg_daily_clicks,
AVG(total_spend) AS avg_daily_spend,
AVG(ctr) AS avg_ctr,
AVG(cpc) AS avg_cpc,
AVG(total_conversions) AS avg_daily_conversions,
AVG(active_campaigns) AS avg_active_campaigns
FROM
daily_metrics
GROUP BY
day_of_week
),
daily_vs_average AS (
SELECT
dm.*,
ws.avg_daily_spend AS typical_daily_spend,
ws.avg_daily_clicks AS typical_daily_clicks,
ws.avg_daily_conversions AS typical_daily_conversions,
ROUND(
(total_spend - ws.avg_daily_spend) * 100.0 / NULLIF(ws.avg_daily_spend, 0),
2
) AS spend_vs_typical,
ROUND(
(total_clicks - ws.avg_daily_clicks) * 100.0 / NULLIF(ws.avg_daily_clicks, 0),
2
) AS clicks_vs_typical,
ROUND(
(total_conversions - ws.avg_daily_conversions) * 100.0 / NULLIF(ws.avg_daily_conversions, 0),
2
) AS conversions_vs_typical
FROM
daily_metrics dm
JOIN weekly_summary ws ON dm.day_of_week = ws.day_of_week
)
SELECT
date,
day_of_week,
total_impressions,
total_reach,
total_clicks,
CAST(total_spend AS DECIMAL(20, 2)) AS total_spend,
CAST(ctr AS DECIMAL(10, 2)) AS ctr,
CAST(cpc AS DECIMAL(10, 2)) AS cpc,
total_conversions,
active_campaigns,
CAST(typical_daily_spend AS DECIMAL(20, 2)) AS typical_daily_spend,
CAST(spend_vs_typical AS DECIMAL(10, 2)) AS spend_vs_typical_pct,
CAST(clicks_vs_typical AS DECIMAL(10, 2)) AS clicks_vs_typical_pct,
CAST(conversions_vs_typical AS DECIMAL(10, 2)) AS conversions_vs_typical_pct
FROM
daily_vs_average
ORDER BY
date DESC
```
```sql theme={null}
WITH
daily_metrics AS (
SELECT
PARSE_DATE ('%Y-%m-%d', ai.date_start) AS date,
FORMAT_DATE ('%A', PARSE_DATE ('%Y-%m-%d', ai.date_start)) AS day_of_week,
SUM(SAFE_CAST (ai.impressions AS INT64)) AS total_impressions,
SUM(SAFE_CAST (ai.reach AS INT64)) AS total_reach,
SUM(SAFE_CAST (ai.clicks AS INT64)) AS total_clicks,
SUM(SAFE_CAST (ai.spend AS BIGNUMERIC)) AS total_spend,
SUM(SAFE_CAST (ai.clicks AS FLOAT64)) * 100.0 / NULLIF(SUM(SAFE_CAST (ai.impressions AS FLOAT64)), 0) AS ctr,
SUM(SAFE_CAST (ai.spend AS FLOAT64)) / NULLIF(SUM(SAFE_CAST (ai.clicks AS FLOAT64)), 0) AS cpc,
COUNT(DISTINCT ai.campaign_id) AS active_campaigns,
SUM(
CASE
WHEN action.action_type = 'purchase' THEN SAFE_CAST (action.value AS BIGNUMERIC)
ELSE 0
END
) AS total_conversions
FROM
`nekt_raw.facebook_ads_adsinsights` AS ai,
UNNEST (ai.actions) AS action
WHERE
PARSE_DATE ('%Y-%m-%d', ai.date_start) >= DATE_SUB (CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
1,
2,
ai.date_start
),
weekly_summary AS (
SELECT
day_of_week,
AVG(total_impressions) AS avg_daily_impressions,
AVG(total_reach) AS avg_daily_reach,
AVG(total_clicks) AS avg_daily_clicks,
AVG(total_spend) AS avg_daily_spend,
AVG(ctr) AS avg_ctr,
AVG(cpc) AS avg_cpc,
AVG(total_conversions) AS avg_daily_conversions,
AVG(active_campaigns) AS avg_active_campaigns
FROM
daily_metrics
GROUP BY
day_of_week
),
daily_vs_average AS (
SELECT
dm.*,
ws.avg_daily_spend AS typical_daily_spend,
ws.avg_daily_clicks AS typical_daily_clicks,
ws.avg_daily_conversions AS typical_daily_conversions,
ROUND(
(total_spend - ws.avg_daily_spend) * 100.0 / NULLIF(ws.avg_daily_spend, 0),
2
) AS spend_vs_typical,
ROUND(
(total_clicks - ws.avg_daily_clicks) * 100.0 / NULLIF(ws.avg_daily_clicks, 0),
2
) AS clicks_vs_typical,
ROUND(
(total_conversions - ws.avg_daily_conversions) * 100.0 / NULLIF(ws.avg_daily_conversions, 0),
2
) AS conversions_vs_typical
FROM
daily_metrics AS dm
INNER JOIN weekly_summary AS ws ON dm.day_of_week = ws.day_of_week
)
SELECT
date,
day_of_week,
total_impressions,
total_reach,
total_clicks,
-- FIX 1: Format currency/metrics as strings to guarantee display format
FORMAT ("%.2f", total_spend) AS total_spend,
FORMAT ("%.2f", ctr) AS ctr,
FORMAT ("%.2f", cpc) AS cpc,
CAST(total_conversions AS BIGINT) AS total_conversions,
active_campaigns,
FORMAT ("%.2f", typical_daily_spend) AS typical_daily_spend,
FORMAT ("%.2f", spend_vs_typical) AS spend_vs_typical_pct,
FORMAT ("%.2f", clicks_vs_typical) AS clicks_vs_typical_pct,
FORMAT ("%.2f", conversions_vs_typical) AS conversions_vs_typical_pct
FROM
daily_vs_average
ORDER BY
date DESC
```
| date | day\_of\_week | total\_impressions | total\_reach | total\_clicks | total\_spend | ctr | cpc | total\_conversions | active\_campaigns | typical\_daily\_spend | spend\_vs\_typical\_pct | clicks\_vs\_typical\_pct | conversions\_vs\_typical\_pct |
| - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 2024-11-27 | Wednesday | 156,234 | 112,450 | 5,892 | 1,178.40 | 3.77 | 0.20 | 78 | 8 | 1,130.50 | 4.2 | 8.5 | 12.8 |
| 2024-11-26 | Tuesday | 148,920 | 108,340 | 5,234 | 1,046.80 | 3.51 | 0.20 | 65 | 8 | 1,072.30 | -2.4 | -3.2 | -8.3 |
| 2024-11-25 | Monday | 142,560 | 102,890 | 4,987 | 997.40 | 3.50 | 0.20 | 58 | 8 | 1,018.60 | -2.1 | -5.8 | -18.5 |
| 2024-11-24 | Sunday | 98,450 | 72,340 | 3,245 | 649.00 | 3.30 | 0.20 | 42 | 7 | 834.20 | -22.2 | -28.4 | -28.6 |
| 2024-11-23 | Saturday | 112,340 | 84,560 | 3,890 | 778.00 | 3.46 | 0.20 | 52 | 7 | 834.20 | -6.7 | -14.2 | -11.2 |
| 2024-11-22 | Friday | 178,560 | 128,450 | 6,234 | 1,246.80 | 3.49 | 0.20 | 95 | 8 | 1,148.90 | 8.5 | 14.8 | 32.4 |
This analysis provides valuable insights into:
* Daily performance trends
* Day-of-week patterns
* Performance against typical metrics
* Campaign activity levels
* Spend and conversion efficiency
You can use these insights to:
* Adjust campaign scheduling
* Optimize budget allocation across days
* Identify and investigate performance anomalies
* Plan campaign launches around high-performing periods
* Improve budget pacing strategies
## Implementation Notes
### Data Quality Considerations
* Ensure you have selected the appropriate attribution windows for your business goals, as this will significantly impact conversion metrics.
* When analyzing performance over time, be mindful of the 28-day lookback window. Data within this window may be subject to change.
* For granular analysis, use the advanced insight reports (breakdowns). Combining these with the core tables will provide a deeper understanding of your ad performance.
### API Limits & Performance
* Enabling advanced reports and selecting a large number of fields in the `AdInsights` stream can lead to longer extraction times and higher API usage.
* To optimize performance, select only the streams and fields necessary for your analysis. Use the `basic` mode for creatives if you don't need detailed media asset information.
### Ad Accounts Permission Requirements
* The **Ad Accounts** stream has two field modes to accommodate different permission levels:
* **Basic mode** (default): Works with standard advertiser permissions. Use this if you're getting permission errors during extraction.
* **Extended mode**: Requires admin or finance access to all connected ad accounts. This mode includes sensitive information like funding source details, tax information, and business manager data.
* If extraction fails with permission errors on the AdAccounts stream, switch to "Basic" mode in the advanced settings.
## Troubleshooting
Most issues with this connector come from permissions inside Meta Business Manager, not from the source configuration in Nekt. Check the authorizing user's access first.
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `OAuthException code 190 (subcode 460) - The session has been invalidated because the user changed their password` | The password of the user who authorized the connection was changed on Facebook. | Re-authorize the source. If several sources use the same account, re-authorize all of them. |
| `USER_PERMISSION_DENIED` | The authorizing user has no permission on that ad account inside Business Manager. | Grant the user access in Meta Ads Manager and re-authorize. |
| The ad account you need does not appear in the list | The account is not accessible to the authorizing user, or a permission was unchecked on Meta's consent screen. | Confirm the account appears in Ads Manager for that user, then authorize again keeping every permission checked. |
| The Instagram asset does not appear in the list | The asset must belong to the same Business Manager used for the authorization. | Add the asset to the Business Manager and authorize again. |
| `TypeError: string indices must be integers` while reading insights | A transient malformed response from the Meta API. | Re-run the pipeline. If it repeats in the same window, contact support with the source slug, the run ID, and the time with timezone. |
| The `campaigns`, `adsets`, `ads`, or `creatives` streams return zero rows | These streams are incremental on the last-modified date, so records not edited since the source was created are not picked up. | Run an initial full sync on those streams. |
| The creative destination URL field comes back empty | The Meta API does not populate this field for most ads, and never for Lead Ads. | Use the destination fields available in the `creatives` stream, such as `link_url`, `object_url`, and `link_destination_display_url`. |
| The authorization flow restarts and leaves sources in draft | The flow was interrupted, usually by opening another tab or switching the interface language midway. | Complete the authorization in a single tab without switching the language, then discard the drafts that were created. |
For permission errors on the **Ad Accounts** stream specifically, see [Ad Accounts Permission Requirements](#ad-accounts-permission-requirements).
## Skills for agents
Facebook Ads connector documentation as plain markdown, for use in AI agent contexts.
# Facebook Leads as a data source
Source: https://docs.nekt.com/sources/facebook-leads
Bring data from Facebook Leads to Nekt.
Facebook Leads is Meta's lead generation platform that helps businesses capture potential customer information through Facebook and Instagram ads. It provides tools for creating lead forms, collecting contact details, and managing prospect data for sales and marketing campaigns.
![]()
## Configuring Facebook Leads as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Facebook Leads** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need Page ID and your Access Token. Check the [linked documentation](https://developers.facebook.com/docs/marketing-api/guides/lead-ads) if you need help on finding any of the credentials.
The following configurations are available:
* **Page ID**: The ID of the Facebook Page associated with your lead forms.
* **Access Token**: Your Meta API access token. The connector automatically detects if this is a User or Page access token. If a User token is provided, it will automatically attempt to exchange it for a Page token (ensure the user has a role on the page and the token has the `pages_show_list` and `leads_retrieval` permissions).
* **API Version**: (Default: `v26.0`) The Facebook Graph API version to request data from.
* **Start Date**: Inform a start date for us to know the date of the earliest record you want to sync with. This will help to avoid processing unnecessary data.
Once you're done, click **Next**.
### 2. Select streams
The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Facebook Leads and their corresponding fields:
Stream containing the information submitted by prospects through Facebook and Instagram lead generation forms.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the record. |
| `created_time` | DateTime | Timestamp when the record was created. |
| `form_id` | String | Identifier of the associated lead form. |
| `campaign_name` | String | Name of the ad campaign. |
| `campaign_id` | String | Identifier of the associated ad campaign. |
| `adset_name` | String | Name of the ad set that captured the lead. |
| `adset_id` | String | Identifier of the associated ad set. |
| `ad_name` | String | Name of the ad that captured the lead. |
| `ad_id` | String | Identifier of the associated ad. |
| `platform` | String | Platform where the lead was submitted. |
| `field_data` | Array of Objects | Custom question and answer pairs from the lead form. |
| `field_data[].name` | String | Name of the lead form field. |
| `field_data[].values` | Array of Strings | Submitted answers for the form field. |
# Implementation Notes
### Authentication and API Access
* **Token Resolution**: The connector automatically handles token resolution at runtime. If a User access token is provided, the system detects it and attempts to derive the appropriate Page token automatically by querying the Graph API (`/me` and `/{page_id}?fields=access_token`).
* **Required Permissions**: To ensure successful token exchange and data extraction, the authenticated user must have a role on the specified page, and the token must include the `pages_show_list` and `leads_retrieval` permissions. If the exchange fails, it will attempt to use the token as-is.
* **API Version**: The connector defaults to using Graph API `v26.0`. Minimum supported/default API versions are periodically bumped to maintain compatibility with Facebook's API lifecycle.
### API Pagination and Discovery
* **Lead Forms Discovery**: The connector automatically handles API pagination through the `leadgen_forms` endpoint, retrieving up to 500 forms per request. This ensures all lead forms associated with your Page ID are successfully discovered and synced, regardless of the account size or previous strict API result limits.
### Reliability and Error Handling
* **Connection Pooling & Retries**: To ensure resilient data extraction, the connector utilizes a shared connection pool and automatic retry logic. It will automatically retry failed requests up to 5 times (with an exponential backoff factor) when encountering API rate limits (HTTP 429) or temporary server errors (HTTP 500, 502, 503, 504), preventing synchronization failures during Facebook Graph API instability.
## Skills for agents
Facebook Leads connector documentation as plain markdown, for use in AI agent contexts.
# Facebook Pages as a data source
Source: https://docs.nekt.com/sources/facebook-pages
Bring data from Facebook Pages to Nekt.
Facebook Pages is Meta's platform for businesses, brands, and public figures to create a presence on Facebook. It enables organizations to share updates, engage with followers, receive reviews, and track page performance through comprehensive insights and analytics.
![]()
## Configuring Facebook Pages as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Facebook Pages option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Facebook Pages data. Click on the `Facebook Authorization` button and log in with your Facebook account. Grant the necessary permissions for the pages you want to extract data from. After authentication, select the specific Page for this source and define a start date for data retrieval.
The user authorizing the connection must have admin or editor access to the Page to successfully exchange the user token for a page access token.
The following configurations are available:
* **Page ID**: The Facebook Page ID to sync data from. This field is automatically populated with the Page ID you select in the dropdown menu.
* **Start Date**: The earliest date from which records will be synced.
* **Lookback Window**: (Default: 28 days) The number of days to look back for incremental sync. Since post engagement metrics can change over time, this ensures recent data is re-fetched to capture updates.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream and a short description.
| Stream | Description |
| - | - |
| Page | General information about your Facebook Page. |
| Posts | Published posts from your Facebook Page, including content and metadata. |
| Post Insights | Lifetime performance metrics for each post. |
| Post Media Views Breakdown | Lifetime post media views broken down by ads and followers. |
| Post Comments | Comments on your Facebook Page posts. |
| Daily Page Insights | Daily page-level performance metrics. |
| Page Media Views Breakdown | Daily page media views broken down by ads and followers. |
| Reviews | Page ratings and recommendations from users. |
## Fields by stream
Below you'll find all available data streams from Facebook Pages and their corresponding fields:
Stream containing general information about your Facebook Page, including profile details, settings, and metadata.
| Field | Type | Description |
| - | - | - |
| `id` | String | The Facebook page ID |
| `page_id` | String | The Facebook page ID (duplicated for consistency) |
| `about` | String | About information of the page |
| `affiliation` | String | The page affiliation |
| `artists_we_like` | String | Artists that the page likes |
| `attire` | String | The attire information for the page |
| `awards` | String | Awards received by the page |
| `band_interests` | String | Band interests of the page |
| `band_members` | String | Band members of the page |
| `bio` | String | Biography of the page |
| `birthday` | String | Birthday of the page |
| `booking_agent` | String | Booking agent information |
| `built` | String | Built date or information |
| `can_checkin` | Boolean | Whether users can check in at this page |
| `can_post` | Boolean | Whether users can post to this page |
| `category` | String | The category of the page |
| `checkins` | Integer | Number of check-ins at the page |
| `company_overview` | String | Company overview information |
| `country_page_likes` | Integer | Number of likes from each country |
| `culinary_team` | String | Culinary team information |
| `current_location` | String | Current location of the page |
| `description` | String | Description of the page |
| `directed_by` | String | Director information |
| `display_subtext` | String | Display subtext of the page |
| `features` | String | Features of the page |
| `followers_count` | Integer | Number of followers |
| `founded` | String | Founded date or information |
| `general_info` | String | General information about the page |
| `general_manager` | String | General manager information |
| `genre` | String | Genre of the page |
| `global_brand_page_name` | String | Global brand page name |
| `global_brand_root_id` | String | Global brand root ID |
| `has_added_app` | Boolean | Whether the page has added an app |
| `has_whatsapp_business_number` | Boolean | Whether the page has a WhatsApp business number |
| `has_whatsapp_number` | Boolean | Whether the page has a WhatsApp number |
| `hometown` | String | Hometown of the page |
| `influences` | String | Influences of the page |
| `is_chain` | Boolean | Whether this is a chain page |
| `is_community_page` | Boolean | Whether this is a community page |
| `is_eligible_for_branded_content` | Boolean | Whether the page is eligible for branded content |
| `is_messenger_bot_get_started_enabled` | Boolean | Whether messenger bot get started is enabled |
| `is_messenger_platform_bot` | Boolean | Whether this is a messenger platform bot |
| `is_owned` | Boolean | Whether the page is owned |
| `is_permanently_closed` | Boolean | Whether the page is permanently closed |
| `is_published` | Boolean | Whether the page is published |
| `is_unclaimed` | Boolean | Whether the page is unclaimed |
| `link` | String | Link to the page |
| `members` | String | Members of the page |
| `merchant_review_status` | String | Merchant review status |
| `mission` | String | Mission of the page |
| `mpg` | String | MPG information |
| `name` | String | Name of the page |
| `network` | String | Network information |
| `personal_info` | String | Personal information |
| `personal_interests` | String | Personal interests |
| `phone` | String | Phone number of the page |
Stream containing published posts from your Facebook Page, including post content, metadata, and engagement indicators.
This stream requests all schema fields at once, including long text and media URLs. To avoid Facebook's "too much data" error, it uses a reduced default page size (25 records per request). If the payload is still too large, the connector will automatically shrink the page size further.
| Field | Type | Description |
| - | - | - |
| `id` | String | The post ID |
| `page_id` | String | The Facebook page ID |
| `allowed_advertising_objects` | String | Allowed advertising objects for the post |
| `created_time` | DateTime | The creation time of the post |
| `full_picture` | String | The full picture URL of the post |
| `icon` | String | The icon URL of the post |
| `message` | String | The message text of the post |
| `is_eligible_for_promotion` | Boolean | Whether the post is eligible for promotion |
| `is_expired` | Boolean | Whether the post has expired |
| `is_hidden` | Boolean | Whether the post is hidden |
| `is_instagram_eligible` | Boolean | Whether the post is eligible for Instagram |
| `is_popular` | Boolean | Whether the post is popular |
| `is_published` | Boolean | Whether the post is published |
| `is_spherical` | Boolean | Whether the post contains spherical media |
| `parent_id` | String | The parent post ID if this is a comment |
| `permalink_url` | String | The permanent URL of the post |
| `story` | String | The story text of the post |
| `shares` | Object | Share information |
| `shares.count` | Integer | The number of shares |
| `subscribed` | Boolean | Whether the user is subscribed to the post |
| `status_type` | String | The status type of the post |
| `updated_time` | DateTime | The last update time of the post |
Stream containing lifetime performance metrics for each post, including engagement, reactions, and video view statistics.
> **Note**: This stream provides lifetime metrics for posts. Video-related metrics will only have values for video posts. Additionally, if your page does not support certain metrics, the connector will automatically exclude them to prevent API errors.
| Field | Type | Description |
| - | - | - |
| `page_id` | String | The Facebook page ID |
| `post_id` | String | The post ID |
| `post_created_time` | DateTime | The post creation time |
| `post_clicks` | Integer | Total post clicks |
| `post_reactions_like_total` | Integer | Total like reactions |
| `post_reactions_love_total` | Integer | Total love reactions |
| `post_reactions_wow_total` | Integer | Total wow reactions |
| `post_reactions_haha_total` | Integer | Total haha reactions |
| `post_reactions_sorry_total` | Integer | Total sorry reactions |
| `post_reactions_anger_total` | Integer | Total anger reactions |
| `post_video_views` | Integer | Total video views |
| `post_video_views_unique` | Integer | Unique video views |
| `post_video_views_organic` | Integer | Organic video views |
| `post_video_views_organic_unique` | Integer | Unique organic video views |
| `post_video_views_paid` | Integer | Paid video views |
| `post_video_views_paid_unique` | Integer | Unique paid video views |
| `post_video_views_autoplayed` | Integer | Autoplayed video views |
| `post_video_views_clicked_to_play` | Integer | Clicked-to-play video views |
| `post_video_views_sound_on` | Integer | Video views with sound on |
| `post_video_view_time` | Integer | Total video view time (ms) |
| `post_video_views_15s` | Integer | 15-second video views |
| `post_video_views_60s_excludes_shorter` | Integer | 60-second video views (excludes shorter videos) |
| `post_video_complete_views_organic` | Integer | Organic complete video views |
| `post_video_complete_views_organic_unique` | Integer | Unique organic complete video views |
| `post_video_complete_views_paid` | Integer | Paid complete video views |
| `post_video_complete_views_paid_unique` | Integer | Unique paid complete video views |
| `post_video_avg_time_watched` | Integer | Average time watched (ms) |
| `post_video_length` | Integer | Video length (ms) |
| `post_total_media_view` | Integer | Number of times the post's media was played or displayed |
| `post_total_media_view_unique` | Integer | Number of unique viewers of the post's media; Meta's replacement for the deprecated unique video view metrics |
Stream containing comments on your Facebook Page posts, including commenter information and engagement metrics.
This stream uses a reduced default page size (20 records per API request) to avoid Facebook's "too much data" error when fetching nested comment structures. If the payload is still too large, the connector will automatically shrink the page size further.
| Field | Type | Description |
| - | - | - |
| `id` | String | The comment ID |
| `post_id` | String | The parent post ID |
| `post_created_time` | DateTime | The parent post creation time |
| `page_id` | String | The Facebook page ID |
| `message` | String | The comment message text |
| `created_time` | DateTime | The comment creation time |
| `from` | Object | The user who made the comment |
| `from.id` | String | The commenter's user ID |
| `from.name` | String | The commenter's name |
| `like_count` | Integer | Number of likes on the comment |
| `comment_count` | Integer | Number of replies to the comment |
| `is_hidden` | Boolean | Whether the comment is hidden |
| `attachment` | Object | Attachment on the comment (if any) |
| `attachment.type` | String | The attachment type |
| `attachment.url` | String | The attachment URL |
| `attachment.media` | Object | Media object for the attachment |
| `attachment.media.image` | Object | Image details |
| `attachment.media.image.src` | String | The image source URL |
Stream containing daily page-level performance metrics, including views, engagement, impressions, reactions, and video statistics.
> **Note**: This stream provides daily aggregated metrics for your entire page. Data is fetched in up to 3-month batches for efficient API usage. If Facebook refuses the payload because the requested date range contains too much data, the connector will automatically narrow the date window (e.g., down to 1 month, 1 week, or 1 day) to prevent extraction failures. If your page does not support certain metrics, the connector will automatically exclude them.
| Field | Type | Description |
| - | - | - |
| `page_id` | String | The Facebook page ID |
| `end_time` | DateTime | The end time for the metric period |
| `page_views_total` | Integer | Total page views |
| `page_post_engagements` | Integer | Total post engagements |
| `page_posts_impressions` | Integer | Total post impressions |
| `page_posts_impressions_unique` | Integer | Unique post impressions |
| `page_posts_impressions_paid` | Integer | Paid post impressions |
| `page_posts_impressions_paid_unique` | Integer | Unique paid post impressions |
| `page_posts_impressions_organic` | Integer | Organic post impressions |
| `page_posts_impressions_organic_unique` | Integer | Unique organic post impressions |
| `page_posts_served_impressions_organic_unique` | Integer | Unique organic impressions served |
| `page_posts_impressions_viral` | Integer | Viral post impressions |
| `page_posts_impressions_viral_unique` | Integer | Unique viral post impressions |
| `page_posts_impressions_nonviral` | Integer | Non-viral post impressions |
| `page_posts_impressions_nonviral_unique` | Integer | Unique non-viral post impressions |
| `page_actions_post_reactions_like_total` | Integer | Total like reactions |
| `page_actions_post_reactions_love_total` | Integer | Total love reactions |
| `page_actions_post_reactions_wow_total` | Integer | Total wow reactions |
| `page_actions_post_reactions_haha_total` | Integer | Total haha reactions |
| `page_actions_post_reactions_sorry_total` | Integer | Total sorry reactions |
| `page_actions_post_reactions_anger_total` | Integer | Total anger reactions |
| `page_actions_post_reactions_total` | Object | Total reactions breakdown |
| `page_actions_post_reactions_total.like` | Integer | Total like reactions |
| `page_actions_post_reactions_total.love` | Integer | Total love reactions |
| `page_actions_post_reactions_total.wow` | Integer | Total wow reactions |
| `page_actions_post_reactions_total.haha` | Integer | Total haha reactions |
| `page_actions_post_reactions_total.sorry` | Integer | Total sorry reactions |
| `page_actions_post_reactions_total.anger` | Integer | Total anger reactions |
| `page_total_actions` | Integer | Total page actions |
| `page_video_views` | Integer | Total video views |
| `page_video_views_paid` | Integer | Paid video views |
| `page_video_views_organic` | Integer | Organic video views |
| `page_video_views_autoplayed` | Integer | Autoplayed video views |
| `page_video_views_click_to_play` | Integer | Click-to-play video views |
| `page_video_views_unique` | Integer | Unique video views |
| `page_video_repeat_views` | Integer | Repeat video views |
| `page_video_view_time` | Integer | Total video view time (ms) |
| `page_video_complete_views_30s` | Integer | 30s complete video views |
| `page_video_complete_views_30s_paid` | Integer | Paid 30s complete video views |
| `page_video_complete_views_30s_organic` | Integer | Organic 30s complete video views |
| `page_video_complete_views_30s_autoplayed` | Integer | Autoplayed 30s complete video views |
| `page_video_complete_views_30s_click_to_play` | Integer | Click-to-play 30s complete video views |
| `page_video_complete_views_30s_unique` | Integer | Unique 30s complete video views |
| `page_video_complete_views_30s_repeat_views` | Integer | Repeat 30s complete video views |
| `page_total_media_view` | Integer | Number of times the page's media (videos, posts, stories, ads) was played or displayed |
| `page_total_media_view_unique` | Integer | Number of unique viewers of the page's media; Meta's replacement for the deprecated unique reach and unique video view metrics |
Stream containing daily page media views broken down by `is_from_ads` and `is_from_followers`. It provides a long format structure with one row per date, breakdown dimension, and breakdown value. This stream is Meta's replacement for the deprecated paid/organic reach metrics.
| Field | Type | Description |
| - | - | - |
| `page_id` | String | The Facebook page ID |
| `end_time` | DateTime | The end time of the metric period (one day) |
| `media_view_total` | Integer | Total media views for the period, summed across every breakdown combination |
| `breakdown` | String | Raw is\_from\_ads / is\_from\_followers breakdown of page media views, serialized as a JSON string. Parse downstream to split paid vs organic and follower vs non-follower views. |
| `breakdown_dimension` | String | Which breakdown this row belongs to: 'is\_from\_ads' or 'is\_from\_followers' |
| `breakdown_value` | String | Value of the breakdown dimension as returned by Facebook: for is\_from\_ads, '1' = views from ads (paid) and '0' = organic; for is\_from\_followers, '1' = follower and '0' = non-follower |
| `media_view` | Integer | Number of media views for this date and breakdown value |
Stream containing lifetime post media views broken down by `is_from_ads` and `is_from_followers`. It provides a long format structure with one row per post, breakdown dimension, and breakdown value.
| Field | Type | Description |
| - | - | - |
| `page_id` | String | The Facebook page ID |
| `post_id` | String | The post ID |
| `post_created_time` | DateTime | The post creation time |
| `media_view_total` | Integer | Total media views for the post, summed across every breakdown combination |
| `breakdown` | String | Raw is\_from\_ads / is\_from\_followers breakdown of post media views, serialized as a JSON string. Parse downstream to split paid vs organic and follower vs non-follower views. |
| `breakdown_dimension` | String | Which breakdown this row belongs to: 'is\_from\_ads' or 'is\_from\_followers' |
| `breakdown_value` | String | Value of the breakdown dimension as returned by Facebook: for is\_from\_ads, '1' = views from ads (paid) and '0' = organic; for is\_from\_followers, '1' = follower and '0' = non-follower |
| `media_view` | Integer | Number of media views for this post and breakdown value |
Stream containing page ratings and recommendations from users.
| Field | Type | Description |
| - | - | - |
| `page_id` | String | The Facebook page ID |
| `created_time` | DateTime | The time the review was created |
| `recommendation_type` | String | The recommendation type (positive or negative) |
| `review_text` | String | The content of the review |
| `reviewer` | Object | The reviewer information |
| `reviewer.id` | String | The Page-scoped ID of the reviewer |
| `reviewer.name` | String | The name of the reviewer |
## API Limitations
Facebook's Graph API has the following limitations that affect the connector:
* **Rate Limits**: The API has standard rate limits. The connector handles pagination automatically and respects these limits.
* **Payload Size Limits**: When fetching records with heavy content or over long date ranges, Facebook may return a "too much data" error if the result set is too large. The connector handles this automatically: for standard streams, it dynamically shrinks the page size; for Insights streams, it dynamically narrows the date window (e.g., from 3 months down to a single day) and retries the request. Heavy streams (like Posts and Post Comments) also use reduced default page sizes (25 and 20 records per request, respectively) to minimize these errors upfront.
* **Unsupported Metrics**: Facebook may return errors if certain insight metrics are invalid, deprecated, or not supported for a specific page. The connector automatically probes and drops these unsupported metrics to ensure the extraction succeeds for the remaining valid data.
* **Permissions**: The user authorizing the connection must have admin or editor access to the Page to successfully retrieve all data streams.
## Stream to Endpoint Mapping
| Stream | Endpoint | Sync Type | Replication Key |
| - | - | - | - |
| Page | `/{page_id}` | Full Table | - |
| Posts | `/{page_id}/published_posts` | Incremental | `created_time` |
| Post Insights | `/{page_id}/published_posts` (with insights) | Incremental | `post_created_time` |
| Post Media Views Breakdown | `/{page_id}/published_posts` (with insights) | Incremental | `post_created_time` |
| Post Comments | `/{page_id}/published_posts` (with comments) | Incremental | `post_created_time` |
| Daily Page Insights | `/{page_id}/insights` | Incremental | `end_time` |
| Page Media Views Breakdown | `/{page_id}/insights` | Incremental | `end_time` |
| Post Media Views Breakdown | `/{page_id}/published_posts` | Incremental | `post_created_time` |
| Reviews | `/{page_id}/ratings` | Full Table | - |
# Data Model
The following diagram illustrates the relationships between the core data streams in Facebook Pages. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Page Information"
Page("Page")
end
subgraph "Content"
Posts("Posts")
PostComments("Post Comments")
end
subgraph "Performance Data"
PostInsights("Post Insights")
PostMediaViewsBreakdown("Post Media Views Breakdown")
DailyPageInsights("Daily Page Insights")
PageMediaViewsBreakdown("Page Media Views Breakdown")
end
subgraph "Engagement"
Reviews("Reviews")
end
Posts -- "page_id" --> Page
PostComments -- "post_id" --> Posts
PostComments -- "page_id" --> Page
PostInsights -- "post_id" --> Posts
PostInsights -- "page_id" --> Page
PostMediaViewsBreakdown -- "post_id" --> Posts
PostMediaViewsBreakdown -- "page_id" --> Page
DailyPageInsights -- "page_id" --> Page
PageMediaViewsBreakdown -- "page_id" --> Page
Reviews -- "page_id" --> Page
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Facebook Pages data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
## Page Performance Analysis
### 1. Daily Page Engagement Overview
Track your page's daily engagement metrics to understand audience interaction patterns.
**Business Value:**
* Monitor daily engagement trends
* Identify high-performing days for content strategy
* Track the balance between organic and paid reach
```sql theme={null}
SELECT
DATE(end_time) AS date,
page_views_total,
page_post_engagements,
page_posts_impressions_organic,
page_posts_impressions_paid,
page_actions_post_reactions_total,
page_video_views
FROM
nekt_raw.facebook_pages_daily_page_insights
WHERE
end_time >= CURRENT_DATE - INTERVAL '30' DAY
ORDER BY
date DESC
```
```sql theme={null}
SELECT
DATE(end_time) AS date,
page_views_total,
page_post_engagements,
page_posts_impressions_organic,
page_posts_impressions_paid,
page_actions_post_reactions_total,
page_video_views
FROM
`nekt_raw.facebook_pages_daily_page_insights`
WHERE
end_time >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
ORDER BY
date DESC
```
| date | page\_views\_total | page\_post\_engagements | page\_posts\_impressions\_organic | page\_posts\_impressions\_paid | page\_actions\_post\_reactions\_total | page\_video\_views |
| - | - | - | - | - | - | - |
| 2024-11-27 | 1,234 | 456 | 8,920 | 2,340 | 189 | 567 |
| 2024-11-26 | 1,156 | 398 | 7,845 | 1,890 | 167 | 423 |
| 2024-11-25 | 987 | 312 | 6,230 | 1,560 | 134 | 289 |
### 2. Post Performance Analysis
Analyze individual post performance to understand what content resonates with your audience.
**Business Value:**
* Identify top-performing content
* Understand reaction distribution across posts
* Optimize content strategy based on engagement data
```sql theme={null}
SELECT
p.id AS post_id,
SUBSTRING(p.message, 1, 100) AS post_preview,
p.created_time,
p.shares.count AS share_count,
pi.post_clicks,
pi.post_reactions_like_total,
pi.post_reactions_love_total,
pi.post_reactions_wow_total,
(COALESCE(pi.post_reactions_like_total, 0) +
COALESCE(pi.post_reactions_love_total, 0) +
COALESCE(pi.post_reactions_wow_total, 0) +
COALESCE(pi.post_reactions_haha_total, 0) +
COALESCE(pi.post_reactions_sorry_total, 0) +
COALESCE(pi.post_reactions_anger_total, 0)) AS total_reactions
FROM
nekt_raw.facebook_pages_posts p
LEFT JOIN nekt_raw.facebook_pages_post_insights pi ON p.id = pi.post_id
WHERE
p.created_time >= CURRENT_DATE - INTERVAL '30' DAY
ORDER BY
total_reactions DESC
LIMIT 20
```
```sql theme={null}
SELECT
p.id AS post_id,
SUBSTR(p.message, 1, 100) AS post_preview,
p.created_time,
p.shares.count AS share_count,
pi.post_clicks,
pi.post_reactions_like_total,
pi.post_reactions_love_total,
pi.post_reactions_wow_total,
(COALESCE(pi.post_reactions_like_total, 0) +
COALESCE(pi.post_reactions_love_total, 0) +
COALESCE(pi.post_reactions_wow_total, 0) +
COALESCE(pi.post_reactions_haha_total, 0) +
COALESCE(pi.post_reactions_sorry_total, 0) +
COALESCE(pi.post_reactions_anger_total, 0)) AS total_reactions
FROM
`nekt_raw.facebook_pages_posts` p
LEFT JOIN `nekt_raw.facebook_pages_post_insights` pi ON p.id = pi.post_id
WHERE
p.created_time >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
ORDER BY
total_reactions DESC
LIMIT 20
```
| post\_id | post\_preview | created\_time | share\_count | post\_clicks | post\_reactions\_like\_total | post\_reactions\_love\_total | post\_reactions\_wow\_total | total\_reactions |
| - | - | - | - | - | - | - | - | - |
| 123456\_789 | 🎉 Exciting news! We're launching our new\... | 2024-11-25T14:30:00Z | 45 | 234 | 156 | 89 | 23 | 278 |
| 123456\_790 | Thank you all for your amazing support... | 2024-11-23T10:00:00Z | 32 | 189 | 134 | 67 | 12 | 219 |
### 3. Comment Engagement Analysis
Analyze comment activity to understand audience engagement and sentiment.
**Business Value:**
* Monitor community engagement
* Identify posts that spark conversation
* Track response patterns and community health
```sql theme={null}
WITH comment_stats AS (
SELECT
post_id,
COUNT(*) AS total_comments,
SUM(like_count) AS total_comment_likes,
SUM(comment_count) AS total_replies,
COUNT(CASE WHEN is_hidden = false THEN 1 END) AS visible_comments
FROM
nekt_raw.facebook_pages_post_comments
WHERE
post_created_time >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
post_id
)
SELECT
p.id AS post_id,
SUBSTRING(p.message, 1, 80) AS post_preview,
p.created_time,
cs.total_comments,
cs.total_comment_likes,
cs.total_replies,
cs.visible_comments
FROM
nekt_raw.facebook_pages_posts p
JOIN comment_stats cs ON p.id = cs.post_id
ORDER BY
cs.total_comments DESC
LIMIT 15
```
```sql theme={null}
WITH comment_stats AS (
SELECT
post_id,
COUNT(*) AS total_comments,
SUM(like_count) AS total_comment_likes,
SUM(comment_count) AS total_replies,
COUNTIF(is_hidden = false) AS visible_comments
FROM
`nekt_raw.facebook_pages_post_comments`
WHERE
post_created_time >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
post_id
)
SELECT
p.id AS post_id,
SUBSTR(p.message, 1, 80) AS post_preview,
p.created_time,
cs.total_comments,
cs.total_comment_likes,
cs.total_replies,
cs.visible_comments
FROM
`nekt_raw.facebook_pages_posts` p
JOIN comment_stats cs ON p.id = cs.post_id
ORDER BY
cs.total_comments DESC
LIMIT 15
```
| post\_id | post\_preview | created\_time | total\_comments | total\_comment\_likes | total\_replies | visible\_comments |
| - | - | - | - | - | - | - |
| 123456\_789 | 🎉 Exciting news! We're launching our new product line... | 2024-11-25T14:30:00Z | 89 | 234 | 45 | 87 |
| 123456\_791 | What's your favorite feature of our service? Let us know\... | 2024-11-24T09:00:00Z | 67 | 156 | 23 | 65 |
### 4. Review Sentiment Analysis
Analyze page reviews to understand customer satisfaction and feedback trends.
**Business Value:**
* Track overall customer satisfaction
* Monitor review volume and sentiment over time
* Identify areas for improvement
```sql theme={null}
SELECT
DATE_TRUNC('week', created_time) AS week,
COUNT(*) AS total_reviews,
COUNT(CASE WHEN recommendation_type = 'positive' THEN 1 END) AS positive_reviews,
COUNT(CASE WHEN recommendation_type = 'negative' THEN 1 END) AS negative_reviews,
ROUND(
COUNT(CASE WHEN recommendation_type = 'positive' THEN 1 END) * 100.0 / COUNT(*),
2
) AS positive_percentage
FROM
nekt_raw.facebook_pages_reviews
WHERE
created_time >= CURRENT_DATE - INTERVAL '90' DAY
GROUP BY
DATE_TRUNC('week', created_time)
ORDER BY
week DESC
```
```sql theme={null}
SELECT
DATE_TRUNC(created_time, WEEK) AS week,
COUNT(*) AS total_reviews,
COUNTIF(recommendation_type = 'positive') AS positive_reviews,
COUNTIF(recommendation_type = 'negative') AS negative_reviews,
ROUND(
COUNTIF(recommendation_type = 'positive') * 100.0 / COUNT(*),
2
) AS positive_percentage
FROM
`nekt_raw.facebook_pages_reviews`
WHERE
created_time >= DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY)
GROUP BY
DATE_TRUNC(created_time, WEEK)
ORDER BY
week DESC
```
| week | total\_reviews | positive\_reviews | negative\_reviews | positive\_percentage |
| - | - | - | - | - |
| 2024-11-25 | 12 | 10 | 2 | 83.33 |
| 2024-11-18 | 8 | 7 | 1 | 87.50 |
| 2024-11-11 | 15 | 12 | 3 | 80.00 |
## Implementation Notes
### Data Quality Considerations
* The **lookback window** is important for post engagement metrics, as reactions and comments can be added to posts days or weeks after publication.
* Daily Page Insights data is fetched in up to 3-month batches to optimize API usage and avoid rate limits. This window dynamically narrows if Facebook refuses the payload due to size limits.
* Post Insights metrics provide lifetime values that accumulate over time, while Daily Page Insights provide daily snapshots.
* The `Page Media Views Breakdown` and `Post Media Views Breakdown` streams are Meta's modern replacements for older unique reach and unique video view metrics. You can extract and parse the JSON string field `breakdown` to segment views by ads (paid) vs followers (organic).
* The Reviews stream does not have incremental sync capability, so it performs a full table sync on each extraction.
* **Unsupported metrics**: If specific insights metrics are unsupported or deprecated for your page, they will be automatically dropped from the extraction. The corresponding fields in your table will appear as null.
### API Limits & Performance
* Facebook's Graph API has rate limits. The connector handles pagination automatically and respects these limits.
* **Dynamic Sizing and Windows**: If Facebook refuses a request because the payload is too large, the connector automatically adjusts to request less data. For cursor-paginated streams, it retries with progressively smaller page sizes (down to 1 record per request). For Insights streams, it dynamically narrows the date window (down to a single day). This ensures extraction succeeds without data loss, though it may increase the extraction time.
* The Post Comments and Posts streams use reduced default page sizes (20 and 25 records per request, respectively) to mitigate payload size errors upfront.
* For pages with high post volumes, consider selecting only the streams you need to optimize extraction times.
* The `page_id` field is included in all streams to enable easy joins and filtering when you have multiple pages.
## Skills for agents
Facebook Pages connector documentation as plain markdown, for use in AI agent contexts.
# Faceponto as a data source
Source: https://docs.nekt.com/sources/faceponto
Bring data from Faceponto to Nekt.
Faceponto is a Brazilian electronic time and attendance platform that records employee clock-ins and clock-outs through facial recognition, on mobile devices and shared tablets. It manages employee registries, work schedules and absences, and validates each punch against the employee's registered photo, location and device.
## Configuring Faceponto as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Faceponto option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the email and password of a Faceponto user for this connection. These are the same credentials used to sign in to Faceponto itself, and the data Nekt can extract is limited to what that user is allowed to see, so we recommend using an account with administrative access.
The following configurations are available:
* **Email**: Email address used to sign in to Faceponto.
* **Password**: Password for the Faceponto account.
* **Start Date**: The earliest date from which punches and absences will be synced. Defaults to 90 days ago when left empty.
* **Lookback Days**: (Default: 30) How many days before the last synced date to re-read on each run. Absences are dated by the day the employee was absent, but are often entered into Faceponto weeks later. Since Faceponto offers no filter on the entry date, re-reading a trailing window is the only way to capture them.
* **Requests Per Minute**: (Default: 9) How many requests per minute the connector is allowed to send. Only raise this if Faceponto has lifted the rate limit for your account.
Once you're done, click **Next**.
Faceponto limits each account to **10 requests per minute** and returns at most **15 records per request**. The connector paces itself to stay within this limit, which means large date ranges take a long time to sync: roughly one hour of syncing per month of punch history. Scheduled runs after the first one only fetch new data and are much faster. Choose your **Start Date** with this in mind.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Colaboradores | `colaboradores` | Employees registered in the account, with their role, work site and payroll number. |
| Escalas | `escalas` | Work schedules (shift patterns) defined in the account, such as 12x36 or 5x2. |
| Escala Horários | `escala_horarios` | The day-by-day expected working hours that make up each work schedule. |
| Justificativas | `justificativas` | Catalogue of reasons an absence can be justified with. |
| Faltas | `faltas` | Absences registered against employees. |
| Falta Detalhes | `falta_detalhes` | Full detail for each absence, including medical fields and the audit trail. |
| Pontos | `pontos` | Individual clock-in and clock-out punches. |
Photos and attachments (`foto`, `anexo`) are returned by Faceponto as temporary links that expire within the hour. The connector stores the stable part of the link, which identifies the file but cannot be opened directly.
## Fields by stream
Below you'll find all available data streams from Faceponto and their corresponding fields:
Employees registered in the account. Synced in full on every run.
**Key Fields:**
* `colaborador_id` - Unique identifier of the employee in Faceponto
* `nome` - Full name of the employee
* `cpf` - Employee's CPF (Brazilian taxpayer number)
* `matricula` - Payroll registration number identifying the employee
**Role and Location:**
* `nome_funcao` - Name of the job role the employee performs
* `nomecontratante` - Name of the contracting entity or work site the employee is assigned to
* `nome_superlocacao` - Name of the parent location grouping the employee's work site
* `nome_sublocacao` - Name of the sub-location within the employee's work site
**Schedule and Status:**
* `carga_id` - Identifier of the working-hours load (carga horária) assigned to the employee
* `folga` - Number of rest days in the employee's work cycle
* `foto` - URL of the employee's reference photo used for facial recognition
* `isAssinatura` - Whether the employee has a registered handwritten signature
* `isTermo` - Whether the employee has accepted the system's terms of use
Work schedules (shift patterns) defined in the account. Synced in full on every run.
**Key Fields:**
* `escala_id` - Unique identifier of the work schedule, as a UUID
* `nome` - Name of the work schedule, such as "Escala 12x36 (Entrada 06:00)"
* `tipo` - Numeric code for the schedule type (weekly, special or on-call shift)
* `qt_colaboradores` - Number of employees currently assigned to this schedule
The day-by-day expected working hours that make up each work schedule. One row per schedule and day.
**Key Fields:**
* `escala_id` - Identifier of the work schedule this day belongs to, matching `escalas.escala_id`
* `rel_escala_horario_id` - Unique identifier of the link between the schedule and this day's hours
* `horario_padrao_id` - Identifier of the standard hours pattern applied on this day, as a UUID
* `dia` - Day the hours apply to
* `horario` - Comma-separated clock times the employee is expected to punch on this day, such as "06:00,12:00,13:00,18:00". Empty on a rest day
**Schedule Attributes:**
* `escala_nome` - Name of the work schedule this day belongs to
* `escala_tipo` - Numeric code for the schedule type
* `escala_tipo_string` - Human-readable schedule type, such as "Semanal", "Especial" or "Plantão"
* `escala_tolerancia` - Tolerance in minutes allowed around the expected punch times before a punch counts as late or early
* `escala_is_show_folha` - Whether rest days are shown on the timesheet for this schedule
Catalogue of reasons an absence can be justified with. A small lookup table that gives `faltas.justificativa_id` its meaning.
**Key Fields:**
* `justificativa_id` - Unique identifier of the justification type
* `nome_justificativa` - Human-readable name of the justification, such as "Atestado Médico" or "Licença de Paternidade"
Absences registered against employees. Supports incremental sync on `data_falta`.
**Key Fields:**
* `falta_id` - Unique identifier of the absence record
* `data_falta` - Day the employee was absent. Used as the incremental replication key. This is the date of the absence itself, not the date the record was created
* `colaborador_id` - Identifier of the absent employee
**Employee Information:**
* `nome` - Full name of the employee
* `cpf` - Employee's CPF (Brazilian taxpayer number)
* `matricula` - Payroll registration number of the employee
* `nome_funcao` - Name of the job role the employee performs
* `nomecontratante` - Name of the contracting entity or work site the employee is assigned to
* `foto` - URL of the employee's reference photo
**Absence Attributes:**
* `justificativa_id` - Identifier of the justification given, matching `justificativas.justificativa_id`
* `isJustificada` - Whether the absence was justified. Unjustified absences are always full-day
* `isAnexo` - Whether a supporting document, such as a medical certificate, is attached
* `isAssinatura` - Whether the absence record carries the employee's signature
* `isTermo` - Whether the employee accepted the terms attached to the absence
Full detail for each absence listed in the `faltas` stream. Requires the `faltas` stream to be selected as well.
**Key Fields:**
* `falta_id` - Unique identifier of the absence record
* `data_falta` - Day the employee was absent
* `data_cadastro` - Moment the absence record was created in Faceponto, often days after the absence itself
* `ativo` - Whether the absence record is still active. Deactivated records remain readable
**Employee Information:**
* `colaborador_id` - Identifier of the absent employee
* `colaborador_nome` - Full name of the absent employee
* `colaborador_cpf` - Absent employee's CPF (Brazilian taxpayer number)
* `colaborador_matricula` - Payroll registration number of the absent employee
* `colaborador_funcao_nome` - Name of the job role the absent employee performs
* `colaborador_locacao_nome` - Name of the work location the absent employee is assigned to
**Absence Attributes:**
* `isDiaInteiro` - Whether the absence covers the entire working day. When false, `duracao` holds the partial length
* `isJustificada` - Whether the absence was justified
* `isDescontar` - Whether the absence should be deducted from the employee's pay
* `duracao` - Length of a partial absence, as HH:mm
* `justificativa_id` - Identifier of the justification given, matching `justificativas.justificativa_id`
* `nome_justificativa` - Name of the justification given for the absence
* `descricao` - Free-text description entered alongside the absence
**Medical Certificate:**
* `cid` - ICD code (Classificação Internacional de Doenças) from the medical certificate
* `crm` - Registration number of the doctor who issued the medical certificate
* `especialidade_id` - Identifier of the medical speciality recorded for the absence
* `especialidade_medica` - Name of the medical speciality recorded for the absence
**Attachments and Audit:**
* `anexo` - URL of the document attached to the absence
* `observacao` - Note attached to the day of the absence, as a JSON object
* `rastreio` - Audit trail recording who created and last changed the absence and when, as a JSON object
Individual clock-in and clock-out punches. Supports incremental sync on `data_ponto`. This is by far the highest-volume stream.
**Key Fields:**
* `ponto_id` - Unique identifier of the punch record
* `data_ponto` - Moment the punch was registered. Used as the incremental replication key
* `tipo` - Direction of the punch: "Entrada" for a clock-in, "Saída" for a clock-out
* `colaborador_id` - Identifier of the employee who registered the punch
**Employee Information:**
* `nome` - Full name of the employee
* `cpf` - Employee's CPF (Brazilian taxpayer number)
* `matricula` - Payroll registration number of the employee
* `nomecontratante` - Name of the contracting entity or work site the employee is assigned to
**Capture Details:**
* `latitude` - Latitude captured by the device when the punch was registered, when location was available
* `longitude` - Longitude captured by the device when the punch was registered, when location was available
* `foto` - URL of the photo taken at the moment of the punch
# Facilita as a data source
Source: https://docs.nekt.com/sources/facilita
Bring data from the Facilita real estate CRM to your Lakehouse.
Facilita is a Brazilian CRM for real estate developers, land developers and agencies. The connector uses the Facilita Public API to extract your deals with their full details, custom fields and activity history, your sales funnels with their stages and statuses, your users and teams, your developments with their units, and your tasks and unit reservations. With it you can analyze your sales funnel, your inventory and your team's performance in your Lakehouse.
## Configuring Facilita as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Facilita option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Facilita gives access to your data through its **Public API**. The Public API is a feature of your Facilita plan: if it is not active in your account, ask Facilita to enable it.
An administrator of your Facilita account generates the credentials in the Facilita panel. You need three values:
* the **instance**: the exact name of your Facilita instance;
* the **API key**;
* the **user token**: the token of an administrator user.
The connector reads what the user of the token can see. Use the token of an administrator so that all deals, users and developments are extracted.
The following configurations are available:
* **Instance**: the exact name of your Facilita instance. Required.
* **API Key**: the Public API key generated in Facilita. Required.
* **User Token**: the token of an administrator user, generated together with the API key. Required.
* **Start Date** (advanced): the earliest last-update date of the deals read on the first sync. Later syncs continue from where the previous one stopped. Leave it empty to read the whole history. The other streams are always extracted in full.
* **Requests Per Minute** (advanced): how fast the connector may call Facilita. The default is 45. Facilita allows 50 to 150 requests per minute depending on your plan; when Facilita asks the connector to slow down, it waits and continues automatically.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Deal Details takes one request per deal. On the first sync of an account with many deals it makes the extraction noticeably longer (at the default pace, about 2,700 deals per hour); later syncs only read the deals that changed. Use **Start Date** to limit how far back the first sync goes.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: deals and deal details are INCREMENTAL: each sync brings the deals that changed since the previous one. The other streams are FULL\_TABLE: each sync replaces the table with what Facilita currently holds. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Facilita and their corresponding fields. API reference: [Facilita Public API](https://documenter.getpostman.com/view/733647/TzsYNpJ9).
Lists and objects are stored as JSON text, and timestamps are converted to UTC. Facilita has no list of customers: customer contact data comes with each deal, and the full customer record is in the `customer` column of Deal Details.
Deals (negócios) of your sales funnels, with their stage, value, customer contact and the people responsible. Incremental: each sync reads the deals updated since the previous one.
Table: `deals` · Primary key: `id` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the deal. |
| `customer_id` | Integer | Identifier of the customer the deal belongs to. |
| `cost` | Number | Value of the deal. |
| `funnel_id` | Integer | Identifier of the sales funnel the deal is in. |
| `funnel_name` | String | Name of the sales funnel the deal is in. |
| `funnel_stage_id` | Integer | Identifier of the funnel stage the deal is in. |
| `stage_name` | String | Name of the funnel stage the deal is in. |
| `funnel_stage_status_id` | Integer | Identifier of the status, within the stage, the deal is in. |
| `status_name` | String | Name of the status, within the stage, the deal is in. |
| `product_id` | Integer | Identifier of the development (empreendimento) of interest. |
| `product_name` | String | Name of the development (empreendimento) of interest. |
| `unity_id` | Integer | Identifier of the unit of interest in the development. |
| `customer_name` | String | Name of the customer. |
| `customer_email` | String | Email address of the customer. |
| `customer_phone` | String | Phone number of the customer. |
| `customer_mobile` | String | Mobile number of the customer. |
| `owner_id` | Integer | User id of the person responsible for the deal. |
| `owner_name` | String | Name of the person responsible for the deal. |
| `owner_email` | String | Email address of the person responsible for the deal. |
| `broker_id` | Integer | Broker (corretor) id responsible for the deal. |
| `manager_id` | Integer | Manager (gerente) id responsible for the deal. |
| `team_id` | Integer | Team (equipe) id responsible for the deal. |
| `sdr_id` | Integer | Pre-sales (SDR) id responsible for the deal. |
| `owners` | String (JSON) | Responsibility record of the deal, with owner, broker, manager, team and SDR (JSON object). |
| `created_at` | Datetime | When the deal was created (UTC). |
| `updated_at` | Datetime | When the deal was last updated (UTC), used as the incremental replication key. |
The full record of each deal read by Deals in the same sync: name, sale date, the custom fields filled on the deal, payment conditions and the activity history. Read with one request per deal, only for the deals that changed. Requires the Deals stream.
Table: `deal_details` · Primary key: `id` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the deal. |
| `name` | String | Name of the deal. |
| `cost` | Number | Value of the deal. |
| `observations` | String | Free-text notes on the deal. |
| `customer_id` | Integer | Identifier of the customer the deal belongs to. |
| `product_id` | Integer | Identifier of the development (empreendimento) of interest. |
| `unity_id` | Integer | Identifier of the unit of interest in the development. |
| `funnel_id` | Integer | Identifier of the sales funnel the deal is in, when Facilita sends it. |
| `funnel_stage_id` | Integer | Identifier of the funnel stage the deal is in. |
| `stage_name` | String | Name of the funnel stage the deal is in. |
| `funnel_stage_status_id` | Integer | Identifier of the status, within the stage, the deal is in. |
| `status_name` | String | Name of the status, within the stage, the deal is in. |
| `origin_id` | Integer | Identifier of the lead origin of the deal. |
| `channel_id` | Integer | Identifier of the channel the deal came from. |
| `queue` | String | Whether the deal came through the lead distribution queue. |
| `allow_fill` | Boolean | Whether the deal's form can be filled in. |
| `simulator_analysis` | Boolean | Whether the deal has a financing simulation under analysis. |
| `proposal_status_erp` | String | Status of the deal's proposal in the integrated ERP. |
| `proposal_provider_erp` | String | ERP the deal's proposal was sent to. |
| `proposal_identifier_erp` | String | Identifier of the deal's proposal in the integrated ERP. |
| `idle_color_id` | Integer | Identifier of the idleness rule currently matching the deal. |
| `idle_color_name` | String | Name of the idleness rule currently matching the deal. |
| `custom_fields` | String (JSON) | Custom fields of the deal with their filled values: group, field id, name, slug, type, value and repeater rows (JSON array). |
| `payment` | String (JSON) | Payment conditions configured for the deal (JSON array). |
| `procedure` | String (JSON) | Procedures (stage actions) of the deal (JSON array). |
| `options` | String (JSON) | Additional options of the deal (JSON). |
| `data` | String (JSON) | Additional data of the deal (JSON). |
| `customer` | String (JSON) | Customer of the deal with contact, documents, origin and owners (JSON object). |
| `owners` | String (JSON) | Responsibility record of the deal, with owner, broker, manager, team and SDR (JSON object). |
| `product` | String (JSON) | Development (empreendimento) of interest (JSON object). |
| `unity` | String (JSON) | Unit of interest in the development (JSON object). |
| `stage` | String (JSON) | Funnel stage the deal is in (JSON object). |
| `status` | String (JSON) | Status, within the stage, the deal is in (JSON object). |
| `tasks` | String (JSON) | Tasks linked to the deal (JSON array). |
| `tags` | String (JSON) | Tags of the deal (JSON array). |
| `historical` | String (JSON) | Activity history of the deal: stage changes, service records and system events (JSON array). |
| `reserves` | String (JSON) | Unit reservations made for the deal (JSON array). |
| `simulation` | String (JSON) | Financing simulations of the deal (JSON array). |
| `files` | String (JSON) | Files attached to the deal (JSON array). |
| `created_at` | Datetime | When the deal was created (UTC). |
| `updated_at` | Datetime | When the deal was last updated (UTC), used as the incremental replication key. |
| `moved_at` | Datetime | When the deal last moved between stages (UTC). |
| `saled_at` | Datetime | When the deal was marked as sold (UTC). |
| `deleted_at` | Datetime | When the deal was deleted (UTC). |
Sales funnels configured in your account. Extracted in full on every sync.
Table: `funnels` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the funnel. |
| `name` | String | Name of the funnel. |
| `slug` | String | URL-friendly name of the funnel. |
| `description` | String | Description of the funnel. |
| `situation` | String | Situation of the funnel (enabled, disabled). |
| `main` | Boolean | Whether this is the account's main funnel. |
| `stages` | String (JSON) | Stages of the funnel, as returned with it (JSON array). |
| `created_at` | Datetime | When the funnel was created (UTC). |
| `updated_at` | Datetime | When the funnel was last updated (UTC). |
| `deleted_at` | Datetime | When the funnel was deleted (UTC). |
Stages (etapas) of the sales funnel. Join `id` with `deals.funnel_stage_id`.
Table: `funnel_stages` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the stage. |
| `funnel_id` | Integer | Identifier of the funnel the stage belongs to, when Facilita sends it. |
| `name` | String | Name shown in the funnel. |
| `position` | Integer | Display order within its parent. |
| `allow_fill` | Boolean | Whether deals in it can have their form filled in. |
| `privacy` | String | Visibility rule (public, private, public\_private, private\_public). |
| `idleness` | String (JSON) | Idleness configuration (JSON). |
| `permissions` | String (JSON) | Users allowed to see it, when restricted (JSON object). |
| `options` | String (JSON) | Additional options (JSON). |
| `win_probability` | String | Win probability setting of the stage. |
Statuses inside each funnel stage. Join `id` with `deals.funnel_stage_status_id`.
Table: `funnel_stage_statuses` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the status. |
| `funnel_stage_id` | Integer | Identifier of the stage the status belongs to. |
| `name` | String | Name shown in the funnel. |
| `position` | Integer | Display order within its parent. |
| `allow_fill` | Boolean | Whether deals in it can have their form filled in. |
| `privacy` | String | Visibility rule (public, private, public\_private, private\_public). |
| `idleness` | String (JSON) | Idleness configuration (JSON). |
| `permissions` | String (JSON) | Users allowed to see it, when restricted (JSON object). |
| `options` | String (JSON) | Additional options (JSON). |
Idleness rules (ociosidades): how long a deal may stay still in a stage before it is flagged.
Table: `idle_rules` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the idleness rule. |
| `name` | String | Name of the idleness rule. |
| `color` | String | Color shown on deals matching the rule (hex). |
| `day` | Integer | Days without update that trigger the rule, for day-based rules. |
| `funnel_stage_id` | Integer | Stage the rule applies to. |
| `funnel_stage_status_id` | Integer | Status the rule applies to, when restricted to one. |
| `time_start` | Integer | Start of the idle time range the rule covers, in minutes. |
| `time_end` | Integer | End of the idle time range the rule covers, in minutes. |
| `created_at` | Datetime | When the rule was created (UTC). |
| `updated_at` | Datetime | When the rule was last updated (UTC). |
Users of your account, with their access group and their links to teams, managers and brokers.
Table: `users` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the user. |
| `first_name` | String | Name of the user. |
| `email` | String | Email address of the user. |
| `phone` | String | Phone number of the user. |
| `cpf` | String | CPF (Brazilian individual taxpayer id) of the user. |
| `creci` | String | CRECI (real estate broker license) of the user. |
| `photo` | String | Path of the user's profile photo. |
| `group` | String | Access group of the user (admin, manager, broker, supervisor or a custom profile). |
| `manager_id` | Integer | Manager id of the user, when the user is a manager. |
| `manager_team_id` | Integer | Team the user manages, when the user is a manager. |
| `manager_team_name` | String | Name of the team the user manages. |
| `broker_id` | Integer | Broker id of the user, when the user is a broker. |
| `broker_manager_id` | Integer | Manager id the broker reports to. |
| `broker_team_id` | Integer | Team the broker belongs to. |
| `broker_team_name` | String | Name of the team the broker belongs to. |
| `supervisor_id` | Integer | Supervisor id of the user, when the user is a supervisor. |
| `supervisor_team_id` | Integer | Team the user supervises. |
| `supervisor_team_name` | String | Name of the team the user supervises. |
Sales teams (equipes).
Table: `teams` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the team. |
| `name` | String | Name of the team. |
Developments (empreendimentos) registered in your account, with their settings and unit summary.
Table: `products` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the development. |
| `name` | String | Name of the development. |
| `code` | String | Internal code of the development. |
| `status` | String | Status of the development (active, inactive). |
| `quantity` | Integer | Number of units of the development. |
| `address` | String | Street address of the development. |
| `neighborhood_id` | Integer | Identifier of the neighborhood of the development. |
| `city` | String | City of the development. |
| `uf` | String | State (UF) of the development. |
| `cep` | String | Postal code (CEP) of the development. |
| `availability_id` | Integer | Identifier of the development's availability (unit inventory). |
| `availability_name` | String | Name of the development's availability. |
| `availability_type` | String | Layout of the availability (vertical for towers, horizontal for lots). |
| `availability_summary` | String (JSON) | Number of units per situation, e.g. available and reserved (JSON object). |
| `cover_id` | Integer | Identifier of the cover image. |
| `file_id` | Integer | Identifier of the main file. |
| `infos` | String (JSON) | Development settings: blocks, stages, reservation and simulation rules (JSON object). |
| `availability` | String (JSON) | Availability of the development with its settings (JSON object). |
| `custom_fields` | String (JSON) | Custom fields of the development (JSON object). |
| `files_json` | String (JSON) | Files of the development (JSON). |
| `google_maps` | String (JSON) | Map location of the development (JSON). |
| `items` | String (JSON) | Items (amenities) of the development (JSON array). |
| `cover` | String (JSON) | Cover image of the development (JSON object). |
| `created_at` | Datetime | When the development was created (UTC). |
| `updated_at` | Datetime | When the development was last updated (UTC). |
Units of each development (apartments, lots, houses), with price and commercial situation. Read for every development that has an availability. Requires the Developments stream.
Table: `availability_units` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the unit. |
| `availability_id` | Integer | Identifier of the availability the unit belongs to. |
| `product_id` | Integer | Identifier of the development the unit belongs to. |
| `number` | String | Number of the unit. |
| `block` | String | Block or tower of the unit. |
| `floor` | String | Floor of the unit. |
| `stage` | String | Construction stage of the unit. |
| `area` | Number | Area of the unit, in square meters. |
| `cost` | Number | Price of the unit. |
| `garage` | String | Parking spaces of the unit. |
| `situation` | String | Commercial situation of the unit (available, pre-reserve, reserved, reserved-permanetly, sale, unavailable). |
| `status` | String | Status of the unit record (active, inactive). |
| `observations` | String | Notes on the unit. |
| `integration_id` | String | Identifier of the unit in an integrated ERP. |
| `coord_x` | Number | Horizontal position of the unit on the development map. |
| `coord_y` | Number | Vertical position of the unit on the development map. |
| `pos_x` | Integer | Column of the unit in the availability grid. |
| `pos_y` | Integer | Row of the unit in the availability grid. |
| `infos` | String (JSON) | Additional unit information, such as the typology (JSON object). |
| `table_data` | String (JSON) | Sales table data of the unit (JSON). |
| `reserves` | String (JSON) | Reservations of the unit, without the embedded deal (JSON array). |
| `created_at` | Datetime | When the unit was created (UTC). |
| `updated_at` | Datetime | When the unit was last updated (UTC). |
Tasks (calls, e-mails, visits, WhatsApp, chats and notes), one row per task and user linked to it.
Table: `tasks` · Primary key: `task_id`, `user_id`, `user_mode` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `task_id` | Integer | Unique identifier of the task. |
| `user_id` | Integer | User linked to the task. |
| `user_mode` | String | Role of the user in the task (e.g. owner). |
| `task_title` | String | Title of the task. |
| `task_type` | String | Type of the task (mail, call, whatsapp, visit, observation, chat). |
| `task_status` | String | Status of the task (open, closed). |
| `task_for_date` | Date | Date the task is scheduled for. |
| `task_for_hour` | String | Time of day the task is scheduled for (HH:MM:SS). |
| `task_repeat` | String | Repetition setting of the task. |
| `task_notes` | String | Notes of the task. |
| `deal_id` | Integer | Deal the task belongs to. |
| `customer_id` | Integer | Customer the task belongs to. |
| `customer_name` | String | Name of the customer the task belongs to. |
| `task_created_at` | Datetime | When the task was created (UTC). |
| `task_updated_at` | Datetime | When the task was last updated (UTC). |
| `user_created_at` | Datetime | When the user was linked to the task (UTC). |
| `user_updated_at` | Datetime | When the user's link to the task was last updated (UTC). |
Unit reservations, with the unit, the customer, the deal and the users involved.
Table: `reserves` · Primary key: `id_reserve` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id_reserve` | Integer | Unique identifier of the reservation. |
| `status_reserve` | String | Status of the reservation (e.g. reserved, canceled). |
| `unity_id` | Integer | Unit reserved. |
| `unit_number` | String | Number of the unit reserved. |
| `unit_block` | String | Block or tower of the unit reserved. |
| `product_id` | Integer | Development the reserved unit belongs to. |
| `product_name` | String | Name of the development the reserved unit belongs to. |
| `product_status` | String | Status of the development the reserved unit belongs to. |
| `deal_id` | Integer | Deal the reservation was made for. |
| `customer_id` | Integer | Customer the reservation was made for. |
| `customer_name` | String | Name of the customer. |
| `customer_email` | String | Email address of the customer. |
| `customer_phone` | String | Phone number of the customer. |
| `customer_mobile` | String | Mobile number of the customer. |
| `customer_origin` | String | Origin of the customer. |
| `requestered_user_id` | Integer | User who requested the reservation. |
| `requestered_user_name` | String | Name of the user who requested the reservation. |
| `managed_user_id` | Integer | User who manages the reservation. |
| `managed_user_name` | String | Name of the user who manages the reservation. |
| `reserved_user_id` | Integer | User the unit is reserved for. |
| `reserved_user_name` | String | Name of the user the unit is reserved for. |
| `reserved_group_id` | Integer | Access group of the user the unit is reserved for. |
| `reserved_group` | String | Name of the access group of the user the unit is reserved for. |
| `expired` | Boolean | Whether the reservation has expired. |
| `expire_date` | String | Expiration date set for the reservation. |
| `expire_time` | String | Reservation duration or expiration time (HH:MM:SS). |
| `order_reservation_expiration_date` | Datetime | When the reservation expires or expired (UTC). |
| `reason` | String | Reason given when the reservation was canceled. |
| `observations` | String | Notes on the reservation. |
| `infos` | String (JSON) | Additional reservation information (JSON). |
| `unit` | String (JSON) | Unit reserved, with its availability (JSON object). |
| `reserve_created_at` | Datetime | When the reservation was created (UTC). |
| `reserve_updated_at` | Datetime | When the reservation was last updated (UTC). |
| `deleted_at` | Datetime | When the reservation was deleted (UTC). |
Definitions of the custom fields of deals, developments, units, teams and users. The values filled on each deal are in the `custom_fields` column of Deal Details.
Table: `custom_fields` · Primary key: `id`, `entity_type` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the custom field. |
| `entity_type` | String | What the field belongs to: Deal\_Main (deals), Product\_Main (developments), Availability\_Units (units), Team or User. |
| `name` | String | Label of the custom field. |
| `slug` | String | Machine name of the custom field. |
| `type` | String | Input type of the field (text, select, date, email, tel, file, textarea, repeater...). |
| `position` | Integer | Display order of the field. |
| `placeholder` | String | Placeholder text of the field. |
| `required` | Boolean | Whether the field is mandatory. |
| `default` | String | Default value of the field. |
| `limit` | Integer | Maximum length of the value. |
| `validation` | String (JSON) | Validation rule of the field (JSON). |
| `options` | String (JSON) | Field options, including the choices of select fields (JSON object). |
| `repeater` | String (JSON) | Sub-fields of a repeater field (JSON array). |
# Fathom as a data source
Source: https://docs.nekt.com/sources/fathom
Bring data from Fathom to your Lakehouse.
Fathom is an AI notetaker that joins calls on Zoom, Google Meet, Microsoft Teams and Slack huddles and turns each meeting into structured content: a summary, a full transcript, action items, highlighted moments and, when a CRM is connected, the CRM records the meeting was matched to. Nekt extracts this data through the Fathom External API.
For more info on the streams and permissions, check the [Fathom API documentation](https://developers.fathom.ai/).
## Configuring Fathom as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Fathom option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The connector authenticates with an API key issued from your own Fathom account. To create one, open Fathom, go to **Settings > API Access** (the direct link is [fathom.video/customize](https://fathom.video/customize#api-access-header)) and generate a key.
API keys are scoped to the user who creates them: the key reaches the meetings that user recorded plus the ones shared with them or their team, and never other users' private meetings. To sync the whole organization, generate the key with an account admin whose [view permissions](https://help.fathom.video/en/articles/10783489#understanding_permissions) cover every team. The `users` stream is admin-only and is skipped with a warning when the key belongs to a regular user.
The following configuration is required:
* **API key**: The key generated under Settings > API Access.
Optionally, under advanced settings:
* **Initial sync date**: The earliest date from which recordings will be synced. When omitted, the first sync fetches the full history.
* **Lookback (days)**: How far back before the last sync each incremental run re-reads, so content Fathom generates after a meeting ends is picked up. Defaults to 1 day.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Incremental sync
The meeting streams are replicated incrementally on `created_at`, the moment the recording was created in Fathom. Because Fathom writes the summary, transcript, action items and highlights only after a meeting ends, every run also re-reads a short lookback window before the last sync so that content is picked up once it exists. Records are keyed by `recording_id`, so re-reading a window never duplicates rows.
## Rate limits
Fathom allows 60 requests per minute overall, and a tighter 30 per minute for the requests that carry summaries and transcripts, a budget Fathom may temporarily lower during periods of high load. The connector paces itself below both limits and retries rate-limited requests, so the `Meeting summaries` and `Meeting transcripts` streams extract more slowly than the others. Selecting only the streams you need keeps extractions short.
# Streams and Fields
Below you'll find all available data streams from Fathom and their corresponding fields. The meeting streams are synced incrementally through the `created_at` field; the organization streams (meeting types, teams, team members, users) are extracted as full tables.
Fields holding nested data (invitees, transcripts and CRM matches) are delivered as JSON strings, ready to be parsed in a Query or Notebook.
Meetings recorded by Fathom, with their calendar, recording and ownership metadata. This is the base stream and the cheapest one to extract. Incremental sync is supported through the `created_at` field.
**Fields:**
* `recording_id` - Unique identifier of the meeting recording, used as the primary key
* `created_at` - Timestamp when the recording was created in Fathom (UTC), used as the incremental replication key
* `title` - Title Fathom gave the recording
* `meeting_title` - Title of the calendar event the meeting was recorded from
* `meeting_type` - Name of the meeting type assigned to this meeting, or empty when none is assigned
* `url` - URL of the recording in the Fathom web app
* `meeting_url` - Join URL of the underlying meeting (Zoom, Google Meet, Microsoft Teams or Slack huddle) taken from the calendar event. Empty when there is no associated calendar meeting
* `share_url` - Public share URL of the recording
* `scheduled_start_time` - Timestamp when the meeting was scheduled to start (UTC)
* `scheduled_end_time` - Timestamp when the meeting was scheduled to end (UTC)
* `recording_start_time` - Timestamp when the recording actually started (UTC)
* `recording_end_time` - Timestamp when the recording actually stopped (UTC)
* `recording_duration_seconds` - Length of the recording in seconds, computed from the recording start and end timestamps
* `scheduled_duration_seconds` - Length the meeting was scheduled for, in seconds, computed from the scheduled start and end timestamps
* `transcript_language` - Language code of the meeting transcript, for example `en`
* `calendar_invitees_domains_type` - Whether the calendar invitee list includes external email domains: `only_internal` or `one_or_more_external`
* `shared_with` - Who the meeting is shared with inside the organization: `no_teams` (private to the recorder), `single_team`, `multiple_teams` or `all_teams`
* `recorded_by_name` - Display name of the Fathom user who recorded the meeting
* `recorded_by_email` - Email address of the Fathom user who recorded the meeting
* `recorded_by_email_domain` - Email domain of the Fathom user who recorded the meeting
* `recorded_by_team` - Team the user who recorded the meeting belongs to
* `calendar_invitees` - People invited to the calendar event, as a JSON array of objects with name, email, email domain and whether they are external
* `invitee_count` - Number of people invited to the calendar event
* `external_invitee_count` - Number of invitees whose email domain is external to the organization
Fathom's generated write-up of each meeting, one row per recording. Fathom produces the summary only after the meeting ends, so it is empty for recordings still in progress. Incremental sync is supported through the `created_at` field.
**Fields:**
* `recording_id` - Unique identifier of the meeting recording, used as the primary key
* `created_at` - Timestamp when the recording was created in Fathom (UTC), used as the incremental replication key
* `title` - Title Fathom gave the recording
* `summary_template_name` - Name of the summary template Fathom used to write the summary, for example `general`
* `summary_markdown` - Summary of the meeting in Markdown. Fathom always writes it in English, regardless of the language spoken in the meeting
Full transcript of each meeting, one row per recording. The speaking turns are available both as a JSON array and as a plain-text rendering. Incremental sync is supported through the `created_at` field.
**Fields:**
* `recording_id` - Unique identifier of the meeting recording, used as the primary key
* `created_at` - Timestamp when the recording was created in Fathom (UTC), used as the incremental replication key
* `title` - Title Fathom gave the recording
* `transcript_language` - Language code of the transcript, for example `en`
* `transcript` - Speaking turns of the meeting, as a JSON array of objects with the speaker (display name and matched calendar invitee email), the spoken text and the timestamp relative to the start of the meeting
* `transcript_text` - Full transcript of the meeting as plain text, with each line prefixed by the timestamp and the name of the speaker
* `speaker_count` - Number of distinct speakers detected in the transcript
* `line_count` - Number of speaking turns in the transcript
Follow-up tasks identified in each meeting, one row per action item. Fathom does not give action items an identifier of their own, so rows are keyed by the recording plus the position of the item in the meeting's list. Incremental sync is supported through the `created_at` field.
**Fields:**
* `recording_id` - Unique identifier of the meeting recording, part of the primary key
* `created_at` - Timestamp when the recording was created in Fathom (UTC), used as the incremental replication key
* `title` - Title Fathom gave the recording
* `action_item_index` - Position of the action item within the meeting's list, part of the primary key
* `description` - What has to be done. Fathom always writes it in English, regardless of the language spoken in the meeting
* `user_generated` - Whether a person created the action item, as opposed to Fathom detecting it automatically
* `completed` - Whether the action item has been marked as done
* `recording_timestamp` - Moment of the recording the action item came from, as HH:MM:SS relative to the start of the recording
* `recording_playback_url` - URL that opens the recording in Fathom at the moment the action item came from
* `assignee_name` - Display name of the person the action item is assigned to
* `assignee_email` - Email address of the person the action item is assigned to
* `assignee_team` - Team of the person the action item is assigned to
Moments bookmarked during each meeting, one row per highlight. Fathom does not give highlights an identifier of their own, so rows are keyed by the recording plus the position of the highlight in the meeting's list. Incremental sync is supported through the `created_at` field.
**Fields:**
* `recording_id` - Unique identifier of the meeting recording, part of the primary key
* `created_at` - Timestamp when the recording was created in Fathom (UTC), used as the incremental replication key
* `title` - Title Fathom gave the recording
* `highlight_index` - Position of the highlight within the meeting's list, part of the primary key
* `type` - Label of the bookmark the highlight was created from, for example `Objection`
* `summary` - Short summary of the highlighted moment
* `text` - Full text of the highlight. Only present when it differs from the summary
* `start_time` - Start of the highlight, in seconds from the start of the recording
* `end_time` - End of the highlight, in seconds from the start of the recording
Contacts, companies and deals Fathom matched to each meeting in the CRM linked to your account or team. When no CRM is connected, the reason is reported in `crm_error` instead of the matches. Incremental sync is supported through the `created_at` field.
**Fields:**
* `recording_id` - Unique identifier of the meeting recording, used as the primary key
* `created_at` - Timestamp when the recording was created in Fathom (UTC), used as the incremental replication key
* `title` - Title Fathom gave the recording
* `crm_contacts` - CRM contacts matched to the meeting, as a JSON array of objects with name, email and the URL of the record in the CRM
* `crm_companies` - CRM companies matched to the meeting, as a JSON array of objects with name and the URL of the record in the CRM
* `crm_deals` - CRM deals matched to the meeting, as a JSON array of objects with name, amount and the URL of the record in the CRM
* `crm_contact_count` - Number of CRM contacts matched to the meeting
* `crm_company_count` - Number of CRM companies matched to the meeting
* `crm_deal_count` - Number of CRM deals matched to the meeting
* `crm_deal_amount_total` - Sum, in dollars, of the amounts of the CRM deals matched to the meeting
* `crm_error` - Reason Fathom could not return CRM matches, for example when no CRM is connected to the workspace
Meeting types configured for the organization, used to categorize meetings. Both active and inactive types are returned: inactive ones are no longer assignable but may still appear on historical meetings. Extracted as a full table.
**Fields:**
* `name` - Name of the meeting type, used as the primary key and as the value that identifies it on a meeting
* `status` - Whether the meeting type is currently assignable: `active` types can be assigned to new meetings, `inactive` ones only appear on historical ones
* `created_at` - Timestamp when the meeting type was created (UTC)
Teams configured in the Fathom organization. Extracted as a full table.
**Fields:**
* `name` - Name of the team, used as the primary key and as the value that identifies it on a meeting
* `created_at` - Timestamp when the team was created (UTC)
People belonging to the teams of the Fathom organization. Extracted as a full table.
**Fields:**
* `email` - Email address of the team member, used as the primary key
* `name` - Display name of the team member
* `created_at` - Timestamp when the team member joined the team (UTC)
Users of the Fathom account with their permissions. Fathom restricts this endpoint to account admins: when the API key belongs to a regular user, the stream is skipped with a warning instead of failing the extraction. Extracted as a full table.
**Fields:**
* `email` - Email address of the user, used as the primary key
* `name` - Display name of the user. Empty for users who have not accepted their invite yet
* `status` - Current state of the user in the account: `active`, `deactivated` or `invited`
* `created_at` - Timestamp when the user joined the account (UTC). For invited users, the date the invite was sent
* `settings_access_level` - How much of the account settings the user can manage: `none`, `team_admin` or `account_admin`. Empty for invited users
* `settings_access_teams` - Teams the settings access applies to: every team for an account admin, the administered teams for a team admin, empty otherwise
* `view_access_level` - Whose meetings the user can view: `own_meetings`, `team`, `multiple_teams` or `all_teams`. Empty for invited users
* `view_access_teams` - Teams whose meetings the user can view: every team for `all_teams`, the granted teams for `team` and `multiple_teams`, empty for `own_meetings`
# Feedz as a data source
Source: https://docs.nekt.com/sources/feedz
Bring data from Feedz to Nekt.
Feedz (TOTVS RH Clima e Engajamento) is a Brazilian HR platform for employee engagement and performance,
covering mood check-ins, engagement and e-NPS surveys, OKRs, celebrations, digital admissions, and gamification.
This connector extracts the company's people data and engagement history through the Feedz integration API.
## 1. Add your Feedz access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Feedz option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Token**: the API v2 integration key of your Feedz account. A Feedz **administrator** can generate it under **Configurações → Integrações → Chave de Integração API v2** (Settings → Integrations → API v2 Integration Key).
* **Start Date** (advanced): earliest date to sync for the incremental streams (`moods` and `notifications`). When omitted, the full history is synced.
* **Base URL** (advanced): override only to target the Feedz sandbox environment.
The integration key can only be generated by a Feedz system administrator. The Feedz API allows up to
100 requests every 30 seconds; the connector paces itself under this limit automatically.
3. Click **Next**.
## 2. Select your Feedz streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Some streams depend on specific Feedz modules (e.g. surveys, OKRs, digital admission, gamification). If a
module is not enabled for your account, the corresponding stream is skipped with a warning instead of failing
the sync; you can also simply leave those streams unselected.
2. Click **Next**.
## 3. Configure your Feedz data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Feedz data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Available streams
| Stream | Slug | Description |
| - | - | - |
| Employees | `employees` | All employees (collaborators) of the company, with department, branch, job and manager. |
| Employee Feedzcoins | `employee_feedzcoins` | Feedzcoins (gamification) balance of each employee. |
| Turnovers | `turnovers` | Dismissed employees with reason, type and last working day. |
| Departments | `departments` | Company departments. |
| Branches | `branches` | Company branches (business units). |
| Job Descriptions | `job_descriptions` | Job positions registered in the company. |
| Job Description Groups | `job_description_groups` | Groups of job positions. |
| Digital Admissions | `digital_admissions` | Digital admission (onboarding) processes. |
| Celebrations | `celebrations` | Celebrations posted on the Feedz feed, with likes and comments. |
| e-NPS Surveys | `enps_surveys` | History of e-NPS survey results. |
| Moods | `moods` | Daily mood check-ins of employees. |
| Notifications | `notifications` | Notifications sent to employees. |
| Objectives | `objectives` | Company objectives (OKRs) with key results and check-ins. |
| Complete Surveys | `complete_surveys` | Complete surveys ("Super Pesquisa") created in the company. |
| Complete Survey Answers | `complete_survey_answers` | Answers submitted to complete surveys. |
| Fast Surveys | `fast_surveys` | Fast surveys (quick polls) created in the company. |
| Fast Survey Answers | `fast_survey_answers` | Answers submitted to fast surveys. |
| Pulse Surveys | `pulse_surveys` | Recurring engagement (pulse) surveys. |
## Fields by stream
Below you'll find all available data streams from Feedz and their corresponding fields:
All employees (collaborators) of the company. **Sync:** Full table. **Primary key:** `employee_id`.
* `employee_id` - Unique identifier of the employee
* `name` - Display name of the employee
* `full_name` - Full legal name of the employee
* `description` - Free-text description of the employee's role and responsibilities
* `email` - Primary email address of the employee
* `cpf` - Brazilian individual taxpayer registry number (CPF) of the employee
* `remuneration` - Salary of the employee, as a decimal string
* `admission_at` - Date the employee was hired
* `birth_at` - Birth date of the employee
* `situation` - Current work situation of the employee (e.g. active, on vacation)
* `manager` - Name of the employee's manager
* `race` - Self-declared race/ethnicity of the employee
* `gender` - Gender of the employee
* `status` - Account status of the employee in Feedz (e.g. active)
* `company` - Company the employee belongs to (name, or JSON object in the expanded form)
* `company_data` - Company the employee belongs to (JSON object with id, name and description)
* `branch` - Branch/unit of the employee (JSON object with id and name)
* `department` - Department of the employee (name, or JSON object in the expanded form)
* `department_data` - Department of the employee (JSON object with id and name)
* `job_description` - Job position of the employee (JSON object with id, title and description)
* `direct_manager` - Direct manager of the employee (JSON object with id, name and email)
* `role` - Feedz access role of the employee (JSON object with id and name)
* `origin` - How the employee record was created (e.g. inserted manually, integration)
* `team` - Profile IDs of the members of the employee's team
* `groups` - Names of the groups the employee belongs to
Feedzcoins (gamification) balance of each employee. **Sync:** Full table. **Primary key:** `employee_id`.
* `employee_id` - Unique identifier of the employee (from the employees stream)
* `cpf` - Brazilian individual taxpayer registry number (CPF) of the employee
* `email` - Primary email address of the employee
* `feedzcoins` - Current Feedzcoins (gamification currency) balance of the employee
Dismissed employees (turnover records). **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the turnover (dismissal) record
* `reason` - Reason given for the dismissal
* `type` - Numeric code of the dismissal type
* `department` - Department the dismissed employee belonged to
* `last_day_working` - Last working day of the dismissed employee
* `profile` - Profile of the dismissed employee (JSON object with id, name, email, admission date and status)
Company departments. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the department
* `name` - Name of the department
Company branches (business units). **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the branch
* `name` - Name of the branch (business unit)
Job positions registered in the company. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the job position
* `title` - Title of the job position
* `description` - Description of the job position
* `id_job_group` - Identifier of the job group this position belongs to
Groups of job positions. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the job group
* `title` - Title of the job group
* `id_company` - Identifier of the company the job group belongs to
* `description` - Description of the job group
* `created_at` - Timestamp when the job group was created
* `updated_at` - Timestamp of the last update to the job group
* `deleted_at` - Timestamp when the job group was deleted, if applicable
* `jobs` - Job positions included in the group (JSON array of objects with id)
Digital admission (onboarding) processes. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the digital admission process
* `name` - Full name of the candidate being admitted
* `email` - Email address of the candidate being admitted
* `deadline_at` - Deadline for the candidate to complete the admission process
* `created_at` - Timestamp when the admission process was created
* `updated_at` - Timestamp of the last update to the admission process
* `profile_department` - Department assigned to the new hire (JSON object with id and name)
* `profile_job` - Job position assigned to the new hire (JSON object with id and name/title)
* `status` - Current status of the admission process (JSON object with id and name)
* `profile` - Feedz profile created for the new hire (JSON object with id and language)
Celebrations posted on the Feedz feed. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the celebration
* `id_profile_from` - Profile ID of the employee who posted the celebration
* `id_profile_to` - Profile ID of the employee being celebrated
* `comment` - HTML content of the celebration post
* `dt_create` - Timestamp when the celebration was posted
* `likes` - Likes received by the celebration (JSON array of objects)
* `comments` - Comments received by the celebration (JSON array of objects)
History of e-NPS survey results. **Sync:** Full table.
* `dt_create` - Date the e-NPS survey was created
* `dt_final` - Closing date of the e-NPS survey
* `departments` - Names of the departments covered by the survey
* `detractors` - Number of detractor responses (score 0-6)
* `passives` - Number of passive responses (score 7-8)
* `promoters` - Number of promoter responses (score 9-10)
* `enps` - Resulting e-NPS score (percentage of promoters minus detractors)
* `status` - Status of the survey (e.g. active)
Daily mood check-ins of employees. **Sync:** Incremental. **Replication key:** `date`.
* `mood` - Mood reported by the employee (e.g. very happy, happy)
* `description` - Optional comment left with the mood check-in
* `department` - Department of the employee at check-in time
* `branch` - Branch of the employee at check-in time
* `name` - Name of the employee who checked in
* `average` - Average mood score associated with the check-in
* `date` - Date of the mood check-in, used as the incremental replication key
Notifications sent to employees. **Sync:** Incremental. **Replication key:** `dt_created`.
* `id` - Unique identifier of the notification
* `link` - Link the notification points to inside Feedz
* `custom_message` - Custom message attached to the notification, when present
* `dt_created` - Timestamp when the notification was created, used as the incremental replication key
* `message` - Text of the notification
* `profile_from` - Profile that triggered the notification (JSON object with id and email)
* `profile_to` - Profile that received the notification (JSON object with id and email)
Company objectives (OKRs) with key results and check-ins. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the objective (OKR)
* `id_parent` - Identifier of the parent objective, or 0 when top-level
* `id_owner` - Profile ID of the objective's owner
* `id_company` - Identifier of the company the objective belongs to
* `id_department` - Identifier of the department the objective belongs to
* `objective` - Title of the objective
* `percentage_status` - Completion percentage of the objective, as a decimal string
* `status` - Numeric status code of the objective
* `private` - Whether the objective is private (1) or public (0)
* `people_allowed` - Profiles allowed to view the objective, when restricted
* `people_edit` - Profiles allowed to edit the objective
* `dt_start` - Start of the objective's period
* `dt_end` - End of the objective's period
* `id_period` - Identifier of the OKR period the objective belongs to
* `signaling` - Numeric signaling (health) indicator of the objective
* `tags` - Tags attached to the objective
* `parent` - Parent objective summary (JSON object with id and objective title)
* `key_results` - Key results of the objective, including their check-ins (JSON array of objects)
* `department` - Department of the objective (JSON object with id and name)
* `checkins` - Check-ins recorded on the objective's key results (JSON array of objects)
* `department_name` - Name of the department the objective belongs to
* `owner_name` - Name of the objective's owner
Complete surveys ("Super Pesquisa") created in the company. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the survey
* `id_company` - Identifier of the company that owns the survey
* `id_profile_created` - Profile ID of the user who created the survey
* `id_departments` - Comma-separated IDs of the departments targeted by the survey
* `id_profiles` - Comma-separated IDs of the profiles targeted by the survey
* `title` - Title of the survey
* `description` - Description of the survey
* `image` - URL of the image attached to the survey, when present
* `video` - URL of the video attached to the survey, when present
* `total_employees` - Number of employees targeted by the survey
* `anonymous` - Whether answers are anonymous (1) or identified (0)
* `dt_deadline` - Deadline for answering the survey
* `dt_created` - Date the survey was created
Answers submitted to complete surveys. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the answer
* `id_company` - Identifier of the company that owns the survey
* `id_survey` - Identifier of the survey the answer belongs to
* `id_question` - Identifier of the question being answered
* `id_profile` - Profile ID of the respondent (null for anonymous surveys)
* `id_profile_department` - Department ID of the respondent
* `id_profile_branch` - Branch ID of the respondent, when available
* `id_profile_role` - Feedz role ID of the respondent
* `answer` - Free-text answer, when the question accepts text
* `answer_option` - Identifier of the selected answer option, for choice questions
* `free_input_text` - Additional free-input text attached to the answer
* `hash` - Anonymous respondent hash, when the survey is anonymous
* `created_at` - Timestamp when the answer was submitted
Fast surveys (quick polls) created in the company. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the fast survey
* `id_company` - Identifier of the company that owns the fast survey
* `id_profile_created` - Profile ID of the user who created the fast survey
* `id_departments` - Comma-separated IDs of the departments targeted by the fast survey
* `question_old` - Previous text of the question, kept for history
* `question` - Text of the question asked
* `description_old` - Previous description of the fast survey, kept for history
* `description` - Description of the fast survey
* `type` - Numeric type code of the fast survey
* `status` - Numeric status code of the fast survey
* `dt_created` - Timestamp when the fast survey was created
* `dt_deadline` - Deadline for answering the fast survey
* `image` - URL of the image attached to the fast survey, when present
* `video` - URL of the video attached to the fast survey, when present
* `total_employees` - Number of employees targeted by the fast survey
* `anonymous` - Whether answers are anonymous (1) or identified (0)
Answers submitted to fast surveys. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the answer
* `id_fast_survey` - Identifier of the fast survey the answer belongs to
* `id_profile` - Profile ID of the respondent
* `score` - Score selected by the respondent, for score-type questions
* `answer_old` - Previous text of the answer, kept for history
* `answer` - Text of the answer
* `answer_option` - Identifier of the selected answer option, when applicable
* `dt_answer` - Timestamp when the answer was submitted
* `profile` - Respondent profile (JSON object with id, name and email)
Recurring engagement (pulse) surveys. **Sync:** Full table. **Primary key:** `id`.
* `id` - Unique identifier of the engagement survey
* `id_company` - Identifier of the company that owns the survey
* `id_profile` - Profile ID of the user who created the survey
* `id_departments` - Comma-separated IDs of the departments targeted by the survey
* `name` - Name of the engagement survey
* `description` - Description of the engagement survey
* `day_of_the_week` - Day of the week the survey is sent (numeric string)
* `number_questions` - Number of questions sent per round
* `free_text` - Whether free-text answers are enabled (1) or not (0)
* `ranges` - Number of points in the answer scale
* `ranges_text` - Labels of the answer scale points (JSON array of objects)
* `favorability` - Scale points considered favorable answers
* `anonymous_checkbox` - Whether respondents may choose to identify themselves (1) or not (0)
* `min_departament_to_show` - Minimum number of respondents required to show department results
* `is_default` - Whether this is the default engagement survey (1) or not (0)
* `is_suvery_climao` - Legacy flag for the climate survey type (kept as returned by the API)
* `is_survey_climao` - Whether this survey is a climate survey (1) or not (0)
* `status` - Numeric status code of the survey
* `minimum_admission_date_to_answer` - Minimum admission date an employee must have to receive the survey
* `created_at` - Timestamp when the survey was created
* `history` - History of survey rounds (JSON array of objects)
* `engagement_level` - Engagement level computed from the survey results
* `can_access` - Whether the requesting token can access the survey results
## Skills for agents
Feedz connector documentation as plain markdown, for use in AI agent contexts.
# Feegow as a data source
Source: https://docs.nekt.com/sources/feegow
Bring data from Feegow Clinic to Nekt.
Feegow Clinic is a Brazilian clinic and practice management platform, part of the Doctoralia group. It covers the appointment agenda, the patient register, staff and their specialties, the clinical catalogue, billing, the financial ledger and stock control. Nekt reads it through Feegow's public REST API.
## Configuring Feegow as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Feegow option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Feegow authenticates with a single **API token** sent on every request. Only the licence's **master user** can issue it; no other role can, and there is no self-service developer portal.
To get it, ask your master user to open Feegow Clinic, release API access for the licence in the settings area, and copy the token that is generated. Paste that value into Nekt.
The token is the licence's only credential and it grants access to every stream listed below, including patient and financial data. Treat it like a password and re-issue it in Feegow Clinic if it is ever exposed.
The following configurations are available:
* **API token**: the value Feegow generates for the licence, sent as the `x-access-token` header.
* **Start date**: (Default: `2020-01-01`) how far back the first sync reads the date-filtered streams (appointments, invoices, sales, quotes and lab reports). Later syncs re-read a trailing window instead of the whole history.
* **Lookback window (days)**: (Default: `90`) how many days back each sync re-reads once history has been loaded. See *How data is kept up to date* below: this setting is what picks up a record edited after it was first read.
* **Future window (days)**: (Default: `365`) how far into the future the agenda and accounts streams read. Appointments, schedule blocks and payables are booked ahead of today, so a window that stopped at today would never see them.
* **Request window (days)**: (Default: `30`) how many days each request to a date-filtered endpoint covers. Smaller windows mean more requests but smaller, faster responses.
* **Requests per minute**: (Default: `120`) the maximum number of requests the sync sends per minute. Feegow publishes no rate limit, so this is a safety belt; lower it if Feegow starts refusing requests.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Two streams, **Patient Dependants** and **Exam Requests**, are off by default because Feegow only answers them one patient at a time. Selecting them costs at least one request per patient, so the first sync walks your whole patient register and can take a long time. Later syncs only cover patients whose record changed.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How data is kept up to date
Feegow offers a "changed since" filter on exactly one endpoint: the patient register. Every other historical endpoint is filtered by the **business date** of the record (the day of the appointment, of the financial movement, of the quote), never by when that record was last edited. Nekt works with that as follows:
* **Patients** resume exactly where the previous sync stopped, using Feegow's own change filter.
* **Every other date-filtered stream** reads the full history on the first sync, from your **Start date**, and then re-reads a trailing window of **Lookback window (days)** on each later sync. Records are matched on their primary key, so re-reading updates rows instead of duplicating them.
* **Agenda and accounts streams** also read forward by **Future window (days)**, because appointments, schedule blocks and payables exist ahead of today.
A record edited long after its own date has left the lookback window is not picked up again. If, for example, someone corrects the status of an appointment from eight months ago and your lookback window is 90 days, that change will not reach your catalog. Widen the lookback window, or schedule an Additional Full Sync, when that matters for your reporting.
## Things to know about Feegow data
* **Nested data is stored as JSON text.** Fields holding a list or an object, such as an appointment's procedures, an insurer's plans, or an invoice's payments, are stored as JSON strings so that the table's columns stay stable when Feegow changes those structures. Use your warehouse's JSON functions to read them.
* **Amounts come in different units.** Feegow reports procedure, package and stock values **in cents**, while quotes and appointments are reported **in reais**. The field descriptions below say which is which; Nekt stores the number Feegow reported without converting it.
* **Staff tables include inactive people.** Feegow requires the active flag on every staff request, so Nekt asks for both values. That keeps historical appointments joinable to the professional who took them, even after that professional leaves the clinic.
* **Some modules are licensed separately.** The two Benefit Card streams read a different Feegow product. If your licence does not contract it, those streams are skipped with a message in the run log and the rest of the sync carries on normally.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Units | `units` | Clinic units (the headquarters and its branches). |
| Locations | `locations` | Rooms and other physical locations registered inside the units. |
| Specialties | `specialties` | Medical specialties offered for scheduling. |
| Insurances | `insurances` | Health insurers (convênios) the clinic accepts, with their plans. |
| Appointment Statuses | `appointment_statuses` | The status codes an appointment can hold, e.g. 3 = "Atendido". |
| Appointment Motives | `appointment_motives` | The reasons a booking can be cancelled or rescheduled with. |
| Appointment Channels | `appointment_channels` | The channels a booking can come in through. |
| Patient Sources | `patient_sources` | How a patient came to know the clinic (origem). |
| Private Price Tables | `private_tables` | Private price tables the clinic bills self-paying patients with. |
| Credit Card Brands | `credit_card_flags` | Credit-card brands accepted in the financial module. |
| Reports | `reports` | Reports the licence can generate from Feegow Clinic. |
| Employees | `employees` | Staff members registered in the clinic. |
| Professionals | `professionals` | Healthcare professionals who take appointments. |
| Professional Insurances | `professional_insurances` | Health insurers each professional accepts. |
| Patients | `patients` | Patients registered in the clinic. |
| Patient Dependants | `patient_dependents` | Dependants registered under a patient. *Off by default.* |
| Appointments | `appointments` | Appointments booked in the clinic. |
| Schedule Blocks | `schedule_locks` | Blocks that take slots off a professional's agenda (holidays, absences). |
| Procedure Types | `procedure_types` | The five kinds a procedure can be (consultation, exam, surgery...). |
| Procedures | `procedures` | The procedure catalogue and its list prices. |
| Procedure Groups | `procedure_groups` | Groups procedures are organised into for billing and reporting. |
| Procedure Packages | `procedure_bundles` | Packages that sell several procedures together at a bundled price. |
| Health Programmes | `health_programs` | Health programmes (programas de saúde) patients can be enrolled in. |
| Lab Reports | `lab_reports` | Lab reports (laudos) issued for patients. |
| Exam Requests | `exam_requests` | Exam orders (pedidos de exames) raised for a patient. *Off by default.* |
| Quotes | `proposals` | Treatment quotes (propostas) presented to patients. |
| Financial Invoices | `financial_invoices` | Accounts payable and receivable, with their payments and line items. |
| Sales | `sales` | Sales totals per invoice, as the financial module reports them. |
| Vouchers | `vouchers` | Discount vouchers issued by the clinic. |
| Suppliers | `suppliers` | Suppliers the clinic buys from. |
| Cost Centres | `cost_centers` | Cost centres the financial module books movements against. |
| Financial Categories | `financial_categories` | The chart of accounts (plano de contas). |
| Current Accounts | `current_accounts` | Bank and cash accounts the clinic settles movements through. |
| Products | `products` | Products registered in the stock module (materials, medicines, vaccines). |
| Stock Positions | `product_positions` | Where the stock sits right now: one row per batch, location and unit. |
| Benefit Card Plans | `benefit_card_plans` | Plans offered by the benefit-card programme. *Off by default.* *Requires the benefit-card module.* |
| Benefit Card Contracts | `benefit_card_contracts` | Benefit-card contracts signed by beneficiaries. *Off by default.* *Requires the benefit-card module.* |
## Fields by stream
Below you'll find all available data streams from Feegow and their corresponding fields:
Clinic units (the headquarters and its branches).
**Sync:** Full table · primary key `unidade_id`.
**Fields:**
* `unidade_id` - Unique identifier of the unit, referenced across the API as unidade\_id
* `is_headquarters` - True when Feegow returned this unit as the licence's headquarters (matriz) rather than as a branch
* `nome_fantasia` - Trading name of the unit
* `cnpj` - Brazilian company registration number (CNPJ) of the unit
* `endereco` - Street name of the unit's address
* `numero` - Street number of the unit's address
* `complemento` - Additional address line of the unit, e.g. floor or suite
* `bairro` - Neighbourhood of the unit's address
* `cidade` - City of the unit's address
* `estado` - Brazilian state (UF) of the unit's address
* `cep` - Postal code (CEP) of the unit's address
* `regiao` - Region the unit is grouped under in Feegow Clinic
* `zona` - Zone the unit is grouped under in Feegow Clinic
* `telefone_1` - Primary phone number of the unit
* `telefone_2` - Secondary phone number of the unit
* `email_1` - Primary e-mail address of the unit
* `email_2` - Secondary e-mail address of the unit
* `foto` - URL of the unit's photo, when one is registered
* `mapa` - URL of the unit's map location, when one is registered
* `exibir_agendamento_online` - Whether the unit is offered for online self-scheduling
Rooms and other physical locations registered inside the units.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the location, referenced by appointments.local\_id
* `local` - Name of the room or location, e.g. 'Sala 01'
* `unidade_id` - Identifier of the unit this location belongs to
Medical specialties offered for scheduling.
**Sync:** Full table · primary key `especialidade_id`.
**Fields:**
* `especialidade_id` - Unique identifier of the specialty
* `nome` - Name of the specialty, e.g. 'Médico urologista'
* `consulta_id` - Identifier of the procedure used for an in-person consultation in this specialty
* `consulta_online_id` - Identifier of the procedure used for an online consultation in this specialty
* `exibir_agendamento_online` - Whether the specialty is offered for online self-scheduling
* `codigo_tiss` - TISS code of the specialty, used when billing health insurers
Health insurers (convênios) the clinic accepts, with their plans.
**Sync:** Full table · primary key `convenio_id`.
**Fields:**
* `convenio_id` - Unique identifier of the health insurer
* `nome` - Name of the health insurer, e.g. 'Bradesco Saúde'
* `exibir_agendamento_online` - Whether the insurer is offered for online self-scheduling
* `permite_telemedicina` - Whether the insurer allows telemedicine appointments
* `registro_ans` - Registration number of the insurer with the Brazilian regulator (ANS)
* `cnpj` - Brazilian company registration number (CNPJ) of the insurer
* `endereco` - Street name of the insurer's address
* `numero` - Street number of the insurer's address
* `complemento` - Additional address line of the insurer
* `bairro` - Neighbourhood of the insurer's address
* `cidade` - City of the insurer's address
* `estado` - Brazilian state (UF) of the insurer's address
* `cep` - Postal code (CEP) of the insurer's address
* `planos` - Plans offered by this insurer, as a JSON array of objects with plano\_id and plano
The status codes an appointment can hold, e.g. 3 = "Atendido".
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the appointment status, referenced by appointments.status\_id
* `status` - Name of the status as shown in Feegow Clinic, e.g. 'Marcado - confirmado' or 'Não compareceu'
The reasons a booking can be cancelled or rescheduled with.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the cancellation or rescheduling reason
* `motivo` - Reason as shown in Feegow Clinic, e.g. 'Solicitado pelo paciente'
The channels a booking can come in through.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the booking channel, referenced by appointments.canal\_id
* `canal` - Name of the booking channel, e.g. 'Agendamento Online' or 'Clínica'
How a patient came to know the clinic (origem).
**Sync:** Full table · primary key `origem_id`.
**Fields:**
* `origem_id` - Unique identifier of the patient source
* `nome_origem` - Name of the source, e.g. 'Eventos' or 'Indicação'
Private price tables the clinic bills self-paying patients with.
**Sync:** Full table · primary key `tabela_id`.
**Fields:**
* `tabela_id` - Unique identifier of the private price table
* `nome_tabela` - Name of the private price table
Credit-card brands accepted in the financial module.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the card brand, referenced by payments as bandeira\_id
* `bandeira` - Name of the card brand, e.g. 'Visa' or 'Elo'
Reports the licence can generate from Feegow Clinic.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the report
* `ct` - Category the report is filed under in Feegow Clinic, e.g. 'Agenda'
* `relatorio` - Display name of the report
* `arquivo` - Slug of the report, the value Feegow expects when generating it
* `sys_active` - Whether the report is active in the system
* `permissoes` - Internal permission key required to run the report
* `status_relatorio_id` - Identifier of the report's availability status
* `nome_status` - Name of the report's availability status, e.g. 'Disponível'
* `cor_status` - Colour Feegow Clinic uses for the status badge
* `habilitado` - Whether the report is enabled for this licence
Staff members registered in the clinic.
**Sync:** Full table · primary key `employee_id`.
**Fields:**
* `employee_id` - Unique identifier of the employee
* `sys_user` - Identifier of the Feegow Clinic system user linked to this employee
* `nome` - Full name of the employee
* `email` - E-mail address of the employee
* `cpf` - Brazilian individual taxpayer number (CPF) of the employee
* `ativo` - Whether the employee is currently active in the clinic
Healthcare professionals who take appointments.
**Sync:** Full table · primary key `profissional_id`.
**Fields:**
* `profissional_id` - Unique identifier of the professional, referenced across the API as profissional\_id
* `sys_user` - Identifier of the Feegow Clinic system user linked to this professional
* `nome` - Full name of the professional
* `tratamento` - Form of address used before the name, e.g. 'Dr.' or 'Dra.'
* `rqe` - Specialist qualification registry number (RQE) of the professional
* `conselho` - Professional council the practitioner is registered with, e.g. CRM or CRO
* `documento_conselho` - Registration number with the professional council
* `uf_conselho` - Brazilian state (UF) that issued the council registration
* `foto` - URL of the professional's photo, when one is registered
* `sexo` - Gender recorded for the professional
* `email` - E-mail address of the professional
* `cpf` - Brazilian individual taxpayer number (CPF) of the professional
* `ativo` - Whether the professional is currently active in the clinic
* `age_restriction` - Patient age range the professional accepts, as a JSON array of objects with age\_from and age\_to
* `especialidades` - Specialties the professional practises, as a JSON array of objects with especialidade\_id, nome\_especialidade and CBOS
Health insurers each professional accepts.
**Sync:** Full table · primary key `profissional_id`, `convenio_id` · read per record of `professionals`.
**Fields:**
* `profissional_id` - Identifier of the professional who accepts this insurer
* `convenio_id` - Identifier of the health insurer accepted by the professional
* `registro_ans` - Registration number of the insurer with the Brazilian regulator (ANS)
* `nome` - Name of the health insurer
Patients registered in the clinic.
**Sync:** Incremental · replication key `alterado_em` · primary key `patient_id`.
**Fields:**
* `patient_id` - Unique identifier of the patient, referenced across the API as paciente\_id
* `nome` - Full name of the patient
* `nome_social` - Social name of the patient, when one is registered
* `nascimento` - Date of birth of the patient
* `bairro` - Neighbourhood of the patient's address
* `tabela_id` - Identifier of the private price table the patient is billed with
* `sexo_id` - Identifier of the patient's registered gender
* `email` - E-mail address of the patient
* `celular` - Mobile phone number of the patient
* `criado_em` - When the patient record was created, converted to UTC
* `alterado_em` - When the patient record was last changed, converted to UTC; used as the incremental replication key
* `programa_de_saude` - Health programmes the patient is enrolled in, as a JSON array of objects (only populated because the tap requests programa\_saude=1)
Dependants registered under a patient.
**Sync:** Full table · primary key `id` · read per record of `patients`.
**Fields:**
* `id` - Unique identifier of the dependant link
* `paciente_id_responsavel` - Identifier of the patient the dependant is registered under
* `nome` - Full name of the dependant
* `paciente_id` - Identifier of the dependant's own patient record
* `responsavel_financeiro` - Marks the dependant as the financially responsible party ('S' when they are)
Appointments booked in the clinic.
**Sync:** Incremental · replication key `agendado_em` · primary key `agendamento_id`.
**Fields:**
* `agendamento_id` - Unique identifier of the appointment
* `data` - Calendar day the appointment is scheduled for
* `horario` - Time of day the appointment starts, as HH:MM:SS in the clinic's local time
* `paciente_id` - Identifier of the patient the appointment belongs to
* `procedimento_id` - Identifier of the main procedure booked
* `status_id` - Identifier of the appointment's current status; see the appointment\_statuses stream
* `local_id` - Identifier of the room the appointment takes place in; see the locations stream
* `profissional_id` - Identifier of the professional who takes the appointment
* `agendado_por` - Name of the person who created the booking
* `notas` - Free-text notes attached to the appointment
* `agendado_em` - When the booking was created, converted to UTC; used as the incremental replication key
* `plano_id` - Identifier of the insurance plan the appointment is booked under
* `canal_id` - Identifier of the channel the booking came in through; see the appointment\_channels stream
* `convenio_id` - Identifier of the health insurer covering the appointment; empty for self-paying patients
* `unidade_id` - Identifier of the unit the appointment takes place in
* `tabela_id` - Identifier of the private price table applied to the appointment
* `nome_fantasia` - Trading name of the unit the appointment takes place in
* `retorno_id` - Identifier of the appointment this one is a follow-up of
* `especialidade_id` - Identifier of the specialty the appointment belongs to
* `dias_limite_retorno` - Number of days the patient has to book a free follow-up
* `permite_retorno` - Whether the procedure entitles the patient to a follow-up appointment
* `retorno_existe` - Whether a follow-up appointment has already been booked
* `retorno` - Whether this appointment is itself a follow-up
* `grupo_procedimento_id` - Identifier of the procedure group the booked procedure belongs to
* `encaixe` - Whether the appointment was squeezed into an already full agenda
* `telemedicina` - Whether the appointment is a telemedicine consultation
* `primeiro_agendamento` - Whether this is the patient's first ever appointment at the clinic
* `valor_total_agendamento` - Total value of the appointment in Brazilian reais, converted from the formatted string Feegow returns (e.g. 'R\$ 300,00')
* `procedimentos` - Every procedure booked in this appointment, as a JSON array of objects with procedimentoID and plano
Blocks that take slots off a professional's agenda (holidays, absences).
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the schedule block
* `date_start` - First day the block applies to
* `date_end` - Last day the block applies to
* `time_start` - Time of day the block starts, as HH:MM:SS in the clinic's local time
* `time_end` - Time of day the block ends, as HH:MM:SS in the clinic's local time
* `holiday_id` - Identifier of the holiday this block was generated from, when it was
* `professional_id` - Identifier of the blocked professional; 0 when the block applies to the whole unit
* `description` - Free-text reason registered for the block
* `week_day` - Days of the week the block repeats on, as a JSON array of weekday numbers
* `units` - Units the block applies to, as a JSON array of unit identifiers
The five kinds a procedure can be (consultation, exam, surgery...).
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the procedure type, referenced by procedures.tipo\_procedimento
* `tipo` - Name of the procedure type, e.g. 'Consulta', 'Exame' or 'Cirurgia'
The procedure catalogue and its list prices.
**Sync:** Full table · primary key `procedimento_id` · read per record of `procedure_types`.
**Fields:**
* `procedimento_id` - Unique identifier of the procedure
* `nome` - Name of the procedure
* `tipo_procedimento` - Identifier of the procedure type; see the procedure\_types stream
* `opcoes_agendamento` - Scheduling options configured for the procedure in Feegow Clinic
* `permite_agendamento_online` - Whether patients may book this procedure themselves online
* `preparo` - Preparation instructions given to the patient before the procedure
* `dias_retorno` - Number of days the patient has to book a free follow-up after this procedure
* `codigo` - Billing code of the procedure, typically the TUSS code
* `grupo_procedimento` - Identifier of the procedure group this procedure belongs to
* `tempo` - Expected duration of the procedure as registered in Feegow Clinic
* `valor` - List price of the procedure, reported by Feegow in cents
* `especialidade_id` - Specialties the procedure belongs to, as a JSON array of specialty identifiers
Groups procedures are organised into for billing and reporting.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the procedure group
* `nome_grupo` - Name of the procedure group
* `descricao_nfse` - Description used for this group on electronic service invoices (NFS-e)
* `procedimentos` - Procedures in the group, as a JSON array of objects with id and procedimento
* `modalidade` - Imaging modality of the group, as a JSON object with id and Modalidade
Packages that sell several procedures together at a bundled price.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the package
* `nome` - Name of the package
* `data_ultima_alteracao` - When the package was last changed, converted to UTC
* `procedimentos` - Procedures in the package with their bundled prices in cents, as a JSON array of objects with itemID, procedimentoID and valor
Health programmes (programas de saúde) patients can be enrolled in.
**Sync:** Full table · primary key `programa_id`.
**Fields:**
* `programa_id` - Unique identifier of the health programme
* `nome_programa` - Name of the health programme
* `tipo_programa_id` - Identifier of the programme type
* `programa_convenio_id` - Identifier of the health insurer the programme runs with
* `data_inicio_programa` - First day the programme is valid for
* `data_fim_programa` - Last day the programme is valid for
Lab reports (laudos) issued for patients.
**Sync:** Incremental · replication key `request_date` · primary key `lab_report_id`.
**Fields:**
* `lab_report_id` - Unique identifier of the lab report
* `request_date` - Day the report was requested; used as the incremental replication key
* `lab_report_date` - Day the report was issued
* `patient_name` - Full name of the patient the report belongs to
* `status_code` - Status code of the report's file, 0 meaning the file is available
* `status_message` - Human-readable status of the report's file, e.g. 'Arquivo disponível'
Exam orders (pedidos de exames) raised for a patient.
**Sync:** Full table · primary key `pedido_exame_id` · read per record of `patients`.
**Fields:**
* `pedido_exame_id` - Unique identifier of the exam order
* `tipo_pedido` - Kind of order: 1 for a standard order, 2 for a TISS SADT order
* `paciente_id` - Identifier of the patient the order was raised for
* `atendimento_id` - Identifier of the encounter the order was raised in
* `executante_id` - Identifier of the professional or unit that carries the exam out
* `exetutante_tipo` - Whether the performer is a 'Profissional' or another kind of provider (Feegow's own spelling of the field)
* `observacao_pedido` - Notes written on the order, as HTML
* `data_pedido` - When the order was raised, converted to UTC
* `indicacao_clinica` - Clinical indication justifying the exam
* `convenio_id` - Identifier of the health insurer covering the order
* `profissional_solicitante_id` - Identifier of the professional who requested the exam
* `data_solicitacao` - When the exam was requested, converted to UTC
* `guia_id` - Identifier of the insurance claim (guia) the order belongs to
* `exame_id` - Identifier of the exam, on orders that carry a single one
* `exame_codigo` - Billing code of the exam, on orders that carry a single one
* `descricao_exame` - Description of the exam, on orders that carry a single one
* `observacao_exame` - Notes on the exam, on orders that carry a single one
* `quantidade_exame` - Quantity requested, on orders that carry a single exam
* `pedido_exame` - Every exam on the order, as a JSON array of objects with ExameID, ExameCodigo, DescricaoExame, ObservacaoExame and QuantidadeExame
Treatment quotes (propostas) presented to patients.
**Sync:** Incremental · replication key `proposal_last_update` · primary key `proposal_id`.
**Fields:**
* `proposal_id` - Unique identifier of the quote
* `proposal_date` - Day the quote was created
* `paciente_id` - Identifier of the patient the quote was presented to
* `value` - Total value of the quote in Brazilian reais
* `proposer_id` - Identifier of the professional or employee who made the quote
* `proposer_name` - Name of the professional or employee who made the quote
* `proposer_role` - Whether the quote was made by a 'Profissional' or by another role
* `status` - Current status of the quote, e.g. 'Aguardando aprovação do cliente' or 'Rejeitada pelo cliente'
* `private_table_name` - Name of the private price table the quote was priced with
* `proposal_last_update` - When the quote was last changed, converted to UTC; used as the incremental replication key
* `procedimentos` - Procedures quoted, as a JSON object with a label and a data array of objects carrying procedimento\_id, nome, valor and desconto
* `unidade` - Unit the quote was issued by, as a JSON object with the unit's identifier and address
Accounts payable and receivable, with their payments and line items.
**Sync:** Incremental · replication key `data` · primary key `invoice_id`.
**Fields:**
* `invoice_id` - Identifier of the invoice, taken from the first entry of the detalhes list
* `movement_id` - Identifier of the financial movement behind the invoice
* `tipo_transacao` - Side of the ledger this invoice was read from: 'C' for accounts receivable, 'D' for accounts payable
* `data` - Date of the financial movement; used as the incremental replication key
* `valor` - Value of the financial movement
* `tipo_conta` - Identifier of the account type the movement belongs to
* `conta_id` - Identifier of the account the movement belongs to
* `responsavel` - Party responsible for the invoice
* `descricao` - Description registered for the movement
* `nfe` - Number of the fiscal invoice (nota fiscal) associated with the movement
* `data_nfe` - Issue date of the associated fiscal invoice
* `detalhes` - Every movement detail on the invoice, as a JSON array of objects
* `pagamentos` - Payments settling the invoice, as a JSON array of objects carrying the amount, method, card brand and transaction identifiers
* `itens` - Items billed on the invoice, as a JSON array of objects carrying the procedure, quantity, discount and execution data
Sales totals per invoice, as the financial module reports them.
**Sync:** Incremental · replication key `timestamp` · primary key `invoice_id`.
**Fields:**
* `invoice_id` - Identifier of the invoice the sale belongs to
* `timestamp` - When the sale was transacted, converted to UTC; used as the incremental replication key
* `amount` - Value of the sale, converted from the decimal string Feegow returns
* `type` - Kind of amount reported, e.g. 'gross'
Discount vouchers issued by the clinic.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the voucher
* `codigo` - Code patients redeem the voucher with
* `de` - First moment the voucher is valid from, converted to UTC
* `ate` - Last moment the voucher is valid until, converted to UTC
* `valor` - Face value of the voucher: an amount or a percentage, depending on tipo\_valor
* `tipo_valor` - Whether valor is an amount ('V') or a percentage
* `unidades` - Units the voucher is valid in, as a JSON array of objects with id and nomeEmpresa
* `limite` - Maximum number of times the voucher may be redeemed
* `expirado` - Whether the voucher's validity period has passed
* `desativado` - Whether the voucher was deactivated by the clinic
* `qtd_utilizado` - Number of times the voucher has been redeemed
* `utilizado` - Whether the voucher has been redeemed at least once
* `disponivel` - Whether the voucher can still be redeemed right now
Suppliers the clinic buys from.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the supplier
* `nome_fornecedor` - Name of the supplier
* `cnpj` - Brazilian company registration number (CNPJ) of the supplier
* `cep` - Postal code (CEP) of the supplier's address
* `endereco` - Street name of the supplier's address
* `numero` - Street number of the supplier's address
* `complemento` - Additional address line of the supplier
* `cidade` - City of the supplier's address
Cost centres the financial module books movements against.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the cost centre
* `name` - Name of the cost centre
The chart of accounts (plano de contas).
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the financial category
* `type` - Whether the category books income or expense
* `name` - Name of the financial category
* `position` - Position of the category in the chart of accounts, e.g. '2.1.4'
* `parent_id` - Identifier of the category this one sits under; 0 at the top level
* `description` - Description registered for the category
Bank and cash accounts the clinic settles movements through.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the account
* `account_name` - Name of the account, e.g. 'Tesouraria (Caixinha)'
* `account_type` - Kind of account: 1, 2 or 3 as defined by Feegow
* `api_enabled` - Whether the account is exposed to Feegow's API integrations
* `percentage_deducted` - Percentage deducted from movements settled through this account
* `empresa` - Identifier of the company (unit) the account belongs to
* `categoriad_taxa_id` - Identifier of the financial category fees on this account are booked to (Feegow's own spelling of the field)
Products registered in the stock module (materials, medicines, vaccines).
**Sync:** Full table · primary key `produto_id`.
**Fields:**
* `produto_id` - Unique identifier of the product
* `nome_produto` - Name of the product
* `codigo_produto` - Code the clinic registered the product under
* `tipo_produto` - Kind of item, e.g. 'Produto'
* `categoria_id` - Identifier of the product's category
* `nome_categoria` - Name of the product's category, e.g. 'Vacina'
* `fabricante_id` - Identifier of the product's manufacturer
* `nome_fabricante` - Name of the product's manufacturer
* `apresentacao_nome` - Name of the presentation the product is sold in
* `apresentacao_quantidade` - Quantity contained in one presentation
* `unidade_medida` - Unit of measure of the product, e.g. '001 - Ampola'
* `principio_ativo` - Active ingredient, for medicines
* `nome_localizacao` - Name of the storage location the product is kept in
* `tiss_cd` - TISS classification of the product, e.g. '02 - Medicamentos'
* `dias_aviso_validade` - How many days before expiry Feegow warns about this product
* `estoque_minimo` - Minimum stock level configured for the product
* `estoque_minimo_tipo` - Unit the minimum stock level is expressed in
* `estoque_maximo` - Maximum stock level configured for the product
* `estoque_maximo_tipo` - Unit the maximum stock level is expressed in
* `preco_compra` - Purchase price registered for the product
* `preco_venda` - Sale price registered for the product
Where the stock sits right now: one row per batch, location and unit.
**Sync:** Full table · primary key `posicao_id`.
**Fields:**
* `posicao_id` - Unique identifier of the stock position
* `produto_id` - Identifier of the product held in this position
* `nome_produto` - Name of the product held in this position
* `lote` - Batch number of the stock held
* `validade` - Expiry date of the batch
* `quantidade` - Quantity held, expressed in the product's presentation
* `quantidade_itens` - Quantity held, expressed in individual items
* `tipo_unidade` - Unit the quantity is expressed in
* `unidade_medida` - Unit of measure of the product
* `localizacao_id` - Identifier of the storage location the stock sits in
* `nome_localizacao` - Name of the storage location
* `unidade_id` - Identifier of the clinic unit holding the stock
* `nome_unidade` - Name of the clinic unit holding the stock
* `categoria_id` - Identifier of the product's category
* `nome_categoria` - Name of the product's category
* `fabricante_id` - Identifier of the product's manufacturer
* `nome_fabricante` - Name of the product's manufacturer
* `apresentacao_nome` - Name of the presentation the stock is counted in
* `cbid` - Bar code registered for the batch
* `responsavel` - Person registered as responsible for the position
* `preco_medio_compra` - Average purchase price of the stock held
* `valor_posicao` - Total value of the stock held in this position
Plans offered by the benefit-card programme.
**Sync:** Full table · primary key `id`.
**Fields:**
* `id` - Unique identifier of the plan (a UUID)
* `name` - Name of the plan
* `membership_value` - Joining fee charged to the plan holder
* `recurrence_value` - Recurring fee charged to the plan holder
* `dependency_membership_value` - Joining fee charged per dependant
* `dependency_recurrence_value` - Recurring fee charged per dependant
* `creation_date` - When the plan was created, converted to UTC
* `count_contract` - Number of contracts currently signed on this plan
Benefit-card contracts signed by beneficiaries.
**Sync:** Full table · primary key `contract_id`.
**Fields:**
* `contract_id` - Unique identifier of the contract
* `contract_date` - Day the contract was created
* `registration_number` - Registration number of the contract
* `contract_person_id` - Unique identifier of the person the contract belongs to
* `account_owner` - Identifier of the owner of the account the contract is billed to
* `name` - Full name of the beneficiary
* `phone` - Phone number of the beneficiary
* `document` - Identity document number of the beneficiary
* `birthdate` - Date of birth of the beneficiary
* `plan_id` - Identifier of the plan the contract is signed on
* `plan` - Name of the plan the contract is signed on
* `status_contract` - Current status of the contract
* `status_contract_id` - Identifier of the contract's current status
* `amount_membership` - Joining amount agreed on the contract
* `amount_recurrence` - Recurring amount agreed on the contract
* `recurrence_id` - Identifier of the recurring charge behind the contract
* `recurrence_value` - Value of the recurring charge
* `membership_value` - Value of the joining charge
* `contract_parameters` - Additional contract parameters, as a JSON object
* `status_recurrence_id` - Identifier of the recurring charge's status
* `status_recurrence_description` - Description of the recurring charge's status
* `eligible` - Whether the contract is currently eligible (Feegow reports 1 for yes, 0 for no)
* `final_status_description` - Final status description of the contract
# Firebird as a data source
Source: https://docs.nekt.com/sources/firebird
Bring data from a Firebird database to Nekt.
Firebird is an open-source relational database management system used by many ERP and accounting systems. A common use case is the **SCI Sistemas Contábeis "Único"** ERP, which exposes its Firebird database for read access through its *BI Connection* module.
## Configuring Firebird as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Firebird option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the connection credentials for your Firebird database.
The database user must have **read access** to the tables and views you want to extract. We recommend using a dedicated read-only user.
Because Nekt runs in the cloud, it needs network access to the Firebird port (default `3050`). If the database is on a private network, expose it securely, ideally through an **SSH tunnel** (see step 3), or a tightly firewalled port forward restricted to Nekt's egress IPs.
### 2. Configuration Options
* **Host**: The hostname or IP address of your Firebird server. Use `127.0.0.1` when connecting through an SSH tunnel.
* **Port**: The port number for your database connection. The default port for Firebird is `3050`.
* **Database File Path**: The path to the database file on the server, for example `C:/SCI/BANCO/VSCI.SDB`.
* **User**: The username for authenticating with the database.
* **Password**: The password for the database user.
#### Advanced Settings
* **Charset**: The connection charset. SCI Único databases use `WIN1252` (the default).
* **Role** (optional): The Firebird role to connect with. Leave blank if not used.
* **Filter Tables** (optional): An allowlist of table/view names to discover (case-insensitive). **Strongly recommended for large databases**, an SCI Único base has around 3,600 tables, and discovering all of them is slow. *Note: Firebird discovery lists every table, granted or not. A read-only user will see tables it cannot `SELECT`, and syncing them fails with `no permission for SELECT`. Use this filter to match your grants.*
* **Filter Table Prefixes** (optional): A list of name prefixes to discover (case-insensitive), for example `TEMPRESAS`, `VSUC`, `VRH`. Combined with Filter Tables as a union. Also highly recommended to restrict discovery to tables your user has permission to access.
* **Chunk Size**: The number of rows to fetch at a time. Default is `20000`. Set to `0` to fetch all rows at once (not recommended for large tables).
* **Use Batch Query**: Use keyset pagination with retry logic for large tables. Requires a primary key.
### 3. SSH Tunnel (optional)
If your Firebird database is behind a firewall or on a private network, you can connect through an SSH tunnel (bastion host).
Toggle the SSH Tunnel option to enabled.
* **SSH Host**: The hostname of your bastion/jump server.
* **SSH Port**: The SSH port (default: `22`).
* **SSH Username**: Your SSH username.
Use either a password or a private key:
* **SSH Password**: Password for the SSH server, OR
* **SSH Private Key**: PEM-formatted private key for key-based authentication. If your key requires a password to decrypt it, you must also provide the **SSH Password**.
When tunneling, set **Host** to `127.0.0.1` so the connection is routed through the tunnel to the remote database.
**Host resolution**: The **Host** field is resolved by the bastion server, not by the Nekt connector. If your SSH key is restricted by a `permitopen="HOST:PORT"` directive on the bastion, the **Host** value you configure must match that string exactly (a hostname vs. IP address mismatch will be refused as "administratively prohibited").
### 4. Select streams
After configuring your connection, Nekt will discover the available tables and views from your Firebird database (scoped by your table filters). Select the tables you want to sync.
Selecting only the columns you need significantly speeds up extraction over remote connections, since Firebird transfers each row over the wire.
### 5. Configure data streams
For each selected table, you can configure:
* **Replication method**: Choose between **Full Table** (complete sync each run) or **Incremental** (only new/updated rows based on a replication key column).
* **Replication key**: For incremental sync, select the column that tracks changes (typically a timestamp or auto-incrementing ID column).
### 6. Verify and save
Review your configuration and click **Save**. Nekt will begin syncing data from your Firebird database on the configured schedule.
## Supported Data Types
Firebird data types are automatically mapped to the appropriate types in Nekt's Catalog:
| Firebird Type | Nekt Type |
| - | - |
| `SMALLINT`, `INTEGER`, `BIGINT`, `INT128` | Integer |
| `FLOAT`, `DOUBLE PRECISION`, `DECFLOAT`, `NUMERIC`, `DECIMAL` | Number |
| `CHAR`, `VARCHAR`, `BLOB SUB_TYPE TEXT` | String |
| `DATE` | Date |
| `TIMESTAMP`, `TIMESTAMP WITH TIME ZONE` | DateTime |
| `TIME`, `TIME WITH TIME ZONE` | String |
| `BOOLEAN` | Boolean |
| `BLOB SUB_TYPE BINARY` | String (hex-encoded) |
## Implementation Notes
### SSH Tunnel Teardown & Authentication
To ensure runs gracefully terminate, the connector strictly handles SSH tunnel teardowns. FirebirdSQL sockets lack timeouts; if an SSH tunnel is dropped before pooled database connections are cleanly disposed, the underlying connection will block indefinitely on a network read, causing syncs to hang.
Additionally, SSH authentication exclusively uses the provided configurations. It intentionally bypasses any local or runtime SSH agents (`allow_agent=False`) and typical host key directories, guaranteeing that connections only succeed using your explicitly provided password or decrypted private key.
# Fireflies as a data source
Source: https://docs.nekt.com/sources/fireflies
Bring data from Fireflies to Nekt.
Fireflies is an AI-powered meeting assistant that records, transcribes, and analyzes conversations. It helps teams capture meeting notes, action items, and insights automatically, improving productivity and collaboration.
![]()
## 1. Add your Fireflies access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Fireflies option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
3. Click **Next**.
## 2. Select your Fireflies streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Fireflies data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Fireflies data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Fireflies connector documentation as plain markdown, for use in AI agent contexts.
# Firestore as a data source
Source: https://docs.nekt.com/sources/firestore
Bring data from Firestore to Nekt.
Firestore is Google's NoSQL cloud database designed for mobile and web applications. It provides real-time data synchronization, offline support, and automatic scaling, making it ideal for building responsive applications that need to work across multiple devices and platforms.
![]()
## Configuring Firestore as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Firestore option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Check the instructions next to each configuration option to discover where you can find the required parameters for the connection. These are the available configurations for this source:
* **Credentials file:** The credentials file for the service account linked to your Firestore project. Make sure the account has access to perform read operations on your collections. You should upload the JSON credentials file directly.
* **Database name:** The database name to extract data from. If not provided, the default database for the account will be used.
* **Batch size:** The number of documents to process in a single stream. Keep in mind higher batch sizes may cause timeouts when reading the document stream.
* **Subcollection extraction mode:** Determines how subcollections are handled by the connector.
* **Nested documents**: Recursively fetches subcollections for each parent collection, embedding subcollections within parent documents.
* **Collection group**: Extracts subcollections as separate streams, using Collection Group queries to speed up the extraction process.
* **None**: Ignores subcollections.
* **Filter collections by name:** This parameter allows you to filter only specific collections from the database. This is useful to speed up the discovery process when you're interested in just a few collections, but your database has a lot of available ones to be discovered.
* **Filter subcollections by name:** Filter nested subcollections to avoid extracting unnecessary data. This is useful when you just need a subset of subcollections from a given document. This configuration depends on the `Subcollection extraction mode` chosen.
* For `Nested documents` mode, you should use the notation `collection.sub_collection`, for example `conversations.messages` if you want to extract the subcollection `messages` from the top-level collection `conversations`. A wildcard is also accepted if you want to get all nested subcollections, for example `conversations.messages.*` will extract all nested subcollections under `conversations -> messages`.
* For `Collection group` mode you can simply enter the name of each subcollection you want to extract. Please note subcollections with the same name under different root level collections will be mapped to the same stream. It's a good practice to use unique names for subcollections to avoid this behavior.
* **Start date:** Starting point for incremental syncs (ISO-8601 format).
Once you're done, click **Next**.
### 2. Select streams
The next step is letting us know which streams you want to bring. Each stream available in that list corresponds to a top-level collection or subcollection on Firestore. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest the same name as the collection, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched. In order for that to work, you need to have a valid datetime or integer incremental field inside your documents.
* **Full table**: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog. However, keep in mind this increases resource usage such as computing time and storage.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Firestore streams correspond dynamically to the collections and subcollections present in your database.
All extracted streams have the following standard properties:
| Field | Type | Description |
| :- | :- | :- |
| `_id` | String | A string field corresponding to the document ID. |
| `document` | String | A stringified JSON version of the document payload. |
| `replication_key` | Datetime / Integer | For incremental syncs, this corresponds to the incremental key selected by the user during the stream configuration. |
| `path` | String | For subcollections extracted using Collection groups, a string indicating the full path to reach the document. |
# Implementation Notes
## Subcollections
The connector supports extracting subcollections. When configuring it, you need to explicitly specify which ones you want to extract.
The most efficient way for extracting subcollections is by using Collection Groups, which allows retrieving subcollections from different parent documents in a single query.
Imagine the following structure:
![]()
When using Collection Groups, the connector would retrieve all reviews from all restaurants in a single query, significantly speeding up the extraction process, especially for large collections with nested subcollections.
## Incremental vs Full Sync
Incremental sync is allowed for both collections and subcollections when using Collection group queries.
**Mixed-Type Replication Keys Warning**
Firestore queries are strictly type-sensitive. If you are using incremental sync with a date-time field, ensure that the replication key is consistently stored as a native Firestore `Timestamp` across all documents.
If your collection contains mixed types for the replication key (e.g., some documents store the date as a native `Timestamp` and others as a string like `"2024-01-01T00:00:00Z"`), a single incremental query will silently skip the documents that do not match the parsed type. Ensure data type consistency across your collections to prevent missing data during incremental runs.
### Best Practices
* **Configure indexes for Collection Groups**: Set up indexes for using `Collection group` queries for subcollections whenever possible. This significantly improves extraction time, saving time and resources.
Step-by-step to configure a collection group index on Firestore:
1. In your Firestore console, click on the `Indexes` tab.
2. Select the `Single field` tab.
3. On the `Exemptions` section click on `Add exemption`.
4. On the `Collection ID` field, enter the name of the subcollection.
5. On the `Field path` field, enter the name of the attribute you want to use for the incremental index (generally a timestamp or date field).
6. In `Query scope` mark the `Collection group` checkbox.
7. Click `Save`.
It takes a while for the changes to propagate, but once it's done you're good to go.
* **Consistent incremental keys**: If configuring your streams as incremental, make sure to include a date-time field that indicates when the document was last updated. This is necessary to ensure data integrity and consistency.
* **Explicit filtering**: Be explicit when defining filters for collections and subcollections to avoid extracting data that won't be useful for you. This helps reduce costs from both the cloud resources needed to perform the data extraction and Firestore itself. You can always add more streams later if needed.
## Skills for agents
Firestore connector documentation as plain markdown, for use in AI agent contexts.
# Flash as a data source
Source: https://docs.nekt.com/sources/flash
Bring time and attendance data from Flash to your Lakehouse.
Flash is a Brazilian HR and benefits platform. The connector uses the Flash API to extract data from the **Controle de Jornada** (time and attendance) module: the time punches of each employee, the payroll budgets (verbas) calculated from them, events such as justified absences, and the timetable (escala) each employee follows. It also extracts the companies of your economic group and their employees, so you can analyze working hours, overtime, and absences in your Lakehouse.
## Configuring Flash as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Flash option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The connector authenticates with a Flash **API key**. To generate it, an administrator of your company in Flash goes to [hros.flashapp.com.br](https://hros.flashapp.com.br/), opens **Configurações > Plataforma > Chaves de acesso programático**, and generates a key.
API access must be enabled by Flash for your company. If the option to generate a key is not available, contact Flash support ([empresa@flashapp.com.br](mailto:empresa@flashapp.com.br)) and ask them to enable API access.
The following configurations are available:
* **API Key**: the API key generated in Flash. Required.
* **Start Date**: the first day of history extracted. Time punches are read from this day on the first sync and continue from the last synced day afterwards. Budgets, events, and timetable allocations are read in full, month by month, from this date on every sync. Required.
* **Company IDs** (advanced): the Flash ids of the companies to extract. Leave it empty to extract every company of the economic group the API key belongs to.
* **Time Punches Lookback Days** (advanced): how many days before today the time punches are read again on every sync, so adjustments and approvals made after the punch are picked up. The default is 35, which covers the whole previous month.
* **Requests Per Minute** (advanced): how fast the connector may call Flash. The default is 120. When Flash asks the connector to slow down, it waits and continues automatically.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Flash returns time punches one day at a time, for one company at a time. On the first sync, the connector makes one request per day since the start date for each company, so an early start date makes that sync noticeably longer. Later syncs only read the recent days.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: time punches are INCREMENTAL: each sync brings the recent days again plus any new ones. The other streams are FULL\_TABLE: each sync replaces the table with what Flash currently holds. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Flash and their corresponding fields. API reference: [Flash API](https://docs.api.flashapp.services/).
Every table has a `raw_payload` column with the whole record exactly as Flash returned it, so information without a column of its own is still available. Lists and objects are stored as JSON text, and timestamps are converted to UTC.
Companies of the economic group the API key belongs to (or only those set in Company IDs). Every other table is read company by company.
Table: `companies` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the company in Flash. |
| `name` | String | Trade name of the company. |
| `legal_name` | String | Legal (registered) name of the company. |
| `registration_number` | String | Company registration number (CNPJ). |
| `active` | Boolean | Whether the company is active in Flash. |
| `logo` | String | URL of the company logo. |
| `address_street` | String | Street of the company address. |
| `address_number` | String | Number of the company address. |
| `address_complement` | String | Complement of the company address (suite, floor). |
| `address_district` | String | District (bairro) of the company address. |
| `address_city` | String | City of the company address. |
| `address_state` | String | State of the company address (e.g. SP). |
| `address_zip_code` | String | Postal code (CEP) of the company address. |
| `raw_payload` | String (JSON) | The whole record exactly as Flash returned it (JSON object), including fields that have no column of their own. |
Employees registered in each company. Use it to add names, e-mails and organization data to the time punches, which only carry the employee id, external id, PIS, and CPF.
Table: `employees` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the employee in Flash. |
| `company_id` | String | Identifier of the Flash company the record belongs to. |
| `name` | String | Full name of the employee. |
| `email` | String | Personal e-mail address of the employee. |
| `corporate_email` | String | Corporate e-mail address of the employee. |
| `document_number` | String | Employee's CPF (Brazilian individual taxpayer number). |
| `external_id` | String | Identifier of the employee in the company's own systems (external id). |
| `phone_number` | String | Phone number of the employee. |
| `status` | String | Status of the employee in Flash (e.g. ACTIVE, INACTIVE). |
| `cost_center_id` | String | Identifier of the cost center the employee belongs to. |
| `manager_id` | String | Identifier of the employee's direct manager (another employee id). |
| `invitation_date` | Datetime | When the invitation to the Flash app is (or was) sent to the employee. |
| `profile_picture` | String | URL of the employee's profile picture. |
| `group_ids` | Array of strings | Identifiers of the groups the employee belongs to. |
| `groups` | String (JSON) | Groups the employee belongs to (JSON array of objects with id and name). |
| `department_ids` | Array of strings | Identifiers of the departments the employee belongs to. |
| `role_ids` | Array of strings | Identifiers of the roles (cargos) assigned to the employee. |
| `raw_payload` | String (JSON) | The whole record exactly as Flash returned it (JSON object), including fields that have no column of their own. |
Time punches (marcações de ponto) of each employee, one row per employee and day; the punches of the day are in the `attendances` column. Each sync reads again the last days set in Time Punches Lookback Days (35 by default), so adjustments and approvals made after the punch are picked up.
Table: `time_punches` · Primary key: `company_id`, `employee_id`, `day` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `company_id` | String | Identifier of the Flash company the record belongs to. |
| `employee_id` | String | Identifier of the employee in Flash. |
| `day` | Date | Day the punches belong to (the day requested from Flash), used as the incremental replication key. |
| `date` | Datetime | Date of the punches as returned by Flash. |
| `external_id` | String | Identifier of the employee in the company's own systems (external id). |
| `pis` | String | Employee's PIS number (Brazilian social integration program id). |
| `document_number` | String | Employee's CPF (Brazilian individual taxpayer number). |
| `attendances` | String (JSON) | Punches of the day (JSON array of objects): time, attendanceStatus, approvalStatus, reason, offline (punched offline), followingDay (punch belongs to the next calendar day), isReceiptAvailable and nsr (sequential registration number of the punch). |
| `raw_payload` | String (JSON) | The whole record exactly as Flash returned it (JSON object), including fields that have no column of their own. |
Payroll budgets (verbas) calculated from the time tracking, per employee and month, such as overtime and absences. Every sync reads every month from the start date to the current month.
Table: `budgets` · Primary key: `company_id`, `employee_id`, `year`, `month`, `event_code`, `date` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `company_id` | String | Identifier of the Flash company the record belongs to. |
| `employee_id` | String | Identifier of the employee in the time and attendance module. |
| `employee_id_api` | String | Identifier of the employee as exposed by the Flash API (employeeIdApi). |
| `external_id` | String | Identifier of the employee in the company's own systems (external id). |
| `year` | Integer | Year of the payroll period. |
| `month` | Integer | Month of the payroll period (1-12). |
| `date` | Datetime | Date of the budget line. |
| `event_code` | String | Code of the payroll event (verba), as configured in Flash. |
| `event_description` | String | Description of the payroll event (e.g. overtime 50%, absences). |
| `event_type` | String | Type of the payroll event, as returned by Flash. |
| `event_value` | String | Value of the event as returned by Flash (text). |
| `event_decimal_value` | Number | Value of the event as a decimal number (e.g. hours as 1.5). |
| `event_value_in_hours_and_minutes` | String | Value of the event in hours and minutes (e.g. 01:30). |
| `raw_payload` | String (JSON) | The whole record exactly as Flash returned it (JSON object), including fields that have no column of their own. |
Time and attendance events, such as justified absences, medical certificates, and time off. Every sync reads every month from the start date to the current month; an event that spans several months appears once.
Table: `events` · Primary key: `company_id`, `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the event in Flash. |
| `company_id` | String | Identifier of the Flash company the record belongs to. |
| `employee_id` | String | Identifier of the employee the event belongs to. |
| `external_id` | String | Identifier of the employee in the company's own systems (external id). |
| `reason_id` | Integer | Identifier of the event reason (type of absence or occurrence). |
| `reason_api_id` | String | Identifier of the event reason as exposed by the Flash API. |
| `period_type` | String | How the event period is expressed (e.g. whole days or hours). |
| `start_date` | Datetime | When the event starts. |
| `end_date` | Datetime | When the event ends. |
| `duration` | Number | Duration of the event. |
| `justification` | String | Justification written for the event. |
| `authorization_text` | String | Text of the authorization given for the event. |
| `employee_authorization_id` | Integer | Identifier of the employee authorization linked to the event. |
| `approval_type` | String | How the event is approved, as returned by Flash. |
| `status` | String | Status of the event (e.g. cancelled). |
| `period_year` | Integer | Year of the period the event is accounted in. |
| `period_month` | Integer | Month of the period the event is accounted in (1-12). |
| `raw_payload` | String (JSON) | The whole record exactly as Flash returned it (JSON object), including fields that have no column of their own. |
Which timetable (escala) each employee follows, and from when. Every sync reads every month from the start date to the end of the current month; an allocation valid in several months appears once.
Table: `timetable_allocations` · Primary key: `company_id`, `employee_id`, `allocation_id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `company_id` | String | Identifier of the Flash company the record belongs to. |
| `employee_id` | String | Identifier of the employee in Flash. |
| `allocation_id` | Integer | Unique identifier of the allocation. |
| `external_id` | String | Identifier of the employee in the company's own systems (external id). |
| `employee_name` | String | Name of the employee. |
| `timetable_id` | Integer | Identifier of the timetable (escala) the employee is allocated to. |
| `timetable_code` | String | Code of the timetable. |
| `timetable_name` | String | Name of the timetable. |
| `allocation_start_date` | String | When the allocation starts, as returned by Flash. |
| `allocation_end_date` | String | When the allocation ends, as returned by Flash (empty when open-ended). |
| `raw_payload` | String (JSON) | The whole record exactly as Flash returned it (JSON object), including fields that have no column of their own. |
# Flowlu as a data source
Source: https://docs.nekt.com/sources/flowlu
Bring data from Flowlu to Nekt.
Flowlu is a comprehensive business management platform that combines CRM, project management, invoicing, and collaboration tools into a single suite. It helps businesses streamline operations, manage customer relationships, and track financial performance.
![]()
## Configuring Flowlu as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Flowlu option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Flowlu API Key and Subdomain for this connection. You can generate an API key in your Flowlu account under `Portal Settings` > `API Settings` > `Create`. Once you have it, add the account access and the connector configuration.
The following configurations are available:
* **API Key**: Your Flowlu API Key.
* **Subdomain**: The subdomain of your Flowlu account (e.g., 'my-company' for `my-company.flowlu.com`).
* **Start Date**: The earliest date from which records will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Flowlu and their corresponding fields:
Stream for managing CRM leads.
**Key Fields:**
* `id`: Unique identifier for the opportunity.
* `name`: Opportunity name.
* `budget`: Estimated opportunity value.
* `customer_id`: Identifier of the associated customer.
* `contact_id`: Identifier of the associated contact.
* `source_id`: Identifier of the opportunity source.
* `assignee_id`: Identifier of the current owner.
* `pipeline_id`: Identifier of the related pipeline.
* `pipeline_stage_id`: Identifier of the pipeline stage.
* `active`: Indicates whether the opportunity is active.
* `closing_status_id`: Identifier of the closing status.
* `start_date`: Timestamp when the opportunity started.
* `deadline`: Deadline for closing the opportunity.
* `closing_date`: Timestamp when the opportunity was closed.
* `created_date`: Timestamp when the opportunity was created.
* `updated_date`: Timestamp of the most recent update.
**Contact Information:**
* `contact_name`: Primary contact name.
* `contact_phone`: Primary contact phone number.
* `contact_mobile`: Primary contact mobile number.
* `contact_email`: Primary contact email address.
* `contact_web`: Primary contact website.
* `contact_position`: Primary contact job title.
* `contact_company`: Primary contact company name.
**Additional Fields:**
* `description`: Opportunity description.
* `merged_to_id`: Identifier of the opportunity into which this record was merged.
* `closing_comment`: Closing notes provided by the user.
* `ref`: External reference code.
* `ref_id`: External reference identifier.
* `created_by`: User who created the opportunity.
* `updated_by`: User who last updated the opportunity.
* `last_activity_id`: Identifier of the last recorded activity.
* `last_activity_time`: Timestamp of the last activity.
* `last_activity_model`: Entity type of the last activity.
* `ordering`: Ordering index among opportunities.
* `auto_calc`: Indicates whether totals are calculated automatically.
* `shipping_charge`: Shipping amount associated with the opportunity.
* `adjustment`: Additional adjustment amount.
* `adjustment_description`: Description of applied adjustments.
* `pricelist_id`: Identifier of the price list in use.
* `deleted_at`: Timestamp when the opportunity was soft-deleted.
* `extra_fields`: Custom fields returned for the opportunity as a JSON string.
Stream for managing CRM accounts.
**Key Fields:**
* `id`: Unique identifier for the CRM account.
* `type`: Account type code.
* `name`: Name of the account.
* `first_name`: First name when the account represents a person.
* `last_name`: Last name.
* `active`: Indicates whether the account is active.
* `created_date`: Timestamp when the account was created.
* `updated_date`: Timestamp of the most recent update.
* `owner_id`: Identifier of the account owner.
* `industry_id`: Identifier of the related industry.
* `account_category_id`: Identifier of the account category.
**Contact Information:**
* `phone`: Primary phone number.
* `phone2`: Secondary phone number.
* `phone3`: Tertiary phone number.
* `email`: Primary email address.
* `email_personal`: Personal email address.
* `web`: Account website URL.
* `telegram`: Telegram contact handle.
* `social_network_link_1`: Social network link 1.
* `social_network_link_2`: Social network link 2.
* `social_network_link_3`: Social network link 3.
* `social_network_link_4`: Social network link 4.
* `social_network_link_5`: Social network link 5.
* `social_network_link_6`: Social network link 6.
**Address Information:**
* `shipping_country`: Shipping country.
* `shipping_state`: Shipping state or region.
* `shipping_city`: Shipping city.
* `shipping_zip`: Shipping postal code.
* `shipping_address_line_1`: Shipping address line 1.
* `shipping_address_line_2`: Shipping address line 2.
* `shipping_address_line_3`: Shipping address line 3.
* `billing_country`: Billing country.
* `billing_state`: Billing state or region.
* `billing_city`: Billing city.
* `billing_zip`: Billing postal code.
* `billing_address_line_1`: Billing address line 1.
* `billing_address_line_2`: Billing address line 2.
* `billing_address_line_3`: Billing address line 3.
* `address`: Primary address string.
* `address_legal`: Legal address of the organization.
**Legal & Financial Information:**
* `name_legal`: Short legal name.
* `name_legal_full`: Full legal name.
* `VAT`: Primary VAT/tax identifier.
* `VAT1`: Secondary VAT/tax identifier 1.
* `VAT2`: Secondary VAT/tax identifier 2.
* `bank_details`: Banking details for the account.
* `organization_revenue`: Reported organization revenue.
* `organization_expenses`: Reported organization expenses.
* `organization_registration_date`: Organization registration date.
**Additional Fields:**
* `honorific_title_id`: Identifier of the honorific title.
* `middle_name`: Middle name.
* `description`: Account description or notes.
* `deleted_at`: Timestamp when the account was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
* `cf_1`: Custom field cf\_1 value.
* `cf_4`: Custom field cf\_4 value.
* `merged_to`: Identifier of the account this record merged into.
* `ref`: External reference code.
* `ref_id`: External reference identifier.
* `guid_1c`: 1C integration GUID.
* `timezone`: Account time zone.
* `organization_employee_count`: Number of employees in the organization.
* `last_activity_id`: Identifier of the last recorded activity.
* `last_activity_time`: Timestamp of the last recorded activity.
* `last_activity_model`: Entity type of the last activity.
Stream for managing CRM account categories.
**Key Fields:**
* `id`: Unique identifier for the account category.
* `name`: Account category name.
* `ordering`: Display ordering for the category.
* `created_by`: User who created the category.
* `created_date`: Timestamp when the category was created.
* `updated_by`: User who last updated the category.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the category was soft-deleted.
* `active`: Indicates whether the category is active.
Stream for managing company absence types.
**Key Fields:**
* `id`: Unique identifier for the absence type.
* `title`: Name of the absence type.
* `color`: Color associated with the absence type.
* `default_absence_duration`: Default duration in days.
* `count_holidays`: Indicates whether holidays are included when computing the absence duration.
* `created_by`: User who created the absence type.
* `created_date`: Timestamp when the type was created.
* `updated_by`: User who last updated the type.
* `updated_date`: Timestamp of the most recent update.
Stream for managing company absences.
**Key Fields:**
* `id`: Unique identifier for the absence record.
* `user_id`: Identifier of the employee who is absent.
* `start_date`: Timestamp when the absence starts.
* `end_date`: Timestamp when the absence ends.
* `original_end_date`: Original absence end timestamp before later adjustments.
* `absence_type_id`: Identifier of the absence type.
* `comment`: Comment provided for the absence.
* `created_by`: User who created the absence record.
* `created_date`: Timestamp when the record was created.
* `updated_by`: User who last updated the record.
* `updated_date`: Timestamp of the last update.
Stream for managing CRM emails.
**Key Fields:**
* `id`: Unique identifier for the email record.
* `email`: Email address associated with the account.
* `type`: Email type (for example, primary or alternate).
* `comment`: Comment added to the email record.
* `account_id`: Identifier of the CRM account linked to the email.
* `deleted_at`: Timestamp when the email record was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing CRM honorific titles.
**Key Fields:**
* `id`: Unique identifier for the honorific title.
* `name`: Name of the honorific title.
* `ordering`: Ordering index for display.
* `created_by`: User who created the honorific title.
* `created_date`: Timestamp when the honorific title was created.
* `updated_by`: User who last updated the honorific title.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the honorific title was soft-deleted.
* `active`: Indicates whether the honorific title is active.
Stream for managing CRM industries.
**Key Fields:**
* `id`: Unique identifier for the industry.
* `name`: Industry name.
* `ordering`: Display ordering for the industry.
* `created_by`: User who created the industry.
* `created_date`: Timestamp when the industry record was created.
* `updated_by`: User who last updated the industry.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the industry was soft-deleted.
* `active`: Indicates whether the industry is active.
Stream for managing CRM lead followers.
**Key Fields:**
* `id`: Unique identifier for the opportunity follower.
* `lead_id`: Identifier of the opportunity.
* `user_id`: Identifier of the user following the opportunity.
* `created_by`: User who created the follower record.
* `created_date`: Timestamp when the follower record was created.
* `updated_by`: User who last updated the follower record.
* `updated_date`: Timestamp of the most recent update.
* `extrafields`: Serialized extra fields returned by Flowlu.
Stream for managing CRM lead items.
**Key Fields:**
* `id`: Unique identifier for the opportunity item.
* `lead_id`: Identifier of the opportunity associated with the item.
* `item_id`: Identifier of the referenced product or service.
* `name`: Name of the product or service.
* `unit_price`: Unit price for the item.
* `quantity`: Quantity of the item.
* `total`: Total amount after taxes and discounts.
**Additional Fields:**
* `ordering`: Ordering index of the item within the opportunity.
* `description`: Description of the product or service.
* `unit`: Unit of measure for the item.
* `unit_id`: Identifier of the unit of measure.
* `tax1`: Tax 1 applied to the item.
* `tax1_included`: Indicates whether Tax 1 is included in the price.
* `tax1_amount`: Amount charged for Tax 1.
* `tax2`: Tax 2 applied to the item.
* `tax2_included`: Indicates whether Tax 2 is included in the price.
* `tax2_amount`: Amount charged for Tax 2.
* `discount_amount`: Absolute discount amount applied.
* `discount`: Discount value or percentage representation.
* `discount_is_percent`: Indicates whether the discount is a percentage.
* `sub_total`: Subtotal amount for the item.
* `tax_total`: Total tax amount applied to the item.
* `tax_displayed`: Tax label displayed on the item.
* `ref`: External reference for the item.
* `ref_id`: External identifier for the item.
* `type`: Item type code.
* `deleted_at`: Timestamp when the item was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing CRM loss reasons.
**Key Fields:**
* `id`: Unique identifier for the loss reason.
* `name`: Name of the loss reason.
* `ordering`: Display ordering for the reason.
* `created_by`: User who created the loss reason.
* `created_date`: Timestamp when the loss reason was created.
* `updated_by`: User who last updated the loss reason.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the reason was soft-deleted.
* `active`: Indicates whether the loss reason is active.
Stream for managing CRM phones.
**Key Fields:**
* `id`: Unique identifier for the phone record.
* `phone`: Phone number associated with the account.
* `phone_masked`: Masked representation of the phone number.
* `type`: Phone type (for example, primary or alternate).
* `old_phone`: Previously used phone number.
* `comment`: Comment associated with the phone record.
* `account_id`: Identifier of the CRM account linked to the phone.
* `deleted_at`: Timestamp when the phone record was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing CRM pipeline stages.
**Key Fields:**
* `id`: Unique identifier for the stage.
* `name`: Name of the stage.
* `pipeline_id`: Identifier of the pipeline associated with the stage.
* `color`: Highlight color used for the stage.
* `ordering`: Ordering index within the pipeline.
* `created_by`: User who created the stage.
* `created_date`: Timestamp when the stage was created.
* `updated_by`: User who last updated the stage.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the stage was soft-deleted.
* `active`: Indicates whether the stage is active.
Stream for managing CRM pipelines.
**Key Fields:**
* `id`: Unique identifier for the pipeline.
* `name`: Pipeline name.
* `description`: Pipeline description.
* `type_access`: Access type for the pipeline.
* `ordering`: Ordering index for display.
* `created_by`: User who created the pipeline.
* `created_date`: Timestamp when the pipeline was created.
* `updated_by`: User who last updated the pipeline.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the pipeline was soft-deleted.
* `active`: Indicates whether the pipeline is active.
Stream for managing CRM relations.
**Key Fields:**
* `id`: Unique identifier for the CRM relation.
* `parent_acc_id`: Identifier of the parent account.
* `child_acc_id`: Identifier of the child account.
* `relation_name`: Name of the relationship.
* `type`: Relationship type code.
* `created_by`: User who created the relationship.
* `created_date`: Timestamp when the relationship was created.
* `updated_by`: User who last updated the relationship.
* `updated_date`: Timestamp of the most recent update.
Stream for managing CRM sources.
**Key Fields:**
* `id`: Unique identifier for the opportunity source.
* `name`: Name of the opportunity source.
* `description`: Description of the source.
* `ordering`: Display order of the source.
* `created_by`: User who created the source.
* `created_date`: Timestamp when the source was created.
* `updated_by`: User who last updated the source.
* `updated_date`: Timestamp of the most recent update.
* `deleted_at`: Timestamp when the source was soft-deleted.
* `active`: Indicates whether the source is active.
Stream for managing project expenses.
**Key Fields:**
* `id`: Unique identifier for the projected expense.
* `project_id`: Identifier of the project associated with the expense.
* `name`: Name of the projected expense stage.
* `total`: Projected amount for the expense.
* `status_paid`: Payment status flag (0=unpaid, 1=partially paid, etc.).
* `org_id`: Identifier of the associated organization.
* `category_id`: Identifier of the expense category.
* `manager_id`: Identifier of the manager responsible for the expense.
* `project_stage_id`: Identifier of the related project stage.
* `created_by`: User who created the expense record.
* `created_date`: Timestamp when the record was created.
* `updated_by`: User who last updated the record.
* `updated_date`: Timestamp of the most recent update.
* `date`: Planned date for realizing the expense.
**Additional Fields:**
* `description`: Additional description for the expense.
* `org_account_id`: Identifier of the organization's financial account.
* `currency_id`: Identifier of the currency used.
* `ordering`: Display order of the projected expense.
* `crm_company_id`: Identifier of the CRM company related to the expense.
Stream for managing project money stages.
**Key Fields:**
* `id`: Unique identifier for the projected revenue stage.
* `project_id`: Identifier of the project associated with the forecast.
* `name`: Name of the projected revenue stage.
* `total`: Projected amount for this stage.
* `status_paid`: Payment status flag (0=unpaid, 1=partially paid, etc.).
* `paid_date`: Date on which the amount was paid.
* `plan_paid_date`: Planned payment date.
* `org_id`: Identifier of the related organization.
* `category_id`: Identifier of the financial category.
* `manager_id`: Identifier of the manager responsible for the forecast.
* `project_stage_id`: Identifier of the related project stage.
* `created_by`: User who created the revenue stage record.
* `created_date`: Timestamp when the record was created.
* `updated_by`: User who last updated the record.
* `updated_date`: Timestamp of the most recent update.
**Additional Fields:**
* `description`: Additional description for the forecast.
* `org_account_id`: Identifier of the organization's bank account.
* `currency_id`: Identifier of the currency used.
* `ordering`: Display order of the revenue stage.
* `crm_company_id`: Identifier of the linked CRM company.
* `invoice_id`: Identifier of the invoice linked to the stage.
Stream for managing project portfolios.
**Key Fields:**
* `id`: Unique identifier for the project portfolio.
* `name`: Name of the project portfolio.
* `description`: Portfolio description.
* `active`: Indicates whether the portfolio is active.
* `status`: Portfolio status code.
* `created_by`: User who created the portfolio.
* `created_date`: Timestamp when the portfolio was created.
* `updated_by`: User who last updated the portfolio.
* `updated_date`: Timestamp of the most recent update.
**Additional Fields:**
* `archive_date`: Timestamp when the portfolio was archived.
* `archive_description`: Reason or details for archiving.
* `ordering`: Ordering index of the portfolio in lists.
* `deleted_at`: Timestamp when the portfolio was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing project stage checkitems.
**Key Fields:**
* `id`: Unique identifier for the stage check item.
* `active`: Indicates whether the check item is active.
* `name`: Name of the check item.
* `ordering`: Ordering index of the check item within the stage.
* `stage_id`: Identifier of the associated project stage.
* `project_id`: Identifier of the associated project, when applicable.
* `checked`: Indicates whether the check item has been completed.
* `enddate`: Deadline for completing the check item.
* `responsible_id`: Identifier of the user responsible for the check item.
* `created_by`: User who created the check item.
* `created_date`: Timestamp when the check item was created.
* `updated_by`: User who performed the last update.
* `updated_date`: Timestamp of the most recent update.
**Additional Fields:**
* `description`: Detailed description of the check item.
* `report`: Default report content linked to the check item.
* `from_checkitem_id`: Identifier of the originating check item when duplicated.
* `deleted_at`: Timestamp when the check item was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing project stages.
**Key Fields:**
* `id`: Unique identifier for the project stage.
* `name`: Short name for the stage.
* `fullname`: Full descriptive name for the stage.
* `project_type_id`: Identifier of the associated project type.
* `ordering`: Ordering index of the stage inside the workflow.
* `color_class`: CSS color class applied in the interface.
* `description`: Detailed description of the project stage.
* `project_id`: Identifier of the specific project when applicable.
**Additional Fields:**
* `ref`: External reference for the stage.
* `ref_id`: External identifier for the stage.
* `color`: Color assigned to the stage.
* `deleted_at`: Timestamp when the stage was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing project types.
**Key Fields:**
* `id`: Unique identifier for the project type.
* `name`: Name of the project type.
* `description`: Description of the project template.
* `ordering`: Ordering index among available templates.
* `is_active`: Indicates whether the template is active.
* `tasks_workflow_id`: Identifier of the default task workflow for projects of this type.
**Additional Fields:**
* `report`: Default report template for the project type.
* `auto_startdate_plus_days`: Number of days automatically added to the start date.
* `auto_enddate_plus_days`: Number of days automatically added to the end date.
* `ref`: External reference for the project type.
* `ref_id`: External identifier for the project type.
* `create_tasks_next_stage`: Indicates whether tasks for the next stage are created automatically.
* `manual_close_checkitems`: Indicates whether check items must be closed manually.
* `description_project`: Default description applied to projects created from this template.
* `deleted_at`: Timestamp when the project type was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing projects.
**Key Fields:**
* `id`: Unique identifier for the project.
* `name`: Project name.
* `estimated_revenue`: Estimated revenue for the project.
* `estimated_expenses`: Estimated expenses for the project.
* `startdate`: Actual start timestamp for the project.
* `enddate`: Actual end timestamp for the project.
* `manager_id`: Identifier of the project manager.
* `stage_id`: Identifier of the current project stage.
* `project_type_id`: Identifier of the project template/type.
* `briefcase_id`: Identifier of the associated portfolio.
* `is_archive`: Indicates whether the project is archived.
* `priority`: Project priority level.
* `customer_id`: Identifier of the customer associated with the project.
* `created_by`: User who created the project.
* `created_date`: Timestamp when the project was created.
* `updated_by`: User who last updated the project.
* `updated_date`: Timestamp of the most recent project update.
**Billing Information:**
* `billing_type`: Billing type used for the project.
* `default_billing_rate`: Default billing rate configured for the project.
* `default_employee_cost_rate`: Default internal cost rate per employee.
* `default_bill_time_type`: Default billable time category.
* `default_time_cost_type`: Default time cost category.
* `use_default_invoice_split_type`: Indicates whether the default invoice split type is used.
* `default_only_completed_task`: Default setting controlling whether only completed tasks are billed.
* `bill_taxes`: Indicates whether taxes are billed for the project.
* `default_invoice_split_type_in_project`: Default invoice split type configured for the project.
* `default_invoice_item_format`: Default format for invoice line items.
* `use_billing_tasks`: Indicates whether dedicated billing tasks are used.
* `use_expenditure_part_tasks`: Indicates whether expenditure part tasks are enabled.
**Additional Fields:**
* `description`: Detailed project description.
* `ordering`: Display order for the project.
* `customer_crm_contact_id`: Identifier of the customer's CRM contact.
* `archive_date`: Timestamp when the project was archived.
* `workspace_id`: Identifier of the workspace that owns the project.
* `archive_description`: Explanation for why the project was archived.
* `crm_lead_id`: Identifier of the associated CRM lead.
* `uuid`: UUID that uniquely identifies the project.
* `ref`: External reference code for the project.
* `ref_id`: External reference identifier for the project.
* `tasks_workflow_id`: Identifier of the default workflow applied to tasks.
* `deleted_at`: Timestamp when the project was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
Stream for managing task collaborators.
**Key Fields:**
* `id`: Unique identifier for the collaborator record.
* `task_id`: Identifier of the related task.
* `user_id`: Identifier of the collaborating user.
* `group_id`: Identifier of the collaborator group.
* `created_by`: User who added the collaborator to the task.
* `created_date`: Timestamp when the collaborator was added.
* `updated_by`: User who last updated the collaborator record.
* `updated_date`: Timestamp of the most recent update.
Stream for managing task invoicing logs.
**Key Fields:**
* `id`: Unique identifier for the invoicing log entry.
* `task_id`: Identifier of the related task.
* `invoice_id`: Identifier of the associated invoice.
* `amount`: Amount captured in the invoicing log.
* `created_by`: User who created the invoicing log entry.
* `created_date`: Timestamp when the entry was created.
* `updated_by`: User who last updated the entry.
* `updated_date`: Timestamp of the most recent update.
Stream for managing task lists.
**Key Fields:**
* `id`: Unique identifier for the task list.
* `name`: Name of the task list.
* `user_id`: Identifier of the user responsible for the list.
* `created_date`: Timestamp when the list was created.
* `created_by`: User who created the list.
* `updated_by`: User who last updated the list.
* `updated_date`: Timestamp of the most recent update.
* `active`: Whether the list is currently active.
**Additional Fields:**
* `deleted_at`: Timestamp when the task list was soft-deleted.
* `color`: Color configured for the list.
* `icon`: Icon associated with the list.
* `ordering`: List position used for ordering.
Stream for managing task stages.
**Key Fields:**
* `id`: Unique identifier for the task stage.
* `name`: Name of the stage.
* `description`: Detailed description of the stage.
* `ordering`: Ordering index for the stage.
* `created_date`: Timestamp when the stage was created.
* `created_by`: User who created the stage.
* `updated_by`: User who last updated the stage.
* `updated_date`: Timestamp of the most recent update.
* `workflow_id`: Identifier of the associated workflow.
**Additional Fields:**
* `color`: Color configured for the stage.
* `task_status`: Task status linked to this stage.
* `deleted_at`: Timestamp when the stage was soft-deleted.
Stream for managing task-to-list links.
**Key Fields:**
* `id`: Unique identifier for the task-to-list link.
* `task_id`: Identifier of the linked task.
* `list_id`: Identifier of the linked list.
* `created_date`: Timestamp when the link was created.
* `created_by`: User who created the link.
* `updated_by`: User who last updated the link.
* `updated_date`: Timestamp of the most recent update.
Stream for managing task workflows.
**Key Fields:**
* `id`: Unique identifier of the workflow.
* `name`: Name of the workflow.
* `description`: Detailed workflow description.
* `ordering`: Display order within the workspace.
* `created_date`: Timestamp when the workflow was created.
* `created_by`: User who created the workflow.
* `updated_by`: User who last updated the workflow.
* `updated_date`: Timestamp of the last update to the workflow.
* `active`: Whether the workflow is currently active.
**Additional Fields:**
* `deleted_at`: Timestamp when the workflow was soft-deleted.
Stream for managing tasks.
**Key Fields:**
* `id`: Unique identifier for the task.
* `name`: Task name.
* `description`: Detailed task description.
* `parent_id`: Identifier of the parent task.
* `deadline`: Due timestamp for the task.
* `plan_start_date`: Planned start timestamp.
* `plan_end_date`: Planned end timestamp.
* `priority`: Task priority level.
* `responsible_id`: Identifier of the user responsible for the task.
* `time_estimate`: Estimated effort in minutes.
* `time_spent`: Time already spent in minutes.
* `created_date`: Timestamp when the task was created.
* `owner_id`: Identifier of the user who created the task.
* `updated_date`: Timestamp of the last update.
* `status`: Current task status code.
* `start_date`: Actual start timestamp.
* `closed_date`: Final closure timestamp.
* `project_stage_id`: Identifier of the related project stage.
* `crm_account_id`: Identifier of the linked CRM account.
* `workflow_id`: Identifier of the workflow associated with the task.
* `workflow_stage_id`: Identifier of the workflow stage.
**Additional Fields:**
* `report`: Task report text.
* `prev_task_id`: Identifier of the previous task in the sequence.
* `deadline_allowchange`: Indicates whether the due date can be changed.
* `task_checkbyowner`: Indicates whether the owner must review the task.
* `cost`: Cost associated with the task.
* `price`: Price associated with the task.
* `cost_type`: Cost type code.
* `price_type`: Price type code.
* `updated_by`: Identifier of the user who last updated the task.
* `status_firstviewdate`: Timestamp when the status was first viewed.
* `first_closed_date`: Timestamp of the first closure event.
* `closed_by`: Identifier of the user who closed the task.
* `return_count`: Number of returns or reopenings.
* `rating`: Task rating score.
* `status_updated_by`: Identifier of the user who updated the status.
* `status_updated_date`: Timestamp when the status was updated.
* `ref`: External reference code.
* `ref_id`: Identifier of the external reference.
* `module`: Associated Flowlu module.
* `model`: Associated Flowlu model.
* `model_id`: Identifier of the associated model.
* `project_checkitem_id`: Identifier of the related checklist item.
* `type`: Task type code.
* `report_complete`: Completion report text.
* `ordering`: Ordering index for presentation.
* `all_day`: Indicates whether the task spans the entire day.
* `uuid`: UUID that uniquely identifies the task.
* `public_template`: Indicates whether the task is a public template.
* `template_id`: Identifier of the associated template.
* `is_repeat`: Indicates whether the task is part of a repeat schedule.
* `is_recurrence`: Indicates whether the task is a recurrence instance.
* `recurrence_id`: Identifier of the recurrence series.
* `recurrence_parent_id`: Identifier of the parent task in the recurrence.
* `recurrence_original_start_date`: Original start timestamp for the recurrence series.
* `event_location`: Location string for the related calendar event.
* `event_color`: Color assigned to the calendar event.
* `event_busy_status`: Busy status value for the event.
* `event_access_type`: Access type setting for the event.
* `event_calendar_id`: Identifier of the calendar that holds the event.
* `event_type`: Event type code.
* `event_etag`: Event ETag used for synchronization.
* `event_sync_type`: Event synchronization type.
* `archive_status`: Archive status code for the task.
* `is_archive`: Indicates whether the task has been archived.
* `is_hidden`: Indicates whether the task is hidden.
* `deleted_at`: Timestamp when the task was soft-deleted.
* `extra_fields`: Additional custom fields as a JSON string.
## Skills for agents
Flowlu connector documentation as plain markdown, for use in AI agent contexts.
# Fly ERP as a data source
Source: https://docs.nekt.com/sources/fly-erp
Bring data from Fly ERP to Nekt.
Fly ERP is a comprehensive enterprise resource planning system that helps businesses manage their operations, including sales, inventory, financial transactions, and customer relationships.
![]()
## Configuring Fly ERP as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Fly ERP option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide the following credentials to connect to Fly ERP:
* **API URL**: The base URL of your Fly ERP instance (e.g., [https://empresa.fly](https://empresa.fly) erp.com.br)
* **Auth Token**: The bearer token to authenticate against the Fly ERP API service. Check [this doc](https://documenter.getpostman.com/view/24704188/2s8Yt1r8wu#authentication) to understand how to generate the token.
* **Start Date**: The earliest record date to sync for `Vendas` and `VendasItens` streams.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Fly ERP and their corresponding fields:
Stream for managing bank accounts and balances.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | Integer | Bank/account code |
| `descricao` | String | Bank/account description |
| `id` | Integer | Bank/account ID |
| `id_filial` | Integer | Branch ID |
| `saldo` | Number | Current balance |
Stream for managing customer and supplier information.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `bairro` | String | Neighborhood |
| `cep` | String | ZIP code |
| `codigo` | String | Customer/Supplier code |
| `codigo_cidade` | String | City code |
| `contato_nome` | String | Contact name |
| `contato_telefone` | String | Contact phone |
| `contatos` | Array | List of contacts (nome, telefone, email, vinculo\_funcao) |
| `cpf_cnpj` | String | CPF/CNPJ (tax ID) |
| `data_cadastro` | String | Registration date |
| `data_ultima_compra` | String | Last purchase date |
| `email` | String | Email address |
| `endereco` | String | Address |
| `enquadramento_pessoa` | String | Person classification |
| `estado` | String | State |
| `id` | Integer | Customer/Supplier ID |
| `id_condicao_pagamento` | Integer | Payment condition ID |
| `id_forma_pagamento` | Integer | Payment method ID |
| `id_importacao` | Integer | Import ID |
| `id_operacao_saida` | Integer | Output operation ID |
| `id_vendedor` | Integer | Sales representative ID |
| `ids_vendedores` | Array | List of sales representative IDs |
| `ie_rg` | String | State registration/ID |
| `nome_fantasia` | String | Trade name |
| `ramo_atividade` | String | Business activity |
| `razao_social` | String | Company name |
| `situacao` | String | Status (A=Active, I=Inactive) |
| `telefone` | String | Primary phone |
| `telefone2` | String | Secondary phone |
Stream for managing payable accounts.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `codigo_banco_conta` | Integer | Bank account code |
| `codigo_categoria` | Integer | Category code |
| `codigo_fornecedor` | Integer | Supplier code |
| `codigo_plano_contas` | Integer | Chart of accounts code |
| `data_emissao` | String | Issue date |
| `data_ultimo_pagamento` | String | Last payment date |
| `data_vencimento` | String | Due date |
| `descricao_banco_conta` | String | Bank account description |
| `descricao_categoria` | String | Category description |
| `descricao_plano_contas` | String | Chart of accounts description |
| `id` | Integer | Account ID |
| `id_filial` | Integer | Branch ID |
| `id_fornecedor` | Integer | Supplier ID |
| `nome_fornecedor` | String | Supplier name |
| `numero` | String | Account number |
| `numero_nota` | String | Invoice number |
| `observacao` | String | Notes |
| `parcela` | Integer | Installment number |
| `valor` | Number | Account value |
| `valor_aberto` | Number | Open amount |
| `valor_desconto_pago` | Number | Paid discount amount |
| `valor_juros_pago` | Number | Paid interest amount |
| `valor_pago` | Number | Paid amount |
Stream for managing receivable accounts.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `codigo_banco_conta` | Integer | Bank account code |
| `codigo_categoria` | Integer | Category code |
| `codigo_cliente` | Integer | Customer code |
| `codigo_plano_contas` | Integer | Chart of accounts code |
| `cpf_cnpj_cliente` | String | Customer CPF/CNPJ |
| `data_emissao` | String | Issue date |
| `data_ultimo_pagamento` | String | Last payment date |
| `data_vencimento` | String | Due date |
| `descricao_banco_conta` | String | Bank account description |
| `descricao_categoria` | String | Category description |
| `descricao_plano_contas` | String | Chart of accounts description |
| `id` | Integer | Account ID |
| `id_cliente` | Integer | Customer ID |
| `id_filial` | Integer | Branch ID |
| `id_nota` | Integer | Invoice ID |
| `link_boleto` | String | Bank slip link |
| `link_fatura` | String | Invoice link |
| `nome_cliente` | String | Customer name |
| `numero` | String | Account number |
| `numero_nota` | String | Invoice number |
| `observacao` | String | Notes |
| `parcela` | Integer | Installment number |
| `url_pagamento` | String | Payment URL |
| `valor` | Number | Account value |
| `valor_aberto` | Number | Open amount |
| `valor_desconto_pago` | Number | Paid discount amount |
| `valor_juros_calculado` | Number | Calculated interest |
| `valor_juros_pago` | Number | Paid interest amount |
| `valor_pago` | Number | Paid amount |
Stream for tracking financial movements.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | Integer | Transaction code |
| `codigo_banco_conta` | Integer | Bank account code |
| `codigo_categoria` | Integer | Category code |
| `codigo_cliente` | Integer | Customer code |
| `codigo_plano_contas` | Integer | Chart of accounts code |
| `complemento` | String | Additional details |
| `data` | String | Transaction date |
| `data_emissao` | String | Issue date |
| `data_ultimo_pagamento` | String | Last payment date |
| `data_vencimento` | String | Due date |
| `descricao_banco_conta` | String | Bank account description |
| `descricao_categoria` | String | Category description |
| `descricao_plano_contas` | String | Chart of accounts description |
| `id` | Integer | Transaction ID |
| `id_cliente` | Integer | Customer ID |
| `id_contas_a_pagar` | Integer | Accounts payable ID |
| `id_contas_a_receber` | Integer | Accounts receivable ID |
| `id_filial` | Integer | Branch ID |
| `link_boleto` | String | Bank slip link |
| `nome_cliente` | String | Customer name |
| `nome_filial` | String | Branch name |
| `numero` | String | Transaction number |
| `numero_nota` | String | Invoice number |
| `observacao` | String | Notes |
| `parcela` | Integer | Installment number |
| `valor` | Number | Transaction value |
| `valor_aberto` | Number | Open amount |
| `valor_desconto_pago` | Number | Paid discount amount |
| `valor_juros_calculado` | Number | Calculated interest |
| `valor_juros_pago` | Number | Paid interest amount |
| `valor_pago` | Number | Paid amount |
Stream for managing product groups and subgroups.
Primary key: `id_grupo`
| Field | Type | Description |
| :- | :- | :- |
| `codigo_grupo` | String | Group code |
| `descricao_grupo` | String | Group description |
| `id_grupo` | Integer | Group ID |
| `subgrupos` | Array | Array of subgroups containing: `codigo_subgrupo`, `descricao_subgrupo`, `id_subgrupo` |
Stream for managing products and services.
Primary key: `id_produto`
| Field | Type | Description |
| :- | :- | :- |
| `codigo_alfa_numerico` | String | Alphanumeric code |
| `codigo_numerico` | String | Numeric code |
| `custo_medio` | Number | Average cost |
| `custo_ultima_compra` | Number | Last purchase cost |
| `estoque` | Number | Stock quantity |
| `id_grupo` | Integer | Group ID |
| `id_marca` | Integer | Brand ID |
| `id_produto` | Integer | Product ID |
| `id_subgrupo` | Integer | Subgroup ID |
| `nome_grupo` | String | Group name |
| `nome_marca` | String | Brand name |
| `nome_produto` | String | Product name |
| `nome_subgrupo` | String | Subgroup name |
| `preco_venda` | Number | Sale price |
| `quantidade_reservada` | Number | Reserved quantity |
| `referencia_principal` | String | Main reference |
| `tipo` | String | Type (product or service) |
| `unidade_medida` | String | Unit of measure |
Stream for managing sales invoices.
Primary key: `id_nota`
| Field | Type | Description |
| :- | :- | :- |
| `codigo_cliente` | Integer | Customer code |
| `data_emissao_nota` | String | Issue date |
| `id_nota` | Integer | Invoice ID |
| `numero_nota` | String | Invoice number |
| `numero_pedido_venda` | String | Sales order number |
| `razao_social_cliente` | String | Customer company name |
| `serie_nota` | String | Invoice series |
| `tipo_nota` | String | Invoice type |
| `valor_nota` | Number | Invoice total value |
Stream for managing sales invoice items.
Primary key: `id_item`
| Field | Type | Description |
| :- | :- | :- |
| `codigo_cliente` | Integer | Customer code |
| `codigo_produto_alfanumerico` | String | Product alphanumeric code |
| `codigo_produto_numerico` | String | Product numeric code |
| `data_emissao_nota` | String | Issue date |
| `id_item` | Integer | Item ID |
| `id_nota` | Integer | Invoice ID |
| `id_produto` | Integer | Product ID |
| `nome_produto` | String | Product name |
| `numero_nota` | String | Invoice number |
| `quantidade_item` | Number | Item quantity |
| `razao_social_cliente` | String | Customer company name |
| `serie_nota` | String | Invoice series |
| `tipo_nota` | String | Invoice type |
| `valor_desconto_item` | Number | Item discount value |
| `valor_desconto_nota_item` | Number | Invoice discount value for item |
| `valor_nota` | Number | Invoice total value |
| `valor_total_item` | Number | Item total value |
| `valor_unitario_item` | Number | Item unit value |
Stream for managing sales representatives.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `ativo` | Boolean | Active status |
| `cnpj_cpf` | String | CPF/CNPJ |
| `codigo` | String | Sales representative code |
| `codigos_condicoes_pagamento` | Array | List of payment condition codes |
| `email` | String | Email address |
| `id` | Integer | Sales representative ID |
| `nome` | String | Sales representative name |
## Data Model
The following diagram illustrates the relationships between the core data streams in Fly ERP. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Clientes("Clientes e Fornecedores")
Produtos("Produtos e Serviços")
Vendedores("Vendedores")
end
subgraph "Sales"
Vendas("Vendas")
VendasItens("Vendas Itens")
end
subgraph "Financial"
ContasReceber("Contas a Receber")
ContasPagar("Contas a Pagar")
Movimentacoes("Movimentações Financeiras")
end
subgraph "Product Organization"
ProdutosGrupos("Produtos Grupos")
end
Vendas -- "codigo_cliente" --> Clientes;
VendasItens -- "id_nota" --> Vendas;
VendasItens -- "id_produto" --> Produtos;
Produtos -- "id_grupo" --> ProdutosGrupos;
ContasReceber -- "id_cliente" --> Clientes;
ContasPagar -- "id_fornecedor" --> Clientes;
Movimentacoes -- "id_cliente" --> Clientes;
Movimentacoes -- "id_contas_a_receber" --> ContasReceber;
Movimentacoes -- "id_contas_a_pagar" --> ContasPagar;
Clientes -- "id_vendedor" --> Vendedores;
```
## Skills for agents
Fly ERP connector documentation as plain markdown, for use in AI agent contexts.
# Freshchat as a data source
Source: https://docs.nekt.com/sources/freshchat
Bring data from Freshchat to Nekt.
Freshchat is a modern messaging platform from Freshworks that enables businesses to engage with customers across web, mobile, and social channels. It provides live chat, chatbots, and messaging capabilities for sales, support, and customer engagement teams. This connector extracts your Freshchat data so you can analyze conversations, agent performance, and user engagement.
![]()
## Configuring Freshchat as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Freshchat option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Freshchat subdomain and API token to authenticate. Freshchat uses Bearer Token authentication.
#### Obtaining Your API Token
Sign in to your Freshchat account at `https://{your-domain}.freshchat.com`.
Go to **Admin** > **CONFIGURE** > **API Tokens**.
Click **Generate Token** and copy the value.
Store your API token securely. Treat it like a password and do not share it publicly.
#### Configuration Fields
The following configurations are available:
* **Freshchat Domain**: Your Freshchat subdomain. For an account hosted at `https://acme.freshchat.com`, this value is `acme`.
* **API Key**: The Bearer token generated from your Freshchat admin panel.
* **Start Date**: The earliest date from which records will be synced for incremental streams (users, conversations).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Chat data typically benefits from hourly or daily refreshes depending on your message volume.
Optionally, you can determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This complements incremental extractions and ensures your data is fully synchronized every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Freshchat and their corresponding fields:
Stream containing your Freshchat account configuration details. Extracted as a full-table sync (single record).
| Field | Type | Description |
| :- | :- | :- |
| `account_id` | Integer | Unique account ID |
| `organisation_id` | Integer | Organization ID |
| `organisation_domain` | String | Organization domain |
| `account_domain` | String | Account domain |
| `app_id` | String | Application ID |
| `bundle_id` | Integer | Bundle ID |
| `bundle_type` | String | Bundle type |
| `plan_type` | String | Plan type |
| `datacenter` | String | Datacenter region |
Stream containing all Freshchat users (visitors and contacts). Supports **incremental sync** based on the `updated_time` field.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique user ID |
| `first_name` | String | First name |
| `last_name` | String | Last name |
| `email` | String | Email address |
| `phone` | String | Phone number |
| `avatar` | String | Avatar URL |
| `reference_id` | String | External reference ID |
| `social_profiles` | String | Social profiles (JSON) |
| `properties` | String | User properties (JSON) |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
Stream containing all conversations, fetched per user from the `/users/{user_id}/conversations` endpoint. Supports **incremental sync** based on `updated_time`.
| Field | Type | Description |
| :- | :- | :- |
| `conversation_id` | String | Unique conversation ID |
| `user_id` | String | User who initiated the conversation |
| `status` | String | Conversation status |
| `app_id` | String | Application ID |
| `channel_id` | String | Channel/topic ID |
| `assigned_agent_id` | String | Assigned agent ID |
| `assigned_group_id` | String | Assigned group ID |
| `messages` | String | Latest messages summary (JSON) |
| `agents` | String | Participating agents (JSON) |
| `users` | String | Participating users (JSON) |
| `properties` | String | Conversation properties (JSON) |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
Stream containing individual messages within conversations, fetched from the `/conversations/{conversation_id}/messages` endpoint. Supports **incremental sync** based on `created_time`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique message ID |
| `conversation_id` | String | Parent conversation ID |
| `actor_type` | String | Sender type (user, agent, system) |
| `actor_id` | String | Sender ID |
| `message_type` | String | Message type |
| `message_parts` | String | Message content parts (JSON) |
| `reply_parts` | String | Reply content parts (JSON) |
| `meta_data` | String | Message metadata (JSON) |
| `created_time` | DateTime | Creation timestamp |
Stream containing conversation property definitions and custom fields. Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique field ID |
| `name` | String | Field name |
| `label` | String | Display label |
| `type` | String | Field type |
| `custom` | Boolean | Whether this is a custom field |
| `editable` | Boolean | Whether the field is editable |
| `position` | Integer | Display position |
| `required` | Boolean | Whether the field is required |
| `choices` | String | Available choices for dropdown fields (JSON) |
Stream containing all Freshchat agents (support staff). Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique agent ID |
| `first_name` | String | First name |
| `last_name` | String | Last name |
| `email` | String | Email address |
| `avatar` | String | Avatar URL |
| `biography` | String | Agent biography |
| `login_status` | Boolean | Whether the agent is logged in |
| `availability_status` | String | Availability status |
| `role_id` | String | Assigned role ID |
| `is_deactivated` | Boolean | Whether the agent is deactivated |
| `skill_id` | String | Skill ID |
| `freshdesk_agent_id` | String | Corresponding Freshdesk agent ID |
| `social_profiles` | String | Social profiles (JSON) |
| `groups` | String | Groups the agent belongs to (JSON) |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
Stream containing all Freshchat groups. Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique group ID |
| `name` | String | Group name |
| `description` | String | Group description |
| `routing_type` | String | Routing type |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
Stream containing all Freshchat channels (topics). Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique channel ID |
| `name` | String | Channel name |
| `enabled` | Boolean | Whether the channel is enabled |
| `public` | Boolean | Whether the channel is public |
| `icon` | String | Channel icon |
| `welcome_message` | String | Welcome message |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
Stream containing all Freshchat roles. Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique role ID |
| `name` | String | Role name |
| `description` | String | Role description |
| `is_default` | Boolean | Whether this is a default role |
| `permissions` | String | Role permissions (JSON) |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
Stream containing outbound messages sent via WhatsApp and other channels. Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique message ID |
| `status` | String | Delivery status |
| `channel` | String | Channel (e.g., whatsapp) |
| `message_type` | String | Message type |
| `message_parts` | String | Message content parts (JSON) |
| `recipients` | String | Message recipients (JSON) |
| `metrics` | String | Delivery metrics (JSON) |
| `created_time` | DateTime | Creation timestamp |
| `updated_time` | DateTime | Last update timestamp |
# Data Model
The following diagram illustrates the relationships between the core data streams in Freshchat. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
erDiagram
Users ||--o{ Conversations : "user_id"
Conversations ||--o{ ConversationMessages : "conversation_id"
Conversations }o--o| Agents : "assigned_agent_id"
Conversations }o--o| Groups : "assigned_group_id"
Conversations }o--o| Channels : "channel_id"
Agents }o--o| Roles : "role_id"
Users {
string id PK
string email
string first_name
datetime updated_time
}
Conversations {
string conversation_id PK
string user_id FK
string assigned_agent_id FK
string assigned_group_id FK
string channel_id FK
string status
datetime updated_time
}
ConversationMessages {
string id PK
string conversation_id FK
string actor_type
string actor_id
datetime created_time
}
Agents {
string id PK
string email
string role_id FK
boolean is_deactivated
}
Groups {
string id PK
string name
}
Channels {
string id PK
string name
}
Roles {
string id PK
string name
}
```
## Implementation Notes
### Data Sync Considerations
* **Incremental sync**: Users and conversations support incremental sync based on `updated_time`. Conversation messages support incremental sync based on `created_time`. All other streams are extracted as full-table syncs.
* **Parent-child hierarchy**: Conversations are nested under users, and messages are nested under conversations. This means syncing conversations will first iterate through all users, and syncing messages will iterate through all conversations.
* **Freshdesk integration**: Agents have a `freshdesk_agent_id` field that maps to the corresponding agent in Freshdesk, enabling cross-product analysis.
### API Limits and Performance
* **Rate limits**: Freshchat enforces rate limits tracked via `X-RateLimit-Limit` and `X-RateLimit-Remaining` headers. The connector respects 429 responses automatically.
* **Pagination**: List endpoints return up to 50 records per page by default.
* **Timestamps**: All timestamps are returned in UTC (ISO 8601).
## Skills for agents
Freshchat connector documentation as plain markdown, for use in AI agent contexts.
# Freshdesk as a data source
Source: https://docs.nekt.com/sources/freshdesk
Bring data from Freshdesk to Nekt.
Freshdesk is a cloud-based customer support platform from Freshworks. It lets teams manage tickets across email, chat, phone, and social channels, track SLAs, and coordinate work between agents and groups. This connector extracts your Freshdesk help desk data so you can analyze ticket volume, agent performance, response times, and customer relationships.
![]()
## Configuring Freshdesk as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Freshdesk option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Freshdesk subdomain and API key to authenticate. Freshdesk uses HTTP Basic authentication with the API key as the username.
#### Obtaining Your API Key
Sign in to your Freshdesk account at `https://{your-domain}.freshdesk.com`.
Click your profile picture in the top-right corner and select **Profile settings**.
On the right side of the profile page, click **View API Key** and copy the value.
Store your API key securely. Treat it like a password and do not share it publicly.
#### Configuration Fields
The following configurations are available:
* **Freshdesk domain**: Your Freshdesk subdomain. For an account hosted at `https://acme.freshdesk.com`, this value is `acme`.
* **API key**: The API key copied from your Freshdesk profile page.
* **Start date**: The earliest date from which records will be synced. This controls the initial window for the `tickets`, `contacts`, and `companies` streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Ticket data typically benefits from hourly or daily refreshes depending on your support volume.
Optionally, you can determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This complements incremental extractions and ensures your data is fully synchronized every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Freshdesk and their corresponding fields:
Stream containing all support tickets, the central fact table for help desk analytics. Supports **incremental sync** based on the `updated_at` field. The stream automatically includes ticket stats (response and resolution timestamps) via the `include=stats` parameter for SLA analysis.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique ticket ID |
| `subject` | String | Ticket subject |
| `description_text` | String | Plain-text description |
| `type` | String | Ticket type (e.g., Question, Incident, Problem, Feature Request) |
| `status` | Integer | Status code: 2=Open, 3=Pending, 4=Resolved, 5=Closed |
| `priority` | Integer | Priority: 1=Low, 2=Medium, 3=High, 4=Urgent |
| `source` | Integer | Source channel (email, portal, phone, chat, etc.) |
| `spam` | Boolean | Whether the ticket is marked as spam |
| `requester_id` | Integer | Contact who created the ticket |
| `responder_id` | Integer | Agent assigned to the ticket |
| `company_id` | Integer | Associated company |
| `group_id` | Integer | Assigned agent group |
| `product_id` | Integer | Product associated with the ticket |
| `email_config_id` | Integer | Email mailbox used |
| `fr_escalated` | Boolean | Whether first response SLA was breached |
| `is_escalated` | Boolean | Whether resolution SLA was breached |
| `fr_due_by` | DateTime | First response due-by timestamp |
| `due_by` | DateTime | Resolution due-by timestamp |
| `nr_due_by` | DateTime | Next response due-by timestamp |
| `nr_escalated` | Boolean | Whether next response SLA was breached |
| `tags` | Array | Ticket tags |
| `cc_emails` | Array | CC emails on the ticket |
| `fwd_emails` | Array | Forwarded emails |
| `reply_cc_emails` | Array | Reply CC emails |
| `to_emails` | Array | To emails on the ticket |
| `due_by_in_hours` | Number | Due-by in hours |
| `sentiment_score` | Number | Latest sentiment score |
| `initial_sentiment_score` | Number | Initial sentiment score |
| `custom_fields` | Object | Account-specific custom fields |
| `stats.agent_responded_at` | DateTime | Last agent response timestamp |
| `stats.requester_responded_at` | DateTime | Last requester response timestamp |
| `stats.first_responded_at` | DateTime | First agent response timestamp |
| `stats.status_updated_at` | DateTime | Last status change timestamp |
| `stats.reopened_at` | DateTime | Reopen timestamp |
| `stats.resolved_at` | DateTime | Resolution timestamp |
| `stats.closed_at` | DateTime | Closure timestamp |
| `stats.pending_since` | DateTime | Pending-since timestamp |
| `created_at` | DateTime | Creation timestamp |
| `updated_at` | DateTime | Last update timestamp |
Stream containing ticket replies and private notes, fetched per ticket from the `/tickets/{ticket_id}/conversations` endpoint. Use this stream to analyze response times, interaction counts, and the full conversation timeline for each ticket.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique conversation ID |
| `ticket_id` | Integer | Parent ticket ID |
| `user_id` | Integer | Author (agent or contact) user ID |
| `from_email` | String | Sender email |
| `to_emails` | Array | To email addresses |
| `cc_emails` | Array | CC email addresses |
| `bcc_emails` | Array | BCC email addresses |
| `body_text` | String | Plain-text message body |
| `incoming` | Boolean | Whether the message is incoming (from customer) |
| `private` | Boolean | Whether this is a private note (agent-only) |
| `source` | Integer | Source channel for the conversation |
| `support_email` | String | Support email address used |
| `category` | Integer | Conversation category |
| `thread_id` | Integer | Thread ID for threaded conversations |
| `thread_message_id` | Integer | Thread message ID |
| `last_edited_at` | DateTime | Last edit timestamp |
| `last_edited_user_id` | Integer | User who last edited |
| `attachments` | Array | Attachment objects (id, name, content\_type, size, attachment\_url, timestamps) |
| `created_at` | DateTime | Creation timestamp |
| `updated_at` | DateTime | Last update timestamp |
Stream containing all customers who have submitted tickets or been added to the help desk. Supports **incremental sync** based on the `updated_at` field.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique contact ID |
| `name` | String | Contact name |
| `email` | String | Primary email |
| `phone` | String | Phone number |
| `mobile` | String | Mobile number |
| `job_title` | String | Job title |
| `address` | String | Address |
| `description` | String | Contact description |
| `language` | String | Preferred language |
| `time_zone` | String | Time zone |
| `twitter_id` | String | Twitter handle |
| `facebook_id` | String | Facebook ID |
| `unique_external_id` | String | External ID from integrated systems |
| `active` | Boolean | Whether the contact is verified/active |
| `blocked` | Boolean | Whether the contact is blocked |
| `blocked_at` | DateTime | Block timestamp |
| `deleted` | Boolean | Whether the contact is soft-deleted |
| `company_id` | Integer | Primary company association |
| `view_all_tickets` | Boolean | Whether contact sees all company tickets |
| `tags` | Array | Contact tags |
| `other_emails` | Array | Additional emails |
| `other_phone_numbers` | Array | Additional phone numbers |
| `other_companies` | String | Secondary companies linked to this contact (JSON) |
| `custom_fields` | String | Account-specific custom fields (JSON) |
| `created_at` | DateTime | Creation timestamp |
| `updated_at` | DateTime | Last update timestamp |
Stream containing organizations that group contacts, commonly used for B2B support. Supports **incremental sync** based on the `updated_at` field.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique company ID |
| `name` | String | Company name |
| `description` | String | Company description |
| `note` | String | Internal note |
| `domains` | Array | Email domains associated with the company |
| `health_score` | String | Customer health score |
| `account_tier` | String | Account tier (e.g., Premium, Basic) |
| `renewal_date` | DateTime | Contract renewal date |
| `industry` | String | Industry |
| `custom_fields` | String | Account-specific custom fields (JSON) |
| `created_at` | DateTime | Creation timestamp |
| `updated_at` | DateTime | Last update timestamp |
Stream containing all support agents with their roles, groups, and skills. Extracted as a full-table sync.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique agent ID |
| `available` | Boolean | Whether the agent is currently available |
| `available_since` | DateTime | Timestamp since current availability state |
| `occasional` | Boolean | Whether the agent is occasional (non-full-time) |
| `signature` | String | HTML signature |
| `ticket_scope` | Integer | 1=Global, 2=Group, 3=Restricted access |
| `type` | String | Agent type (support\_agent, field\_agent, collaborator) |
| `focus_mode` | Boolean | Whether focus mode is enabled |
| `deactivated` | Boolean | Whether the agent has been deactivated |
| `last_active_at` | DateTime | Last active timestamp |
| `group_ids` | Array | Groups the agent belongs to |
| `role_ids` | Array | Roles assigned to the agent |
| `skill_ids` | Array | Skills assigned to the agent |
| `contact` | String | Embedded user/contact record for the agent (JSON) |
| `created_at` | DateTime | Agent record creation timestamp |
| `updated_at` | DateTime | Agent record last update timestamp |
# Data Model
The following diagram illustrates the relationships between the core data streams in Freshdesk. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
erDiagram
Tickets ||--o{ Conversations : "ticket_id"
Tickets }o--|| Contacts : "requester_id"
Tickets }o--|| Agents : "responder_id"
Tickets }o--o| Companies : "company_id"
Contacts }o--o| Companies : "company_id"
Tickets {
int id PK
int requester_id FK
int responder_id FK
int company_id FK
int status
int priority
datetime created_at
datetime updated_at
}
Conversations {
int id PK
int ticket_id FK
int user_id
boolean private
datetime created_at
}
Contacts {
int id PK
string email
int company_id FK
datetime updated_at
}
Companies {
int id PK
string name
datetime updated_at
}
Agents {
int id PK
string type
boolean deactivated
}
```
## Implementation Notes
### Data Sync Considerations
* **Incremental sync**: Tickets, contacts, and companies support incremental sync based on `updated_at`. Agents and conversations are extracted as full-table syncs.
* **Ticket stats**: The tickets stream automatically requests the `stats` block from the Freshdesk API, giving you response and resolution timestamps without a second API call.
* **Conversations**: Conversations are nested under tickets. Any ticket included in a sync will have its conversations re-fetched, so keep this in mind when tuning the start date for high-volume accounts.
### API Limits & Performance
* **Page cap**: The Freshdesk list endpoints cap at 300 pages of 100 records (30,000 records per filter window). For accounts exceeding this, use a recent start date and increase the sync frequency so each incremental window stays within the cap.
* **Rate limits**: Freshdesk enforces per-minute rate limits that vary by plan. The connector retries on 429 responses using the `Retry-After` header.
* **Timestamps**: All timestamps are returned in UTC (ISO 8601).
## Skills for agents
Freshdesk connector documentation as plain markdown, for use in AI agent contexts.
# Funraisin as a data source
Source: https://docs.nekt.com/sources/funraisin
Bring data from Funraisin to Nekt.
Funraisin is an online fundraising platform that enables nonprofits and charities to create and manage fundraising campaigns, events, and peer-to-peer fundraising pages. It provides tools for donation management, participant tracking, team coordination, and event organization.
## Configuring Funraisin as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Funraisin option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Funraisin API credentials. To obtain an API key, create a user with "API Access" enabled in the User Admin section of your Funraisin dashboard.
The following configurations are available:
* **Domain**: Your Funraisin domain (e.g., `yourorg.funraisin.co`).
* **API Key**: The API key generated from your Funraisin admin panel for a user with API Access enabled.
* **Start Date**: (Optional) The earliest date from which records will be synced. Only applies to incremental streams (donations, sales, transactions).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Funraisin and their corresponding fields:
| Field | Type | Description |
| :- | :- | :- |
| `donation_id` | String | Unique identifier for the donation |
| `member_id` | String | Identifier for the member |
| `event_id` | String | Identifier for the event |
| `team_id` | String | Identifier for the team |
| `history_id` | String | Identifier for the history record |
| `d_amount` | Number | Donation amount |
| `d_email` | String | Donor's email address |
| `d_fname` | String | Donor's first name |
| `d_lname` | String | Donor's last name |
| `d_status` | String | Donation status |
| `d_type` | String | Donation type |
| `d_anonymous` | String | Indicates if the donation is anonymous |
| `d_message` | String | Donation message |
| `d_phone` | String | Donor's phone number |
| `d_address` | String | Donor's address |
| `d_city` | String | Donor's city |
| `d_state` | String | Donor's state |
| `d_postcode` | String | Donor's postal code |
| `d_country` | String | Donor's country |
| `d_company` | String | Donor's company |
| `date_paid` | DateTime | Date the donation was paid |
| `date_created` | DateTime | Date the donation was created |
| `crm_donor_id` | String | CRM identifier for the donor |
| `transaction_id` | String | Identifier for the transaction |
| `payment_type` | String | Type of payment |
| `receipt_number` | String | Receipt number |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `donation_id` | String | Donation identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `d_amount` | Number | Donation amount |
| `d_email` | String | Donor's email address |
| `d_fname` | String | Donor's first name |
| `d_lname` | String | Donor's last name |
| `donation_frequency` | String | Frequency of the donation |
| `donation_status` | String | Status of the donation |
| `date_created` | DateTime | Creation date |
| `date_next` | DateTime | Next donation date |
| `date_end` | DateTime | End date of the donation |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `donation_id` | String | Donation identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `d_amount` | Number | Donation amount |
| `d_email` | String | Donor's email address |
| `d_fname` | String | Donor's first name |
| `d_lname` | String | Donor's last name |
| `donation_frequency` | String | Frequency of the donation |
| `donation_status` | String | Status of the donation |
| `date_created` | DateTime | Creation date |
| `date_next` | DateTime | Next donation date |
| `date_end` | DateTime | End date of the donation |
| Field | Type | Description |
| :- | :- | :- |
| `member_id` | String | Member identifier |
| `m_fname` | String | Member's first name |
| `m_lname` | String | Member's last name |
| `m_email` | String | Member's email |
| `m_phone` | String | Member's phone |
| `m_address` | String | Member's address |
| `m_city` | String | Member's city |
| `m_state` | String | Member's state |
| `m_postcode` | String | Member's postal code |
| `m_country` | String | Member's country |
| `m_dob` | String | Member's date of birth |
| `m_gender` | String | Member's gender |
| `m_status` | String | Member's status |
| `m_company` | String | Member's company |
| `date_created` | DateTime | Creation date |
| `date_updated` | DateTime | Update date |
| `crm_member_id` | String | CRM member identifier |
| Field | Type | Description |
| :- | :- | :- |
| `history_id` | String | History identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `team_id` | String | Team identifier |
| `m_target` | Number | Target amount |
| `m_raised` | Number | Amount raised |
| `total_paid` | Number | Total paid |
| `total_pending` | Number | Total pending |
| `is_paid` | String | Indicates if the participation is paid |
| `m_status` | Integer | Member status |
| `date_created` | DateTime | Creation date |
| `date_updated` | DateTime | Update date |
| `crm_history_id` | String | CRM history identifier |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `history_id` | String | History identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `option_name` | String | Option name |
| `option_value` | String | Option value |
| `option_type` | String | Option type |
| `option_price` | Number | Option price |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `history_id` | String | History identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `fitness_provider` | String | Fitness provider |
| `fitness_type` | String | Fitness type |
| `fitness_value` | Number | Fitness value |
| `fitness_unit` | String | Fitness unit |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `history_id` | String | History identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `activity_type` | String | Activity type |
| `activity_value` | Number | Activity value |
| `activity_unit` | String | Activity unit |
| `activity_date` | DateTime | Activity date |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `team_id` | String | Team identifier |
| `t_name` | String | Team name |
| `t_target` | Number | Team target amount |
| `t_raised` | Number | Team amount raised |
| `event_id` | String | Event identifier |
| `captain_id` | String | Captain identifier |
| `t_status` | String | Team status |
| `t_members` | Integer | Number of team members |
| `date_created` | DateTime | Creation date |
| `date_updated` | DateTime | Update date |
| `crm_team_id` | String | CRM team identifier |
| Field | Type | Description |
| :- | :- | :- |
| `event_id` | String | Event identifier |
| `event_name` | String | Event name |
| `event_type` | String | Event type |
| `event_date` | DateTime | Event date |
| `event_end_date` | DateTime | Event end date |
| `event_target` | Number | Event target amount |
| `event_raised` | Number | Event amount raised |
| `event_status` | String | Event status |
| `event_location` | String | Event location |
| `event_url` | String | Event URL |
| `date_created` | DateTime | Creation date |
| `date_updated` | DateTime | Update date |
| `crm_event_id` | String | CRM event identifier |
| Field | Type | Description |
| :- | :- | :- |
| `product_id` | String | Product identifier |
| `event_id` | String | Event identifier |
| `product_name` | String | Name of the product |
| `product_price` | Number | Price of the product |
| `product_description` | String | Description of the product |
| `is_mandatory` | String | Indicates if the product is mandatory |
| `product_status` | Integer | Status of the product |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `wave_id` | String | Wave identifier |
| `event_id` | String | Event identifier |
| `wave_name` | String | Wave name |
| `wave_date` | DateTime | Wave date |
| `wave_price` | Number | Wave price |
| `wave_capacity` | Integer | Wave capacity |
| `wave_registered` | Integer | Number of registrations |
| `wave_status` | String | Wave status |
| `date_created` | DateTime | Creation date |
| `crm_wave_id` | String | CRM wave identifier |
| Field | Type | Description |
| :- | :- | :- |
| `category_id` | String | Category identifier |
| `category_name` | String | Category name |
| `category_description` | String | Category description |
| `category_status` | String | Category status |
| `crm_category_id` | String | CRM category identifier |
| Field | Type | Description |
| :- | :- | :- |
| `product_id` | String | Product identifier |
| `product_name` | String | Product name |
| `product_description` | String | Product description |
| `product_price` | Number | Product price |
| `product_status` | String | Product status |
| `product_stock` | Integer | Product stock quantity |
| `date_created` | DateTime | Creation date |
| `crm_product_id` | String | CRM product identifier |
| Field | Type | Description |
| :- | :- | :- |
| `option_id` | String | Option identifier |
| `product_id` | String | Product identifier |
| `option_name` | String | Option name |
| `option_price` | Number | Option price |
| `option_stock` | Integer | Option stock quantity |
| `option_status` | String | Option status |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `sale_id` | String | Sale identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `total` | Number | Total sale amount |
| `sale_status` | String | Sale status |
| `date_paid` | DateTime | Date paid |
| `date_created` | DateTime | Creation date |
| `crm_sale_id` | String | CRM sale identifier |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `sale_id` | String | Sale identifier |
| `product_id` | String | Product identifier |
| `option_id` | String | Option identifier |
| `quantity` | Integer | Quantity sold |
| `price` | Number | Price |
| `total` | Number | Total amount |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `transaction_id` | String | Transaction identifier |
| `member_id` | String | Member identifier |
| `event_id` | String | Event identifier |
| `transaction_type` | String | Transaction type |
| `transaction_value` | Number | Transaction value |
| `transaction_status` | String | Transaction status |
| `payment_type` | String | Payment type |
| `payment_reference` | String | Payment reference |
| `transaction_notes` | String | Transaction notes |
| `date_created` | DateTime | Creation date |
| `date_updated` | DateTime | Update date |
| `crm_transaction_id` | String | CRM transaction identifier |
| Field | Type | Description |
| :- | :- | :- |
| `raffle_id` | String | Raffle identifier |
| `event_id` | String | Event identifier |
| `raffle_name` | String | Raffle name |
| `raffle_description` | String | Raffle description |
| `raffle_price` | Number | Raffle price |
| `raffle_status` | String | Raffle status |
| `date_created` | DateTime | Creation date |
| `crm_raffle_id` | String | CRM raffle identifier |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `raffle_id` | String | Raffle identifier |
| `member_id` | String | Member identifier |
| `ticket_number` | String | Ticket number |
| `ticket_status` | String | Ticket status |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `sale_id` | String | Sale identifier |
| `raffle_id` | String | Raffle identifier |
| `member_id` | String | Member identifier |
| `quantity` | Integer | Quantity of tickets |
| `total` | Number | Total amount |
| `sale_status` | String | Sale status |
| `date_created` | DateTime | Creation date |
| `crm_sale_id` | String | CRM sale identifier |
| Field | Type | Description |
| :- | :- | :- |
| `ticket_id` | String | Ticket identifier |
| `event_id` | String | Event identifier |
| `member_id` | String | Member identifier |
| `ticket_type` | String | Ticket type |
| `ticket_price` | Number | Ticket price |
| `ticket_status` | String | Ticket status |
| `ticket_quantity` | Integer | Ticket quantity |
| `date_created` | DateTime | Creation date |
| Field | Type | Description |
| :- | :- | :- |
| `guest_id` | String | Guest identifier |
| `ticket_id` | String | Ticket identifier |
| `event_id` | String | Event identifier |
| `member_id` | String | Member identifier |
| `g_fname` | String | Guest's first name |
| `g_lname` | String | Guest's last name |
| `g_email` | String | Guest's email |
| `g_phone` | String | Guest's phone |
| `g_status` | String | Guest's status |
| `date_created` | DateTime | Creation date |
| `crm_guest_id` | String | CRM guest identifier |
# Gainsight as a data source
Source: https://docs.nekt.com/sources/gainsight
Bring data from Gainsight NXT to your Lakehouse.
Gainsight NXT is a customer success platform: it keeps your customers' companies and relationships, the people you work with at each account, the activities your team logs on the Timeline, Cockpit CTAs and Success Plans, health scores, and any custom object your Gainsight admins create. This connector extracts all of them. The streams and columns you get mirror exactly how your Gainsight tenant is modeled, custom objects and fields included.
## Configuring Gainsight as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Gainsight option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Gainsight authenticates API calls with an **Access Key**. A Gainsight **Super Admin** finds it in **Administration > Connectors 2.0**, in the **Gainsight API** connection. For more information, check the official [documentation](https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs/Generate_REST_API/Generate_REST_API_Key).
Gainsight allows only **one Access Key per tenant**. If your company already uses it for another integration, reuse the same key. Do **not** regenerate it: regenerating creates a new key and stops every integration that still uses the old one.
The following configurations are available:
* **Gainsight Domain**: the address of your Gainsight tenant, the same one you use to open Gainsight, e.g. `https://yourcompany.gainsightcloud.com`. Required.
* **Access Key**: the Gainsight REST API access key. Required.
* **Objects** (advanced): the API names of the Gainsight objects to sync, e.g. `company`, `relationship`, `activity_timeline` or `my_object__gc`. Leave it empty to sync every standard and custom object of your tenant. You can find the API names in Gainsight's **Administration > Data Management**.
* **Initial sync date** (advanced): for objects that track modifications, only records modified after this date are extracted on the first sync. Objects without a modification date are always extracted in full. Leave it empty to extract the full history.
* **Requests Per Minute** (advanced): how fast the connector may call Gainsight. Gainsight allows 100 requests per minute per tenant, shared with your other Gainsight integrations. The default is 80.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
If you don't see an object you were expecting, add its API name to the **Objects** setting. When that setting is filled, the connector reads exactly the objects you list, including Gainsight system objects.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: objects that have a modification date (`modified_date`) support INCREMENTAL; the others are FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Incremental syncs pick up new and changed records, but they cannot see records that were deleted in Gainsight. Schedule an **Additional Full Sync** to remove deleted records from your tables.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Streams are discovered from your tenant's data model: the connector creates **one stream per Gainsight object**, standard and custom alike, and one column per field. API reference: [Gainsight NXT API](https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs/About/API_Documentation_Overview).
Stream names are the snake\_case version of the object's API name, so the custom object `nps_feedback__gc` becomes `nps_feedback_gc`. Standard objects you will typically find:
| Stream | Description |
| - | - |
| `company` | Your customer companies (accounts) and their attributes, such as stage, status, ARR and renewal date. |
| `relationship` | Relationships: products, business units or projects you manage inside a company. |
| `person` | People known to Gainsight, with their contact details. |
| `company_person` | Links between people and the companies they are associated with. |
| `relationship_person` | Links between people and relationships. |
| `gsuser` | Gainsight users (your team members). |
| `activity_timeline` | Activities logged on the Timeline: calls, meetings, emails, notes and updates. |
| `call_to_action` | Cockpit Calls to Action (CTAs). |
| `cta_group` | Success Plans. |
Any other object of your tenant, including the ones your admins created, shows up as an additional stream automatically. SYSTEM objects (internal Gainsight tables such as raw email events) are only synced when named in the **Objects** setting.
About the columns you'll see in the Catalog:
* Column names are the snake\_case version of the Gainsight field API names, e.g. `ModifiedDate` becomes `modified_date` and `Renewal_Date__gc` becomes `renewal_date_gc`. Column descriptions come from the field labels in Gainsight.
* `gsid`, the Gainsight ID of the record, is the primary key. `modified_date` is the incremental replication key of the objects that have it.
* Dates and date-times arrive as dates and UTC timestamps.
* Dropdown fields and lookup fields (like a company's CSM) hold the **GSID** of the selected item or referenced record. Join them with the referenced object's `gsid` column to get the name.
* Values that are lists or objects arrive as JSON strings, ready to be parsed in a Query or Notebook.
Gainsight does not allow some field types to be exported through its API. When a field is refused, the connector leaves that column empty, syncs everything else and adds a warning to the run log naming the field. An object that the API cannot read at all is skipped with a warning.
## Skills for agents
Gainsight connector documentation as plain markdown, for use in AI agent contexts.
# Gestão Click as a data source
Source: https://docs.nekt.com/sources/gestao-click
Bring data from Gestão Click to Nekt.
Gestão Click is a Brazilian cloud ERP for small and mid-sized businesses. It covers sales, quotes and service orders, product and service catalogs, inventory, purchasing, Brazilian fiscal documents (NF-e, NFC-e and NFS-e), and the accounts payable and receivable ledger.
## Configuring Gestão Click as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Gestão Click option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an Access Token and a Secret Access Token, generated inside Gestão Click.
In Gestão Click, go to **Configurações > Integração API**.
Generate the tokens and copy both values. They are shown together on the same screen.
Fill in the two fields below with the values you copied.
Clicking **Gerar token novamente** in Gestão Click issues a new pair and invalidates the previous one. If you regenerate the tokens, update them here as well or the extraction will start failing with an authentication error.
The following configurations are available:
* **Access Token**: The Access Token generated under Configurações > Integração API.
* **Secret Access Token**: The Secret Access Token issued alongside the Access Token on the same screen.
* **Start Date**: The earliest document date from which records will be synced. This applies to quotes, sales, service orders, invoices, payables and receivables. Registrations and reference tables (customers, products, cities and so on) are always synced in full, regardless of this setting.
* **Lookback Days**: (Default: 30 days) How many days before the last synced date to re-read on each run. Gestão Click can only filter records by their own document date, so this overlap window is what allows a document to be picked up again after it is edited. Increase it if documents in your operation are commonly revised more than 30 days after being issued.
* **Requests Per Second**: (Default: 3) The maximum request rate. Gestão Click allows 3 requests per second per company. Lower this only if other integrations share the same token pair.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How data is synced
Gestão Click's API can only filter records by their own document date. It offers no "modified since" filter, which shapes how each stream is synced:
* **Quotes, sales, service orders and the three invoice streams** sync incrementally from the document's issue date, re-reading the last **Lookback Days** so that documents edited after being issued are picked up again.
* **Payments and receipts** are synced in full on every run. Their dates are due and settlement dates, which do not move forward with time: an instalment can be due two years from now, and an overdue one can be years in the past. Reading the full ledger each run is what guarantees that overdue open entries are never missed.
* **Registrations and reference tables** are synced in full on every run. They are small, and the API's date filter on these endpoints matches the record's creation date, which would permanently hide long-standing customers or products that are still in use.
Because payments, receipts and the registration streams are read in full, choosing INCREMENTAL as the sync type for them will still append the complete current state on each run. Full table is usually the more useful choice for these.
## Nested fields
Fields that hold a list or a structure are delivered as **JSON strings**, not as nested columns. This applies to document line items (`produtos`, `servicos`), addresses (`enderecos`), contacts (`contatos`), instalments (`pagamentos`), attachments (`anexos`), product variations (`variacoes`) and custom field values (`atributos`).
Query them with SQL JSON functions. Note that Gestão Click wraps each item in a single-key object, so the line items of a sale look like `[{"produto": {...}}, {"produto": {...}}]`:
```sql theme={null}
select
id,
codigo,
json_value(item, '$.produto.nome_produto') as produto,
json_value(item, '$.produto.quantidade') as quantidade,
json_value(item, '$.produto.valor_venda') as valor_unitario
from sales, unnest(json_query_array(produtos, '$')) as item
```
## Rate limits
Gestão Click allows **3 requests per second** and **30,000 requests per day**, counted per company rather than per integration. The connector stays within the per-second limit automatically and retries with backoff if the API throttles it.
Every request returns at most 100 records, so the daily limit translates to roughly 3 million records per day. If you run several integrations against the same token pair, lower **Requests Per Second** so they share the budget.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API), how it replicates, and a short description.
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Customers | `customers` | Full table | People and companies you sell to, with their addresses, contacts and custom fields. |
| Suppliers | `suppliers` | Full table | People and companies you buy from, with their addresses, contacts and custom fields. |
| Carriers | `carriers` | Full table | Freight carriers used on sales and purchases. |
| Employees | `employees` | Full table | Staff records, including the commission rate used on sales. |
| Users | `users` | Full table | ERP user accounts. |
| Stores | `stores` | Full table | Stores (units) registered in the account. |
| Products | `products` | Full table | Products for sale, with cost and selling prices, stock on hand, tax classification and variations. |
| Product Groups | `product_groups` | Full table | The product category hierarchy. |
| Services | `services` | Full table | Services offered for sale. |
| Quotes | `quotes` | Incremental | Quotes (orçamentos) issued to customers, with their line items and payment terms. |
| Sales | `sales` | Incremental | Sales (vendas), covering product sales, service sales and counter sales. |
| Service Orders | `service_orders` | Incremental | Service orders (ordens de serviço), including the customer equipment attached to them. |
| Purchases | `purchases` | Full table | Purchase orders placed with suppliers. |
| Product Invoices | `product_invoices` | Incremental | NF-e (model 55) product invoices, with issuer, recipient, tax totals and line items. |
| Consumer Invoices | `consumer_invoices` | Incremental | NFC-e (model 65) consumer invoices, same shape as NF-e. |
| Service Invoices | `service_invoices` | Incremental | NFS-e service invoices, with ISS tax base and withholding details. |
| Payments | `payments` | Full table | Accounts payable: every payable entry, open and settled. |
| Receipts | `receipts` | Full table | Accounts receivable: every receivable entry, open and settled. |
| Payment Methods | `payment_methods` | Full table | Payment methods, including bank slip and card acquirer configuration. |
| Bank Accounts | `bank_accounts` | Full table | Bank accounts entries are settled into. |
| Chart of Accounts | `chart_of_accounts` | Full table | The chart of accounts hierarchy. |
| Cost Centers | `cost_centers` | Full table | Cost centres used to classify documents and ledger entries. |
| Quote Statuses | `quote_statuses` | Full table | Status values quotes can take. |
| Sale Statuses | `sale_statuses` | Full table | Status values sales can take, and whether each one posts to finance and inventory. |
| Service Order Statuses | `service_order_statuses` | Full table | Status values service orders can take. |
| Purchase Statuses | `purchase_statuses` | Full table | Status values purchases can take. |
| Registration Custom Fields | `registration_custom_fields` | Full table | Definitions of the user-defined fields on customers, suppliers and carriers. |
| Product Custom Fields | `product_custom_fields` | Full table | Definitions of the user-defined fields on products. |
| Quote Custom Fields | `quote_custom_fields` | Full table | Definitions of the user-defined fields on quotes. |
| Sale Custom Fields | `sale_custom_fields` | Full table | Definitions of the user-defined fields on sales. |
| Service Order Custom Fields | `service_order_custom_fields` | Full table | Definitions of the user-defined fields on service orders. |
| Financial Custom Fields | `financial_custom_fields` | Full table | Definitions of the user-defined fields on payable and receivable entries. |
| Contact Types | `contact_types` | Full table | Contact type labels used inside registration contact lists. |
| Address Types | `address_types` | Full table | Address type labels used inside registration address lists. |
| States | `states` | Full table | Brazilian states, the IBGE reference table. |
| Cities | `cities` | Full table | Brazilian municipalities, the IBGE reference table. |
## Stream details
People and companies you sell to, with their addresses, contacts and custom fields.
**Primary key:** `id`\
**Replication:** full table
**Fields (27):**
* `id` - Unique identifier of the customer
* `tipo_pessoa` - Entity type: PF for an individual, PJ for a company, ES for a foreign entity
* `nome` - Display name of the customer (trade name for companies)
* `razao_social` - Registered legal name, for companies
* `cnpj` - Brazilian company tax ID (CNPJ), formatted
* `inscricao_estadual` - State tax registration number
* `inscricao_municipal` - Municipal tax registration number
* `tipo_contribuinte` - ICMS taxpayer classification
* `responsavel` - Name of the person responsible for the account
* `cpf` - Brazilian individual tax ID (CPF), formatted
* `rg` - Brazilian national ID document number
* `data_nascimento` - Date of birth (individuals) or incorporation date (companies), as YYYY-MM-DD
* `sexo` - Gender recorded for an individual customer
* `loja_virtual_ativo` - Whether the customer has online-store access enabled: 1 for yes, 0 for no
* `email_acesso` - Email the customer uses to sign in to the online store
* `telefone` - Landline phone number
* `celular` - Mobile phone number
* `fax` - Fax number
* `email` - Primary email address
* `ativo` - Whether the customer record is active: 1 for active, 0 for inactive
* `vendedor_id` - Identifier of the salesperson assigned to this customer
* `nome_vendedor` - Name of the salesperson assigned to this customer
* `cadastrado_em` - Timestamp the customer was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the customer, as YYYY-MM-DD HH:MM:SS
* `contatos` - Additional contact people for this record, as a JSON array of objects (each wrapped in a 'contato' key) holding the contact's name, role, phone and email
* `enderecos` - Registered addresses, as a JSON array of objects (each wrapped in an 'endereco' key) holding street, number, complement, district, postal code, city and state
* `atributos` - Values of the account's user-defined registration fields, as a JSON array of objects (each wrapped in an 'atributo' key); the field definitions are in the registration\_custom\_fields stream
People and companies you buy from, with their addresses, contacts and custom fields.
**Primary key:** `id`\
**Replication:** full table
**Fields (20):**
* `id` - Unique identifier of the supplier
* `tipo_pessoa` - Entity type: PF for an individual, PJ for a company, ES for a foreign entity
* `nome` - Display name of the supplier (trade name for companies)
* `razao_social` - Registered legal name, for companies
* `cnpj` - Brazilian company tax ID (CNPJ), formatted
* `inscricao_estadual` - State tax registration number
* `inscricao_municipal` - Municipal tax registration number
* `tipo_contribuinte` - ICMS taxpayer classification
* `cpf` - Brazilian individual tax ID (CPF), formatted
* `rg` - Brazilian national ID document number
* `data_nascimento` - Date of birth (individuals) or incorporation date (companies), as YYYY-MM-DD
* `telefone` - Landline phone number
* `celular` - Mobile phone number
* `email` - Primary email address
* `ativo` - Whether the supplier record is active: 1 for active, 0 for inactive
* `cadastrado_em` - Timestamp the supplier was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the supplier, as YYYY-MM-DD HH:MM:SS
* `contatos` - Additional contact people for this record, as a JSON array of objects (each wrapped in a 'contato' key) holding the contact's name, role, phone and email
* `enderecos` - Registered addresses, as a JSON array of objects (each wrapped in an 'endereco' key) holding street, number, complement, district, postal code, city and state
* `atributos` - Values of the account's user-defined registration fields, as a JSON array of objects (each wrapped in an 'atributo' key); the field definitions are in the registration\_custom\_fields stream
Freight carriers used on sales and purchases.
**Primary key:** `id`\
**Replication:** full table
**Fields (19):**
* `id` - Unique identifier of the carrier
* `tipo_pessoa` - Entity type: PF for an individual, PJ for a company, ES for a foreign entity
* `nome` - Display name of the carrier (trade name for companies)
* `razao_social` - Registered legal name, for companies
* `cnpj` - Brazilian company tax ID (CNPJ), formatted
* `inscricao_estadual` - State tax registration number
* `inscricao_municipal` - Municipal tax registration number
* `cpf` - Brazilian individual tax ID (CPF), formatted
* `rg` - Brazilian national ID document number
* `data_nascimento` - Date of birth (individuals) or incorporation date (companies), as YYYY-MM-DD
* `telefone` - Landline phone number
* `celular` - Mobile phone number
* `email` - Primary email address
* `observacoes` - Free-text notes about the carrier
* `ativo` - Whether the carrier record is active: 1 for active, 0 for inactive
* `cadastrado_em` - Timestamp the carrier was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the carrier, as YYYY-MM-DD HH:MM:SS
* `contatos` - Additional contact people for this record, as a JSON array of objects (each wrapped in a 'contato' key) holding the contact's name, role, phone and email
* `enderecos` - Registered addresses, as a JSON array of objects (each wrapped in an 'endereco' key) holding street, number, complement, district, postal code, city and state
Staff records, including the commission rate used on sales.
**Primary key:** `id`\
**Replication:** full table
**Fields (19):**
* `id` - Unique identifier of the employee
* `nome` - Full name of the employee
* `email_secundario` - Secondary email address
* `cpf` - Brazilian individual tax ID (CPF), formatted
* `rg` - Brazilian national ID document number
* `data_nascimento` - Date of birth, as YYYY-MM-DD
* `sexo` - Gender recorded for the employee
* `telefone` - Phone number
* `cep` - Postal code (CEP) of the employee's address
* `logradouro` - Street name of the employee's address
* `numero` - Street number of the employee's address
* `complemento` - Address complement, such as apartment or suite
* `bairro` - District or neighbourhood of the employee's address
* `cidade_id` - Identifier of the city, joinable to the cities stream
* `nome_cidade` - Name of the city
* `estado` - Two-letter Brazilian state abbreviation
* `comissao` - Commission percentage earned by the employee on sales
* `ativo` - Whether the employee record is active: 1 for active, 0 for inactive
* `atributos` - Values of the account's user-defined registration fields, as a JSON array of objects (each wrapped in an 'atributo' key); the field definitions are in the registration\_custom\_fields stream
ERP user accounts.
**Primary key:** `id`\
**Replication:** full table
**Fields (17):**
* `id` - Unique identifier of the ERP user
* `nome` - Full name of the user
* `cpf` - Brazilian individual tax ID (CPF), formatted
* `rg` - Brazilian national ID document number
* `data_nascimento` - Date of birth, as YYYY-MM-DD
* `sexo` - Gender recorded for the user
* `grupo_id` - Identifier of the permission group the user belongs to
* `email` - Email address used to sign in to the ERP
* `observacoes` - Free-text notes about the user
* `telefone` - Landline phone number
* `celular1` - Primary mobile phone number
* `celular2` - Secondary mobile phone number
* `hora_entrada` - Scheduled start of the working day, as HH:MM:SS
* `hora_almoco_entrada` - Scheduled start of the lunch break, as HH:MM:SS
* `hora_almoco_saida` - Scheduled end of the lunch break, as HH:MM:SS
* `hora_saida` - Scheduled end of the working day, as HH:MM:SS
* `ativo` - Whether the user account is active: 1 for active, 0 for inactive
Stores (units) registered in the account.
**Primary key:** `id`\
**Replication:** full table
**Fields (2):**
* `id` - Unique identifier of the store
* `nome` - Name of the store, for example Matriz for the head office
Products for sale, with cost and selling prices, stock on hand, tax classification and variations.
**Primary key:** `id`\
**Replication:** full table
**Fields (25):**
* `id` - Unique identifier of the product
* `nome` - Name of the product
* `codigo_interno` - Internal SKU or reference code assigned by the company
* `codigo_barra` - Barcode (EAN/GTIN) of the product
* `possui_variacao` - Whether the product has variations such as size or colour: 1 for yes, 0 for no
* `possui_composicao` - Whether the product is assembled from other products: 1 for yes, 0 for no
* `movimenta_estoque` - Whether sales and purchases of this product move inventory: 1 for yes, 0 for no
* `peso` - Weight of the product, in kilograms
* `largura` - Width of the product, in centimetres
* `altura` - Height of the product, in centimetres
* `comprimento` - Length of the product, in centimetres
* `ativo` - Whether the product is active: 1 for active, 0 for inactive
* `grupo_id` - Identifier of the product group, joinable to the product\_groups stream
* `nome_grupo` - Name of the product group
* `descricao` - Long description of the product
* `estoque` - Current inventory quantity on hand
* `valor_custo` - Cost price of the product
* `valor_venda` - Default selling price of the product
* `cadastrado_em` - Timestamp the product was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the product, as YYYY-MM-DD HH:MM:SS
* `valores` - Price tiers for the product, as a JSON array of objects holding the price-type identifier and name, the markup applied, and the resulting cost and selling prices
* `variacoes` - Product variations, as a JSON array of objects (each wrapped in a 'variacao' key) holding the variation's own code, attributes, prices and stock
* `fiscal` - Tax classification of the product, as a JSON object holding the NCM and CEST codes, net and gross weight, and the approximate tax amount
* `fotos` - Product images, as a JSON array of objects holding the image URLs
* `atributos` - Values of the account's user-defined product fields, as a JSON array of objects; the field definitions are in the product\_custom\_fields stream
The product category hierarchy.
**Primary key:** `id`\
**Replication:** full table
**Fields (6):**
* `id` - Unique identifier of the product group
* `grupo_pai_id` - Identifier of the parent group, empty for a top-level group
* `nome` - Name of the product group
* `url` - URL slug used for the group in the online store
* `meta_palavras_chaves` - Meta keywords used for the group in the online store
* `meta_descricao` - Meta description used for the group in the online store
Services offered for sale.
**Primary key:** `id`\
**Replication:** full table
**Fields (5):**
* `id` - Unique identifier of the service
* `codigo` - Reference code assigned to the service
* `nome` - Name of the service
* `valor_venda` - Default selling price of the service
* `observacoes` - Free-text notes about the service
Quotes (orçamentos) issued to customers, with their line items and payment terms.
**Primary key:** `id`\
**Replication:** incremental on `data`
**Fields (52):**
* `id` - Unique identifier of the quote
* `codigo` - Sequential number shown to users for this quote
* `cliente_id` - Identifier of the customer, joinable to the customers stream
* `nome_cliente` - Name of the customer
* `vendedor_id` - Identifier of the salesperson, joinable to the employees stream
* `nome_vendedor` - Name of the salesperson
* `tecnico_id` - Identifier of the assigned technician, joinable to the employees stream
* `nome_tecnico` - Name of the assigned technician
* `situacao_id` - Identifier of the quote status, joinable to the matching status stream
* `nome_situacao` - Name of the quote status, for example Em aberto or Finalizado
* `cor_situacao` - Hex colour the ERP uses to display the status
* `valor_total` - Total value of the quote, after discounts and freight
* `valor_produtos` - Subtotal of the product line items
* `valor_servicos` - Subtotal of the service line items
* `valor_custo` - Total cost of the line items, used to derive margin
* `valor_frete` - Freight amount charged
* `desconto_valor` - Discount applied as an absolute amount
* `desconto_porcentagem` - Discount applied as a percentage
* `transportadora_id` - Identifier of the freight carrier, joinable to the carriers stream
* `nome_transportadora` - Name of the freight carrier
* `centro_custo_id` - Identifier of the cost centre, joinable to the cost\_centers stream
* `nome_centro_custo` - Name of the cost centre
* `nome_canal_venda` - Name of the sales channel the document came through
* `nome_loja` - Name of the store that owns the document
* `aos_cuidados_de` - Attention-to name the document is addressed to
* `validade` - Number of days the document remains valid
* `introducao` - Introductory text printed at the top of the document
* `observacoes` - Notes visible to the customer
* `observacoes_interna` - Internal notes, not shown to the customer
* `condicao_pagamento` - Payment terms agreed for the document
* `forma_pagamento_id` - Identifier of the payment method, joinable to the payment\_methods stream
* `nome_forma_pagamento` - Name of the payment method
* `data_primeira_parcela` - Due date of the first instalment, as YYYY-MM-DD
* `numero_parcelas` - Number of instalments the payment was split into
* `intervalo_dias` - Number of days between instalments
* `situacao_financeiro` - Whether the document has been posted to the financial ledger
* `situacao_estoque` - Whether the document has moved inventory
* `nota_fiscal_id` - Identifier of the product invoice issued for this document, joinable to the product\_invoices stream
* `nota_fiscal_servico_id` - Identifier of the service invoice issued for this document, joinable to the service\_invoices stream
* `exibir_endereco` - Whether the customer address is printed on the document: 1 for yes, 0 for no
* `hash` - Opaque token used to build the document's public share link
* `cadastrado_em` - Timestamp the quote was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the quote, as YYYY-MM-DD HH:MM:SS
* `produtos` - Product line items, as a JSON array of objects (each wrapped in a 'produto' key) holding the product identifier, name, quantity, unit price and total
* `servicos` - Service line items, as a JSON array of objects (each wrapped in a 'servico' key) holding the service identifier, name, quantity, unit price and total
* `equipamentos` - Customer equipment attached to the document, as a JSON array of objects (each wrapped in an 'equipamento' key) holding the equipment description, brand, model and serial number
* `pagamentos` - Instalments generated for the document, as a JSON array of objects (each wrapped in a 'pagamento' key) holding the due date, amount and payment method
* `enderecos` - Delivery and billing addresses for the document, as a JSON array of objects (each wrapped in an 'endereco' key)
* `atributos` - Values of the account's user-defined fields for this document, as a JSON array of objects (each wrapped in an 'atributo' key)
* `anexos` - Files attached to the document, as a JSON array of objects (each wrapped in an 'anexo' key) holding the file name and URL
* `data` - Date the quote was issued, as YYYY-MM-DD; this is the field the API's data\_inicio filter matches and the incremental replication key
* `previsao_entrega` - Expected delivery date, as YYYY-MM-DD
Sales (vendas), covering product sales, service sales and counter sales.
**Primary key:** `id`\
**Replication:** incremental on `data`
**Fields (52):**
* `id` - Unique identifier of the sale
* `codigo` - Sequential number shown to users for this sale
* `cliente_id` - Identifier of the customer, joinable to the customers stream
* `nome_cliente` - Name of the customer
* `vendedor_id` - Identifier of the salesperson, joinable to the employees stream
* `nome_vendedor` - Name of the salesperson
* `tecnico_id` - Identifier of the assigned technician, joinable to the employees stream
* `nome_tecnico` - Name of the assigned technician
* `situacao_id` - Identifier of the sale status, joinable to the matching status stream
* `nome_situacao` - Name of the sale status, for example Em aberto or Finalizado
* `cor_situacao` - Hex colour the ERP uses to display the status
* `valor_total` - Total value of the sale, after discounts and freight
* `valor_produtos` - Subtotal of the product line items
* `valor_servicos` - Subtotal of the service line items
* `valor_custo` - Total cost of the line items, used to derive margin
* `valor_frete` - Freight amount charged
* `desconto_valor` - Discount applied as an absolute amount
* `desconto_porcentagem` - Discount applied as a percentage
* `transportadora_id` - Identifier of the freight carrier, joinable to the carriers stream
* `nome_transportadora` - Name of the freight carrier
* `centro_custo_id` - Identifier of the cost centre, joinable to the cost\_centers stream
* `nome_centro_custo` - Name of the cost centre
* `nome_canal_venda` - Name of the sales channel the document came through
* `nome_loja` - Name of the store that owns the document
* `aos_cuidados_de` - Attention-to name the document is addressed to
* `validade` - Number of days the document remains valid
* `introducao` - Introductory text printed at the top of the document
* `observacoes` - Notes visible to the customer
* `observacoes_interna` - Internal notes, not shown to the customer
* `condicao_pagamento` - Payment terms agreed for the document
* `forma_pagamento_id` - Identifier of the payment method, joinable to the payment\_methods stream
* `nome_forma_pagamento` - Name of the payment method
* `data_primeira_parcela` - Due date of the first instalment, as YYYY-MM-DD
* `numero_parcelas` - Number of instalments the payment was split into
* `intervalo_dias` - Number of days between instalments
* `situacao_financeiro` - Whether the document has been posted to the financial ledger
* `situacao_estoque` - Whether the document has moved inventory
* `nota_fiscal_id` - Identifier of the product invoice issued for this document, joinable to the product\_invoices stream
* `nota_fiscal_servico_id` - Identifier of the service invoice issued for this document, joinable to the service\_invoices stream
* `exibir_endereco` - Whether the customer address is printed on the document: 1 for yes, 0 for no
* `hash` - Opaque token used to build the document's public share link
* `cadastrado_em` - Timestamp the sale was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the sale, as YYYY-MM-DD HH:MM:SS
* `produtos` - Product line items, as a JSON array of objects (each wrapped in a 'produto' key) holding the product identifier, name, quantity, unit price and total
* `servicos` - Service line items, as a JSON array of objects (each wrapped in a 'servico' key) holding the service identifier, name, quantity, unit price and total
* `equipamentos` - Customer equipment attached to the document, as a JSON array of objects (each wrapped in an 'equipamento' key) holding the equipment description, brand, model and serial number
* `pagamentos` - Instalments generated for the document, as a JSON array of objects (each wrapped in a 'pagamento' key) holding the due date, amount and payment method
* `enderecos` - Delivery and billing addresses for the document, as a JSON array of objects (each wrapped in an 'endereco' key)
* `atributos` - Values of the account's user-defined fields for this document, as a JSON array of objects (each wrapped in an 'atributo' key)
* `anexos` - Files attached to the document, as a JSON array of objects (each wrapped in an 'anexo' key) holding the file name and URL
* `data` - Date the sale was issued, as YYYY-MM-DD; this is the field the API's data\_inicio filter matches and the incremental replication key
* `prazo_entrega` - Agreed delivery deadline, as YYYY-MM-DD
Service orders (ordens de serviço), including the customer equipment attached to them.
**Primary key:** `id`\
**Replication:** incremental on `data_entrada`
**Fields (52):**
* `id` - Unique identifier of the service order
* `codigo` - Sequential number shown to users for this service order
* `cliente_id` - Identifier of the customer, joinable to the customers stream
* `nome_cliente` - Name of the customer
* `vendedor_id` - Identifier of the salesperson, joinable to the employees stream
* `nome_vendedor` - Name of the salesperson
* `tecnico_id` - Identifier of the assigned technician, joinable to the employees stream
* `nome_tecnico` - Name of the assigned technician
* `situacao_id` - Identifier of the service order status, joinable to the matching status stream
* `nome_situacao` - Name of the service order status, for example Em aberto or Finalizado
* `cor_situacao` - Hex colour the ERP uses to display the status
* `valor_total` - Total value of the service order, after discounts and freight
* `valor_produtos` - Subtotal of the product line items
* `valor_servicos` - Subtotal of the service line items
* `valor_custo` - Total cost of the line items, used to derive margin
* `valor_frete` - Freight amount charged
* `desconto_valor` - Discount applied as an absolute amount
* `desconto_porcentagem` - Discount applied as a percentage
* `transportadora_id` - Identifier of the freight carrier, joinable to the carriers stream
* `nome_transportadora` - Name of the freight carrier
* `centro_custo_id` - Identifier of the cost centre, joinable to the cost\_centers stream
* `nome_centro_custo` - Name of the cost centre
* `nome_canal_venda` - Name of the sales channel the document came through
* `nome_loja` - Name of the store that owns the document
* `aos_cuidados_de` - Attention-to name the document is addressed to
* `validade` - Number of days the document remains valid
* `introducao` - Introductory text printed at the top of the document
* `observacoes` - Notes visible to the customer
* `observacoes_interna` - Internal notes, not shown to the customer
* `condicao_pagamento` - Payment terms agreed for the document
* `forma_pagamento_id` - Identifier of the payment method, joinable to the payment\_methods stream
* `nome_forma_pagamento` - Name of the payment method
* `data_primeira_parcela` - Due date of the first instalment, as YYYY-MM-DD
* `numero_parcelas` - Number of instalments the payment was split into
* `intervalo_dias` - Number of days between instalments
* `situacao_financeiro` - Whether the document has been posted to the financial ledger
* `situacao_estoque` - Whether the document has moved inventory
* `nota_fiscal_id` - Identifier of the product invoice issued for this document, joinable to the product\_invoices stream
* `nota_fiscal_servico_id` - Identifier of the service invoice issued for this document, joinable to the service\_invoices stream
* `exibir_endereco` - Whether the customer address is printed on the document: 1 for yes, 0 for no
* `hash` - Opaque token used to build the document's public share link
* `cadastrado_em` - Timestamp the service order was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the service order, as YYYY-MM-DD HH:MM:SS
* `produtos` - Product line items, as a JSON array of objects (each wrapped in a 'produto' key) holding the product identifier, name, quantity, unit price and total
* `servicos` - Service line items, as a JSON array of objects (each wrapped in a 'servico' key) holding the service identifier, name, quantity, unit price and total
* `equipamentos` - Customer equipment attached to the document, as a JSON array of objects (each wrapped in an 'equipamento' key) holding the equipment description, brand, model and serial number
* `pagamentos` - Instalments generated for the document, as a JSON array of objects (each wrapped in a 'pagamento' key) holding the due date, amount and payment method
* `enderecos` - Delivery and billing addresses for the document, as a JSON array of objects (each wrapped in an 'endereco' key)
* `atributos` - Values of the account's user-defined fields for this document, as a JSON array of objects (each wrapped in an 'atributo' key)
* `anexos` - Files attached to the document, as a JSON array of objects (each wrapped in an 'anexo' key) holding the file name and URL
* `data_entrada` - Date the service order was opened, as YYYY-MM-DD; this is the field the API's data\_inicio filter matches and the incremental replication key
* `data_saida` - Date the service order was closed and the item returned, as YYYY-MM-DD; empty while the order is still open
Purchase orders placed with suppliers.
**Primary key:** `id`\
**Replication:** full table
**Fields (31):**
* `id` - Unique identifier of the purchase
* `codigo` - Sequential number shown to users for this purchase
* `fornecedor_id` - Identifier of the supplier, joinable to the suppliers stream
* `nome_fornecedor` - Name of the supplier
* `data_emissao` - Date the purchase was issued, as YYYY-MM-DD
* `situacao_id` - Identifier of the purchase status, joinable to the purchase\_statuses stream
* `nome_situacao` - Name of the purchase status
* `centro_custo_id` - Identifier of the cost centre, joinable to the cost\_centers stream
* `nome_centro_custo` - Name of the cost centre
* `numero_nfe` - Number of the supplier's product invoice (NF-e) covering this purchase
* `transportadora_id` - Identifier of the freight carrier, joinable to the carriers stream
* `nome_transportadora` - Name of the freight carrier
* `valor_frete` - Freight amount on the purchase
* `pagar_frete` - Whether the freight is paid by the buyer: 1 for yes, 0 for no
* `valor_produtos` - Subtotal of the purchased product line items
* `valor_impostos` - Total taxes on the purchase
* `desconto_valor` - Discount applied as an absolute amount
* `desconto_porcentagem` - Discount applied as a percentage
* `valor_total` - Total value of the purchase, after taxes, freight and discounts
* `forma_pagamento` - Name of the payment method agreed with the supplier
* `forma_pagamento_id` - Identifier of the payment method, joinable to the payment\_methods stream
* `numero_parcelas` - Number of instalments the payment was split into
* `intervalo_dias` - Number of days between instalments
* `data_primeira_parcela` - Due date of the first instalment, as YYYY-MM-DD
* `observacoes` - Notes visible to the supplier
* `observacoes_interna` - Internal notes, not shared with the supplier
* `cadastrado_em` - Timestamp the purchase was created, as YYYY-MM-DD HH:MM:SS
* `pagamentos` - Instalments generated for the purchase, as a JSON array of objects (each wrapped in a 'pagamento' key) holding the due date, amount and payment method
* `situacoes` - Status history of the purchase, as a JSON array of objects (each wrapped in a 'situacao' key) holding the status and when it was applied
* `produtos` - Purchased product line items, as a JSON array of objects (each wrapped in a 'produto' key) holding the product identifier, name, quantity, unit cost and total
* `campos_extras` - Values of the account's user-defined purchase fields, as a JSON array of objects
NF-e (model 55) product invoices, with issuer, recipient, tax totals and line items.
**Primary key:** `id`\
**Replication:** incremental on `data_emissao`
**Fields (97):**
* `id` - Unique identifier of the product invoice
* `empresa_id` - Identifier of the company that issued the document
* `loja_id` - Identifier of the issuing store, joinable to the stores stream
* `nome_loja` - Name of the issuing store
* `pedido_id` - Identifier of the sale this document was issued for, joinable to the sales stream
* `compra_id` - Identifier of the purchase this document was issued for, joinable to the purchases stream
* `chave` - 44-digit access key assigned to the document by SEFAZ
* `protocolo` - SEFAZ protocol number returned when the document was authorized
* `cfop_id` - Internal identifier of the CFOP operation code
* `codigo_cfop` - CFOP code classifying the fiscal operation
* `descricao_cfop` - Description of the CFOP operation code
* `natureza_operacao` - Nature of the operation, for example Venda de mercadoria
* `indicador_pagamento` - Payment indicator: 0 for cash, 1 for instalments
* `modelo` - Fiscal document model number, 55 for NF-e and 65 for NFC-e
* `serie` - Series of the fiscal document
* `numero_nf` - Sequential number of the fiscal document within its series
* `data_emissao` - Date the document was issued, as YYYY-MM-DD; this is the field the API's data\_inicio filter matches and the incremental replication key
* `hora_emissao` - Time the document was issued, as HH:MM:SS
* `tipo_nf` - Direction of the document: 0 for inbound, 1 for outbound
* `indicador_destinatario` - Recipient's ICMS taxpayer status
* `cidade_id` - Identifier of the city where the operation took place, joinable to the cities stream
* `cidade_nome` - Name of the city where the operation took place
* `codigo_uf` - IBGE numeric code of the state where the operation took place
* `tipo_emissao` - Issuance mode: 1 for normal, other values for contingency
* `tipo_ambiente` - SEFAZ environment: 1 for production, 2 for homologation
* `finalidade_nf` - Purpose of the document: 1 normal, 2 complementary, 3 adjustment, 4 return
* `consumidor_final` - Whether the recipient is the final consumer: 1 for yes, 0 for no
* `tipo_atendimento` - How the sale was served, for example on premises or delivery
* `tipo_documento_emitente` - Whether the issuer is identified by CNPJ or CPF
* `cnpj_emitente` - Issuer's company tax ID (CNPJ)
* `cpf_emitente` - Issuer's individual tax ID (CPF)
* `nome_emitente` - Issuer's registered legal name
* `fantasia_emitente` - Issuer's trade name
* `logradouro_emitente` - Street name of the issuer's address
* `numero_emitente` - Street number of the issuer's address
* `complemento_emitente` - Address complement of the issuer's address
* `bairro_emitente` - District of the issuer's address
* `codigo_municipio_emitente` - IBGE code of the issuer's municipality
* `municipio_emitente` - Name of the issuer's municipality
* `uf_emitente` - Two-letter state abbreviation of the issuer's address
* `cep_emitente` - Postal code (CEP) of the issuer's address
* `codigo_pais_emitente` - Country code of the issuer's address, 1058 for Brazil
* `pais_emitente` - Country name of the issuer's address
* `telefone_emitente` - Issuer's phone number
* `ie_emitente` - Issuer's state tax registration (inscrição estadual)
* `iest_emitente` - Issuer's substitute-taxpayer state registration
* `im_emitente` - Issuer's municipal tax registration
* `destinatario_id` - Identifier of the recipient customer, joinable to the customers stream
* `destinatario_nome` - Name of the recipient customer
* `destinatario_fornecedor_id` - Identifier of the recipient supplier, joinable to the suppliers stream
* `destinatario_fornecedor_nome` - Name of the recipient supplier
* `destinatario_tipo_documento` - Whether the recipient is identified by CNPJ or CPF
* `destinatario_cnpj` - Recipient's company tax ID (CNPJ)
* `destinatario_cpf` - Recipient's individual tax ID (CPF)
* `destinatario_id_estrangeiro` - Recipient's foreign identification number, for exports
* `destinatario_logradouro` - Street name of the recipient's address
* `destinatario_numero` - Street number of the recipient's address
* `destinatario_complemento` - Address complement of the recipient's address
* `destinatario_bairro` - District of the recipient's address
* `destinatario_cidade_id` - Identifier of the recipient's city, joinable to the cities stream
* `destinatario_municipio_codigo` - IBGE code of the recipient's municipality
* `destinatario_municipio_nome` - Name of the recipient's municipality
* `destinatario_uf` - Two-letter state abbreviation of the recipient's address
* `destinatario_cep` - Postal code (CEP) of the recipient's address
* `destinatario_pais_codigo` - Country code of the recipient's address, 1058 for Brazil
* `destinatario_pais_nome` - Country name of the recipient's address
* `destinatario_telefone` - Recipient's phone number
* `destinatario_ie` - Recipient's state tax registration (inscrição estadual)
* `destinatario_im` - Recipient's municipal tax registration
* `destinatario_email` - Recipient's email address the document was sent to
* `valor_total_nf` - Total value of the fiscal document
* `valor_produtos` - Sum of the product line item values, before freight and discounts
* `valor_frete` - Freight amount on the document
* `valor_seguro` - Insurance amount on the document
* `valor_desconto` - Discount amount on the document
* `valor_outros` - Other accessory expenses on the document
* `base_icms` - ICMS tax base
* `valor_icms` - ICMS tax amount
* `valor_fcp` - Poverty-reduction fund (FCP) amount
* `base_icms_st` - Tax base for ICMS under tax substitution
* `valor_icms_st` - ICMS amount under tax substitution
* `valor_fcp_st` - FCP amount under tax substitution
* `valor_fcp_st_retido` - FCP amount previously withheld under tax substitution
* `valor_imposto_importacao` - Import duty amount
* `valor_ipi` - IPI tax amount
* `valor_ipi_devolvido` - IPI amount refunded on returned goods
* `valor_pis` - PIS contribution amount
* `valor_cofins` - COFINS contribution amount
* `valor_icms_desonerado` - ICMS amount exempted
* `valor_fcp_uf_destino` - FCP amount due to the destination state, on interstate sales
* `valor_icms_uf_remetente` - ICMS amount due to the origin state, on interstate sales
* `informacoes_complementares` - Additional information printed on the document
* `situacao_nf` - Status of the document, for example authorized, cancelled or pending
* `cadastrado_em` - Timestamp the product invoice was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the product invoice, as YYYY-MM-DD HH:MM:SS
* `produtos` - Product line items, as a JSON array of objects holding the product identifier and code, name, CFOP, unit, quantity, unit price, NCM and barcode
* `pagamento` - Payment details, as a JSON array of objects holding the instalment number, payment method, due date, amount, card brand, acquirer CNPJ and authorization code
NFC-e (model 65) consumer invoices, same shape as NF-e.
**Primary key:** `id`\
**Replication:** incremental on `data_emissao`
**Fields (97):**
* `id` - Unique identifier of the consumer invoice
* `empresa_id` - Identifier of the company that issued the document
* `loja_id` - Identifier of the issuing store, joinable to the stores stream
* `nome_loja` - Name of the issuing store
* `pedido_id` - Identifier of the sale this document was issued for, joinable to the sales stream
* `compra_id` - Identifier of the purchase this document was issued for, joinable to the purchases stream
* `chave` - 44-digit access key assigned to the document by SEFAZ
* `protocolo` - SEFAZ protocol number returned when the document was authorized
* `cfop_id` - Internal identifier of the CFOP operation code
* `codigo_cfop` - CFOP code classifying the fiscal operation
* `descricao_cfop` - Description of the CFOP operation code
* `natureza_operacao` - Nature of the operation, for example Venda de mercadoria
* `indicador_pagamento` - Payment indicator: 0 for cash, 1 for instalments
* `modelo` - Fiscal document model number, 55 for NF-e and 65 for NFC-e
* `serie` - Series of the fiscal document
* `numero_nf` - Sequential number of the fiscal document within its series
* `data_emissao` - Date the document was issued, as YYYY-MM-DD; this is the field the API's data\_inicio filter matches and the incremental replication key
* `hora_emissao` - Time the document was issued, as HH:MM:SS
* `tipo_nf` - Direction of the document: 0 for inbound, 1 for outbound
* `indicador_destinatario` - Recipient's ICMS taxpayer status
* `cidade_id` - Identifier of the city where the operation took place, joinable to the cities stream
* `cidade_nome` - Name of the city where the operation took place
* `codigo_uf` - IBGE numeric code of the state where the operation took place
* `tipo_emissao` - Issuance mode: 1 for normal, other values for contingency
* `tipo_ambiente` - SEFAZ environment: 1 for production, 2 for homologation
* `finalidade_nf` - Purpose of the document: 1 normal, 2 complementary, 3 adjustment, 4 return
* `consumidor_final` - Whether the recipient is the final consumer: 1 for yes, 0 for no
* `tipo_atendimento` - How the sale was served, for example on premises or delivery
* `tipo_documento_emitente` - Whether the issuer is identified by CNPJ or CPF
* `cnpj_emitente` - Issuer's company tax ID (CNPJ)
* `cpf_emitente` - Issuer's individual tax ID (CPF)
* `nome_emitente` - Issuer's registered legal name
* `fantasia_emitente` - Issuer's trade name
* `logradouro_emitente` - Street name of the issuer's address
* `numero_emitente` - Street number of the issuer's address
* `complemento_emitente` - Address complement of the issuer's address
* `bairro_emitente` - District of the issuer's address
* `codigo_municipio_emitente` - IBGE code of the issuer's municipality
* `municipio_emitente` - Name of the issuer's municipality
* `uf_emitente` - Two-letter state abbreviation of the issuer's address
* `cep_emitente` - Postal code (CEP) of the issuer's address
* `codigo_pais_emitente` - Country code of the issuer's address, 1058 for Brazil
* `pais_emitente` - Country name of the issuer's address
* `telefone_emitente` - Issuer's phone number
* `ie_emitente` - Issuer's state tax registration (inscrição estadual)
* `iest_emitente` - Issuer's substitute-taxpayer state registration
* `im_emitente` - Issuer's municipal tax registration
* `destinatario_id` - Identifier of the recipient customer, joinable to the customers stream
* `destinatario_nome` - Name of the recipient customer
* `destinatario_fornecedor_id` - Identifier of the recipient supplier, joinable to the suppliers stream
* `destinatario_fornecedor_nome` - Name of the recipient supplier
* `destinatario_tipo_documento` - Whether the recipient is identified by CNPJ or CPF
* `destinatario_cnpj` - Recipient's company tax ID (CNPJ)
* `destinatario_cpf` - Recipient's individual tax ID (CPF)
* `destinatario_id_estrangeiro` - Recipient's foreign identification number, for exports
* `destinatario_logradouro` - Street name of the recipient's address
* `destinatario_numero` - Street number of the recipient's address
* `destinatario_complemento` - Address complement of the recipient's address
* `destinatario_bairro` - District of the recipient's address
* `destinatario_cidade_id` - Identifier of the recipient's city, joinable to the cities stream
* `destinatario_municipio_codigo` - IBGE code of the recipient's municipality
* `destinatario_municipio_nome` - Name of the recipient's municipality
* `destinatario_uf` - Two-letter state abbreviation of the recipient's address
* `destinatario_cep` - Postal code (CEP) of the recipient's address
* `destinatario_pais_codigo` - Country code of the recipient's address, 1058 for Brazil
* `destinatario_pais_nome` - Country name of the recipient's address
* `destinatario_telefone` - Recipient's phone number
* `destinatario_ie` - Recipient's state tax registration (inscrição estadual)
* `destinatario_im` - Recipient's municipal tax registration
* `destinatario_email` - Recipient's email address the document was sent to
* `valor_total_nf` - Total value of the fiscal document
* `valor_produtos` - Sum of the product line item values, before freight and discounts
* `valor_frete` - Freight amount on the document
* `valor_seguro` - Insurance amount on the document
* `valor_desconto` - Discount amount on the document
* `valor_outros` - Other accessory expenses on the document
* `base_icms` - ICMS tax base
* `valor_icms` - ICMS tax amount
* `valor_fcp` - Poverty-reduction fund (FCP) amount
* `base_icms_st` - Tax base for ICMS under tax substitution
* `valor_icms_st` - ICMS amount under tax substitution
* `valor_fcp_st` - FCP amount under tax substitution
* `valor_fcp_st_retido` - FCP amount previously withheld under tax substitution
* `valor_imposto_importacao` - Import duty amount
* `valor_ipi` - IPI tax amount
* `valor_ipi_devolvido` - IPI amount refunded on returned goods
* `valor_pis` - PIS contribution amount
* `valor_cofins` - COFINS contribution amount
* `valor_icms_desonerado` - ICMS amount exempted
* `valor_fcp_uf_destino` - FCP amount due to the destination state, on interstate sales
* `valor_icms_uf_remetente` - ICMS amount due to the origin state, on interstate sales
* `informacoes_complementares` - Additional information printed on the document
* `situacao_nf` - Status of the document, for example authorized, cancelled or pending
* `cadastrado_em` - Timestamp the consumer invoice was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the consumer invoice, as YYYY-MM-DD HH:MM:SS
* `produtos` - Product line items, as a JSON array of objects holding the product identifier and code, name, CFOP, unit, quantity, unit price, NCM and barcode
* `pagamento` - Payment details, as a JSON array of objects holding the instalment number, payment method, due date, amount, card brand, acquirer CNPJ and authorization code
NFS-e service invoices, with ISS tax base and withholding details.
**Primary key:** `id`\
**Replication:** incremental on `data_emissao`
**Fields (121):**
* `id` - Unique identifier of the service invoice
* `empresa_id` - Identifier of the company that issued the invoice
* `loja_id` - Identifier of the issuing store, joinable to the stores stream
* `nome_loja` - Name of the issuing store
* `contrato_id` - Identifier of the contract the invoice was issued against
* `pedido_id` - Identifier of the sale or service order the invoice was issued for
* `codigo` - Internal sequential code of the invoice
* `rps` - Number of the provisional service receipt (RPS) the invoice replaced
* `numero` - Number assigned to the invoice by the municipality
* `codigo_verificacao` - Verification code used to validate the invoice on the municipal portal
* `serie` - Series of the invoice
* `data_emissao` - Date the invoice was issued, as YYYY-MM-DD; this is the field the API's data\_inicio filter matches and the incremental replication key
* `data_emissao_ano` - Year the invoice was issued
* `data_emissao_mes` - Month the invoice was issued
* `hora_emissao` - Time the invoice was issued, as HH:MM:SS
* `codigo_natureza_operacao` - Code for the nature of the operation
* `nome_natureza_operacao` - Description of the nature of the operation
* `natureza_operacao` - Nature of the operation, as returned on records using the short field spelling
* `codigo_incidencia_issqn` - Code of the municipality where ISSQN is levied
* `cidade_incidencia_issqn` - Name of the municipality where ISSQN is levied
* `emitente_cnpj` - Issuer's company tax ID (CNPJ)
* `emitente_inscricao_municipal` - Issuer's municipal tax registration
* `emit_cnpj` - Issuer's company tax ID (CNPJ), as returned on records using the short field spelling
* `emit_inscricao_municipal` - Issuer's municipal tax registration, as returned on records using the short field spelling
* `destinatario_id_cliente` - Identifier of the recipient customer, joinable to the customers stream
* `destinatario_nome_cliente` - Name of the recipient customer
* `destinatario_tipo_pessoa` - Recipient entity type: PF for an individual, PJ for a company
* `destinatario_razao_social` - Recipient's registered legal name
* `destinatario_cnpj` - Recipient's company tax ID (CNPJ)
* `destinatario_cpf` - Recipient's individual tax ID (CPF)
* `destinatario_inscricao_municipal` - Recipient's municipal tax registration
* `destinatario_cep` - Postal code (CEP) of the recipient's address
* `destinatario_logradouro_endereco` - Street name of the recipient's address
* `destinatario_numero_endereco` - Street number of the recipient's address
* `destinatario_complemento_endereco` - Address complement of the recipient's address
* `destinatario_bairro_endereco` - District of the recipient's address
* `destinatario_cidade_id_endereco` - Identifier of the recipient's city, joinable to the cities stream
* `destinatario_nome_cidade_endereco` - Name of the recipient's city
* `destinatario_codigo_cidade_endereco` - IBGE code of the recipient's city
* `destinatario_estado_endereco` - Two-letter state abbreviation of the recipient's address
* `dest_codigo_siafi` - SIAFI code of the recipient's municipality
* `destinatario_telefone` - Recipient's phone number
* `destinatario_email` - Recipient's email address
* `dest_cliente_id` - Identifier of the recipient customer, as returned on records using the short field spelling
* `dest_nome_cliente` - Name of the recipient customer, as returned on records using the short field spelling
* `dest_tipo_pessoa` - Recipient entity type, as returned on records using the short field spelling
* `dest_razao_social` - Recipient's registered legal name, as returned on records using the short field spelling
* `dest_cnpj` - Recipient's CNPJ, as returned on records using the short field spelling
* `dest_cpf` - Recipient's CPF, as returned on records using the short field spelling
* `dest_inscricao_municipal` - Recipient's municipal tax registration, as returned on records using the short field spelling
* `dest_cep` - Recipient's postal code, as returned on records using the short field spelling
* `dest_logradouro` - Recipient's street name, as returned on records using the short field spelling
* `dest_numero` - Recipient's street number, as returned on records using the short field spelling
* `dest_complemento` - Recipient's address complement, as returned on records using the short field spelling
* `dest_bairro` - Recipient's district, as returned on records using the short field spelling
* `dest_cidade_id` - Recipient's city identifier, as returned on records using the short field spelling
* `dest_nome_cidade` - Recipient's city name, as returned on records using the short field spelling
* `dest_codigo_cidade` - Recipient's IBGE city code, as returned on records using the short field spelling
* `dest_estado` - Recipient's state abbreviation, as returned on records using the short field spelling
* `dest_telefone` - Recipient's phone number, as returned on records using the short field spelling
* `dest_email` - Recipient's email address, as returned on records using the short field spelling
* `prestador_cep` - Postal code (CEP) of the service provider's address
* `prestador_logradouro` - Street name of the service provider's address
* `prestador_numero` - Street number of the service provider's address
* `prestador_complemento` - Address complement of the service provider's address
* `prestador_bairro` - District of the service provider's address
* `prestador_cidade_id` - Identifier of the service provider's city, joinable to the cities stream
* `prestador_nome_cidade` - Name of the service provider's city
* `prestador_codigo_cidade` - IBGE code of the service provider's city
* `prestador_estado` - Two-letter state abbreviation of the service provider's address
* `prest_cep` - Provider's postal code, as returned on records using the short field spelling
* `prest_logradouro` - Provider's street name, as returned on records using the short field spelling
* `prest_numero` - Provider's street number, as returned on records using the short field spelling
* `prest_complemento` - Provider's address complement, as returned on records using the short field spelling
* `prest_bairro` - Provider's district, as returned on records using the short field spelling
* `prest_cidade_id` - Provider's city identifier, as returned on records using the short field spelling
* `prest_nome_cidade` - Provider's city name, as returned on records using the short field spelling
* `prest_codigo_cidade` - Provider's IBGE city code, as returned on records using the short field spelling
* `prest_estado` - Provider's state abbreviation, as returned on records using the short field spelling
* `descricao` - Description of the service rendered, as printed on the invoice
* `atividade_id` - Internal identifier of the service activity
* `codigo_atividade` - Municipal service code for the activity
* `atividade_descricao` - Description of the service activity
* `atividade_codigo_tributacao` - Municipal taxation code for the activity
* `atividade_cnae` - CNAE code for the activity
* `descricao_atividade` - Description of the service activity, as returned on records using the short field spelling
* `codigo_tributacao_atividade` - Municipal taxation code, as returned on records using the short field spelling
* `cnae_atividade` - CNAE code, as returned on records using the short field spelling
* `valor_servico` - Gross value of the service rendered
* `valor_base_calculo` - ISS tax base
* `valor_iss` - ISS tax amount
* `iss_retido` - Whether ISS was withheld by the recipient: 1 for yes, 0 for no
* `valor_deducoes` - Deductions applied before the ISS tax base
* `valor_aliquota` - ISS rate applied, as a percentage
* `descontos` - Discounts applied to the invoice
* `valor_total` - Net total of the invoice after taxes and discounts
* `construcao_civil` - Whether the service is civil construction: 1 for yes, 0 for no
* `codigo_obra` - Construction site registration code (CNO), for civil construction
* `codigo_art` - Technical responsibility note (ART) code, for civil construction
* `aliquota_cofins` - COFINS rate applied, as a percentage
* `aliquota_pis` - PIS rate applied, as a percentage
* `aliquota_csll` - CSLL rate applied, as a percentage
* `aliquota_ir` - Income tax rate applied, as a percentage
* `aliquota_inss` - INSS rate applied, as a percentage
* `valor_cofins` - COFINS amount
* `valor_pis` - PIS amount
* `valor_csll` - CSLL amount
* `valor_ir` - Income tax amount
* `valor_inss` - INSS amount
* `cofins_retido` - Whether COFINS was withheld: 1 for yes, 0 for no
* `pis_retido` - Whether PIS was withheld: 1 for yes, 0 for no
* `csll_retido` - Whether CSLL was withheld: 1 for yes, 0 for no
* `ir_retido` - Whether income tax was withheld: 1 for yes, 0 for no
* `inss_retido` - Whether INSS was withheld: 1 for yes, 0 for no
* `outras_informacoes` - Additional information printed on the invoice
* `situacao` - Status of the invoice, for example issued or cancelled
* `hash` - Opaque token used to build the invoice's public link
* `usuario_id` - Identifier of the ERP user who issued the invoice, joinable to the users stream
* `nome_usuario` - Name of the ERP user who issued the invoice
* `cadastrado_em` - Timestamp the invoice was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the invoice, as YYYY-MM-DD HH:MM:SS
Accounts payable: every payable entry, open and settled.
**Primary key:** `id`\
**Replication:** full table
**Fields (37):**
* `id` - Unique identifier of the payable entry
* `codigo` - Sequential number shown to users for this payable entry
* `descricao` - Description of what the payable entry is for
* `valor` - Face amount of the entry, before interest, discounts and fees
* `juros` - Interest or late-payment charges added to the entry
* `desconto` - Discount granted on the entry
* `taxa_banco` - Bank fee charged on the entry
* `taxa_operadora` - Card acquirer fee charged on the entry
* `valor_total` - Net settled amount, after interest, discounts and fees
* `liquidado` - Settlement state of the entry: 1 when confirmed, 0 while still open
* `data_vencimento` - Due date of the entry, as YYYY-MM-DD
* `data_liquidacao` - Date the entry was actually settled, as YYYY-MM-DD; empty while it is still open. This is the date the API's data\_inicio filter matches, falling back to the due date for entries that are still open
* `data_competencia` - Accrual date used for accounting purposes, as YYYY-MM-DD
* `plano_contas_id` - Identifier of the chart-of-accounts entry, joinable to the chart\_of\_accounts stream
* `nome_plano_conta` - Name of the chart-of-accounts entry
* `centro_custo_id` - Identifier of the cost centre, joinable to the cost\_centers stream
* `nome_centro_custo` - Name of the cost centre
* `conta_bancaria_id` - Identifier of the bank account, joinable to the bank\_accounts stream
* `nome_conta_bancaria` - Name of the bank account
* `forma_pagamento_id` - Identifier of the payment method, joinable to the payment\_methods stream
* `nome_forma_pagamento` - Name of the payment method
* `entidade` - Which kind of payee the entry is settled with, telling you which of the party identifier fields below is populated
* `cliente_id` - Identifier of the customer, when the entry is settled with a customer
* `nome_cliente` - Name of the customer, when the entry is settled with a customer
* `fornecedor_id` - Identifier of the supplier, when the entry is settled with a supplier
* `nome_fornecedor` - Name of the supplier, when the entry is settled with a supplier
* `transportadora_id` - Identifier of the carrier, when the entry is settled with a carrier
* `nome_transportadora` - Name of the carrier, when the entry is settled with a carrier
* `funcionario_id` - Identifier of the employee, when the entry is settled with an employee
* `nome_funcionario` - Name of the employee, when the entry is settled with an employee
* `usuario_id` - Identifier of the ERP user who recorded the entry, joinable to the users stream
* `nome_usuario` - Name of the ERP user who recorded the entry
* `loja_id` - Identifier of the store the entry belongs to, joinable to the stores stream
* `nome_loja` - Name of the store the entry belongs to
* `cadastrado_em` - Timestamp the payable entry was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the payable entry, as YYYY-MM-DD HH:MM:SS
* `atributos` - Values of the account's user-defined financial fields, as a JSON array of objects (each wrapped in an 'atributo' key); the field definitions are in the financial\_custom\_fields stream
Accounts receivable: every receivable entry, open and settled.
**Primary key:** `id`\
**Replication:** full table
**Fields (37):**
* `id` - Unique identifier of the receivable entry
* `codigo` - Sequential number shown to users for this receivable entry
* `descricao` - Description of what the receivable entry is for
* `valor` - Face amount of the entry, before interest, discounts and fees
* `juros` - Interest or late-payment charges added to the entry
* `desconto` - Discount granted on the entry
* `taxa_banco` - Bank fee charged on the entry
* `taxa_operadora` - Card acquirer fee charged on the entry
* `valor_total` - Net settled amount, after interest, discounts and fees
* `liquidado` - Settlement state of the entry: 1 when confirmed, 0 while still open
* `data_vencimento` - Due date of the entry, as YYYY-MM-DD
* `data_liquidacao` - Date the entry was actually settled, as YYYY-MM-DD; empty while it is still open. This is the date the API's data\_inicio filter matches, falling back to the due date for entries that are still open
* `data_competencia` - Accrual date used for accounting purposes, as YYYY-MM-DD
* `plano_contas_id` - Identifier of the chart-of-accounts entry, joinable to the chart\_of\_accounts stream
* `nome_plano_conta` - Name of the chart-of-accounts entry
* `centro_custo_id` - Identifier of the cost centre, joinable to the cost\_centers stream
* `nome_centro_custo` - Name of the cost centre
* `conta_bancaria_id` - Identifier of the bank account, joinable to the bank\_accounts stream
* `nome_conta_bancaria` - Name of the bank account
* `forma_pagamento_id` - Identifier of the payment method, joinable to the payment\_methods stream
* `nome_forma_pagamento` - Name of the payment method
* `entidade` - Which kind of payer the entry is settled with, telling you which of the party identifier fields below is populated
* `cliente_id` - Identifier of the customer, when the entry is settled with a customer
* `nome_cliente` - Name of the customer, when the entry is settled with a customer
* `fornecedor_id` - Identifier of the supplier, when the entry is settled with a supplier
* `nome_fornecedor` - Name of the supplier, when the entry is settled with a supplier
* `transportadora_id` - Identifier of the carrier, when the entry is settled with a carrier
* `nome_transportadora` - Name of the carrier, when the entry is settled with a carrier
* `funcionario_id` - Identifier of the employee, when the entry is settled with an employee
* `nome_funcionario` - Name of the employee, when the entry is settled with an employee
* `usuario_id` - Identifier of the ERP user who recorded the entry, joinable to the users stream
* `nome_usuario` - Name of the ERP user who recorded the entry
* `loja_id` - Identifier of the store the entry belongs to, joinable to the stores stream
* `nome_loja` - Name of the store the entry belongs to
* `cadastrado_em` - Timestamp the receivable entry was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the receivable entry, as YYYY-MM-DD HH:MM:SS
* `atributos` - Values of the account's user-defined financial fields, as a JSON array of objects (each wrapped in an 'atributo' key); the field definitions are in the financial\_custom\_fields stream
Payment methods, including bank slip and card acquirer configuration.
**Primary key:** `id`\
**Replication:** full table
**Fields (44):**
* `id` - Unique identifier of the payment method
* `nome` - Name of the payment method, for example Dinheiro or Cartão de crédito
* `tipo` - Type of the payment method, for example cash, card or bank slip
* `conta_bancaria_id` - Identifier of the bank account entries settle into, joinable to the bank\_accounts stream
* `nome_conta_bancaria` - Name of the bank account entries settle into
* `movimentacao` - Whether using this method moves the bank account balance: 1 for yes, 0 for no
* `confirmar_financeiro` - Whether entries are settled automatically when created: 1 for yes, 0 for no
* `maximo_parcelas` - Maximum number of instalments allowed
* `intervalo_parcelas` - Number of days between instalments
* `intervalo_primeira_parcela` - Number of days until the first instalment falls due
* `disponivel_pdv` - Whether the method is offered at the point of sale: 1 for yes, 0 for no
* `bandeira_cartao` - Card brand, for card payment methods
* `cnpj_credenciadora` - Company tax ID (CNPJ) of the card acquirer
* `taxa_banco` - Bank fee charged per transaction, as a percentage
* `taxa_operadora` - Card acquirer fee charged per transaction, as a percentage
* `boleto` - Whether the method issues a bank slip (boleto): 1 for yes, 0 for no
* `banco` - Bank code used for bank slip issuance
* `cnab` - CNAB layout used for the bank remittance file
* `carteira` - Bank portfolio (carteira) code used for bank slips
* `agencia` - Bank branch number
* `agencia_dv` - Check digit of the bank branch number
* `conta` - Bank account number
* `conta_dv` - Check digit of the bank account number
* `codigo_cliente` - Account holder code assigned by the bank
* `codigo_cliente_dv` - Check digit of the account holder code
* `taxa_multa_atraso` - Late-payment penalty rate applied on bank slips, as a percentage
* `taxa_mora_atraso` - Daily late-payment interest rate applied on bank slips, as a percentage
* `dias_juros` - Number of days after the due date before interest starts accruing
* `aceite` - Acceptance indicator printed on the bank slip
* `especie_doc` - Document kind code printed on the bank slip
* `protestar` - Whether unpaid bank slips are sent for protest: 1 for yes, 0 for no
* `dias_protesto` - Number of days after the due date before an unpaid slip is protested
* `ultimo_boleto` - Sequential number of the last bank slip issued
* `ultima_remessa` - Sequential number of the last remittance file generated
* `versao_layout_header` - CNAB file header layout version
* `versao_layout_header_lote` - CNAB batch header layout version
* `ambiente` - Bank integration environment: production or homologation
* `instrucao1` - First instruction line printed on the bank slip
* `instrucao2` - Second instruction line printed on the bank slip
* `instrucao3` - Third instruction line printed on the bank slip
* `instrucao4` - Fourth instruction line printed on the bank slip
* `local_pagamento1` - Place-of-payment text printed on the bank slip
* `demonstrativo1` - First statement line printed on the bank slip
* `demonstrativo2` - Second statement line printed on the bank slip
Bank accounts entries are settled into.
**Primary key:** `id`\
**Replication:** full table
**Fields (2):**
* `id` - Unique identifier of the bank account
* `nome` - Name of the bank account as shown in the ERP
The chart of accounts hierarchy.
**Primary key:** `id`\
**Replication:** full table
**Fields (7):**
* `id` - Unique identifier of the chart-of-accounts entry
* `conta_mae_id` - Identifier of the parent account, empty for a top-level account
* `nome_conta_mae` - Name of the parent account
* `classificacao` - Hierarchical classification code of the account, for example 1.1.02
* `tipo` - Account direction: D for debit (expense), C for credit (revenue)
* `nome_tipo` - Human-readable account direction, for example Débito or Crédito
* `nome` - Name of the account
Cost centres used to classify documents and ledger entries.
**Primary key:** `id`\
**Replication:** full table
**Fields (3):**
* `id` - Unique identifier of the cost centre
* `nome` - Name of the cost centre
* `cadastrado_em` - Timestamp the cost centre was created, as YYYY-MM-DD HH:MM:SS
Status values quotes can take.
**Primary key:** `id`\
**Replication:** full table
**Fields (3):**
* `id` - Unique identifier of the status, referenced by situacao\_id on quotes
* `nome` - Name of the status as shown in the ERP
* `padrao` - Whether this is the default status for new records: 1 for yes, 0 for no
Status values sales can take, and whether each one posts to finance and inventory.
**Primary key:** `id`\
**Replication:** full table
**Fields (6):**
* `id` - Unique identifier of the status, referenced by situacao\_id on sales
* `nome` - Name of the status as shown in the ERP
* `padrao` - Whether this is the default status for new records: 1 for yes, 0 for no
* `cor` - Hex colour the ERP uses to display this status
* `lancar` - Whether reaching this status posts the sale to the financial ledger and inventory: 1 for yes, 0 for no
* `editar` - Whether a sale in this status can still be edited: 1 for yes, 0 for no
Status values service orders can take.
**Primary key:** `id`\
**Replication:** full table
**Fields (3):**
* `id` - Unique identifier of the status, referenced by situacao\_id on service orders
* `nome` - Name of the status as shown in the ERP
* `padrao` - Whether this is the default status for new records: 1 for yes, 0 for no
Status values purchases can take.
**Primary key:** `id`\
**Replication:** full table
**Fields (4):**
* `id` - Unique identifier of the status, referenced by situacao\_id on purchases
* `nome` - Name of the status as shown in the ERP
* `padrao` - Whether this is the default status for new records: 1 for yes, 0 for no
* `tipo_lancamento` - What reaching this status posts, for example inventory, financial entries or both
Definitions of the user-defined fields on customers, suppliers and carriers.
**Primary key:** `id`\
**Replication:** full table
**Fields (6):**
* `id` - Unique identifier of the user-defined field, referenced by atributo\_id inside the atributos array on customers, suppliers and carriers
* `nome` - Label of the user-defined field
* `tipo` - Input type of the field, for example texto, data, check\_list or select
* `obrigatorio` - Whether filling the field is mandatory: 1 for yes, 0 for no
* `cadastrado_em` - Timestamp the field was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the field, as YYYY-MM-DD HH:MM:SS
Definitions of the user-defined fields on products.
**Primary key:** `id`\
**Replication:** full table
**Fields (6):**
* `id` - Unique identifier of the user-defined field, referenced by atributo\_id inside the atributos array on products
* `nome` - Label of the user-defined field
* `tipo` - Input type of the field, for example texto, data, check\_list or select
* `obrigatorio` - Whether filling the field is mandatory: 1 for yes, 0 for no
* `cadastrado_em` - Timestamp the field was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the field, as YYYY-MM-DD HH:MM:SS
Definitions of the user-defined fields on quotes.
**Primary key:** `id`\
**Replication:** full table
**Fields (7):**
* `id` - Unique identifier of the user-defined field, referenced by atributo\_id inside the atributos array on quotes
* `nome` - Label of the user-defined field
* `tipo` - Input type of the field, for example texto, data, check\_list or select
* `obrigatorio` - Whether filling the field is mandatory: 1 for yes, 0 for no
* `cadastrado_em` - Timestamp the field was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the field, as YYYY-MM-DD HH:MM:SS
* `exibir_impressao` - Whether the field is printed on the document: 1 for yes, 0 for no
Definitions of the user-defined fields on sales.
**Primary key:** `id`\
**Replication:** full table
**Fields (7):**
* `id` - Unique identifier of the user-defined field, referenced by atributo\_id inside the atributos array on sales
* `nome` - Label of the user-defined field
* `tipo` - Input type of the field, for example texto, data, check\_list or select
* `obrigatorio` - Whether filling the field is mandatory: 1 for yes, 0 for no
* `cadastrado_em` - Timestamp the field was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the field, as YYYY-MM-DD HH:MM:SS
* `exibir_impressao` - Whether the field is printed on the document: 1 for yes, 0 for no
Definitions of the user-defined fields on service orders.
**Primary key:** `id`\
**Replication:** full table
**Fields (7):**
* `id` - Unique identifier of the user-defined field, referenced by atributo\_id inside the atributos array on service orders
* `nome` - Label of the user-defined field
* `tipo` - Input type of the field, for example texto, data, check\_list or select
* `obrigatorio` - Whether filling the field is mandatory: 1 for yes, 0 for no
* `cadastrado_em` - Timestamp the field was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the field, as YYYY-MM-DD HH:MM:SS
* `exibir_impressao` - Whether the field is printed on the document: 1 for yes, 0 for no
Definitions of the user-defined fields on payable and receivable entries.
**Primary key:** `id`\
**Replication:** full table
**Fields (6):**
* `id` - Unique identifier of the user-defined field, referenced by atributo\_id inside the atributos array on payable and receivable entries
* `nome` - Label of the user-defined field
* `tipo` - Input type of the field, for example texto, data, check\_list or select
* `obrigatorio` - Whether filling the field is mandatory: 1 for yes, 0 for no
* `cadastrado_em` - Timestamp the field was created, as YYYY-MM-DD HH:MM:SS
* `modificado_em` - Timestamp of the last change to the field, as YYYY-MM-DD HH:MM:SS
Contact type labels used inside registration contact lists.
**Primary key:** `id`\
**Replication:** full table
**Fields (2):**
* `id` - Unique identifier of the contact type, referenced by tipo\_id inside the contatos array on party records
* `nome` - Name of the contact type, for example Financeiro or Comercial
Address type labels used inside registration address lists.
**Primary key:** `id`\
**Replication:** full table
**Fields (2):**
* `id` - Unique identifier of the address type, referenced by tipo\_id inside the enderecos array on party records
* `nome` - Name of the address type, for example Cobrança or Entrega
Brazilian states, the IBGE reference table.
**Primary key:** `id`\
**Replication:** full table
**Fields (4):**
* `id` - Unique identifier of the state in Gestão Click
* `codigo` - IBGE numeric code of the state
* `nome` - Full name of the state
* `sigla` - Two-letter abbreviation of the state, for example SP or MG
Brazilian municipalities, the IBGE reference table.
**Primary key:** `id`\
**Replication:** full table
**Fields (3):**
* `id` - Unique identifier of the city in Gestão Click, referenced by cidade\_id across the other streams
* `codigo` - IBGE numeric code of the municipality
* `nome` - Name of the municipality
# GitHub as a data source
Source: https://docs.nekt.com/sources/github
Bring repository, pull request, and commit data from GitHub to Nekt.
GitHub is a software development platform for source control, collaboration, and code review. The Nekt GitHub connector uses the GitHub REST API to extract repository metadata, pull requests, and commits into your Catalog.
![]()
## Configuring GitHub as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **GitHub** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a GitHub Personal Access Token (classic or fine-grained) with permission to read the repositories you want to sync.
The following configurations are available:
* **Personal Access Token**: Your GitHub Personal Access Token (classic or fine-grained) with repo scope. This field is required and stored securely.
* **Repositories**: Optional list of repositories in `owner/repo` format (one per line). If provided, only these repositories are synced. If left empty, the connector syncs all repositories accessible by the token.
* **Start Date**: Optional starting point used by incremental commit syncs. Only sync records created or updated after this date.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync:
* **repositories**
* **pull\_requests**
* **commits**
For faster extractions, select only the streams you need.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: Choose the layer where extracted GitHub tables will be created.
* **Folder**: Optionally group all GitHub tables inside a folder.
* **Table name**: A default name is suggested, but you can customize it. You can also add a prefix to all tables.
* **Sync Type**: Choose between **INCREMENTAL** and **FULL\_TABLE**.
* **Incremental**: Recommended for `commits`, using `committed_at` as the replication key.
* **Full table**: Useful for one-off backfills or full refreshes.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often your repositories change:
* **Hourly / every few hours** for active engineering analytics.
* **Daily** for standard operational reporting.
* **Weekly** for low-change repositories.
Optionally, you can define:
* **Delta Log Retention**: How long Nekt keeps previous table states. See [Resource control](https://docs.nekt.com/get-started/core-concepts/resource-control).
* **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**: Periodic full syncs in addition to incrementals.
When you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the extraction by clicking on the arrow button. Once a run completes successfully, your data appears in the Catalog.
You need at least one successful source run to see the tables in your [Catalog](https://app.nekt.ai/catalog).
# Streams and Fields
Below you'll find the available GitHub streams and their core fields.
Repository metadata for all repositories accessible by the token (or only the configured list in `repositories`).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Repository numeric ID (primary key) |
| `full_name` | String | Repository name in `owner/repo` format |
| `private` | Boolean | Indicates whether the repository is private |
| `visibility` | String | Repository visibility (`public`, `private`, etc.) |
| `default_branch` | String | Default branch |
| `language` | String | Primary detected language |
| `stargazers_count` | Integer | Number of stars |
| `forks_count` | Integer | Number of forks |
| `open_issues_count` | Integer | Number of open issues |
| `created_at` | DateTime | Timestamp when the repository was created |
| `updated_at` | DateTime | Timestamp when the repository was last updated |
| `pushed_at` | DateTime | Timestamp when the repository was last pushed to |
**Notes:**
* Primary key: `id`
* Replication: full-table style (no replication key)
* Child context: each repository emits `owner` and `repo` context used by `pull_requests` and `commits`
Pull requests for each repository. The connector fetches all pull request states (`open`, `closed`, and `merged`).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Pull request ID (primary key) |
| `number` | Integer | Pull request number inside the repository |
| `title` | String | Pull request title |
| `body` | String | Pull request description/body |
| `state` | String | Current state (`open`, `closed`) |
| `draft` | Boolean | Indicates if it is a draft PR |
| `locked` | Boolean | Indicates if the PR is locked |
| `user` | Object | Pull request author |
| `head` | Object | Source branch metadata |
| `base` | Object | Target branch metadata |
| `merged_at` | DateTime | Timestamp when merged |
| `closed_at` | DateTime | Timestamp when closed |
| `created_at` | DateTime | Timestamp when created |
| `updated_at` | DateTime | Timestamp when last updated |
| `additions` | Integer | Lines added |
| `deletions` | Integer | Lines deleted |
| `changed_files` | Integer | Number of files changed |
| `comments` | Integer | Number of comments |
| `review_comments` | Integer | Number of review comments |
| `commits` | Integer | Number of commits in the PR |
| `_sdc_repository` | String | Repository context in `owner/repo` format |
**Notes:**
* Primary key: `id`
* Replication: full-table style (no replication key)
* Includes repository context fields (`owner`, `repo`, `_sdc_repository`) for easier joins
Commits for each repository. This stream supports incremental sync using commit timestamp.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `sha` | String | Git SHA hash value of the object (primary key). |
| `node_id` | String | GraphQL node identifier of the record. |
| `html_url` | String | Web URL for this resource in GitHub. |
| `url` | String | API URL for this resource. |
| `commit.message` | String | Commit message. |
| `commit.author` | Object | Author metadata embedded in the commit (name, email, date). |
| `commit.committer` | Object | Committer metadata embedded in the commit (name, email, date). |
| `commit.tree` | Object | Git tree object referenced by this commit. |
| `commit.verification` | Object | Verification details (verified, reason, signature, payload). |
| `author` | Object | GitHub user object for the author (when available). |
| `committer` | Object | GitHub user object for the committer (when available). |
| `parents` | Array | Parent commit references. |
| `stats.additions` | Integer | Lines added. |
| `stats.deletions` | Integer | Lines deleted. |
| `stats.total` | Integer | Total lines changed. |
| `committed_at` | DateTime | Replication key (derived from `commit.committer.date`). |
| `_sdc_repository` | String | Repository context in `owner/repo` format. |
**Notes:**
* Primary key: `sha`
* Replication key: `committed_at`
* Incremental sync sends `since` to GitHub API based on state bookmark (or `start_date` when state is not available)
# Data Model
The connector follows a repository-centered model:
```mermaid theme={null}
graph TD;
Repositories["repositories"];
PullRequests["pull_requests"];
Commits["commits"];
Repositories -- "owner + repo context" --> PullRequests;
Repositories -- "owner + repo context" --> Commits;
```
# Use Cases for Data Analysis
This section includes practical SQL examples you can run in [Explorer](https://app.nekt.ai/explorer).
### 1. Pull Request throughput by repository
Measure how many pull requests are created, closed, and merged by repository.
```sql theme={null}
SELECT
_sdc_repository AS repository,
COUNT(*) AS total_prs,
SUM(CASE WHEN state = 'open' THEN 1 ELSE 0 END) AS open_prs,
SUM(CASE WHEN closed_at IS NOT NULL THEN 1 ELSE 0 END) AS closed_prs,
SUM(CASE WHEN merged_at IS NOT NULL THEN 1 ELSE 0 END) AS merged_prs
FROM
nekt_raw.github_pull_requests
GROUP BY
1
ORDER BY
total_prs DESC;
```
```sql theme={null}
SELECT
_sdc_repository AS repository,
COUNT(*) AS total_prs,
SUM(CASE WHEN state = 'open' THEN 1 ELSE 0 END) AS open_prs,
SUM(CASE WHEN closed_at IS NOT NULL THEN 1 ELSE 0 END) AS closed_prs,
SUM(CASE WHEN merged_at IS NOT NULL THEN 1 ELSE 0 END) AS merged_prs
FROM
`nekt_raw.github_pull_requests`
GROUP BY
1
ORDER BY
total_prs DESC;
```
### 2. Commit activity in the last 30 days
Track commit volume and active contributors by repository.
```sql theme={null}
SELECT
_sdc_repository AS repository,
COUNT(*) AS commits_last_30d,
COUNT(DISTINCT COALESCE(author.login, commit.author.email)) AS active_authors
FROM
nekt_raw.github_commits
WHERE
CAST(committed_at AS timestamp) >= current_timestamp - interval '30' day
GROUP BY
1
ORDER BY
commits_last_30d DESC;
```
```sql theme={null}
SELECT
_sdc_repository AS repository,
COUNT(*) AS commits_last_30d,
COUNT(DISTINCT COALESCE(author.login, JSON_VALUE(TO_JSON_STRING(commit), '$.author.email'))) AS active_authors
FROM
`nekt_raw.github_commits`
WHERE
TIMESTAMP(committed_at) >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY
1
ORDER BY
commits_last_30d DESC;
```
## Skills for agents
GitHub connector documentation as plain markdown, for use in AI agent contexts.
# Gleap as a data source
Source: https://docs.nekt.com/sources/gleap
Bring data from Gleap to Nekt.
Gleap is a customer feedback and bug reporting platform that helps product teams collect, manage, and analyze user feedback through in-app surveys, bug reports, feature requests, and live chat. It provides a unified inbox for managing all customer interactions and product insights.
![]()
## Configuring Gleap as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Gleap** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Gleap API credentials. You can find them in your Gleap dashboard under **Project Settings > Security > API Key**.
The following configurations are available:
* **API Key**: Your Gleap API key used to authenticate requests. This field is required.
* **Project ID**: The Gleap project ID found in your project settings. This field is required.
* **Start Date**: The earliest date from which records will be synced. Used for the first full sync and when no previous state exists. Format: `YYYY-MM-DDTHH:mm:ssZ`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables once the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between **INCREMENTAL** and **FULL\_TABLE**.
* **Incremental**: Each run fetches only new or updated records since the last replicated timestamp. Recommended for ongoing syncs so you keep every event without re-reading full history.
* **Full table**: Each run re-exports all data from the configured start date. Use when you need to backfill or fully refresh the dataset.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated.
Optionally, you can define some additional settings:
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Gleap and their corresponding fields:
| Field | Type | Description |
| - | - | - |
| id | string | Unique ticket identifier |
| bugId | integer | Numeric bug ID |
| title | string | Ticket title |
| description | string | Ticket description |
| type | string | Ticket type (BUG, FEATURE\_REQUEST, etc.) |
| status | string | Current status (OPEN, CLOSED, etc.) |
| priority | string | Priority level (HIGH, MEDIUM, LOW) |
| sentiment | string | Detected sentiment |
| createdAt | datetime | When the ticket was created |
| updatedAt | datetime | When the ticket was last updated |
| closedAt | datetime | When the ticket was closed |
| dueDate | datetime | Due date for the ticket |
| processingUser | string | Assigned user ID |
| processingTeam | string | Assigned team ID |
| session | string | Associated session ID |
| tags | array | Tags assigned to the ticket |
| archived | boolean | Whether the ticket is archived |
| isSpam | boolean | Whether the ticket is marked as spam |
| conversationClosed | boolean | Whether the conversation is closed |
| conversationRating | string | Rating given by the user |
| hasAgentReply | boolean | Whether an agent has replied |
| upvotesCount | integer | Number of upvotes |
| duplicatesCount | integer | Number of duplicates |
| score | number | Ticket score |
| channel | string | Channel information |
| aiSummary | string | AI-generated summary |
| shareToken | string | Share token for public access |
| formData | string | Form data (JSON) |
| customData | string | Custom data (JSON) |
| metaData | string | Metadata (JSON) |
| slaBreached | boolean | Whether SLA was breached |
| snoozed | boolean | Whether the ticket is snoozed |
| snoozedUntil | datetime | Snooze end date |
| firstAssignmentAt | datetime | When the ticket was first assigned |
| notificationsUnread | integer | Count of unread notifications |
| Field | Type | Description |
| - | - | - |
| id | string | Unique session identifier |
| gleapId | string | Gleap internal ID |
| gleapHash | string | Gleap hash |
| userId | string | User identifier |
| name | string | User display name |
| email | string | User email address |
| phone | string | User phone number |
| lang | string | User language |
| location | string | User location |
| avatar | string | Avatar URL |
| companyId | string | Company identifier |
| companyName | string | Company name |
| deviceType | string | Device type |
| platform | string | Platform (web, iOS, Android) |
| plan | string | User plan |
| value | number | User value |
| sla | number | SLA value |
| sessions | number | Number of sessions |
| blocked | boolean | Whether the user is blocked |
| unsubscribed | boolean | Whether the user unsubscribed |
| emailOnly | boolean | Whether it is an email-only contact |
| firstActivity | datetime | First activity date |
| lastActivity | datetime | Last activity date |
| lastHeardFrom | datetime | Last heard from date |
| lastPageView | string | Last page viewed |
| countFeedbackReports | integer | Number of feedback reports |
| countFeedbackComments | integer | Number of feedback comments |
| tags | array | Tags assigned to the session |
| customData | string | Custom data (JSON) |
| eventData | string | Event data (JSON) |
| Field | Type | Description |
| - | - | - |
| id | string | Unique message identifier |
| ticket | string | Associated ticket ID |
| type | string | Message type |
| status | string | Message status |
| bot | boolean | Whether the message is from a bot |
| isNote | boolean | Whether it is an internal note |
| kaiChat | boolean | Whether it is from the AI assistant |
| outbound | boolean | Whether it is an outbound message |
| sendToChannel | boolean | Whether it was sent to a channel |
| sessionNotificationsUnread | boolean | Unread notification flag |
| createdAt | datetime | When the message was created |
| updatedAt | datetime | When the message was last updated |
| data | string | Message content (JSON) |
| translatedData | string | Translated content (JSON) |
| comment | string | Comment text |
| rating | string | Message rating |
| user | string | Sender user ID |
| session | string | Associated session ID |
| emailId | string | Associated email ID |
| messageRef | string | Message reference |
| replyTo | string | Reply-to message ID |
| outboundTemplate | string | Outbound template ID |
| outboundSubject | string | Outbound email subject |
| attachments | string | Attachments (JSON) |
| actions | string | Actions (JSON) |
| channel | string | Channel information (JSON) |
| aiData | string | AI data (JSON) |
| Field | Type | Description |
| - | - | - |
| id | string | Unique team identifier |
| name | string | Team name |
| assignmentMethod | string | How tickets are assigned (random, balanced, manual, circular) |
| members | array | List of team member IDs |
| selectOnlyOnlineMembers | boolean | Whether to assign only to online members |
| limitTicketAmount | boolean | Whether there is a ticket limit per member |
| ticketAmount | number | Maximum tickets per member |
| createdAt | datetime | When the team was created |
| updatedAt | datetime | When the team was last updated |
| Field | Type | Description |
| - | - | - |
| id | string | Unique survey identifier |
| name | string | Survey name |
| type | string | Survey type |
| status | string | Survey status |
| createdAt | datetime | When the survey was created |
| updatedAt | datetime | When the survey was last updated |
| config | string | Survey configuration (JSON) |
| Field | Type | Description |
| - | - | - |
| id | string | Unique response identifier |
| survey\_id | string | Parent survey ID |
| session | string | Respondent session ID |
| createdAt | datetime | When the response was submitted |
| updatedAt | datetime | When the response was last updated |
| responses | string | Response data (JSON) |
| read | boolean | Whether the response has been read |
# Data Model
The following diagram illustrates the relationships between the core data streams in Gleap. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
erDiagram
SESSIONS {
string id PK
}
TICKETS {
string id PK
string session FK
}
MESSAGES {
string id PK
string ticket FK
string session FK
}
TEAMS {
string id PK
}
SURVEYS {
string id PK
}
SURVEY_RESPONSES {
string id PK
string survey_id FK
string session FK
}
SESSIONS ||--o{ TICKETS : "has"
SESSIONS ||--o{ MESSAGES : "sends"
SESSIONS ||--o{ SURVEY_RESPONSES : "submits"
TICKETS ||--o{ MESSAGES : "contains"
SURVEYS ||--o{ SURVEY_RESPONSES : "receives"
```
## Implementation Notes & Troubleshooting
### Incremental Sync & Pagination
* The connector natively supports incremental extractions (`state` capability enabled) using cursor-based pagination and URL-encoded date filtering for high reliability. This ensures no data is missed across extraction windows.
* Ensure that the start date format matches `YYYY-MM-DDTHH:mm:ssZ` if you decide to alter it via configuration.
### Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Authentication errors | Wrong API Key or Project ID | Verify your API credentials in your Gleap dashboard under **Project Settings > Security > API Key**. |
| Missing recent records | State configuration issue | For incremental syncs, make sure the prior runs finished successfully. The connector uses a URL-encoded start date marker automatically tracking the last fetched object to continue the process accurately. |
| Nested JSON objects | Unparsed properties | Data structures such as `customData`, `metaData`, or `eventData` are delivered as JSON strings. Apply proper JSON parsing (e.g. `JSON_EXTRACT`) in your downstream SQL queries inside Nekt Explorer to unnest them into individual fields. |
## Skills for agents
Gleap connector documentation as plain markdown, for use in AI agent contexts.
# Gmail as a data source
Source: https://docs.nekt.com/sources/gmail
Bring data from Gmail to Nekt.
Gmail is Google's email service. This connector extracts mailbox metadata and full message content via the Gmail API so you can analyze labels, headers, and message structure in your catalog.
![]()
## Configuring Gmail as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Gmail option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Authorize Nekt to access your Gmail data. Click the **Google Authorization** button and sign in with the Google account whose mailbox you want to sync. Grant the scopes required for reading messages and labels.
The following configurations are available:
* **Start date**: Only messages received on or after this date are included. The connector uses Gmail search (`after:YYYY/MM/DD`) based on this value.
* **Include SPAM and Trash**: When enabled, messages in Spam and Trash are included in the extraction. When disabled, they follow Gmail's default visibility for listings.
* **Enable email attachments stream**: (Default: `false`) When enabled, the source adds an email attachments stream that downloads every message's attachments and uploads them to a Nekt volume. This makes extractions significantly slower and heavier: each attachment is downloaded from Gmail and re-uploaded to Nekt. Only enable this if you need the email files stored in a Nekt volume.
* **Attachments Volume**: Nekt volume to upload email attachments to. Required when the email attachments stream is enabled.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental** (recommended for **Email**): each run brings new or updated messages based on the replication key (`internalDate`). Fits keeping a growing history of mail in the catalog.
* **Full table** (typical for **Label**): each run replaces the table with the current set of labels and their counts from Gmail.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Gmail and their corresponding fields:
Full Gmail messages for the authenticated user. The connector lists message IDs, then fetches each message resource so rows include headers, MIME structure, and nested parts. Sync is incremental on **`internalDate`** (message receive time).
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the record. |
| `threadId` | String | Identifier of the thread this message belongs to. |
| `labelIds` | Array | Identifiers of labels applied to the message. |
| `snippet` | String | Short excerpt of the message text. |
| `internalDate` | DateTime | Timestamp when the message was received by Gmail. |
| `historyId` | String | Identifier of the last history record that modified this message. |
| `sizeEstimate` | Integer | Estimated size of the message in bytes. |
| `payload` | Object | Parsed email structure including headers, body, and nested parts. |
| `payload.partId` | String | Identifier of the MIME message part. |
| `payload.mimeType` | String | MIME type of the message part. |
| `payload.filename` | String | Original filename of the attachment if present. |
| `payload.headers` | Array | RFC 2822 headers on this message part. |
| `payload.headers[].name` | String | Name of the email header field. |
| `payload.headers[].value` | String | Value of the email header field. |
| `payload.body` | Object | Body of this message part. |
| `payload.body.size` | Integer | Size of the body in bytes. |
| `payload.parts` | Array | Child MIME parts for multipart messages. |
| `payload.parts[].partId` | String | Identifier of the MIME message part. |
| `payload.parts[].mimeType` | String | MIME type of the message part. |
| `payload.parts[].filename` | String | Original filename of the attachment if present. |
| `payload.parts[].headers` | Array | RFC 2822 headers on this message part. |
| `payload.parts[].headers[].name` | String | Name of the email header field. |
| `payload.parts[].headers[].value` | String | Value of the email header field. |
| `payload.parts[].body` | Object | Body of this message part. |
| `payload.parts[].body.size` | Integer | Size of the body in bytes. |
| `payload.parts[].body.data` | String | Body data as a base64url-encoded string before decoding. |
| `payload.parts[].body.attachmentId` | String | Identifier of the attachment when fetched separately from the API. |
Downloads email attachments and uploads them to a Nekt volume. This stream is only active when "Enable email attachments stream" is enabled in the source configuration. It makes one API call per attachment to fetch its bytes and emits one metadata record per file.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `message_id` | String | ID of the message the attachment belongs to. |
| `internalDate` | DateTime | Timestamp when the message was received by Gmail. |
| `attachment_id` | String | Gmail identifier of the attachment. |
| `file_name` | String | Original filename of the attachment. |
| `unique_file_name` | String | Collision-free, filesystem-safe volume key for the attachment. |
All labels in the mailbox (system and user-defined), including visibility settings and aggregate counts.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the record. |
| `name` | String | Name of the record. |
| `messageListVisibility` | String | Whether the label is shown in the message list. |
| `labelListVisibility` | String | Whether the label is shown in the label list. |
| `type` | String | Type classification of the record. |
| `messagesTotal` | Integer | Total number of messages with this label. |
| `messagesUnread` | Integer | Number of unread messages with this label. |
| `threadsTotal` | Integer | Total number of threads with this label. |
| `threadsUnread` | Integer | Number of unread threads with this label. |
| `color` | Object | Display colors for the label in the Gmail UI. |
| `color.textColor` | String | Hex color for the label text. |
| `color.backgroundColor` | String | Hex color for the label background. |
# Data Model
```mermaid theme={null}
graph LR;
Email("Email");
EmailAttachments("Email Attachments");
Label("Label");
Email -- "labelIds[] contains label.id" --> Label;
EmailAttachments -- "message_id = Email.id" --> Email;
```
# Use Cases for Data Analysis
**Join messages to label names.** Resolve `labelIds` on each email to human-readable names using the **Label** table.
```sql theme={null}
SELECT
e.id AS message_id,
e.threadId,
e.internalDate,
l.name AS label_name
FROM nekt_raw.gmail_email e
CROSS JOIN nekt_raw.gmail_label l
WHERE contains(e.labelIds, l.id)
ORDER BY e.internalDate DESC
LIMIT 100;
```
Replace schema and table names (`nekt_raw`, `gmail_email`, `gmail_label`) with the layer and table names you configured for the source. On BigQuery, use `JOIN LATERAL UNNEST(e.labelIds) AS label_id` (or equivalent) instead of `contains`.
## Implementation Notes
### Extraction Behavior
* **Email** uses the Gmail `messages.list` query with `maxResults: 500` per page and `messages.get` per message, so runtime grows with mailbox volume.
* The **Start date** filter applies to the list query; ensure OAuth and API quotas are sufficient for initial backfills.
* **Label** is a single API call (`users.labels.list`); it does not use incremental state.
* The **Email Attachments** stream extracts each message's attachment parts and makes a separate API call per attachment to fetch its bytes before uploading to the Nekt volume. Because this process is heavy, it should only be enabled when file extraction is explicitly required.
### Payload and Timestamps
* `internalDate` is normalized to Unix time in **seconds** in post-processing (API returns milliseconds).
* Top-level `payload.body` in the schema exposes size only; inline and attachment content typically appears under `payload.parts` with optional decoded `data`. Only text body parts are decoded inline; file attachments carry an `attachmentId` and are fetched separately.
* Threading and deduplication are based on `threadId` and `id` as returned by Gmail.
## Skills for agents
Gmail connector documentation as plain markdown, for use in AI agent contexts.
# HighLevel as a data source
Source: https://docs.nekt.com/sources/gohighlevel
Bring data from HighLevel (GoHighLevel) to Nekt.
HighLevel (GoHighLevel) is a business management platform for service-based businesses. It provides customer relationship management, marketing automation, and operations tooling to help teams capture leads, manage pipelines, and run campaigns.
![]()
The connector is built for **agency** accounts: you authenticate with OAuth, then sync one or more **location IDs** (sub-accounts) in a single source. All location-scoped streams include `location_id` so you can filter or join across sub-accounts in the catalog.
## Configuring HighLevel as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the HighLevel option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Authorize Nekt with OAuth using an agency user that can access the locations you want to sync.
The following configurations are available:
* **OAuth (refresh token)**: Sign in and grant access so Nekt can call the HighLevel API on your behalf.
* **Location IDs**: The sub-account location IDs to include in the sync. See [HighLevel help](https://help.gohighlevel.com/support/solutions/articles/48001204848-how-do-i-find-my-client-s-location-id-) for how to find a location ID.
* **Start date**: Earliest point in time for **incremental** streams (`contacts`, `opportunities`). Records updated on or after this window are considered for historical loads and incremental bookmarks.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental** (`contacts`, `opportunities`, `notes`, `tasks`): each run fetches updates since the last successful bookmark (driven by API update timestamps and your **Start date**). Child streams (`notes`, `tasks`) automatically merge new or updated records based on their parent contacts.
* **Full table** (`locations`, `custom_fields`, `campaigns`, `pipelines`, `tags`, `forms`): each run replaces the current snapshot for that table.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from HighLevel and their corresponding fields:
Reference stream that emits one row per configured **Location ID**, with the agency **company** identifier. Child streams (contacts, opportunities, etc.) run in the context of each location.
**Replication:** FULL\_TABLE (not incremental). In the tap catalog this stream is **off by default**; enable it if you want an explicit dimension table of configured locations.
| Field | Type | Description |
| :- | :- | :- |
| `location_id` | String | Sub-account location identifier |
| `company_id` | String | Agency company identifier associated with the OAuth token |
Contacts for each location, loaded via the search API. **Incremental** replication uses `date_updated`.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Contact identifier (primary key) |
| `location_id` | String | Location scope |
| `first_name`, `last_name`, `first_name_lower_case`, `last_name_lower_case` | String | Contact name profile attributes |
| `email`, `valid_email`, `phone`, `phone_label` | String | Contact info |
| `company_name`, `business_name`, `business_id`, `website` | String | Business profile attributes |
| `address`, `city`, `state`, `country`, `postal_code` | String | Address profile attributes |
| `date_of_birth` | Integer | Unix timestamp from the API |
| `type`, `source`, `assigned_to`, `followers`, `tags`, `dnd` | String / Array | Other profile attributes |
| `additional_emails` | Array | Array of objects (`email`, `valid_email_date`) |
| `additional_phones` | Array | Array of objects (`phone`, `phone_label`) |
| `custom_fields` | Array | Array of objects (`id`, `value`). Non-string values are coerced to string/JSON. |
| `opportunities` | String | JSON string of opportunity payloads returned on the contact |
| `date_added`, `date_updated` | String | Timestamps (`date_updated` is the replication key) |
Notes attached to contacts, with their text content.
One API call per contact, so this stream is **off by default** in the catalog; enable it when note text is needed in the warehouse.
**Replication:** INCREMENTAL (merges on primary key using `date_added`).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Note identifier (primary key) |
| `contact_id` | String | Related contact identifier |
| `date_added` | String | Creation timestamp (used as replication key to ensure the loader merges records) |
Tasks attached to contacts, with their text content.
One API call per contact, so this stream is **off by default** in the catalog; enable it when task text is needed in the warehouse.
**Replication:** INCREMENTAL (merges on primary key using `due_date`).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Task identifier (primary key) |
| `contact_id` | String | Related contact identifier |
| `due_date` | String | Deadline timestamp (used as replication key to ensure the loader merges records) |
Custom field definitions for the location (for example fields attached to contacts or opportunities).
**Replication:** FULL\_TABLE.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Field definition identifier |
| `location_id` | String | Location scope |
| `name`, `field_key`, `model`, `data_type`, `placeholder`, `position` | String / Integer | Field metadata |
| `document_type`, `parent_id`, `standard` | String / Boolean | Field structure attributes |
| `picklist_options` | Array | Allowed labels (normalized from API options) |
| `date_added` | String | Timestamp |
Catalog of contact tags defined in each sub-account. `contacts.tags` only carries tag names; this stream lists every tag of the location (including ones no contact currently has) with its identifier. The endpoint needs the `locations/tags.readonly` scope, which older connections may not have granted (in which case it safely skips).
**Replication:** FULL\_TABLE.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the tag. |
| `name` | String | Name of the tag, as it appears on contacts. |
| `location_id` | String | Identifier of the associated location. |
Marketing campaigns for the location.
**Replication:** FULL\_TABLE.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Campaign identifier |
| `name` | String | Campaign name |
| `status` | String | Campaign status from the API |
| `location_id` | String | Location scope |
Opportunity pipelines and their stages for the location.
**Replication:** FULL\_TABLE.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Pipeline identifier |
| `name` | String | Pipeline name |
| `location_id` | String | Location scope |
| `show_in_funnel`, `show_in_pie_chart` | Boolean | UI flags |
| `origin_id` | String | Origin identifier |
| `date_added`, `date_updated` | String | Timestamps |
| `stages` | Array | Nested array of stages (`id`, `name`, `position`, `show_in_pie_chart`, `show_in_funnel`, `origin_id`) |
Forms defined in each sub-account.
**Replication:** FULL\_TABLE.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the form. |
| `name` | String | Name of the form. |
| `location_id` | String | Identifier of the associated location. |
Opportunities (deals) per location from the search API. **Incremental** replication uses `updated_at` (filtered via API `date_updated`).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Opportunity identifier |
| `location_id` | String | Location scope |
| `contact_id` | String | Related contact |
| `pipeline_id`, `pipeline_stage_id`, `pipeline_stage_u_id` | String | Pipeline associations |
| `name`, `status`, `source`, `assigned_to`, `followers` | String / Array | Core attributes |
| `monetary_value` | String | Monetary value (cast explicitly in SQL for numeric aggregations) |
| `lost_reason_id`, `index_version` | String | Loss tracking and versioning |
| `created_at`, `updated_at` | String | Timestamps (`updated_at` is the replication key) |
| `last_status_change_at`, `last_stage_change_at`, `last_action_date` | String | Status/stage change timestamps |
| `contact` | Object | Embedded contact summary (`id`, `name`, `company_name`, `email`, `phone`, `tags`) |
| `notes`, `tasks`, `calendar_events` | Array | Arrays of string identifiers or payloads from the API |
| `custom_fields` | Array | Array of objects (`id`, `field_value`) |
| `relations` | Array | Array of related records |
# Data Model
Relationships are driven by **location** scope, then CRM keys between opportunities, contacts, pipelines, and field definitions.
```mermaid theme={null}
graph TD;
subgraph "Scope"
Locations["locations"];
end
subgraph "Reference"
CustomFields["custom_fields"];
Tags["tags"];
Campaigns["campaigns"];
Pipelines["pipelines"];
Forms["forms"];
end
subgraph "Core CRM"
Contacts["contacts"];
Opportunities["opportunities"];
Notes["notes"];
Tasks["tasks"];
end
CustomFields -- "location_id" --> Locations;
Tags -- "location_id" --> Locations;
Campaigns -- "location_id" --> Locations;
Pipelines -- "location_id" --> Locations;
Forms -- "location_id" --> Locations;
Contacts -- "location_id" --> Locations;
Opportunities -- "location_id" --> Locations;
Opportunities -- "contact_id" --> Contacts;
Opportunities -- "pipeline_id" --> Pipelines;
Notes -- "contact_id" --> Contacts;
Tasks -- "contact_id" --> Contacts;
Contacts -- "custom_fields[].id" --> CustomFields;
Opportunities -- "custom_fields[].id" --> CustomFields;
```
# Use Cases for Data Analysis
Examples below use placeholder schema and table names. Replace `nekt_raw` and the table names with the **layer** and **table names** you configured for this source (for example `nekt_raw.hilevel_opportunities`).
### 1. Pipeline overview by stage
Summarize open opportunities by pipeline and stage using `monetary_value` (cast in SQL because it is stored as a string in the catalog).
```sql theme={null}
SELECT
o.pipeline_id,
o.pipeline_stage_id,
o.status,
COUNT(*) AS opportunity_count,
SUM(TRY_CAST(o.monetary_value AS DOUBLE)) AS total_value
FROM
nekt_raw.hilevel_opportunities o
WHERE
o.location_id = 'YOUR_LOCATION_ID'
GROUP BY
o.pipeline_id,
o.pipeline_stage_id,
o.status
ORDER BY
total_value DESC NULLS LAST
```
```sql theme={null}
SELECT
o.pipeline_id,
o.pipeline_stage_id,
o.status,
COUNT(*) AS opportunity_count,
SUM(SAFE_CAST(o.monetary_value AS FLOAT64)) AS total_value
FROM
`nekt_raw.hilevel_opportunities` o
WHERE
o.location_id = 'YOUR_LOCATION_ID'
GROUP BY
1,
2,
3
ORDER BY
total_value DESC
```
## Implementation Notes
### Replication and state
* Incremental streams partition bookmark state by `location_id`, so each sub-account progresses independently.
* **`locations`** exists mainly to drive per-location extraction in the Singer tap; enable it in the catalog only if you want that reference table materialized.
### Child streams (`notes` and `tasks`)
* **Replication and merging**: `notes` and `tasks` are fetched individually for each contact updated in the sync window. Because these API endpoints do not accept a date filter, the connector reads all of a contact's child records. They declare a replication key (`date_added` for notes, `due_date` for tasks) strictly to keep the extraction mode **INCREMENTAL** in the catalog, ensuring the data warehouse merges records on their primary keys rather than dropping data for contacts that didn't change in the window.
* **Deleted contacts mid-sync**: The extraction searches for updated contacts and subsequently fetches their notes/tasks. If a contact is deleted in GoHighLevel between the search and the child stream fetch, the API returns a `400 Contact not found` error. The connector safely skips these deleted contacts rather than failing the overall sync.
### Data shape
* **`contacts.opportunities`** and some custom field values are stored as **strings** (JSON text) where the API returns nested structures or mixed types.
* **`opportunities.monetary_value`** is modeled as a string; cast explicitly in SQL for numeric aggregations.
### API usage
* Selecting fewer streams reduces runtime and API load. Prefer incremental streams with a tight **Start date** for large locations.
## Skills for agents
HighLevel connector documentation as plain markdown, for use in AI agent contexts.
# Google Ads as a data source
Source: https://docs.nekt.com/sources/google-ads
Bring data from Google Ads to Nekt.
Google Ads is Google's online advertising platform that allows businesses to display ads on Google's search results pages and across the Google Display Network. It provides tools for creating, managing, and optimizing digital advertising campaigns to reach target audiences and drive conversions.
![]()
## Configuring Google Ads as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Ads option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Google Ads data. Click on the `Google Authorization` button and log in with your Google account. Grant the necessary permissions for the ad accounts you want to extract data from.
The following configurations are available:
* **Customer Account ID**: The Google Ads customer ID from the account you want to fetch data from. Check [Google Ads' documentation](https://support.google.com/google-ads/answer/1704344?hl=en) to discover where you can find this parameter.
* **Start Date**: The earliest date from which records will be synced.
* **Lookback Window**: (Default: 0 days) This parameter allows you to define the time window during which you consider that a conversion was influenced by an ad. For example, sometimes a click conversion happens X days after an ad is launched. The lookback retrieves data from X days ago to ensure it captures all conversions.
* **Manager Account ID**: Fill this in case you authenticated using a manager account that is different from the informed Customer Account ID.
If the account is accessed through a manager account (MCC), **Manager Account ID** is required. Leaving it empty is the most common reason a Google Ads connection fails to complete.
* **Enable Hotel Insights**: (Default: `false`) When enabled, this option unlocks the `Hotel Insights` stream, providing performance data specific to hotel campaigns.
* **Enable change events (Change History)**: (Default: `false`) When enabled, this option unlocks the `change_events` stream: the account Change History, with who changed what and when. Google keeps change events for 30 days only, so the history accumulates in Nekt from the first sync onwards.
* **Enable change status (resources changed in the last 90 days)**: (Default: `false`) When enabled, this option unlocks the `change_status` stream: one row per resource changed in the account, with the date and time of its most recent change. It does not say who made the change, but Google keeps it for 90 days instead of 30.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked *optional* are only discovered when the corresponding setting is enabled.
| Stream | Slug | Description |
| - | - | - |
| Campaigns | `campaigns` | Ad campaigns and their settings (status, channel type, budget, bidding). |
| Ad Groups | `ad_groups` | Ad groups within campaigns. |
| Ads | `ad` | Individual ads and their metadata and settings. |
| Account Budget | `account_budget` | Account-level budget configuration. |
| Campaign Budget | `campaign_budget` | Budgets assigned to campaigns. |
| Conversion Actions | `conversion_actions` | Conversion actions configured for the account. |
| Campaign Conversion Actions | `campaign_conversion_actions` | Conversion metrics broken down at the campaign level. |
| Ad Group Conversion Actions | `ad_group_conversion_actions` | Conversion metrics broken down at the ad group level. |
| Ad Conversion Actions | `ad_conversion_actions` | Conversion metrics broken down at the ad level. |
| Keywords | `keywords` | Keywords (ad group criteria) and their attributes. |
| Geo Target Constant | `geo_target_constant` | Reference table of geographic targeting constants (locations). |
| Campaign Performance | `campaign_performance` | Daily performance metrics per campaign (impressions, clicks, cost, conversions). |
| Campaign Reach & Frequency | `campaign_reach_frequency` | Reach and frequency metrics per campaign. |
| Ad Group Performance | `ad_group_performance` | Daily performance metrics per ad group. |
| Ad Performance | `ad_performance` | Daily performance metrics per ad. |
| Keyword Performance | `keyword_performance` | Daily performance metrics per keyword. |
| Geographic Performance | `geographic_performance` | Performance metrics broken down by geographic location. |
| User Location Performance | `user_location_performance` | Performance metrics broken down by the user's physical location. |
| Videos | `videos` | Video assets used in campaigns. |
| Video Performance | `video_performance` | Performance metrics for video assets. |
| Search Term Performance | `search_term_performance` | Performance metrics broken down by the search terms that triggered the ads. |
| Age Range Performance | `age_range_performance` | Performance metrics broken down by the age range criterion of the ad group. |
| Gender Performance | `gender_performance` | Performance metrics broken down by the gender criterion of the ad group. |
| Campaign Performance (Hourly) | `campaign_performance_hourly` | *Optional (deprecated).* Campaign performance segmented by hour of day. |
| Geographic Performance (Detailed) | `geographic_performance_detailed` | *Optional.* Like `geographic_performance`, additionally broken down by region and city. |
| Hotel Insights | `hotel_insights` | *Optional.* Insights for hotel campaigns. |
| Detail Placement Performance | `detail_placement_performance` | *Optional.* Performance metrics broken down by the automatic placement (Display Network website, YouTube channel/video or mobile app) where ads were shown. Can produce very large volumes on Display-heavy accounts. |
| Change Events | `change_events` | *Optional.* Account Change History: who changed what and when, for the last 30 days. |
| Change Status | `change_status` | *Optional.* Every resource changed in the last 90 days, with the time of its most recent change. |
| Assets | `assets` | *Optional.* Account assets (images, YouTube videos, texts) with their media URLs and metadata, resolving the asset references carried by the `ad` stream. |
## Fields by stream
Below you'll find all available data streams from Google Ads and their corresponding fields:
Stream for managing ad campaigns and their settings.
**Key Fields:**
* `id` - Unique identifier for the campaign
* `name` - Name of the campaign
* `status` - The status of the campaign (e.g., ENABLED, PAUSED, REMOVED)
* `resource_name` - The resource name of the campaign
* `start_date` - The start date of the campaign
* `end_date` - The end date of the campaign
**Advertising Settings:**
* `advertising_channel_type` - The primary channel type (e.g., SEARCH, DISPLAY, VIDEO, SHOPPING)
* `advertising_channel_sub_type` - The sub-type of the advertising channel
* `ad_serving_optimization_status` - Ad serving optimization status
* `primary_status` - The primary status of the campaign
* `primary_status_reasons` - Array of reasons for the primary status
* `serving_status` - The serving status of the campaign
**Budget & Bidding:**
* `campaign_budget` - Reference to the campaign budget
* `bidding_strategy` - Reference to the bidding strategy
* `bidding_strategy_type` - The type of bidding strategy (e.g., TARGET\_CPA, TARGET\_ROAS, MAXIMIZE\_CONVERSIONS)
* `bidding_strategy_system_status` - System status of the bidding strategy
* `accessible_bidding_strategy` - Reference to an accessible bidding strategy
* `target_cpa` - Target CPA settings including:
* `target_cpa_micros` - Target CPA in micros
* `cpc_bid_ceiling_micros` - Maximum CPC bid limit
* `cpc_bid_floor_micros` - Minimum CPC bid limit
* `target_roas` - Target ROAS settings including:
* `target_roas` - Target return on ad spend
* `cpc_bid_ceiling_micros` - Maximum CPC bid limit
* `cpc_bid_floor_micros` - Minimum CPC bid limit
* `target_spend` - Target spend settings
* `maximize_conversions` - Maximize conversions settings
* `maximize_conversion_value` - Maximize conversion value settings
* `target_impression_share` - Target impression share settings
* `target_cpm` - Target CPM settings with frequency goals
* `manual_cpc` - Manual CPC settings with enhanced CPC option
* `percent_cpc` - Percent CPC settings
* `payment_mode` - Payment mode for the campaign
**Network Settings:**
* `network_settings` - Network targeting settings including:
* `target_google_search` - Target Google Search network
* `target_search_network` - Target search network partners
* `target_content_network` - Target display network
* `target_partner_search_network` - Target partner search network
* `target_youtube` - Target YouTube
* `target_google_tv_network` - Target Google TV network
**Geo Targeting:**
* `geo_target_type_setting` - Geo targeting settings including:
* `positive_geo_target_type` - Positive geo target type
* `negative_geo_target_type` - Negative geo target type
**Shopping Settings:**
* `shopping_setting` - Shopping campaign settings including:
* `merchant_id` - Merchant Center ID
* `feed_label` - Feed label
* `campaign_priority` - Campaign priority
* `enable_local` - Enable local inventory ads
* `use_vehicle_inventory` - Use vehicle inventory
* `advertising_partner_ids` - Partner IDs
* `disable_product_feed` - Disable product feed
**App Campaign Settings:**
* `app_campaign_setting` - App campaign settings including:
* `app_id` - App ID
* `app_store` - App store (e.g., GOOGLE\_APP\_STORE, APPLE\_APP\_STORE)
* `bidding_strategy_goal_type` - Bidding strategy goal type
**Dynamic Search Ads Settings:**
* `dynamic_search_ads_setting` - DSA settings including:
* `domain_name` - Domain name
* `language_code` - Language code
* `use_supplied_urls_only` - Use only supplied URLs
**Hotel Settings:**
* `hotel_setting` - Hotel campaign settings
* `hotel_property_asset_set` - Hotel property asset set
**Travel Settings:**
* `travel_campaign_settings` - Travel campaign settings
**Local Campaign Settings:**
* `local_campaign_setting` - Local campaign settings
* `local_services_campaign_settings` - Local services settings
**Performance Max:**
* `performance_max_upgrade` - Performance Max upgrade status
**Advanced Settings:**
* `optimization_score` - Campaign optimization score
* `optimization_goal_setting` - Optimization goal settings
* `experiment_type` - Experiment type
* `frequency_caps` - Frequency capping settings
* `targeting_setting` - Targeting settings with restrictions
* `selective_optimization` - Selective optimization settings
* `real_time_bidding_setting` - Real-time bidding settings
* `demand_gen_campaign_settings` - Demand Gen campaign settings
* `audience_setting` - Audience settings
* `asset_automation_settings` - Asset automation settings
* `vanity_pharma` - Vanity pharma settings
* `video_brand_safety_suitability` - Video brand safety setting
**Tracking & URLs:**
* `tracking_url_template` - Tracking URL template
* `tracking_setting` - Tracking settings
* `final_url_suffix` - Final URL suffix
* `url_custom_parameters` - Custom URL parameters
* `url_expansion_opt_out` - URL expansion opt-out setting
**Labels & Metadata:**
* `labels` - Array of labels attached to the campaign
* `base_campaign` - Base campaign reference
* `campaign_group` - Campaign group reference
* `listing_type` - Listing type
* `excluded_parent_asset_field_types` - Excluded parent asset field types
* `excluded_parent_asset_set_types` - Excluded parent asset set types
Stream for managing ad groups, which control targeting and bidding for a group of ads within a campaign.
**Key Fields:**
* `id` - Unique identifier for the ad group
* `name` - Name of the ad group
* `status` - The status of the ad group (e.g., ENABLED, PAUSED, REMOVED)
* `campaign` - Reference to the parent campaign
* `resource_name` - The resource name of the ad group
* `type` - The type of ad group (e.g., SEARCH\_STANDARD, DISPLAY\_STANDARD)
* `primary_status` - The primary status of the ad group
* `primary_status_reasons` - Array of reasons for the primary status
* `base_ad_group` - Base ad group reference
**Bidding:**
* `cpc_bid_micros` - CPC bid in micros
* `cpm_bid_micros` - CPM bid in micros
* `cpv_bid_micros` - CPV bid in micros
* `percent_cpc_bid_micros` - Percent CPC bid in micros
* `target_cpa_micros` - Target CPA in micros
* `target_cpm_micros` - Target CPM in micros
* `target_roas` - Target return on ad spend
* `effective_target_cpa_micros` - Effective target CPA
* `effective_target_cpa_source` - Source of effective target CPA
* `effective_target_roas` - Effective target ROAS
* `effective_target_roas_source` - Source of effective target ROAS
**Targeting:**
* `targeting_setting` - Targeting settings including:
* `target_restrictions` - Array of targeting restrictions with:
* `targeting_dimension` - Dimension being restricted
* `bid_only` - Whether it's bid-only targeting
* `optimized_targeting_enabled` - Whether optimized targeting is enabled
* `display_custom_bid_dimension` - Custom bid dimension for display
**Rotation & URLs:**
* `ad_rotation_mode` - Ad rotation mode (e.g., OPTIMIZE, ROTATE\_FOREVER)
* `tracking_url_template` - Tracking URL template
* `final_url_suffix` - Final URL suffix
* `url_custom_parameters` - Custom URL parameters
**Labels & Metadata:**
* `labels` - Array of labels attached to the ad group
* `excluded_parent_asset_field_types` - Excluded parent asset field types
* `excluded_parent_asset_set_types` - Excluded parent asset set types
Stream for managing individual ads within an ad group, including all ad formats and creative content.
**Key Fields:**
* `id` - Unique identifier for the ad
* `ad_group` - Reference to the parent ad group
* `status` - The status of the ad (e.g., ENABLED, PAUSED, REMOVED)
* `resource_name` - The resource name of the ad group ad
* `ad_strength` - Ad strength indicator (e.g., EXCELLENT, GOOD, AVERAGE, POOR)
* `primary_status` - The primary status of the ad
* `primary_status_reasons` - Array of reasons for the primary status
* `action_items` - Suggested action items for the ad
**Ad Content (nested under `ad` object):**
*Common Fields:*
* `ad.name` - Name of the ad
* `ad.type` - Type of ad (e.g., RESPONSIVE\_SEARCH\_AD, RESPONSIVE\_DISPLAY\_AD, VIDEO\_AD)
* `ad.resource_name` - Resource name of the ad
* `ad.display_url` - Display URL
* `ad.final_urls` - Array of final URLs
* `ad.final_mobile_urls` - Array of final mobile URLs
* `ad.final_app_urls` - Array of final app URLs
* `ad.final_url_suffix` - Final URL suffix
* `ad.tracking_url_template` - Tracking URL template
* `ad.url_custom_parameters` - Custom URL parameters
* `ad.url_collections` - URL collections
* `ad.added_by_google_ads` - Whether the ad was added by Google Ads
* `ad.device_preference` - Device preference
* `ad.system_managed_resource_source` - System managed resource source
*Responsive Search Ad:*
* `ad.responsive_search_ad.headlines` - Array of headlines with text and performance labels
* `ad.responsive_search_ad.descriptions` - Array of descriptions with text and performance labels
* `ad.responsive_search_ad.path1` - First path
* `ad.responsive_search_ad.path2` - Second path
*Responsive Display Ad:*
* `ad.responsive_display_ad.marketing_images` - Marketing images
* `ad.responsive_display_ad.square_marketing_images` - Square marketing images
* `ad.responsive_display_ad.logo_images` - Logo images
* `ad.responsive_display_ad.square_logo_images` - Square logo images
* `ad.responsive_display_ad.headlines` - Headlines
* `ad.responsive_display_ad.long_headline` - Long headline
* `ad.responsive_display_ad.descriptions` - Descriptions
* `ad.responsive_display_ad.youtube_videos` - YouTube videos
* `ad.responsive_display_ad.business_name` - Business name
* `ad.responsive_display_ad.main_color` - Main color
* `ad.responsive_display_ad.accent_color` - Accent color
* `ad.responsive_display_ad.allow_flexible_color` - Allow flexible color
* `ad.responsive_display_ad.call_to_action_text` - Call to action text
* `ad.responsive_display_ad.price_prefix` - Price prefix
* `ad.responsive_display_ad.promo_text` - Promotional text
* `ad.responsive_display_ad.format_setting` - Format setting
* `ad.responsive_display_ad.control_spec` - Control specifications
*Video Ad:*
* `ad.video_ad.video` - Video asset reference
* `ad.video_ad.in_stream` - In-stream video settings
* `ad.video_ad.bumper` - Bumper ad settings
* `ad.video_ad.out_stream` - Out-stream video settings
* `ad.video_ad.non_skippable` - Non-skippable video settings
* `ad.video_ad.in_feed` - In-feed video settings
*Video Responsive Ad:*
* `ad.video_responsive_ad.headlines` - Headlines
* `ad.video_responsive_ad.long_headlines` - Long headlines
* `ad.video_responsive_ad.descriptions` - Descriptions
* `ad.video_responsive_ad.call_to_actions` - Call to actions
* `ad.video_responsive_ad.videos` - Videos
* `ad.video_responsive_ad.companion_banners` - Companion banners
* `ad.video_responsive_ad.breadcrumb1` - First breadcrumb
* `ad.video_responsive_ad.breadcrumb2` - Second breadcrumb
*Expanded Text Ad (Legacy):*
* `ad.expanded_text_ad.headline_part1` - First headline
* `ad.expanded_text_ad.headline_part2` - Second headline
* `ad.expanded_text_ad.headline_part3` - Third headline
* `ad.expanded_text_ad.description` - First description
* `ad.expanded_text_ad.description2` - Second description
* `ad.expanded_text_ad.path1` - First path
* `ad.expanded_text_ad.path2` - Second path
*Call Ad:*
* `ad.call_ad.country_code` - Country code
* `ad.call_ad.phone_number` - Phone number
* `ad.call_ad.business_name` - Business name
* `ad.call_ad.headline1` - First headline
* `ad.call_ad.headline2` - Second headline
* `ad.call_ad.description1` - First description
* `ad.call_ad.description2` - Second description
* `ad.call_ad.call_tracked` - Whether calls are tracked
* `ad.call_ad.disable_call_conversion` - Disable call conversion
* `ad.call_ad.phone_number_verification_url` - Phone verification URL
* `ad.call_ad.conversion_action` - Conversion action reference
* `ad.call_ad.conversion_reporting_state` - Conversion reporting state
*Image Ad:*
* `ad.image_ad.image_asset` - Image asset reference
* `ad.image_ad.image_url` - Image URL
* `ad.image_ad.pixel_width` - Pixel width
* `ad.image_ad.pixel_height` - Pixel height
* `ad.image_ad.mime_type` - MIME type
* `ad.image_ad.name` - Image name
* `ad.image_ad.preview_image_url` - Preview image URL
* `ad.image_ad.preview_pixel_width` - Preview pixel width
* `ad.image_ad.preview_pixel_height` - Preview pixel height
*App Ads:*
* `ad.app_ad` - App ad content
* `ad.app_engagement_ad` - App engagement ad content
* `ad.app_pre_registration_ad` - App pre-registration ad content
*Shopping Ads:*
* `ad.shopping_product_ad` - Shopping product ad
* `ad.shopping_smart_ad` - Shopping smart ad
* `ad.shopping_comparison_listing_ad` - Shopping comparison listing ad
*Demand Gen Ads:*
* `ad.demand_gen_product_ad` - Demand gen product ad
* `ad.demand_gen_carousel_ad` - Demand gen carousel ad
* `ad.demand_gen_multi_asset_ad` - Demand gen multi-asset ad
* `ad.demand_gen_video_responsive_ad` - Demand gen video responsive ad
*Other Ad Types:*
* `ad.local_ad` - Local ad content
* `ad.smart_campaign_ad` - Smart campaign ad content
* `ad.text_ad` - Legacy text ad content
* `ad.display_upload_ad` - Display upload ad
* `ad.expanded_dynamic_search_ad` - Expanded dynamic search ad
* `ad.legacy_responsive_display_ad` - Legacy responsive display ad
**Policy & Approval:**
* `policy_summary.approval_status` - Approval status
* `policy_summary.review_status` - Review status
**Labels:**
* `labels` - Array of labels attached to the ad
Stream for retrieving daily performance metrics at the campaign level.
**Identifiers:**
* `id` - Unique identifier for the record (hash of campaign\_id + date)
* `campaign_id` - The ID of the campaign
* `campaign_name` - Name of the campaign
* `campaign_status` - Status of the campaign
* `date` - The date of the metrics
**Basic Metrics:**
* `metrics_impressions` - Number of times ads were shown
* `metrics_clicks` - Number of clicks
* `metrics_ctr` - Click-through rate
* `metrics_interactions` - Number of interactions
* `metrics_interaction_rate` - Interaction rate
* `metrics_engagements` - Number of engagements
* `metrics_engagement_rate` - Engagement rate
**Cost Metrics:**
* `metrics_cost_micros` - Total cost in micros (divide by 1,000,000 for actual value)
* `metrics_average_cost` - Average cost per interaction
* `metrics_average_cpc` - Average cost per click
* `metrics_average_cpe` - Average cost per engagement
* `metrics_average_cpm` - Average cost per thousand impressions
* `metrics_average_cpv` - Average cost per view
**Conversion Metrics:**
* `metrics_conversions` - Number of conversions
* `metrics_conversions_value` - Total value of conversions
* `metrics_conversions_by_conversion_date` - Conversions by conversion date
* `metrics_conversions_value_by_conversion_date` - Conversion value by conversion date
* `metrics_conversions_from_interactions_rate` - Conversion rate from interactions
* `metrics_cost_per_conversion` - Cost per conversion
* `metrics_all_conversions` - All conversions (including cross-device)
* `metrics_all_conversions_value` - All conversions value
* `metrics_all_conversions_from_interactions_rate` - All conversions rate from interactions
* `metrics_cost_per_all_conversions` - Cost per all conversions
* `metrics_cross_device_conversions` - Cross-device conversions
* `metrics_view_through_conversions` - View-through conversions
**Value Metrics:**
* `metrics_value_per_conversion` - Value per conversion
* `metrics_value_per_all_conversions` - Value per all conversions
* `metrics_value_per_conversions_by_conversion_date` - Value per conversion by conversion date
* `metrics_value_per_all_conversions_by_conversion_date` - Value per all conversions by conversion date
* `metrics_new_customer_lifetime_value` - New customer lifetime value
* `metrics_all_new_customer_lifetime_value` - All new customer lifetime value
**Search Impression Share:**
* `metrics_search_impression_share` - Search impression share
* `metrics_search_absolute_top_impression_share` - Absolute top impression share
* `metrics_search_top_impression_share` - Top impression share
* `metrics_search_click_share` - Search click share
* `metrics_search_exact_match_impression_share` - Exact match impression share
* `metrics_search_budget_lost_impression_share` - Budget lost impression share
* `metrics_search_budget_lost_absolute_top_impression_share` - Budget lost absolute top impression share
* `metrics_search_budget_lost_top_impression_share` - Budget lost top impression share
* `metrics_search_rank_lost_impression_share` - Rank lost impression share
* `metrics_search_rank_lost_absolute_top_impression_share` - Rank lost absolute top impression share
* `metrics_search_rank_lost_top_impression_share` - Rank lost top impression share
**Content Network Metrics:**
* `metrics_content_impression_share` - Content network impression share
* `metrics_content_budget_lost_impression_share` - Content budget lost impression share
* `metrics_content_rank_lost_impression_share` - Content rank lost impression share
**Position Metrics:**
* `metrics_top_impression_percentage` - Top impression percentage
* `metrics_absolute_top_impression_percentage` - Absolute top impression percentage
**Video Metrics:**
* `metrics_video_views` - Number of video views
* `metrics_video_view_rate` - Video view rate
* `metrics_video_quartile_p25_rate` - 25% quartile rate
* `metrics_video_quartile_p50_rate` - 50% quartile rate
* `metrics_video_quartile_p75_rate` - 75% quartile rate
* `metrics_video_quartile_p100_rate` - 100% quartile rate (completed views)
**Active View Metrics:**
* `metrics_active_view_impressions` - Active View impressions
* `metrics_active_view_cpm` - Active View CPM
* `metrics_active_view_ctr` - Active View CTR
* `metrics_active_view_viewability` - Active View viewability
* `metrics_active_view_measurability` - Active View measurability
* `metrics_active_view_measurable_impressions` - Active View measurable impressions
* `metrics_active_view_measurable_cost_micros` - Active View measurable cost
**Gmail Metrics:**
* `metrics_gmail_forwards` - Gmail forwards
* `metrics_gmail_saves` - Gmail saves
* `metrics_gmail_secondary_clicks` - Gmail secondary clicks
**Phone Metrics:**
* `metrics_phone_calls` - Phone calls
* `metrics_phone_impressions` - Phone impressions
* `metrics_phone_through_rate` - Phone through rate
**Website Engagement:**
* `metrics_average_page_views` - Average page views
* `metrics_average_time_on_site` - Average time on site
* `metrics_bounce_rate` - Bounce rate
* `metrics_percent_new_visitors` - Percent new visitors
**Shopping & Commerce:**
* `metrics_orders` - Number of orders
* `metrics_average_cart_size` - Average cart size
* `metrics_average_order_value_micros` - Average order value in micros
* `metrics_units_sold` - Units sold
* `metrics_revenue_micros` - Revenue in micros
* `metrics_cost_of_goods_sold_micros` - Cost of goods sold
* `metrics_gross_profit_micros` - Gross profit
* `metrics_gross_profit_margin` - Gross profit margin
* `metrics_cross_sell_revenue_micros` - Cross-sell revenue
* `metrics_cross_sell_units_sold` - Cross-sell units sold
* `metrics_lead_revenue_micros` - Lead revenue
* `metrics_lead_units_sold` - Lead units sold
**Attribution:**
* `metrics_current_model_attributed_conversions` - Current model attributed conversions
* `metrics_current_model_attributed_conversions_value` - Current model attributed conversions value
* `metrics_cost_per_current_model_attributed_conversion` - Cost per current model attributed conversion
**Click Quality:**
* `metrics_invalid_clicks` - Invalid clicks
* `metrics_invalid_click_rate` - Invalid click rate
* `metrics_relative_ctr` - Relative CTR
**Mobile & App:**
* `metrics_sk_ad_network_installs` - SKAdNetwork installs
* `metrics_sk_ad_network_total_conversions` - SKAdNetwork total conversions
Stream for retrieving daily performance metrics at the ad group level.
**Identifiers:**
* `id` - Unique identifier for the record (hash of campaign\_id + ad\_group\_id + date)
* `campaign_id` - The ID of the campaign
* `campaign_name` - Name of the campaign
* `ad_group_id` - The ID of the ad group
* `ad_group_name` - Name of the ad group
* `date` - The date of the metrics
**Metrics:**
All metrics from Campaign Performance are available at the ad group level, prefixed with `metrics_`. This allows for granular analysis of performance within each ad group.
Key metrics include:
* `metrics_impressions`, `metrics_clicks`, `metrics_ctr`
* `metrics_cost_micros`, `metrics_average_cpc`, `metrics_average_cpm`
* `metrics_conversions`, `metrics_conversions_value`, `metrics_cost_per_conversion`
* `metrics_video_views`, `metrics_video_view_rate`
* All impression share and position metrics
* All conversion and value metrics
Stream for retrieving daily performance metrics at the individual ad level.
**Identifiers:**
* `id` - Unique identifier for the record (hash of campaign\_id + ad\_group\_id + ad\_id + date)
* `campaign_id` - The ID of the campaign
* `campaign_name` - Name of the campaign
* `campaign_start_date` - Campaign start date
* `campaign_end_date` - Campaign end date
* `campaign_status` - Campaign status
* `ad_group_id` - The ID of the ad group
* `ad_group_name` - Name of the ad group
* `ad_group_ad_ad_id` - The ID of the ad
* `ad_group_ad_ad_name` - Name of the ad
* `date` - The date of the metrics
**Metrics:**
All metrics from Campaign Performance are available at the ad level, prefixed with `metrics_`. This provides the most granular view of ad performance.
Key metrics include:
* `metrics_impressions`, `metrics_clicks`, `metrics_ctr`
* `metrics_cost_micros`, `metrics_average_cpc`, `metrics_average_cpm`
* `metrics_conversions`, `metrics_conversions_value`, `metrics_cost_per_conversion`
* `metrics_video_views`, `metrics_video_view_rate`, `metrics_video_quartile_p*_rate`
* All engagement and interaction metrics
* All conversion and value metrics
Stream for retrieving performance metrics specific to hotel campaigns. This stream is only available when "Enable Hotel Insights" is turned on in the source configuration.
**Identifiers:**
* `id` - Unique identifier for the record
* `date` - The date of the metrics
* `hotel_performance_view_resource_name` - Resource name of the hotel performance view
**Hotel Segments:**
* `segments_hotel_booking_window_days` - Number of days between booking and check-in
* `segments_hotel_center_id` - Hotel Center ID
* `segments_hotel_check_in_date` - Check-in date
* `segments_hotel_check_in_day_of_week` - Day of week for check-in
* `segments_hotel_city` - City of the hotel
* `segments_hotel_class` - Hotel class (star rating)
* `segments_hotel_country` - Country of the hotel
* `segments_hotel_date_selection_type` - Date selection type
* `segments_hotel_length_of_stay` - Length of stay in days
* `segments_hotel_state` - State/region of the hotel
**Hotel-Specific Metrics:**
* `metrics_hotel_average_lead_value_micros` - Average lead value for hotel
* `metrics_hotel_eligible_impressions` - Hotel eligible impressions
* `metrics_hotel_price_difference_percentage` - Price difference percentage
**Standard Metrics:**
* `metrics_impressions`, `metrics_clicks`, `metrics_ctr`
* `metrics_cost_micros`, `metrics_average_cpc`, `metrics_average_cpm`
* `metrics_conversions`, `metrics_conversions_value`, `metrics_cost_per_conversion`
* `metrics_all_conversions`, `metrics_all_conversions_value`
* `metrics_cross_device_conversions`
* All search impression share metrics
* `metrics_value_per_conversion`, `metrics_value_per_all_conversions`
Stream with the account Change History: one row per change made to a campaign, ad group, ad, keyword, budget, asset or other resource, with the user and client that made it. This stream is only available when "Enable change events (Change History)" is turned on in the source configuration.
The Google Ads API only keeps change events for the last 30 days. The first sync loads those 30 days and every later sync adds the new events, so the history accumulates in Nekt beyond the window offered by the API. Use it with the INCREMENTAL sync type; a Full sync can only rebuild the last 30 days.
**Identifiers:**
* `resource_name` - Unique identifier of the change event (primary key)
* `change_date_time` - Date and time at which the change was committed (replication key)
**Who and how:**
* `user_email` - Email of the user who made the change
* `client_type` - Client through which the change was made, such as `GOOGLE_ADS_WEB_CLIENT`, `GOOGLE_ADS_EDITOR`, `GOOGLE_ADS_API`, `GOOGLE_ADS_SCRIPTS`, `GOOGLE_ADS_AUTOMATED_RULE` or `GOOGLE_ADS_RECOMMENDATIONS`. Use it to separate changes made by people from automated ones.
**What changed:**
* `change_resource_type` - Type of the changed resource, such as `CAMPAIGN`, `CAMPAIGN_BUDGET`, `AD_GROUP`, `AD_GROUP_AD`, `AD_GROUP_CRITERION` or `ASSET`
* `change_resource_name` - Resource name of the changed resource
* `resource_change_operation` - `CREATE`, `UPDATE` or `REMOVE`
* `changed_fields` - Comma-separated list of the fields modified by the change
* `old_resource` - JSON string with the changed fields before the change
* `new_resource` - JSON string with the changed fields after the change
**Where it happened:**
* `campaign_id`, `campaign_name` - Campaign the changed resource belongs to, when applicable
* `ad_group_id`, `ad_group_name` - Ad group the changed resource belongs to, when applicable
* `campaign`, `ad_group`, `asset` - Resource names of the campaign, ad group and asset the change is attributed to
Each sync reads at most 10,000 events per account, the maximum the Google Ads API allows per query. Events are read in chronological order, so on very active accounts the remaining events arrive on the following syncs without losing the position.
Stream listing every resource changed in the account, with the date and time of its most recent change. It complements `change_events`: it does not say who made the change or what the values were, but the Google Ads API keeps it for 90 days instead of 30. This stream is only available when "Enable change status" is turned on in the source configuration.
Each row is a resource, not a change. A resource changed several times appears once, with the time of its latest change, so the row is updated on every sync in which the resource changes again. Use it with the INCREMENTAL sync type.
**Identifiers:**
* `resource_name` - Unique identifier of the change status entry, one per changed resource (primary key)
* `last_change_date_time` - Date and time of the most recent change to the resource (replication key)
**What changed:**
* `resource_type` - Type of the changed resource, such as `CAMPAIGN`, `AD_GROUP`, `AD_GROUP_AD`, `AD_GROUP_CRITERION`, `CAMPAIGN_BUDGET` or `ASSET`. It defines which of the resource name columns below are filled.
* `resource_status` - `ADDED`, `CHANGED` or `REMOVED`
**Affected resource (resource names, only the ones matching `resource_type` are filled):**
* `campaign`, `ad_group`, `ad_group_ad`, `ad_group_criterion`, `ad_group_bid_modifier`
* `campaign_criterion`, `campaign_budget`
* `asset`, `customer_asset`, `campaign_asset`, `ad_group_asset`, `asset_group`, `asset_set`, `campaign_asset_set`
* `shared_set`, `campaign_shared_set`, `combined_audience`
**Where it happened:**
* `campaign_id`, `campaign_name` - Campaign the changed resource belongs to, when applicable
* `ad_group_id`, `ad_group_name` - Ad group the changed resource belongs to, when applicable
Each sync reads at most 10,000 rows per account, the maximum the Google Ads API allows per query, in chronological order of the last change. On very active accounts the remaining rows arrive on the following syncs.
Stream containing all conversion actions configured in your Google Ads account. Conversion actions define the types of conversions you want to track.
**Key Fields:**
* `id` - Unique identifier for the conversion action
* `name` - Name of the conversion action
* `status` - Status of the conversion action (e.g., ENABLED, REMOVED)
* `type` - Type of conversion action (e.g., WEBPAGE, PHONE\_CALL, UPLOAD\_CLICKS)
* `category` - Category of the conversion (e.g., PURCHASE, LEAD, SIGN\_UP)
* `primary_for_goal` - Whether this is a primary conversion for the goal
* `include_in_conversions_metric` - Whether this action is included in the main conversions metric
* `origin` - Origin of the conversion action (e.g., WEBSITE, APP)
Stream containing campaign budget configurations and recommendations.
**Key Fields:**
* `id` - Unique identifier for the budget
* `name` - Name of the budget
* `resource_name` - The resource name of the budget
* `status` - Status of the budget (e.g., ENABLED, REMOVED)
* `type` - Type of budget (e.g., STANDARD, FIXED\_CPA)
* `period` - Period for the budget (e.g., DAILY)
* `delivery_method` - Delivery method (e.g., STANDARD, ACCELERATED)
**Budget Amounts:**
* `amount_micros` - Budget amount in micros (divide by 1,000,000 for actual value)
* `total_amount_micros` - Total budget amount in micros
**Sharing & Reference:**
* `explicitly_shared` - Whether the budget is explicitly shared across campaigns
* `reference_count` - Number of campaigns using this budget
* `aligned_bidding_strategy_id` - ID of the aligned bidding strategy
**Budget Recommendations:**
* `has_recommended_budget` - Whether Google has a budget recommendation
* `recommended_budget_amount_micros` - Recommended budget amount in micros
* `recommended_budget_estimated_change_weekly_clicks` - Estimated weekly click change with recommended budget
* `recommended_budget_estimated_change_weekly_cost_micros` - Estimated weekly cost change
* `recommended_budget_estimated_change_weekly_interactions` - Estimated weekly interaction change
* `recommended_budget_estimated_change_weekly_views` - Estimated weekly view change
Stream for retrieving conversion metrics broken down by conversion action at the campaign level. This allows you to analyze which types of conversions each campaign is driving.
**Identifiers:**
* `id` - Unique identifier for the record (hash of campaign\_id + date + conversion\_action)
* `campaign_id` - The ID of the campaign
* `campaign_name` - Name of the campaign
* `campaign_status` - Status of the campaign
* `date` - The date of the metrics
* `conversion_action_category` - Category of the conversion action
* `conversion_action_name` - Name of the conversion action
**Conversion Metrics:**
* `metrics_conversions` - Number of conversions
* `metrics_conversions_by_conversion_date` - Conversions by conversion date
* `metrics_conversions_value` - Total value of conversions
* `metrics_conversions_value_by_conversion_date` - Conversion value by conversion date
* `metrics_all_conversions` - All conversions (including cross-device)
* `metrics_all_conversions_by_conversion_date` - All conversions by conversion date
* `metrics_all_conversions_value` - All conversions value
* `metrics_all_conversions_value_by_conversion_date` - All conversions value by conversion date
**Value Metrics:**
* `metrics_value_per_conversion` - Value per conversion
* `metrics_value_per_conversions_by_conversion_date` - Value per conversion by conversion date
* `metrics_value_per_all_conversions` - Value per all conversions
* `metrics_value_per_all_conversions_by_conversion_date` - Value per all conversions by conversion date
* `metrics_view_through_conversions` - View-through conversions
Stream for retrieving conversion metrics broken down by conversion action at the ad group level. This provides granular insight into which ad groups are driving specific types of conversions.
**Identifiers:**
* `id` - Unique identifier for the record (hash of campaign\_id + ad\_group\_id + date + conversion\_action)
* `campaign_id` - The ID of the campaign
* `campaign_name` - Name of the campaign
* `ad_group_id` - The ID of the ad group
* `ad_group_name` - Name of the ad group
* `date` - The date of the metrics
* `conversion_action_category` - Category of the conversion action
**Conversion Metrics:**
* `metrics_conversions` - Number of conversions
* `metrics_conversions_by_conversion_date` - Conversions by conversion date
* `metrics_conversions_value` - Total value of conversions
* `metrics_conversions_value_by_conversion_date` - Conversion value by conversion date
* `metrics_all_conversions` - All conversions (including cross-device)
* `metrics_all_conversions_by_conversion_date` - All conversions by conversion date
* `metrics_all_conversions_value` - All conversions value
* `metrics_all_conversions_value_by_conversion_date` - All conversions value by conversion date
**Value Metrics:**
* `metrics_value_per_conversion` - Value per conversion
* `metrics_value_per_conversions_by_conversion_date` - Value per conversion by conversion date
* `metrics_value_per_all_conversions` - Value per all conversions
* `metrics_value_per_all_conversions_by_conversion_date` - Value per all conversions by conversion date
* `metrics_view_through_conversions` - View-through conversions
Stream for retrieving conversion metrics broken down by conversion action at the individual ad level. This provides the most granular view of which ads are driving specific types of conversions.
**Identifiers:**
* `id` - Unique identifier for the record
* `campaign_id` - The ID of the campaign
* `campaign_name` - Name of the campaign
* `campaign_start_date` - Campaign start date
* `campaign_end_date` - Campaign end date
* `campaign_status` - Campaign status
* `ad_group_id` - The ID of the ad group
* `ad_group_name` - Name of the ad group
* `ad_group_ad_ad_id` - The ID of the ad
* `ad_group_ad_ad_name` - Name of the ad
* `date` - The date of the metrics
* `conversion_action_category` - Category of the conversion action
**Conversion Metrics:**
* `metrics_conversions` - Number of conversions
* `metrics_conversions_by_conversion_date` - Conversions by conversion date
* `metrics_conversions_value` - Total value of conversions
* `metrics_conversions_value_by_conversion_date` - Conversion value by conversion date
* `metrics_all_conversions` - All conversions (including cross-device)
* `metrics_all_conversions_by_conversion_date` - All conversions by conversion date
* `metrics_all_conversions_value` - All conversions value
* `metrics_all_conversions_value_by_conversion_date` - All conversions value by conversion date
**Value Metrics:**
* `metrics_value_per_conversion` - Value per conversion
* `metrics_value_per_conversions_by_conversion_date` - Value per conversion by conversion date
* `metrics_value_per_all_conversions` - Value per all conversions
* `metrics_value_per_all_conversions_by_conversion_date` - Value per all conversions by conversion date
* `metrics_view_through_conversions` - View-through conversions
# Data Model
The following diagram illustrates the relationships between the core data streams in Google Ads. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Campaigns("Campaigns");
AdGroups("Ad Groups");
Ads("Ads");
CampaignBudget("Campaign Budget");
ConversionActions("Conversion Actions");
end
subgraph "Performance Data"
CampaignPerformance("Campaign Performance");
AdGroupPerformance("Ad Group Performance");
AdPerformance("Ad Performance");
HotelInsights("Hotel Insights");
end
subgraph "Conversion Action Reports"
CampaignConvActions("Campaign Conversion Actions");
AdGroupConvActions("Ad Group Conversion Actions");
AdConvActions("Ad Conversion Actions");
end
AdGroups -- "campaign" --> Campaigns;
Ads -- "ad_group" --> AdGroups;
Campaigns -- "campaign_budget" --> CampaignBudget;
CampaignPerformance -- "campaign_id" --> Campaigns;
AdGroupPerformance -- "campaign_id" --> Campaigns;
AdGroupPerformance -- "ad_group_id" --> AdGroups;
AdPerformance -- "campaign_id" --> Campaigns;
AdPerformance -- "ad_group_id" --> AdGroups;
AdPerformance -- "ad_group_ad_ad_id" --> Ads;
CampaignConvActions -- "campaign_id" --> Campaigns;
CampaignConvActions -- "conversion_action_name" --> ConversionActions;
AdGroupConvActions -- "campaign_id" --> Campaigns;
AdGroupConvActions -- "ad_group_id" --> AdGroups;
AdConvActions -- "campaign_id" --> Campaigns;
AdConvActions -- "ad_group_id" --> AdGroups;
AdConvActions -- "ad_group_ad_ad_id" --> Ads;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Google Ads data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Campaign Performance Overview
Track the overall performance of your campaigns with key metrics including spend, conversions, and efficiency ratios.
**Business Value:**
* Identify which campaigns deliver the best return on investment
* Compare performance across different campaign types
* Optimize budget allocation based on efficiency metrics
```sql theme={null}
WITH campaign_metrics AS (
SELECT
c.name AS campaign_name,
c.advertising_channel_type,
c.status AS campaign_status,
c.bidding_strategy_type,
SUM(cp.metrics_impressions) AS total_impressions,
SUM(cp.metrics_clicks) AS total_clicks,
SUM(cp.metrics_cost_micros) / 1000000.0 AS total_spend,
SUM(cp.metrics_conversions) AS total_conversions,
SUM(cp.metrics_conversions_value) AS total_conversion_value,
AVG(cp.metrics_ctr) AS avg_ctr,
SUM(cp.metrics_cost_micros) / NULLIF(SUM(cp.metrics_clicks), 0) / 1000000.0 AS avg_cpc,
SUM(cp.metrics_cost_micros) / NULLIF(SUM(cp.metrics_conversions), 0) / 1000000.0 AS cost_per_conversion,
SUM(cp.metrics_conversions_value) / NULLIF(SUM(cp.metrics_cost_micros) / 1000000.0, 0) AS roas
FROM
nekt_raw.google_ads_campaigns c
LEFT JOIN nekt_raw.google_ads_campaign_performance cp
ON CAST(c.id AS VARCHAR) = cp.campaign_id
WHERE
cp.date >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
c.name,
c.advertising_channel_type,
c.status,
c.bidding_strategy_type
)
SELECT
campaign_name,
advertising_channel_type,
campaign_status,
bidding_strategy_type,
total_impressions,
total_clicks,
ROUND(total_spend, 2) AS total_spend,
CAST(total_conversions AS INTEGER) AS total_conversions,
ROUND(total_conversion_value, 2) AS total_conversion_value,
ROUND(avg_ctr * 100, 2) AS ctr_percent,
ROUND(avg_cpc, 2) AS avg_cpc,
ROUND(cost_per_conversion, 2) AS cost_per_conversion,
ROUND(roas, 2) AS roas
FROM
campaign_metrics
ORDER BY
total_spend DESC
```
```sql theme={null}
WITH
campaign_metrics AS (
SELECT
c.name AS campaign_name,
c.advertising_channel_type,
c.status AS campaign_status,
c.bidding_strategy_type,
-- Sum core metrics
SUM(cp.metrics_impressions) AS total_impressions,
SUM(cp.metrics_clicks) AS total_clicks,
-- Convert cost_micros to standard currency (BigQuery handles division on BigInts smoothly)
SUM(cp.metrics_cost_micros) / 1000000.0 AS total_spend,
SUM(cp.metrics_conversions) AS total_conversions,
SUM(cp.metrics_conversions_value) AS total_conversion_value,
-- Average of daily CTRs
AVG(cp.metrics_ctr) AS avg_ctr,
-- Calculated average CPC: Total Cost / Total Clicks. Using SAFE_DIVIDE for BigQuery.
SAFE_DIVIDE (
SUM(cp.metrics_cost_micros),
SUM(cp.metrics_clicks)
) / 1000000.0 AS avg_cpc,
-- Calculated Cost Per Conversion: Total Cost / Total Conversions.
SAFE_DIVIDE (
SUM(cp.metrics_cost_micros),
SUM(cp.metrics_conversions)
) / 1000000.0 AS cost_per_conversion,
-- Calculated ROAS: Conversion Value / Total Spend.
SAFE_DIVIDE (
SUM(cp.metrics_conversions_value),
SUM(cp.metrics_cost_micros) / 1000000.0
) AS roas
FROM
`nekt_raw.google_ads_campaigns` c
LEFT JOIN `nekt_raw.google_ads_campaign_performance` cp ON CAST(c.id AS STRING) = cp.campaign_id
WHERE
DATE(cp.date) >= DATE_SUB (CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
1,
2,
3,
4
)
SELECT
campaign_name,
advertising_channel_type,
campaign_status,
bidding_strategy_type,
total_impressions,
total_clicks,
ROUND(total_spend, 2) AS total_spend,
CAST(total_conversions AS INT64) AS total_conversions,
ROUND(total_conversion_value, 2) AS total_conversion_value,
ROUND(avg_ctr * 100, 2) AS ctr_percent,
ROUND(avg_cpc, 2) AS avg_cpc,
ROUND(cost_per_conversion, 2) AS cost_per_conversion,
ROUND(roas, 2) AS roas
FROM
campaign_metrics
ORDER BY
total_spend DESC
```
| campaign\_name | advertising\_channel\_type | campaign\_status | bidding\_strategy\_type | total\_impressions | total\_clicks | total\_spend | total\_conversions | total\_conversion\_value | ctr\_percent | avg\_cpc | cost\_per\_conversion | roas |
| - | - | - | - | - | - | - | - | - | - | - | -: | -: |
| Summer Sale 2024 | SEARCH | ENABLED | TARGET\_ROAS | 1,245,890 | 42,156 | 8,542.30 | 856 | 42,800.00 | 3.38 | 0.20 | 9.98 | 5.01 |
| Brand Awareness | DISPLAY | ENABLED | TARGET\_CPA | 3,892,450 | 28,430 | 5,686.00 | 312 | 9,360.00 | 0.73 | 0.20 | 18.22 | 1.65 |
| Product Launch Q4 | SEARCH | ENABLED | MAXIMIZE\_CONVERSIONS | 892,340 | 31,245 | 4,998.50 | 423 | 21,150.00 | 3.50 | 0.16 | 11.82 | 4.23 |
| Retargeting Campaign | DISPLAY | ENABLED | TARGET\_CPA | 2,145,670 | 18,920 | 3,784.00 | 245 | 12,250.00 | 0.88 | 0.20 | 15.44 | 3.24 |
| Video Promos | VIDEO | PAUSED | MAXIMIZE\_CONVERSIONS | 567,230 | 8,456 | 1,691.20 | 89 | 2,670.00 | 1.49 | 0.20 | 19.00 | 1.58 |
### 2. Time-Based Performance Trends
Track performance trends over time to identify patterns, seasonality, and optimization opportunities.
**Business Value:**
* Understand daily and weekly performance patterns
* Identify performance trends and anomalies
* Optimize campaign scheduling and budget pacing
* Compare current performance against historical averages
```sql theme={null}
WITH daily_metrics AS (
SELECT
DATE(cp.date) AS report_date,
DATE_FORMAT(DATE(cp.date), '%W') AS day_of_week,
DATE_FORMAT(DATE(cp.date), '%Y-%m') AS month,
SUM(cp.metrics_impressions) AS impressions,
SUM(cp.metrics_clicks) AS clicks,
SUM(cp.metrics_cost_micros) / 1000000.0 AS spend,
SUM(cp.metrics_conversions) AS conversions,
SUM(cp.metrics_conversions_value) AS conversion_value,
COUNT(DISTINCT cp.campaign_id) AS active_campaigns
FROM
nekt_raw.google_ads_campaign_performance cp
WHERE
cp.date >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
DATE(cp.date),
DATE_FORMAT(DATE(cp.date), '%W'),
DATE_FORMAT(DATE(cp.date), '%Y-%m')
),
weekly_averages AS (
SELECT
day_of_week,
AVG(impressions) AS avg_impressions,
AVG(clicks) AS avg_clicks,
AVG(spend) AS avg_spend,
AVG(conversions) AS avg_conversions,
AVG(conversion_value) AS avg_conversion_value
FROM
daily_metrics
GROUP BY
day_of_week
),
daily_with_comparison AS (
SELECT
dm.*,
wa.avg_spend AS typical_daily_spend,
wa.avg_conversions AS typical_daily_conversions,
ROUND((dm.spend - wa.avg_spend) / NULLIF(wa.avg_spend, 0) * 100, 1) AS spend_vs_typical_pct,
ROUND((dm.conversions - wa.avg_conversions) / NULLIF(wa.avg_conversions, 0) * 100, 1) AS conversions_vs_typical_pct,
-- Calculate 7-day moving averages
AVG(dm.spend) OVER (ORDER BY dm.report_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS spend_7day_avg,
AVG(dm.conversions) OVER (ORDER BY dm.report_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS conversions_7day_avg,
-- Calculate CPA and ROAS
dm.spend / NULLIF(dm.conversions, 0) AS cpa,
dm.conversion_value / NULLIF(dm.spend, 0) AS roas
FROM
daily_metrics dm
JOIN weekly_averages wa ON dm.day_of_week = wa.day_of_week
)
SELECT
report_date,
day_of_week,
impressions,
clicks,
ROUND(spend, 2) AS spend,
ROUND(conversions, 2) AS conversions,
ROUND(conversion_value, 2) AS conversion_value,
active_campaigns,
ROUND(cpa, 2) AS cpa,
ROUND(roas, 2) AS roas,
ROUND(spend_7day_avg, 2) AS spend_7day_avg,
ROUND(conversions_7day_avg, 2) AS conversions_7day_avg,
spend_vs_typical_pct,
conversions_vs_typical_pct,
CASE
WHEN conversions_vs_typical_pct > 20 THEN 'Above Average'
WHEN conversions_vs_typical_pct < -20 THEN 'Below Average'
ELSE 'Normal'
END AS performance_indicator
FROM
daily_with_comparison
ORDER BY
report_date DESC
```
```sql theme={null}
WITH
daily_metrics AS (
SELECT
DATE(cp.date) AS report_date,
FORMAT_DATE ('%A', DATE(cp.date)) AS day_of_week,
FORMAT_DATE ('%Y-%m', DATE(cp.date)) AS MONTH,
SUM(cp.metrics_impressions) AS impressions,
SUM(cp.metrics_clicks) AS clicks,
SUM(cp.metrics_cost_micros) / 1000000.0 AS spend,
SUM(cp.metrics_conversions) AS conversions,
SUM(cp.metrics_conversions_value) AS conversion_value,
COUNT(DISTINCT cp.campaign_id) AS active_campaigns
FROM
`nekt_raw.google_ads_campaign_performance` cp
WHERE
DATE(cp.date) >= DATE_SUB (CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
1,
2,
3
),
weekly_averages AS (
SELECT
day_of_week,
AVG(impressions) AS avg_impressions,
AVG(clicks) AS avg_clicks,
AVG(spend) AS avg_spend,
AVG(conversions) AS avg_conversions,
AVG(conversion_value) AS avg_conversion_value
FROM
daily_metrics
GROUP BY
day_of_week
),
daily_with_comparison AS (
SELECT
dm.*,
wa.avg_spend AS typical_daily_spend,
wa.avg_conversions AS typical_daily_conversions,
ROUND(
SAFE_DIVIDE (dm.spend - wa.avg_spend, wa.avg_spend) * 100,
1
) AS spend_vs_typical_pct,
ROUND(
SAFE_DIVIDE (
dm.conversions - wa.avg_conversions,
wa.avg_conversions
) * 100,
1
) AS conversions_vs_typical_pct,
AVG(dm.spend) OVER (
ORDER BY
dm.report_date ROWS BETWEEN 6 PRECEDING
AND CURRENT ROW
) AS spend_7day_avg,
AVG(dm.conversions) OVER (
ORDER BY
dm.report_date ROWS BETWEEN 6 PRECEDING
AND CURRENT ROW
) AS conversions_7day_avg,
SAFE_DIVIDE (dm.spend, dm.conversions) AS cpa,
SAFE_DIVIDE (dm.conversion_value, dm.spend) AS roas
FROM
daily_metrics dm
JOIN weekly_averages wa ON dm.day_of_week = wa.day_of_week
)
SELECT
report_date,
day_of_week,
impressions,
clicks,
ROUND(spend, 2) AS spend,
ROUND(conversions, 2) AS conversions,
ROUND(conversion_value, 2) AS conversion_value,
active_campaigns,
ROUND(cpa, 2) AS cpa,
ROUND(roas, 2) AS roas,
ROUND(spend_7day_avg, 2) AS spend_7day_avg,
ROUND(conversions_7day_avg, 2) AS conversions_7day_avg,
spend_vs_typical_pct,
conversions_vs_typical_pct,
CASE
WHEN conversions_vs_typical_pct > 20 THEN 'Above Average'
WHEN conversions_vs_typical_pct < -20 THEN 'Below Average'
ELSE 'Normal'
END AS performance_indicator
FROM
daily_with_comparison
ORDER BY
report_date DESC
```
| report\_date | day\_of\_week | impressions | clicks | spend | conversions | conversion\_value | active\_campaigns | cpa | roas | spend\_7day\_avg | conversions\_7day\_avg | spend\_vs\_typical\_pct | conversions\_vs\_typical\_pct | performance\_indicator |
| - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 2024-11-27 | Wednesday | 156,234 | 5,892 | 1,178.40 | 78 | 3,900.00 | 8 | 15.11 | 3.31 | 1,145.20 | 72.5 | 4.2 | 12.8 | Normal |
| 2024-11-26 | Tuesday | 148,920 | 5,234 | 1,046.80 | 65 | 3,250.00 | 8 | 16.10 | 3.10 | 1,132.50 | 70.2 | -2.5 | -8.3 | Normal |
| 2024-11-25 | Monday | 142,560 | 4,987 | 997.40 | 58 | 2,900.00 | 8 | 17.20 | 2.91 | 1,125.80 | 68.4 | -7.8 | -18.5 | Normal |
| 2024-11-24 | Sunday | 98,450 | 3,245 | 649.00 | 42 | 2,100.00 | 7 | 15.45 | 3.24 | 1,098.60 | 65.8 | -22.4 | -28.6 | Below Average |
| 2024-11-23 | Saturday | 112,340 | 3,890 | 778.00 | 52 | 2,600.00 | 7 | 14.96 | 3.34 | 1,112.40 | 67.2 | -6.8 | -11.2 | Normal |
| 2024-11-22 | Friday | 178,560 | 6,234 | 1,246.80 | 95 | 4,750.00 | 8 | 13.12 | 3.81 | 1,156.30 | 74.8 | 8.5 | 32.4 | Above Average |
This analysis provides valuable insights into:
* Daily performance trends with moving averages
* Day-of-week patterns for optimization
* Performance against typical metrics
* CPA and ROAS trends over time
You can use these insights to:
* Adjust campaign scheduling for high-performing days
* Optimize budget allocation across the week
* Identify and investigate performance anomalies
* Improve budget pacing strategies
## Implementation Notes
### Data Quality Considerations
* All cost and value metrics are provided in micros (divide by 1,000,000 for actual values in queries)
* Use the lookback window configuration to ensure conversion data is complete before analysis
* For accurate trend analysis, ensure you have at least 30 days of data
### API Limits & Performance
* Selecting all streams and metrics can increase extraction times
* For faster extractions, select only the streams necessary for your analysis
* The Hotel Insights stream is only available for hotel campaigns and requires enabling it in the source configuration
* The `change_events` and `change_status` streams are optional and must be enabled in the source configuration. Google keeps change events for 30 days and change status for 90 days, and each sync reads at most 10,000 rows per account for each of them
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| The connection does not complete for an account accessed through an MCC | The **Manager Account ID** is missing; the **Customer Account ID** alone is not enough. | Fill in **Manager Account ID** in the source settings and validate again. |
| `USER_PERMISSION_DENIED` | The authorizing user has no permission on that Google Ads account. | Grant the user access in Google Ads and re-authorize the source. |
| The account does not appear in the selection list | The user lacks permission, or the link to the manager account was created recently. | Confirm the access in Google Ads. Recent links can take a few hours to become visible. |
| The app is not authorized in the customer's Google organization | The organization blocks third-party apps on the consent screen. | Ask your Google Workspace administrator to allow the app. |
| An error mentioning an unsupported API version | The request was not accepted by the API version in use. | Contact support with the source slug, the run ID, and the time with timezone. |
For extraction time and API usage, see [API Limits & Performance](#api-limits-performance).
## Skills for agents
Google Ads connector documentation as plain markdown, for use in AI agent contexts.
# Google Analytics GA4 as a data source
Source: https://docs.nekt.com/sources/google-analytics-ga4
Bring data from Google Analytics GA4 to Nekt.
Google Analytics GA4 (Google Analytics 4) is Google's web analytics platform that provides comprehensive insights into website and app performance. It tracks user interactions, conversions, and engagement metrics, helping businesses understand audience behavior and optimize their digital presence.
![]()
## 1. Add your Google Analytics GA4 access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Analytics GA4 CRM option from the list of connectors.
2. Click **Next** and you'll be prompted to fill the configuration.
**Default configuration**
* **Authorization**: first of all, authorize Nekt through the **Google Authorization** button.
* **Property ID**: add the property ID associated with your website. Check [GA4 documentation](https://developers.google.com/analytics/devguides/reporting/data/v1/property-id#what_is_my_property_id) to discover where you can find this information.
and select the date of the first reports you want to retrieve.
* **Start date**: the date of the first report you want to retrieve.
**Advanced configuration**
By default, the connector is configured to generate the following reports:
| Report Name | Dimensions | Metrics |
| - | - | - |
| **Website Overview** | date | activeUsers
newUsers
sessions
sessionsPerUser
averageSessionDuration
screenPageViews
screenPageViewsPerSession
bounceRate
engagementRate |
| **Traffic Sources** | date
source
medium
sourcePlatform | activeUsers
sessions
sessionsPerUser
bounceRate
engagementRate |
| **Locations** | date
country
countryId
region
city
cityId | activeUsers
newUsers
sessions
sessionsPerUser
averageSessionDuration
screenPageViews
screenPageViewsPerSession
bounceRate
engagementRate |
| **Four Weekly Active Users** | date | active28DayUsers |
| **Weekly Active Users** | date | active7DayUsers |
| **Daily Active Users** | date | active1DayUsers |
| **Devices** | date
deviceCategory
deviceModel
operatingSystem
browser | activeUsers
newUsers
sessions
sessionsPerUser
averageSessionDuration
screenPageViews
screenPageViewsPerSession
bounceRate
engagementRate |
| **Transactions** | date | activeUsers
averageRevenuePerUser
newUsers
purchaseRevenue
sessions
totalRevenue
transactions |
| **Pages** | date
hostName
pagePath
sessionMedium
sessionSource | activeUsers
bounceRate
engagedSessions
engagementRate
eventCount
screenPageViews
screenPageViewsPerUser
screenPageViewsPerSession
userEngagementDuration |
However, you also have the ability to build your own reports. You can configure that by clicking on **Advanced settings** and defining the following parameters:
1. **Report name**: the name of the report, which will be converted into a stream name. It should have spaces and special characters.
2. **Dimensions**: the list of dimensions you want to add to the report.
3. **Metrics**: the list of metrics you want to add to the report.
[Here's the list](https://developers.google.com/analytics/devguides/reporting/data/v1/api-schema) of available Dimensions & Metrics from Google Analytics.
We also support custom dimensions. In that case, you should add to the **Dimensions** list using the prefix `customEvent:` or `customUser:` followed by the name of the event. Which prefix to use depends on the scope defined for the custom dimension (user vs event). For example, of your custom event name has an `Event` scope and the name of the event is `page_view`, you should include this dimension as `customEvent:page_view`.
**IMPORTANT:** in order for custom dimensions to be available in the API, you have to ensure the dimension was created in the Google Analytics panel and has at least 1 event associated with it. It may take up to 24h for the event to propagate.
The name of the dimensions and metrics should match the exact name from the
[Google Analytics documentation](https://developers.google.com/analytics/devguides/reporting/data/v1/api-schema).
There are certain combinations of dimensions and metrics that are not allowed
by Google. You can check and validate it on [this link](https://ga-dev-tools.google/ga4/dimensions-metrics-explorer/)
before building your the report.
3. Click **Next**.
## 2. Select your Google Analytics GA4 streams
1. The next step is letting us know which streams you want to bring. The streams available are related to the reports mentioned in the previous section. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Google Analytics GA4 data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Google Analytics GA4 data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| Extraction fails on quota and fails again when re-run shortly after | The query quota is set by Google, per property and per hour, and is consumed even by attempts that fail. | Wait at least one hour before trying again, and reduce the number of dimensions on the stream. |
| Page reports consume far more quota than the others | Reports with many dimensions cost more tokens per request. | Split those reports into dedicated streams or sources with their own schedule. |
| Recreating the source does not fix the quota failure | The quota belongs to the property in Google Analytics, not to the configuration in Nekt. | Wait for the window to renew, or consider a GA 360 property, which has a larger quota. See [Google's quota documentation](https://developers.google.com/analytics/devguides/reporting/data/v1/quotas). |
| The app is not authorized in the Google organization | The organization blocks third-party apps on the consent screen. | Ask your Google Workspace administrator to allow the app. |
### Reports returning 0 records (Data Thresholding)
If your reports are returning 0 records, the issue might be related to **data thresholding**, a privacy protection mechanism in Google Analytics GA4.
**What is data thresholding?**
GA4 hides data when there are fewer than approximately 40-50 users per segment to protect user privacy. This happens when your report combines multiple dimensions that create very granular segments, making individual users potentially identifiable.
**Dimensions commonly affected by thresholding:**
* `userAgeBracket` (age demographics)
* `userGender`
* Other user demographic dimensions
**Example scenario:**
If your report combines dimensions like:
* `userAgeBracket` (thresholded dimension)
* `deviceCategory` → `operatingSystem`
* `country` → `region` → `city`
* `date`
This creates very specific segments (e.g., "18-24 year olds using Windows in Paris on 2025-01-01") that likely fall below the \~50 user threshold, causing all data to be hidden.
**How to resolve:**
1. **Reduce dimension granularity**: Remove or consolidate dimensions to create larger user segments
2. **Remove demographic dimensions**: Consider removing `userAgeBracket`, `userGender`, or similar thresholded dimensions
3. **Use broader time ranges**: Instead of daily reports, try weekly or monthly aggregations
4. **Simplify location data**: Use `country` alone instead of `country` → `region` → `city`
Data thresholding is a Google Analytics privacy feature and cannot be disabled. The only solution is to adjust your report configuration to create less granular segments.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Google Analytics GA4 connector documentation as plain markdown, for use in AI agent contexts.
# Google Calendar as a data source
Source: https://docs.nekt.com/sources/google-calendar
Bring data from Google Calendar to Nekt.
Google Calendar is Google's time management and scheduling application that helps users organize events, meetings, and appointments. It provides features for calendar sharing, event reminders, and integration with other Google services for seamless scheduling and time management.
![]()
## Configuring Google Calendar as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Calendar option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Google Calendar data. Click on the `Google Authorization` button and log in with your Google account.
The following configurations are available:
* **Start Date**: The earliest date from which event records will be synced.
* **Calendar IDs Filter**: (Optional) A list of specific calendar IDs to extract. If empty, all calendars available to the authenticated account are included.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. Google Calendar streams are generated dynamically, one per calendar, with names following the pattern `events_`.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is useful when you want to keep historical records.
* Full table: every time the extraction happens, we'll get the current state of the data, which is useful when you do not want deleted records in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequency you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Google Calendar and their corresponding fields:
Stream for retrieving details about the calendars available to the authenticated user.
| Field | Type | Description |
| :- | :- | :- |
| `kind` | String | Type classification of the record. |
| `etag` | String | ETag of the calendar resource. |
| `id` | String | Unique identifier of the record. |
| `description` | String | Description of the calendar. |
| `summary` | String | Display title of the calendar. |
| `time_zone` | String | Time zone configured for the calendar. |
| `color_id` | String | Identifier of the calendar color. |
| `background_color` | String | Background color of the calendar in hex format. |
| `foreground_color` | String | Foreground color of the calendar in hex format. |
| `selected` | Boolean | Indicates whether the calendar is selected in the UI. |
| `access_role` | String | Access role granted to the authenticated user. |
| `default_reminders` | Array | Default reminders applied to events in the calendar. |
| `notification_settings` | Object | Settings that control calendar notifications. |
| `primary` | Boolean | Indicates whether this is the user's primary calendar. |
| `conference_properties` | Object | Conference settings available for the calendar. |
Stream for retrieving calendar events from each accessible Google Calendar.
**Stream naming:**
* `events_` - One stream is created per calendar (for example, `events_marketing_team`)
**Selection behavior:**
* If `calendar_ids_filter` is empty, streams are created for all calendars the authenticated user can access
* If `calendar_ids_filter` is provided, only matching calendar IDs are used
**Sync behavior:**
* Primary key: `id`
* Replication key: `updated` (incremental sync supported)
* Deleted events are included in extraction (`showDeleted=true`)
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `anyone_can_add_self` | Boolean | Indicates whether any attendee can invite themselves. |
| `attendees_omitted` | Boolean | Indicates whether attendee details were omitted from the response. |
| `kind` | String | Type classification of the record. |
| `location` | String | Geographic location of the event. |
| `etag` | String | ETag of the event resource. |
| `id` | String | Unique identifier of the record. |
| `description` | String | Description of the event. |
| `color_id` | String | Identifier of the event color. |
| `status` | String | Current status of the record. |
| `html_link` | String | URL to view the event in Google Calendar. |
| `created` | Datetime | Timestamp when the event was created. |
| `updated` | Datetime | Timestamp when the record was last updated. |
| `summary` | String | Title of the event. |
| `start` | Object | Start date and time information for the event. |
| `end` | Object | End date and time information for the event. |
| `creator` | Object | Information about the user who created the event. |
| `organizer` | Object | Information about the event organizer. |
| `original_start_time` | Object | Original start information for a recurring event instance. |
| `i_cal_uid` | String | iCalendar UID of the event. |
| `sequence` | Integer | Revision sequence number of the event. |
| `attendees` | Array | List of attendees invited to the event. |
| `attachments` | Array | Files attached to the event. |
| `hangout_link` | String | URL to join the Google Meet associated with the event. |
| `conference_data` | Object | Conference metadata attached to the event. |
| `reminders` | Object | Reminder settings for the event. |
| `event_type` | String | Type classification of the record. |
| `end_time_unspecified` | Boolean | Indicates whether the event has an unspecified end time. |
| `guests_can_invite_others` | Boolean | Indicates whether guests can invite other attendees. |
| `guests_can_modify` | Boolean | Indicates whether guests can modify the event. |
| `guests_can_see_other_guests` | Boolean | Indicates whether guests can view other attendees. |
| `recurring_event_id` | String | Identifier of the associated recurring event. |
| `transparency` | String | Transparency setting that determines event availability behavior. |
| `visibility` | String | Visibility level of the event. |
| `source` | Object | Original source information for the event. |
| `working_location_properties` | Object | Working location metadata for the event. |
## Implementation Notes
### Data behavior considerations
* Stream names are dynamic and depend on calendar names. Renaming a calendar can change the discovered stream name.
* The connector uses Google Calendar list data internally to discover calendars before creating event streams.
* Event payloads can vary by event type (single event, recurring instance, working location), so nested fields may be sparse depending on your account usage.
### Incremental sync limitations & fallbacks
* **Automatic Full Sync for Holiday Calendars**: Google's Holiday calendars (e.g., calendars ending in `#holiday@group.v.calendar.google.com`) are read-only and load events in bulk with a single `updated` timestamp. Because their bookmark quickly falls outside of Google's incremental sync window limit (`updatedMin`), incremental sync is not meaningful for them. The connector automatically identifies these calendars and will **always perform a full sync**, overriding the incremental configuration.
* **Expired Incremental Window (`updatedMinTooLongAgo`)**: The Google Calendar API enforces a limit on how far in the past an incremental sync request (`updatedMin`) can be made. If a calendar hasn't been synced in a long time and the API returns a `410 Gone` error with the reason `updatedMinTooLongAgo`, the connector will seamlessly detect this rejection and automatically **fall back to a full sync** for that particular run. This built-in resiliency prevents pipeline failures and guarantees your catalog's data integrity.
### API limits & performance
* If your account has many calendars, selecting all streams can increase extraction times.
* Use `calendar_ids_filter` to reduce the number of discovered streams and improve run performance.
## Skills for agents
Google Calendar connector documentation as plain markdown, for use in AI agent contexts.
# Google Drive as a data source
Source: https://docs.nekt.com/sources/google-drive
Bring data from Excel and CSV files, or raw files, stored in Google Drive to Nekt.
Google Drive is Google's cloud storage and collaboration platform. This connector supports two operating modes:
* **Records mode** (default): Reads **Microsoft Excel files** (`.xlsx`, `.xlsm`, and `.xls`) and **CSV files** (`.csv` and `.tsv`) from Google Drive and turns them into data streams, so you can load tabular data from Drive, including files in **Shared Drives**, into Nekt.
* **Unstructured mode**: Downloads **any file type** (PDFs, images, XML, etc.) from a Google Drive folder and uploads them to a [Nekt volume](https://docs.nekt.com/get-started/core-concepts/catalog-layers), emitting one metadata record per file. Useful for materializing a catalog of unstructured assets into your warehouse.
CSV files are read as a stream, without being held in memory, so file size is not a limitation: a multi-gigabyte export loads the same way a small one does.
For native Google Sheets (not Excel or CSV files), use the [Google Sheets](https://docs.nekt.com/sources/google-sheets) connector instead.
## Records mode (Excel and CSV extraction)
### Configuring Google Drive as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the **Add source** button on the top right, then select **Google Drive** from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You need to authorize Nekt to read files from Google Drive and, optionally, pick the file or folder to extract from.
* **Authentication**: Use the **Google Authorization** flow. Sign in with a Google account that has access to the Drive (and, if applicable, the Shared Drive) where your files are stored. The connector uses OAuth and stores a refresh token so it can keep accessing Drive without re-authorizing.
* **File or folder**: Use the in-app picker to select the file or folder; the connector will resolve the correct ID.
Under **Advanced Settings** you can optionally set:
* **CSV delimiter**: The character separating fields in your CSV files, such as `,` or `;`. Leave it empty to have the connector detect it per file; set it only when a file is being read wrong, since one value applies to every CSV in the source.
* **Include subfolders**: Also extract files from folders inside the selected folder. When off (default), only files placed directly in the selected folder are read.
* **Subfolder depth**: Available when **Include subfolders** is enabled. How many levels of subfolders to read. 1 reads the folders directly inside the selected folder, 2 also reads their subfolders, and so on. Deeper levels mean more files and longer runs. Default is 5.
Once you're done, click **Next**.
### 2. Select streams
The connector discovers streams **dynamically** from the file or folder you selected, and how a file maps to streams depends on its type:
* **Excel files, one stream per sheet**: Each worksheet in each Excel file becomes one stream, named `{file_name}_{sheet_name}` (lowercase, special characters replaced with underscores), e.g. `monthly_sales_january`.
* **CSV files, one stream per file**: A CSV has no sheets, so each file becomes a single stream named after the file, e.g. `base_clientes`.
* **Skip rows**: Number of rows to skip from the top before treating the next row as the header. Use this when the file has title or empty rows above the data. Works for both Excel sheets and CSV files. If headers are on the first row, leave this at 0.
* **Range** (Excel only): A sheet range (e.g. `A:D` or `A1:E100`) so only part of the sheet is read. If not set, the full sheet is used.
Files that are neither spreadsheets nor CSVs are skipped, and the run log names them so you can tell the difference between "ignored" and "missing".
When per-stream settings are provided, only the streams you configure use their custom range or skip rows; the others read the full sheet or file with no skip.
Choose which streams you want to sync. You can select all or a subset.
> Tip: You can search for a stream by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want the data to appear in your catalog: layer, folder, table names, and sync type.
* **Layer**: Choose the [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) where the new tables will live.
* **Folder**: Optionally create or select a folder inside the layer to group tables from this source.
* **Table name**: A default name is suggested per stream; you can change it or add a **prefix** for all tables at once.
* **Sync type**: Only **FULL\_TABLE** is supported. Each run re-reads the selected files from scratch, so your tables always reflect the current content. There is no incremental mode in Records mode: a large file is read in full on every run, not just the first.
Click **Next** when done.
### 4. Configure data source
Add a short description of the source (e.g. what data it brings or which team owns it), and define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers) (how often the extraction runs).
Optionally:
* Configure **Delta Log Retention** for how long old table states are kept. See [Resource control](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Schedule an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)** if you want periodic full refreshes in addition to your normal schedule.
Click **Next** to finish.
### 5. Check your new source
Your new source appears on the [Sources](https://app.nekt.ai/sources) page. Trigger a run manually if needed; after a successful run, the tables will appear in your [Catalog](https://app.nekt.ai/catalog).
You need at least one successful source run to see the tables in your [Catalog](https://app.nekt.ai/catalog).
***
## Unstructured mode (raw file extraction)
Unstructured mode downloads files from a Google Drive folder, uploads them to a Nekt volume, and emits one metadata record per file through a `google_drive_files` stream. This is useful for extracting PDFs, images, XML files, or any other file type that doesn't fit a tabular format.
### Configuring unstructured mode
In the [Sources](https://app.nekt.ai/sources) tab, click **Add source** and select **Google Drive** from the list of connectors.
#### 1. Add account access
Same as Records mode: authorize Nekt via **Google Authorization** and select the folder containing your files.
#### 2. Select mode
Under **Advanced Settings**, set the **Mode** to **Unstructured**.
#### 3. Configure volume
When Unstructured mode is selected, the **Attachments Volume** setting becomes available.
* **Attachments Volume**: Use the in-app volume picker to directly select the destination Nekt volume where you want the downloaded Google Drive files to be uploaded. This is required when operating in Unstructured mode.
#### 4. Optional: filter files and subfolders
You can optionally set a **File name filter** (under Advanced Settings) to only process files matching a wildcard pattern (e.g. `*.pdf`, `invoice_*`). Files that don't match the pattern are skipped.
You can also enable **Include subfolders** (and configure **Subfolder depth**) to extract files from directories inside the selected folder.
#### 5. Configure and finish
Complete the remaining steps (data source description, trigger schedule) as with any other source.
### How it works
1. The connector lists all files in the selected Google Drive folder (with pagination for large folders).
2. If a **File name filter** is set, only matching files are processed.
3. Each file is downloaded and uploaded to the configured Nekt volume.
4. On subsequent runs, only files modified after the last extraction are processed (incremental replication via `modified_at`).
***
# Streams and Fields
Each selected Excel file contributes one stream per **sheet** (tab). The stream name is built from the file name (without extension) and the sheet name, sanitized (e.g. `revenue_2024_q1`).
**Schema:** Column names and types are **inferred** from the first 1,000 rows of the sheet (after applying skip rows and any range, if configured). Column headers in the file are slugified: spaces and special characters become underscores, and names are lowercased (e.g. `Revenue (USD)` → `revenue_usd`).
**Field types:** The connector maps Excel/pandas types to schema types:
| Inferred type | Schema type |
| :- | :- |
| datetime | DateTime |
| number | Number |
| integer | Integer |
| boolean | Boolean |
| other | String |
**Data behavior:**
* Rows where every cell is empty are dropped.
* Excel blanks become `null` in the output.
* Records are cleansed (e.g. invalid values normalized) before being written.
**Large files:** Sheets are read in memory and emitted in chunks (e.g. 5,000 rows per chunk) for efficiency. The file is only parsed once per stream.
Each selected CSV file becomes **one stream**, named after the file without its extension (e.g. `base_clientes.csv` → `base_clientes`).
**Schema:** Column names come from the header row, slugified the same way as Excel headers (lowercase, underscores). Blank header cells are dropped; repeated names are de-duplicated with a numeric suffix.
**Field types: every column is typed as String.** This is deliberate. A CSV carries no type information, and the files are often too large to inspect in full, so a type guessed from the first rows could be contradicted millions of rows later, silently turning an unexpected value into `null`. Emitting the raw text keeps the data intact; cast the columns you need in a downstream transformation.
**Encoding:** Detected automatically from the start of each file. UTF-8 (with or without a byte-order mark) is used when the file decodes as UTF-8; otherwise the connector falls back to latin-1, which covers the accented characters produced by Brazilian and European spreadsheet exports.
**Separator:** Detected automatically per file, from `,`, `;`, tab and `|`. Detection compares how consistently each candidate splits the rows, so a semicolon-separated file whose fields contain commas (e.g. `Rua São João, 100`) is read correctly. Set **CSV delimiter** in the connector settings to override detection.
**Data behavior:**
* Quoted fields may contain the separator, and may span several lines; both are preserved as a single value.
* Rows with more fields than the header declares have the extra values dropped; rows with fewer get `null` for the missing columns.
* Null bytes are stripped, since they cannot be parsed as CSV.
**Large files:** The file is read as a stream, one line at a time, and never buffered whole or written to disk. Memory use is constant regardless of file size, so a multi-gigabyte CSV is supported; expect the run to take proportionally longer.
The unstructured mode emits a single stream `google_drive_files` with one metadata record per uploaded file.
| Field | Type | Description |
| - | - | - |
| `file_id` | String | Google Drive file ID (primary key) |
| `file_name` | String | Original file name |
| `file_size` | Integer | File size in bytes |
| `mime_type` | String | MIME type (e.g. `application/pdf`, `image/png`) |
| `google_drive_url` | String | Web view URL for the file in Google Drive |
| `created_at` | DateTime | File creation timestamp |
| `modified_at` | DateTime | Last modification timestamp (replication key) |
| `uploaded_at` | DateTime | Timestamp of when the file was uploaded to the Nekt volume |
| `nekt_file_id` | String | ID assigned by Nekt after upload |
| `nekt_layer` | String | Destination layer slug |
| `nekt_volume` | String | Destination volume slug |
# Implementation notes
### Authentication
* **Google OAuth**: The connector uses Google OAuth (client ID, client secret, refresh token) to obtain access tokens for the Google Drive API. Credentials are stored securely.
* **Scopes**: The connected account must have read access to the chosen file or folder (and to the Shared Drive, if applicable).
* **Shared Drives**: Supported via `supportsAllDrives` and `includeItemsFromAllDrives` when resolving the item and listing folder contents.
### File and folder behavior
* **Records mode**: Excel (`.xlsx`, `.xlsm`, `.xls`) and CSV (`.csv`, `.tsv`) files are processed. Other files in a selected folder are skipped, and the run log lists them by name. If the selection points to a single file, it must be one of those types. If it points to a folder, the connector lists its full contents (paginated, including from Shared Drives, excluding items in the trash), then discovers one stream per sheet for each Excel file and one stream per CSV file. File type is determined by extension first and MIME type second, because Drive records the MIME type of an uploaded CSV inconsistently. By default, it only reads files placed directly in the selected folder; enable **Include subfolders** to extract from nested folders up to a specific depth.
* **Unstructured mode**: Any file type is accepted. The connector lists all files in the folder (with pagination), optionally filters by `search_pattern`, downloads each file, uploads it to a Nekt volume via the SDK's Volume Attachment system, and emits one metadata record per file. Google Workspace files (Docs, Sheets, Slides) that cannot be downloaded as binary are skipped. Like Records mode, subfolders are skipped unless **Include subfolders** is enabled.
* **Subfolder extraction**: When **Include subfolders** is enabled, the connector traverses directories using a breadth-first search down to the specified **Subfolder depth** (default 5). If files in different subfolders resolve to the same table name (e.g., `2025/vendas.csv` and `2026/vendas.csv` both map to `vendas`), the first one discovered is kept and subsequent ones are skipped to prevent them from overwriting each other in the warehouse.
### Schema and per-stream settings
* **Discovery (Excel)**: Schema is built from the first 1,000 rows, after skip rows and any optional range. If your header row is not the first row, set **Skip rows** so the correct row is used as the header.
* **Discovery (CSV)**: Only the first few hundred kilobytes of the file are read, to detect the encoding, the separator and the header row. The rest of the file is never touched during discovery, so adding a very large CSV does not make setup slow.
* **Per-stream settings**: Optional, keyed by **stream name** (e.g. `my_file_my_sheet` or `base_clientes`). Each can set skip rows, and for Excel a range. Streams without settings read the full sheet or file with no skip.
* **Column names**: All column names are slugified (lowercase, underscores) for consistency and compatibility with the catalog.
* **Column types**: Excel types are inferred from the data; CSV columns are always String. See the CSV stream notes above for why.
### Sync type
* **Records mode (FULL\_TABLE)**: There is no replication key. Every run re-reads the file(s) in full, so sync type is effectively full table, including for large CSVs, which are re-read on every run rather than only when they change.
* **Unstructured mode (INCREMENTAL)**: Uses `modified_at` as the replication key. Only files modified after the last successful run are downloaded and uploaded. Unchanged files are skipped, and the run log reports how many were skipped and how many will be uploaded.
### Best practices
1. **Use a dedicated service account or folder**: Prefer a Google account or folder used only for this integration, so permissions are clear and revocable.
2. **Set skip rows when needed**: If the first rows of the file are empty or contain titles, set **Skip rows** so the header row is detected correctly.
3. **Use range for large sheets**: If only a subset of columns or rows is needed, set **Range** to reduce payload and improve performance.
4. **Pick only needed streams**: Selecting only the sheets and files you need keeps runs faster and the catalog simpler.
5. **Schedule according to updates**: Run the source as often as your files change (e.g. daily or weekly). Because every run re-reads the files in full, scheduling a very large CSV more often than it actually changes costs time without adding data.
6. **Check the first run's log for CSVs**: The log reports the columns, separator and encoding detected for each CSV file. Confirming those once is the quickest way to catch a file that needs an explicit **CSV delimiter** or a **Skip rows** value.
7. **Limit subfolder depth**: When using **Include subfolders**, set the **Subfolder depth** only as deep as necessary to avoid scanning unnecessary folders and increasing discovery time.
### Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Auth or token errors | Invalid or expired OAuth credentials | Re-run Google Authorization and ensure the account has access to the file or folder. |
| "No spreadsheet or CSV files were found in the selected folder" | The folder holds only file types this mode cannot read | The message lists how many files were skipped. Add a supported file, or use Unstructured mode to capture the folder's raw files instead. |
| "The selected file is not a spreadsheet or a CSV file" | The selection points at a single unsupported file | Select an `.xlsx`, `.xlsm`, `.xls`, `.csv` or `.tsv` file, or the folder containing it. |
| "No files were found directly in the selected folder, but it contains X subfolder(s)..." | Files are inside subfolders, but recursion is disabled | Turn on **Include subfolders** in the source settings to extract from them, or select the specific subfolder that holds your files. |
| "No files were found in the selected folder or in the X subfolder(s) scanned Y level(s) deep." | Files sit deeper than the configured subfolder depth | Raise **Subfolder depth** in the source settings. |
| "Subfolder depth must be at least 1..." | Depth is set to less than 1 with recursion enabled | Set it to 1 to read the folders directly inside the selected folder, or turn off **Include subfolders**. |
| "Skipping '\[file]': it maps to the same table name..." | Multiple files with the same name in different subfolders | Two files in different subfolders cannot share one table name (derived from the file name). Rename one of them, or narrow the folder selection. |
| A CSV loaded as a single column | The separator was detected wrong, or the header sits below a preamble | The run log warns when this happens. Set **CSV delimiter** explicitly, or set **Skip rows** to the number of lines above the header. |
| Wrong columns or headers | Header not in first row, or wrong range | Set **Skip rows** and/or **Range** for that stream. |
| Accented characters look corrupted in a CSV | The file uses an encoding the automatic detection could not identify | Re-save the file as UTF-8 from your spreadsheet tool and re-run. |
| CSV numbers or dates arrive as text | Expected: every CSV column is loaded as String | Cast the columns in a downstream transformation. See the CSV stream notes above for why the connector does not guess types. |
| Missing or wrong types (Excel) | Schema inferred from first 1,000 rows | Ensure the first 1,000 rows are representative; mixed or later-format changes can cause type mismatches. |
| Slow run | Very large file, or many sheets and files | Every run re-reads the files in full. Reduce the number of selected streams, use **Range** for Excel, or lower the trigger frequency to match how often the files actually change. |
| Empty or partial data | Range too narrow or skip rows too high | Check **Range** and **Skip rows** for the stream; verify the file has data in the selected area. |
| A file in the folder is not picked up | The file is in the trash, or is not a supported type | Trashed files are ignored. The run log names every file that was skipped and why. |
| No files processed in unstructured mode | File name filter too restrictive | Check the **File name filter** pattern; use `*` to match all files. The run log reports how many files were excluded by the pattern. |
| The target folder is not found or not accessible | The authorizing user has no access to the folder, or the folder sits on a shared drive where the user's role is insufficient | Grant the user access to the folder and re-authorize. |
## Skills for agents
Google Drive connector documentation as plain markdown, for use in AI agent contexts.
# Google Local Services as a data source
Source: https://docs.nekt.com/sources/google-local-services
Bring Google Local Services Ads performance and lead data to Nekt.
Google Local Services Ads (LSA) helps local service businesses (plumbers, electricians, locksmiths, and more) appear at the top of Google Search and connect with nearby customers. Advertisers pay per lead rather than per click, and Google surfaces verified providers with ratings and reviews. This connector brings your Local Services account performance and lead-level data into Nekt.
## Configuring Google Local Services as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Local Services option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Local Services data. Using the UI, you can click on the `Google Authorization` button and log in with the Google account that has read access to your Local Services manager account. Alternatively, you can configure your own OAuth app credentials.
The following configurations are available:
* **Manager customer ID**: (Required) The Google Ads manager (MCC) customer ID that owns the linked Local Services accounts. Use digits only, without dashes.
* **Customer ID filter**: (Optional) Restrict the reports to a single linked Local Services customer ID. Use digits only, without dashes.
* **Start date**: (Optional) The earliest lead creation date to sync for the detailed lead reports stream. Defaults to one year ago.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. The following streams are available:
* **account\_reports**: Performance and metrics for each linked Local Services account, comparing the current period against the previous one (charged leads, phone calls, cost, ratings, and reviews).
* **detailed\_lead\_reports**: Lead-level detail for each linked account, including lead type, category, geography, charge status, and per-lead-type contact information.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is useful when you want to keep historical records.
* Full table: every time the extraction happens, we'll get the current state of the data, which is useful when you do not want deleted records in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Google Local Services and their corresponding fields:
Stream for retrieving performance and metrics for each linked Local Services account. Each row compares the current period against the previous period of equal length.
**Sync behavior:**
* Primary key: `account_id`
* Replication: Full table
| Field | Type | Description |
| :- | :- | :- |
| `account_id` | String | Unique identifier of the Local Services account. |
| `business_name` | String | Business name of the account. |
| `currency_code` | String | Currency code of the account. |
| `average_weekly_budget` | Number | Average weekly budget in the local currency. |
| `average_five_star_rating` | Number | Average review rating (1-5 stars). |
| `total_review` | Integer | Total number of reviews. |
| `impressions_last_two_days` | Integer | Impressions in the last two days. |
| `phone_lead_responsiveness` | Number | Phone lead responsiveness (0-1). |
| `current_period_charged_leads` | Integer | Charged leads in the current period. |
| `current_period_phone_calls` | Integer | Phone calls in the current period. |
| `current_period_connected_phone_calls` | Integer | Connected phone calls in the current period. |
| `current_period_total_cost` | Number | Total cost in the current period. |
| `previous_period_charged_leads` | Integer | Charged leads in the previous period. |
| `previous_period_phone_calls` | Integer | Phone calls in the previous period. |
| `previous_period_connected_phone_calls` | Integer | Connected phone calls in the previous period. |
| `previous_period_total_cost` | Number | Total cost in the previous period. |
| `aggregator_provider_id` | String | Aggregator provider ID, if the account is managed by an aggregator. |
Stream for retrieving lead-level detail for each linked Local Services account. Per-lead-type fields are populated according to the `lead_type` (phone, message, or booking).
**Sync behavior:**
* Primary key: `lead_id`
* Replication key: `lead_creation_timestamp` (incremental sync supported)
| Field | Type | Description |
| :- | :- | :- |
| `lead_id` | String | Unique identifier of the lead. |
| `account_id` | String | Local Services account that received the lead. |
| `google_ads_lead_id` | String | Google Ads lead ID. |
| `business_name` | String | Business name of the account. |
| `lead_creation_timestamp` | Datetime | Timestamp the lead was created. |
| `lead_type` | String | Lead type: MESSAGE, PHONE\_CALL, or BOOKING. |
| `lead_category` | String | Category of the service for the lead. |
| `geo` | String | Geographic region of the lead. |
| `currency_code` | String | Currency code of the lead price. |
| `lead_price` | Number | Price charged for the lead. |
| `charge_status` | String | Charge status: CHARGED or NOT\_CHARGED. |
| `dispute_status` | String | Dispute status of the lead. |
| `aggregator_provider_id` | String | Aggregator provider ID, if applicable. |
| `timezone_id` | String | IANA timezone ID of the account. |
| `timezone_version` | String | IANA timezone database version. |
| `phone_lead_charged_call_timestamp` | Datetime | Timestamp of the charged phone call. |
| `phone_lead_charged_connected_call_duration_seconds` | String | Duration of the connected charged call (e.g. `65s`). |
| `phone_lead_consumer_phone_number` | String | Consumer phone number for the phone lead. |
| `message_lead_consumer_phone_number` | String | Consumer phone number for the message lead. |
| `message_lead_customer_name` | String | Customer name for the message lead. |
| `message_lead_job_type` | String | Job type for the message lead. |
| `message_lead_postal_code` | String | Postal code for the message lead. |
| `booking_lead_booking_appointment_timestamp` | Datetime | Appointment timestamp for the booking lead. |
| `booking_lead_consumer_email` | String | Consumer email for the booking lead. |
| `booking_lead_consumer_phone_number` | String | Consumer phone number for the booking lead. |
| `booking_lead_customer_name` | String | Customer name for the booking lead. |
| `booking_lead_job_type` | String | Job type for the booking lead. |
## Implementation Notes
### Data behavior considerations
* **Account reports** are a full-table snapshot. Each run reports a rolling 30-day current period alongside the equivalent previous period, so the table always reflects recent performance rather than historical rows.
* **Detailed lead reports** support true incremental sync on `lead_creation_timestamp`. The Local Services API limits each query to a 31-day range, so the connector automatically splits the sync window (from your start date or last bookmark up to today) into ≤30-day chunks.
* The per-lead-type fields (`phone_lead_*`, `message_lead_*`, `booking_lead_*`) are only populated for leads of the matching `lead_type`; the others will be empty.
### API access & credentials
* The connector authenticates with Google OAuth using the `https://www.googleapis.com/auth/adwords` scope, against the Local Services API (`localservices.googleapis.com`).
* The manager account must run Local Services Ads, and Local Services API access may need to be enabled/approved by Google. If reports come back empty, confirm the account has API access and that the manager customer ID is correct.
* Google has marked the `detailedLeadReports` endpoint as deprecated in favor of the Google Ads API. It remains functional today; account reports are unaffected.
## Skills for agents
Google Local Services connector documentation as plain markdown, for use in AI agent contexts.
# Google Meet as a data source
Source: https://docs.nekt.com/sources/google-meet
Bring your meetings, transcripts and Gemini notes from Google Meet to your Lakehouse.
Google Meet is Google's video conferencing service. This source brings your meetings and everything they produce into your Lakehouse: who attended, the automatic transcript, the recording link, and the full text of the meeting summaries generated by Gemini's "Take notes for me".
Transcripts and Gemini notes are saved by Google as documents in Drive, and this source extracts their text into a column, so meeting content is queryable alongside the rest of your data.
Google keeps meetings available for only **30 days** after they end. There is no historical backfill: meetings older than 30 days cannot be extracted, no matter which start date you set. Your Lakehouse becomes the permanent archive, so schedule this source to run at least once a month, and preferably once a day. A meeting that is never extracted within its 30-day window is lost.
## Before you start
* Only meetings the connected Google account **took part in** can be extracted.
* The **Gemini notes** stream only returns data if the meeting organizer's Google Workspace has a Gemini license and note taking was switched on during the call. An empty table means the feature was not used, not that the extraction failed.
* Transcripts and notes are written by Google **minutes to hours after a meeting ends**. This source automatically re-reads the last couple of days on every run so late documents are still picked up.
## 1. Add your Google Meet access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Meet option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access. Authorize Nekt through the **Google Authorization** button, signing in with the account that attends the meetings you want to extract.
Accept every permission requested, including access to Meet recordings and transcripts in Google Drive. Without it, meetings and participants are still extracted, but the text of transcripts and Gemini notes is not.
3. The following configurations are available:
* **Start Date**: The earliest meeting start date to extract. Because of the 30-day retention, an earlier date does not return more data.
* **Extract document text**: (Default: enabled) Extracts the full text of transcripts and Gemini notes. Turn it off to bring only metadata and document links.
* **Lookback (days)**: (Default: 2) How many days before the last extracted meeting to read again on every run, so documents Google finished writing after the previous run are picked up.
* **Maximum document length**: (Default: 1,000,000) Maximum number of characters kept from each document. Longer documents are cut and flagged in the `content_status` column. Set to 0 to remove the limit.
4. Click **Next**.
## 2. Select your Google Meet streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Google Meet data streams
1. Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: choose **INCREMENTAL** for every Google Meet stream.
Do not use FULL\_TABLE with this source. Google only returns the last 30 days of meetings, so a full sync would delete every older meeting already stored in your Lakehouse.
2. Click **Next**.
## 4. Configure your Google Meet data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. A daily trigger is recommended, so no meeting expires before being extracted.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
# Streams and Fields
Below you'll find all available data streams from Google Meet and their corresponding fields. Every stream is keyed by `name`, the Google resource identifier of the record.
One record per meeting held in a Google Meet space. This is the central stream: every other stream joins back to it.
**Key Fields:**
* `name` - Resource name of the meeting, in the format `conferenceRecords/`. Primary key of this table
* `conference_record_id` - Short identifier of the meeting, used to join with the other tables
* `start_time` - When the meeting started. Used as the incremental replication key
* `end_time` - When the meeting ended. Empty while the meeting is still in progress
**Space:**
* `space` - Resource name of the Meet space where the meeting was held, in the format `spaces/`
* `space_id` - Short identifier of the space. Groups every occurrence of a recurring meeting
**Duration and retention:**
* `duration_seconds` - How long the meeting lasted, in seconds. Empty while the meeting is still in progress
* `expire_time` - When Google deletes this meeting from its API, 30 days after it ends. After this moment the extracted data is the only remaining copy
One record per person who attended a meeting.
**Key Fields:**
* `name` - Resource name of the participation. Primary key of this table
* `participant_id` - Short identifier of the participation within this meeting
**Meeting reference:**
* `conference_record` - Resource name of the meeting attended
* `conference_record_id` - Short identifier of the meeting attended
* `conference_start_time` - When the meeting started. Used as the incremental replication key
**Identity:**
* `user_type` - How the person joined: `signed_in` for a Google account or meeting-room device, `anonymous` for someone not signed in, `phone` for someone who dialed in
* `user_id` - Unique identifier of the signed-in user, in the format `users/`. Interoperable with the Google Admin SDK and People APIs. Empty for anonymous and phone participants
* `display_name` - Name shown during the meeting. For phone participants, a partially redacted phone number
**Attendance:**
* `earliest_start_time` - When the participant first joined the meeting
* `latest_end_time` - When the participant left for the last time. Empty while the meeting is still in progress
One record per Gemini "Take notes for me" summary, including its full text. Requires a Gemini license on the meeting organizer's Google Workspace.
**Key Fields:**
* `name` - Resource name of the smart notes. Primary key of this table
* `smart_note_id` - Short identifier of the notes session
* `state` - `STARTED` while the session runs, `ENDED` once it stopped but before the file exists, and `FILE_GENERATED` when the document is ready
**Meeting reference:**
* `conference_record` - Resource name of the meeting
* `conference_record_id` - Short identifier of the meeting
* `conference_start_time` - When the meeting started. Used as the incremental replication key
**Content:**
* `content` - Full text of the notes, exported from Google Docs as Markdown
* `content_status` - Why the text is filled or empty. See the table below
* `document_id` - Identifier of the Google Doc holding the notes
* `export_uri` - Link to open the notes in Google Docs
**Timing:**
* `start_time` - When the notes session started
* `end_time` - When the notes session stopped
One record per transcription session of a meeting, including its full text. Transcripts are generated independently from recordings.
**Key Fields:**
* `name` - Resource name of the transcript. Primary key of this table
* `transcript_id` - Short identifier of the transcription session
* `state` - `STARTED` while the session runs, `ENDED` once it stopped but before the file exists, and `FILE_GENERATED` when the document is ready
**Meeting reference:**
* `conference_record` - Resource name of the meeting
* `conference_record_id` - Short identifier of the meeting
* `conference_start_time` - When the meeting started. Used as the incremental replication key
**Content:**
* `content` - Full text of the transcript, exported from Google Docs as Markdown
* `content_status` - Why the text is filled or empty. See the table below
* `document_id` - Identifier of the Google Doc holding the transcript
* `export_uri` - Link to open the transcript in Google Docs
**Timing:**
* `start_time` - When the transcription session started
* `end_time` - When the transcription session stopped
One record per recording session of a meeting. Only metadata and the Drive link are extracted; the video file itself stays in Google Drive.
**Key Fields:**
* `name` - Resource name of the recording. Primary key of this table
* `recording_id` - Short identifier of the recording session
* `state` - `STARTED` while recording, `ENDED` once it stopped but before the file exists, and `FILE_GENERATED` when the video is ready
**Meeting reference:**
* `conference_record` - Resource name of the meeting
* `conference_record_id` - Short identifier of the meeting
* `conference_start_time` - When the meeting started. Used as the incremental replication key
**File:**
* `drive_file_id` - Identifier of the MP4 file in Google Drive
* `export_uri` - Link to play the recording in the browser
**Timing:**
* `start_time` - When the recording started
* `end_time` - When the recording ended
## Understanding the `content_status` column
The Smart Notes and Transcripts tables always explain why their text column is filled or empty:
| Value | Meaning |
| - | - |
| `exported` | The text was retrieved successfully |
| `truncated` | The text was retrieved but cut at the configured maximum document length |
| `not_generated` | Google has not finished writing the document yet. It will be picked up on a later run |
| `no_document` | No document was attached to this artifact |
| `permission_denied` | The connection lacks permission to read Meet files in Google Drive. Reconnect the source and accept all permissions |
| `not_found` | The document was deleted from Google Drive |
| `too_large` | The document is over the 10 MB that Google allows to be exported. Use `export_uri` to open it |
| `disabled` | Document text extraction is turned off in the source settings |
| `error` | Google Drive kept failing to return the document |
If you also need the meeting **video files** in your Lakehouse, use the [Google Drive](./google-drive) source in unstructured mode with the `drive_file_id` from the Recordings table.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Google Merchant Center as a data source
Source: https://docs.nekt.com/sources/google-merchant-center
Bring data from Google Merchant Center to your Lakehouse.
Google Merchant Center is where retailers upload and manage the product data that powers Shopping ads, free listings and local inventory ads across Google. This connector brings that catalog into your Lakehouse: the accounts you manage, the feeds behind them, every processed product with its full attribute set and review status, and the regional and in-store inventory overrides applied on top.
This source is built on Google's **Merchant API**, which replaced the Content API for Shopping when Google shut the latter down on 18 August 2026.
## 1. Add your Google Merchant Center access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Merchant Center option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **Google Authorization:** Click the authorization button and sign in with a Google account that has access to the Merchant Center account you want to extract. Every account that user can read becomes available for extraction, including the sub-accounts of an advanced (multi-client) account.
* **Merchant Center account IDs:** (Optional, advanced) A comma-separated list of account IDs, if you want to restrict the extraction to specific accounts. Leave it empty to extract everything the authorized user can read. You can find the ID at the top right of the [Merchant Center](https://merchants.google.com/) interface. A sub-account ID works here as well.
3. Click **Next**.
Merchant Center permissions are per account and per user. If an account you expected is missing, check that the Google user you authorized with is listed under **Settings → People and access** of that Merchant Center account.
## 2. Select your Google Merchant Center streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
**Regional Inventories** and **Local Inventories** are the expensive ones. Google only serves inventory data one product at a time, so these streams make one request per product in the catalog. On a large catalog, they take considerably longer than the rest and consume a much larger share of the daily API quota. Select them only if the account actually uses regional pricing or local inventory ads.
If you don't see a stream you were expecting to find, please check if the authorized user has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Google Merchant Center data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Every stream of this source is extracted in full on each run; see [How the data is replicated](#how-the-data-is-replicated) below. **FULL\_TABLE** keeps your tables mirroring the current state of Merchant Center, which is what most analyses of a catalog need. Choose INCREMENTAL only if you deliberately want to keep the history of products that have since been removed.
2. Click **Next**.
## 4. Configure your Google Merchant Center data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## How the data is replicated
The Merchant API offers no way to ask for "everything that changed since". None of its catalog resources accept a date filter, and products carry a last-update timestamp but cannot be sorted or filtered by it. Every stream is therefore extracted **in full** on each run, and each run reflects the state of the account at that moment.
Products do expose `last_update_date` and `creation_date`, so you can still follow how the catalog evolves inside your Lakehouse, but they describe the row; they don't limit what Google sends.
Newly uploaded or edited products take a few minutes to appear in the processed catalog this source reads. A sync started immediately after a feed upload may still return the previous version.
## How the tables are organized
Every resource is identified by a `name`, the full path Google uses for it, for example `accounts/123456/products/en~BR~sku123`. That is the primary key of each table. Alongside it, `account_id` is stamped on every row so you can join the whole source back to `accounts` without parsing paths.
Products are the widest table by far. The Merchant API nests the entire [product data specification](https://support.google.com/merchants/answer/7052112) under a `productAttributes` object and the review state under `productStatus`; this connector lifts both to the top level, so `title`, `brand`, `availability` or `last_update_date` are plain columns rather than nested structures. The rules are consistent across the whole source:
| What Google returns | What you get |
| - | - |
| A `camelCase` field | The same name in `snake_case` |
| A price | Two columns: `_amount_micros` (an integer, one million micros = one unit of currency) and `_currency_code` |
| A 64-bit number, which the API sends as text | An integer column |
| A nested object or a list of objects | A single column holding it as a JSON string |
| A list of plain values | A list column |
The custom labels are the one deliberate exception to the automatic renaming: they are delivered as `custom_label_0` through `custom_label_4`, the names used in Merchant Center and Google Ads.
Prices are expressed in micros, the unit the Merchant API uses. Divide by 1,000,000 to get the currency amount: `price_amount_micros = 19990000` means 19.99 in `price_currency_code`.
Product issues live in the `item_level_issues` column of `products`, as a JSON list: one entry per problem Google found, with its code, severity, and the destinations it affects. Account-wide problems are a separate stream, `account_issues`.
## Limits and quotas
Google sets the quota per Merchant Center account and per group of methods, and recomputes it daily: the products group is set to roughly twice the number of offers in the account, and the accounts group to the maximum number of sub-accounts allowed. Both a per-minute and a daily limit apply.
| Limit | Effect |
| - | - |
| Per-minute rate limit | The connector backs off and retries automatically. The run takes longer, but no data is lost. |
| Daily limit | The run stops with a message saying the quota was exhausted. It resets every day at 12:00 UTC. |
The daily quota is shared by everything that calls the Merchant API for that account, including your own feed automation and other tools. Selecting the inventory streams on a large catalog is what most often exhausts it, since each product costs one extra request per inventory stream.
If you need a larger quota, request it through Google Merchant Center support.
# Streams and Fields
Below you'll find all available data streams from Google Merchant Center and their corresponding fields.
Every Merchant Center account the authorized user can read, including the sub-accounts of an advanced account.
**Fields:**
* `account_id` - The ID of the account
* `account_name` - A human-readable name of the account. Don't use punctuation, capitalization, or non- alphanumeric symbols such as the "/" or "\_" symbols. See Adding a business name for more information
* `adult_content` - Whether this account contains adult content
* `language_code` - The account's BCP-47 language code, such as en-US or sr-Latn
* `name` - The resource name of the account. Format: `accounts/{account}`
* `test_account` - Whether this is a test account
* `time_zone` - The IANA time zone the account reports in, for example America/Sao\_Paulo
Address, phone and customer service details declared on each account.
**Fields:**
* `address` - The address of the business. Only region\_code, address\_lines, postal\_code, administrative\_area and locality fields are supported. All other fields are ignored. Serialized as a JSON object string
* `customer_service` - The customer service of the business. Serialized as a JSON object string
* `korean_business_registration_number` - The 10-digit Korean business registration number separated with dashes in the format: XXX-XX-XXXXX
* `name` - The resource name of the business info. Format: `accounts/{account}/businessInfo`
* `phone` - The phone number of the business. Serialized as a JSON object string
* `phone_verification_state` - The phone verification state of the business. Possible values: PHONE\_VERIFICATION\_STATE\_VERIFIED, PHONE\_VERIFICATION\_STATE\_UNVERIFIED
* `account_id` - ID of the Merchant Center account this business info belongs to
Account-level issues Merchant Center reports, such as policy warnings or missing information.
**Fields:**
* `detail` - Further localized details about the issue
* `documentation_uri` - Link to Merchant Center Help Center providing further information about the issue and how to fix it
* `impacted_destinations` - The impact this issue has on various destinations. Serialized as a JSON array string
* `name` - The resource name of the account issue. Format: `accounts/{account}/issues/{id}.` For example, accounts/123456/issues/misrepresentation-of-self-or-products- unacceptable- business-practice-policy
* `severity` - The overall severity of the issue. Possible values: CRITICAL, ERROR, SUGGESTION
* `title` - The localized title of the issue
* `account_id` - ID of the Merchant Center account the issue was reported on
Which Merchant Center programs each account takes part in, and what is still missing to qualify.
**Fields:**
* `active_region_codes` - The regions in which the account is actively participating in the program. Active regions are defined as those where all program requirements affecting the regions have been met. Region codes are…
* `documentation_uri` - The URL of a Merchant Center help page describing the program
* `name` - The resource name of the program. Format: `accounts/{account}/programs/{program}`
* `state` - The participation state of the account in the program. Possible values: NOT\_ELIGIBLE, ELIGIBLE, ENABLED
* `unmet_requirements` - The requirements that the account has not yet satisfied that are affecting participation in the program. Serialized as a JSON array string
* `account_id` - ID of the Merchant Center account enrolled in the program
Regions defined on the account, referenced by regional inventory and shipping settings.
**Fields:**
* `display_name` - The display name of the region
* `geotarget_area` - A list of geotargets that defines the region area. Serialized as a JSON object string
* `name` - The resource name of the region. Format: `accounts/{account}/regions/{region}`
* `postal_code_area` - A list of postal codes that defines the region area. Serialized as a JSON object string
* `radius_area` - A radius area that defines the region area. Serialized as a JSON object string
* `regional_inventory_eligible` - Indicates if the region is eligible for use in the Regional Inventory configuration
* `shipping_eligible` - Indicates if the region is eligible for use in the Shipping Services configuration
* `account_id` - ID of the Merchant Center account the region was defined on
The feeds each account pulls its product data from, primary and supplemental.
**Fields:**
* `data_source_id` - The data source id
* `display_name` - The displayed data source name in the Merchant Center UI
* `file_input` - The field is used only when data is managed through a file. Serialized as a JSON object string
* `input` - Determines the type of input to the data source. Based on the input some settings might not work. Only generic data sources can be created through the API. Possible values: API, FILE, UI, AUTOFEED
* `local_inventory_data_source` - The local inventory data source. Serialized as a JSON object string
* `merchant_review_data_source` - The merchant review data source. Serialized as a JSON object string
* `name` - The name of the data source. Format: `accounts/{account}/dataSources/{datasource}`
* `primary_product_data_source` - The primary data source for local and online products. Serialized as a JSON object string
* `product_review_data_source` - The product review data source. Serialized as a JSON object string
* `promotion_data_source` - The promotion data source. Serialized as a JSON object string
* `regional_inventory_data_source` - The regional inventory data source. Serialized as a JSON object string
* `supplemental_product_data_source` - The supplemental data source for local and online products. Serialized as a JSON object string
* `account_id` - ID of the Merchant Center account the data source belongs to
The processed catalog: what Merchant Center shows after applying rules and supplemental feeds, one row per product.
**Fields:**
* `archived` - Determines whether the product is archived. To archive or restore your product, visit Merchant Center products page. Learn also more about offer visibility
* `automated_discounts` - The automated discounts information for the product. Serialized as a JSON object string
* `base64_encoded_name` - The **unpadded base64url encoded name** of the product. Format: `accounts/{account}/products/{product}` where the last section product is the unpadded base64url encoding of the…
* `content_language` - The two-letter ISO 639-1 language code for the product
* `custom_attributes` - A list of custom (merchant-provided) attributes. It can also be used to submit any attribute of the data specification in its generic form (for example, `{"name": "size type", "value": "regular"}`).…
* `data_source` - The primary data source of the product
* `feed_label` - The feed label lets you categorize and identify your products. The maximum allowed characters is 20 and the supported characters areA-Z, 0-9, hyphen and underscore. The feed label must not include…
* `legacy_local` - Determines whether the product is **only** targeting local destinations and whether the product name should be distinguished with a local\~ prefix. For example, accounts/123/products/local~~en~~US\~sku123
* `name` - The name of the product. Format: `accounts/{account}/products/{product}` where the last section product consists of: content\_language~~feed\_label~~offer\_id example for product name is…
* `offer_id` - Your unique identifier for the product. This is the same for the product input and processed product. Leading and trailing whitespaces are stripped and multiple whitespaces are replaced by a single…
* `version_number` - Represents the existing version (freshness) of the product, which can be used to preserve the right order when multiple updates are done at the same time. If set, the insertion is prevented when…
* `creation_date` - Date on which the item has been created, in ISO 8601 format
* `destination_statuses` - The intended destinations for the product. Serialized as a JSON array string
* `google_expiration_date` - Date on which the item expires, in ISO 8601 format
* `item_level_issues` - A list of all issues associated with the product. Serialized as a JSON array string
* `last_update_date` - Date on which the item has been last updated, in ISO 8601 format
* `additional_image_links` - Additional URLs of images of the item
* `ads_grouping` - Used to group items in an arbitrary way. Only for CPA%, discouraged otherwise. For more information, see Display ads attribute
* `ads_labels` - Similar to ads\_grouping, but only works on CPC
* `ads_redirect` - Allows advertisers to override the item URL when the product is shown within the context of Product ads
* `adult` - Set to true if the item is targeted towards adults
* `age_group` - Target age group of the item. Possible values: ADULT, KIDS, TODDLER, INFANT, NEWBORN
* `amenity_feature` - The amenity features for the property
* `auto_pricing_min_price_amount_micros` - A safeguard in the \[automated discounts] ([https://support.google.com/merchants/answer/10295759](https://support.google.com/merchants/answer/10295759)) and "Dynamic Promotions" ([https://support.google.com/merchants/answer/13949249](https://support.google.com/merchants/answer/13949249)) projects, ensuring that…
* `auto_pricing_min_price_currency_code` - A safeguard in the \[automated discounts] ([https://support.google.com/merchants/answer/10295759](https://support.google.com/merchants/answer/10295759)) and "Dynamic Promotions" ([https://support.google.com/merchants/answer/13949249](https://support.google.com/merchants/answer/13949249)) projects, ensuring that…
* `availability` - Availability status of the item. Possible values: IN\_STOCK, OUT\_OF\_STOCK, PREORDER, LIMITED\_AVAILABILITY, BACKORDER
* `availability_date` - The day a pre-ordered product becomes available for delivery, in ISO 8601 format
* `body_style` - The body style of the vehicle. Possible values: ATV\_SPORT, ATV\_TOURING, ATV\_UTILITY, ATV\_YOUTH, CITY\_CAR, CLASS\_A\_MOTORHOME, CLASS\_B\_MOTORHOME, CLASS\_C\_MOTORHOME,
* `brand` - Brand of the item. For example, "Google"
* `canonical_link` - URL for the canonical version of your item's landing page
* `carrier_shipping` - Rules for carrier-based shipping. Serialized as a JSON array string
* `certifications` - Product Certifications, for example for energy efficiency labeling of products recorded in the EU EPREL database. See the Help Center article for more information. Serialized as a JSON array string
* `certified_pre_owned` - Whether the vehicle is OEM certified pre-owned
* `cloud_export_additional_properties` - Extra fields to export to the Cloud Retail program. Serialized as a JSON array string
* `co2_emissions` - The co2 emission of the vehicle. Serialized as a JSON object string
* `color` - Color of the item. For example, "red"
* `condition` - Condition or state of the item. Possible values: NEW, USED, REFURBISHED
* `cost_of_goods_sold_amount_micros` - Cost of goods sold. Used for gross profit reporting. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `cost_of_goods_sold_currency_code` - Cost of goods sold. Used for gross profit reporting. Three-letter ISO 4217 currency code
* `custom_label_0` - Custom label 0 for custom grouping of items in a Shopping campaign
* `custom_label_1` - Custom label 1 for custom grouping of items in a Shopping campaign
* `custom_label_2` - Custom label 2 for custom grouping of items in a Shopping campaign
* `custom_label_3` - Custom label 3 for custom grouping of items in a Shopping campaign
* `custom_label_4` - Custom label 4 for custom grouping of items in a Shopping campaign
* `date_first_registered` - The date the vehicle was first registered. Format: YYYY-MM. See the Date first registered for more information
* `description` - Description of the item
* `disclosure_date` - The date time when an offer becomes visible in search results across Google's YouTube surfaces, in ISO 8601 format. See Disclosure date for more information
* `display_address` - The display address of the property. Serialized as a JSON object string
* `display_ads_id` - An identifier for an item for dynamic remarketing campaigns
* `display_ads_link` - URL directly to your item's landing page for dynamic remarketing campaigns
* `display_ads_similar_ids` - Advertiser-specified recommendations. For more information, see Display ads attribute specification
* `display_ads_title` - Title of an item for dynamic remarketing campaigns
* `display_ads_value` - Offer margin for dynamic remarketing campaigns. For more information, see Display ads attribute
* `document_links` - Contains a list of PDF document URLs for the product. Examples are training manuals, user guides, assembly instructions, package inserts, etc. Must start with "http\://" or "https\://"), ASCII…
* `electric_range` - The electric range of the vehicle in miles/kms. Serialized as a JSON object string
* `emissions_standard` - The emission standard of the vehicle. Possible values: ZERO\_EMISSIONS, EURO1, EURO2, EURO3, EURO4, EURO5, EURO5B, EURO6,
* `energy_consumption` - The energy consumption of the vehicle. Serialized as a JSON object string
* `energy_efficiency_class` - The energy efficiency class as defined in EU directive 2010/30/EU. Possible values: APPP, APP, AP, A, B, C, D, E,
* `engine` - The engine type of the vehicle. Possible values: GASOLINE, DIESEL, ELECTRIC, HYBRID, PLUG\_IN\_HYBRID, NATURAL\_GAS, LPG, METHANE,
* `excluded_destinations` - The list of destinations to exclude for this target (corresponds to unchecked check boxes in Merchant Center). For more information, see Excluded destination. Note: We recommend setting destinations…
* `expiration_date` - Date on which the item should expire, as specified upon insertion, in ISO 8601 format. The actual expiration date is exposed in productstatuses as googleExpirationDate and might be earlier if…
* `external_seller_id` - Required for multi-seller accounts. Use this attribute if you're a marketplace uploading products for various sellers to your multi-seller account
* `free_shipping_threshold` - Conditions to be met for a product to have free shipping. Serialized as a JSON array string
* `fuel_consumption` - The fuel consumption of the vehicle. Serialized as a JSON object string
* `fuel_consumption_discharged_battery` - The fuel consumption of the vehicle when the hybrid battery is discharged. See the Help Center article for more information. Serialized as a JSON object string
* `gender` - Target gender of the item. Possible values: MALE, FEMALE, UNISEX
* `google_product_category` - Google's category of the item (see Google product taxonomy). When querying products, this field will contain the user provided value. There is currently no way to get back the auto assigned google…
* `gtins` - Global Trade Item Numbers (GTIN) of the item. You can provide up to 10 GTINs
* `handling_cutoff_times` - The handling cutoff times for shipping. Serialized as a JSON array string
* `identifier_exists` - Set this value to false when the item does not have unique product identifiers appropriate to its category, such as GTIN, MPN, and brand. Defaults to true, if not provided
* `image_link` - URL of an image of the item
* `included_destinations` - The list of destinations to include for this target (corresponds to checked check boxes in Merchant Center). Default destinations are always included unless provided in excludedDestinations. For more…
* `installment` - Number and amount of installments to pay for an item. Serialized as a JSON object string
* `is_bundle` - Whether the item is a business-defined sub-API. A \[sub-API] ([https://support.google.com/merchants/answer/6324449](https://support.google.com/merchants/answer/6324449)) is a custom grouping of different products sold by a business for a single price
* `item_group_id` - Shared identifier for all variants of the same product
* `item_group_title` - Represents the title of the product group to which this variant product belongs. This can be used along with the item group id attribute. It lets you perform better grouping of variant products, and…
* `latitude` - The latitude of the property. The value must be between -90 (inclusive) and 90 (inclusive), up to 6 decimal places
* `lease_term` - The lease term of the property. Serialized as a JSON object string
* `lifestyle_image_links` - Additional URLs of lifestyle images of the item, used to explicitly identify images that showcase your item in a real-world context. See the Help Center article for more information
* `link` - URL directly linking to your item's page on your online store
* `link_template` - Link template for business hosted local storefront
* `longitude` - The longitude of the property. The value must be between -180 (inclusive) and 180 (inclusive), up to 6 decimal places
* `loyalty_points` - Loyalty points that users receive after purchasing the item. Japan only. Serialized as a JSON object string
* `loyalty_programs` - A list of loyalty program information that is used to surface loyalty benefits (for example, better pricing, points, etc) to the user of this item. Serialized as a JSON array string
* `material` - The material of which the item is made. For example, "Leather" or "Cotton"
* `max_energy_efficiency_class` - The energy efficiency class as defined in EU directive 2010/30/EU. Possible values: APPP, APP, AP, A, B, C, D, E,
* `max_handling_time` - Maximal product handling time (in business days)
* `maximum_retail_price_amount_micros` - Maximum retail price (MRP) of the item. Applicable to India only. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `maximum_retail_price_currency_code` - Maximum retail price (MRP) of the item. Applicable to India only. Three-letter ISO 4217 currency code
* `mileage` - The number of miles/kms on the vehicle. See the Mileage for more information. Serialized as a JSON object string
* `min_energy_efficiency_class` - The energy efficiency class as defined in EU directive 2010/30/EU. Possible values: APPP, APP, AP, A, B, C, D, E,
* `min_handling_time` - Minimal product handling time (in business days)
* `minimum_order_values` - The minimum value in the cart before a customer can initiate checkout. Supports multiple minimum order values. Different minimum order values can be specified per country, service and surface.…
* `mobile_link` - URL for the mobile-optimized version of your item's landing page
* `mobile_link_template` - Link template for business hosted local storefront optimized for mobile devices
* `model` - The Model of the vehicle, such as LX, EX, and others
* `mpn` - Manufacturer Part Number (MPN) of the item
* `multipack` - The number of identical products in a business-defined multipack
* `neighborhood` - The neighborhood (locality) of the property, such as Wallingford, Greenwood, etc
* `number_of_bathrooms` - The number of bathrooms in the property. The value must be greater than 0 and a multiple of 0.5
* `number_of_bedrooms` - The number of bedrooms in the property. The value must be greater than or equal to 0 and a multiple of 1.0
* `number_of_units` - The number of units available for a specific floor plan of the property. The value must be greater than 0
* `pattern` - The item's pattern. For example, polka dots
* `pause` - Publication of this item will be temporarily paused. Possible values: ADS, ALL
* `pet_policy` - The pet policy for the property. Serialized as a JSON object string
* `pickup_cost` - The pickup cost for an item when a customer buys it online and picks it up at a store. Serialized as a JSON object string
* `pickup_method` - The pickup option for the item. Possible values: NOT\_SUPPORTED, BUY, RESERVE, SHIP\_TO\_STORE
* `pickup_sla` - Item store pickup timeline. For more information, see Pickup SLA. Possible values: SAME\_DAY, NEXT\_DAY, TWO\_DAY, THREE\_DAY, FOUR\_DAY, FIVE\_DAY, SIX\_DAY, MULTI\_WEEK
* `popularity_rank` - Indicates the popularity of the product in a merchant's inventory. Using a scale of 0.0 (lowest) to 100.0 (highest)
* `price_amount_micros` - Price of the item. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `price_currency_code` - Price of the item. Three-letter ISO 4217 currency code
* `product_details` - Technical specification or additional product details. Serialized as a JSON array string
* `product_fee` - The product fee for the property. Serialized as a JSON array string
* `product_height` - The height of the product in the units provided. The value must be between 0 (exclusive) and 3000 (inclusive). Serialized as a JSON object string
* `product_highlights` - Bullet points describing the most relevant product highlights
* `product_length` - The length of the product in the units provided. The value must be between 0 (exclusive) and 3000 (inclusive). Serialized as a JSON object string
* `product_types` - Categories of the item (formatted as in product data specification)
* `product_weight` - The weight of the product in the units provided. The value must be between 0 (exclusive) and 2000 (inclusive). Serialized as a JSON object string
* `product_width` - The width of the product in the units provided. The value must be between 0 (exclusive) and 3000 (inclusive). Serialized as a JSON object string
* `promotion_ids` - The unique ID of a promotion
* `property_name` - The name of the property
* `property_type` - The type of property. Possible values: APARTMENT, CONDO, LOFT, MULTI\_FAMILY\_HOME, PENTHOUSE, ROOM, SINGLE\_FAMILY\_HOME, STUDIO,
* `questions_and_answers` - Contains user-, merchant-, and manufacturer-authored questions and answers about the product. Max 30 question and answer pairs. Max 10000 characters total. Each question can have max 1000 characters.…
* `related_products` - Specifies how other products are related to this product. Serialized as a JSON array string
* `return_policy_label` - The return label of the product, used to group products in account-level return policies. Max. 100 characters. For more information, see Return policy label
* `returns` - Return rules for the product. Serialized as a JSON array string
* `sale_price_amount_micros` - Advertised sale price of the item. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `sale_price_currency_code` - Advertised sale price of the item. Three-letter ISO 4217 currency code
* `sale_price_effective_date` - Date range during which the item is on sale, see product data specification. Serialized as a JSON object string
* `sell_on_google_quantity` - The quantity of the product that is available for selling on Google. Supported only for online products
* `shipping` - Shipping rules. Serialized as a JSON array string
* `shipping_handling_business_days` - The business days during which orders can be handled. If not provided, Monday to Friday business days will be assumed. Serialized as a JSON array string
* `shipping_height` - Height of the item for shipping. Serialized as a JSON object string
* `shipping_label` - The shipping label of the product, used to group products in account-level shipping rules. Max. 100 characters. For more information, see Shipping label
* `shipping_length` - Length of the item for shipping. Serialized as a JSON object string
* `shipping_transit_business_days` - The business days during which orders are in transit. If not provided, Monday to Friday business days will be assumed. Serialized as a JSON array string
* `shipping_weight` - Weight of the item for shipping. Serialized as a JSON object string
* `shipping_width` - Width of the item for shipping. Serialized as a JSON object string
* `shopping_ads_excluded_countries` - List of country codes (ISO 3166-1 alpha-2) to exclude the offer from Shopping Ads destination. Countries from this list are removed from countries configured in data source settings
* `short_title` - The short title of the item
* `size` - Size of the item. Only one value is allowed. For variants with different sizes, insert a separate product for each size with the same itemGroupId value, see Size
* `size_system` - System in which the size is specified. Recommended for apparel items. For more information, see Size system. Possible values: AU, BR, CN, DE, EU, FR, IT, JP,
* `size_types` - The cut of the item. It can be used to represent combined size types for apparel items. Maximum two of size types can be provided, see Size type
* `specialty_housing_type` - The specialty housing type for the property. Possible values: CORPORATE, LOW\_INCOME, MILITARY, SENIOR, SHORT\_TERM, STUDENT
* `structured_description` - Structured description, for algorithmically (AI)-generated descriptions. Serialized as a JSON object string
* `structured_title` - Structured title, for algorithmically (AI)-generated titles. Serialized as a JSON object string
* `subscription_cost` - Number of periods (weeks, months or years) and amount of payment per period for an item with an associated subscription contract. Serialized as a JSON object string
* `sustainability_incentives` - The list of sustainability incentive programs. Serialized as a JSON array string
* `title` - Title of the item
* `transit_time_label` - The transit time label of the product, used to group product in account-level transit time tables
* `trim` - The Trim of the vehicle model, such as S, SV, SL and others
* `unit_area` - The unit area of the property, such as 1000 sqft. Serialized as a JSON object string
* `unit_pricing_base_measure` - The preference of the denominator of the unit price. Serialized as a JSON object string
* `unit_pricing_measure` - The measure and dimension of an item. Serialized as a JSON object string
* `utilities_included` - The utilities included for the property
* `variant_options` - Contains the list of all variant-identifying options of this product. Serialized as a JSON array string
* `vehicle_all_in_price_amount_micros` - The all-in advertised price for a vehicle, including the costs of the following: any accessories attached to the vehicle, environmental levies, extra warranty, fuel, freight, pre-delivery inspection…
* `vehicle_all_in_price_currency_code` - The all-in advertised price for a vehicle, including the costs of the following: any accessories attached to the vehicle, environmental levies, extra warranty, fuel, freight, pre-delivery inspection…
* `vehicle_expenses_amount_micros` - The miscellaneous expenses like insurance and registration fees of the vehicle. See the Vehicle expenses for more information. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `vehicle_expenses_currency_code` - The miscellaneous expenses like insurance and registration fees of the vehicle. See the Vehicle expenses for more information. Three-letter ISO 4217 currency code
* `vehicle_mandatory_inspection_included` - Whether the vehicle is sold with mandatory inspection and maintenance performed before delivery. See the Vehicle mandatory inspection included for more information
* `vehicle_msrp_amount_micros` - The MSRP (Manufacturer Suggested Retail Price) for the vehicle in its current configuration. See the Vehicle MSRP for more information. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `vehicle_msrp_currency_code` - The MSRP (Manufacturer Suggested Retail Price) for the vehicle in its current configuration. See the Vehicle MSRP for more information. Three-letter ISO 4217 currency code
* `vehicle_price_type` - The price type of the vehicle. Possible values: ALL\_IN\_PRICE, DRIVE\_AWAY\_PRICE, ESTIMATED\_DRIVE\_AWAY\_PRICE, EXCLUDING\_GOVERNMENT\_CHARGES\_PRICE, VEHICLE\_BASE\_PRICE
* `video_links` - A list of video URLs for the item. Use this attribute to provide more visuals for your product beyond your image attributes. See the Help Center article for more information
* `vin` - The Vehicle Identification Number (VIN) of the vehicle
* `virtual_model_link` - URL of the 3D image of the item. See the Help Center article for more information
* `warranty` - The warranty of the vehicle. Serialized as a JSON object string
* `year` - The Year of the vehicle model
* `account_id` - ID of the Merchant Center account the product belongs to
* `product_id` - Identifier of the product within the account, the last section of the resource name, in the form content\_language~~feed\_label~~offer\_id
Price and availability overrides that apply to a product in a given region. One row per product and region.
**Fields:**
* `account` - The account that owns the product. This field will be ignored if set by the client
* `base64_encoded_name` - The unpadded base64url encoded name of the RegionalInventory resource. Format: `accounts/{account}/products/{product}/regionalInventories/{region}` where the `{product}` segment is the unpadded base64url…
* `name` - The name of the RegionalInventory resource. Format: `accounts/{account}/products/{product}/regionalInventories/{region}` The `{product}` segment is a unique identifier for the product. This identifier…
* `region` - ID of the region for this RegionalInventory resource. See the Regional availability and pricing for more details
* `availability` - Availability of the product in this region. Possible values: IN\_STOCK, OUT\_OF\_STOCK
* `loyalty_programs` - An optional list of loyalty programs containing applicable loyalty member prices for this product in this region. This field is used to show region-specific member prices on Product Listing Ads…
* `price_amount_micros` - Price of the product in this region. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `price_currency_code` - Price of the product in this region. Three-letter ISO 4217 currency code
* `sale_price_amount_micros` - Sale price of the product in this region. Mandatory if salePriceEffectiveDate is defined. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `sale_price_currency_code` - Sale price of the product in this region. Mandatory if salePriceEffectiveDate is defined. Three-letter ISO 4217 currency code
* `sale_price_effective_date` - The TimePeriod of the sale price in this region. Serialized as a JSON object string
* `account_id` - ID of the Merchant Center account the product belongs to
* `product_name` - Resource name of the product this inventory belongs to, in the form `accounts/{account}/products/{product}`
* `product_id` - Identifier of the product within the account, in the form content\_language~~feed\_label~~offer\_id
Price, availability and pickup options of a product at a physical store. One row per product and store.
**Fields:**
* `account` - The account that owns the product. This field will be ignored if set by the client
* `base64_encoded_name` - The unpadded base64url encoded name of the LocalInventory resource. Format: `accounts/{account}/products/{product}/localInventories/{store_code}` where the `{product}` segment is the unpadded base64url…
* `name` - The name of the LocalInventory resource. Format: `accounts/{account}/products/{product}/localInventories/{store_code}` The `{product}` segment is a unique identifier for the product. This identifier must…
* `store_code` - Store code (the store ID from your Business Profile) of the physical store the product is sold in. See the Local product inventory data specification for more information
* `availability` - Availability of the product at this store. Possible values: IN\_STOCK, LIMITED\_AVAILABILITY, ON\_DISPLAY\_TO\_ORDER, OUT\_OF\_STOCK
* `custom_attributes` - A list of custom (merchant-provided) attributes. It can also be used for submitting any attribute of the data specification in its generic form (for example, `{"name": "size type", "value": "regular"}`).
* `instore_product_location` - Location of the product inside the store. Maximum length is 20 bytes
* `local_shipping_label` - Specifies a label associated with the shipping for the LocalInventory resource. Can be used to group local shipping services to this particular inventory item. For accepted field format, see the…
* `loyalty_programs` - An optional list of loyalty programs containing applicable loyalty member prices for this product at this store. This field is used to show store-specific member prices on Local Inventory Ads (LIA).…
* `pickup_method` - Supported pickup method for this product. Unless the value is "not supported", this field must be submitted together with pickupSla. Possible values: BUY, RESERVE, SHIP\_TO\_STORE, NOT\_SUPPORTED
* `pickup_sla` - Relative time period from the order date for an order for this product, from this store, to be ready for pickup. Must be submitted with pickupMethod. See more details here. Possible values: SAME\_DAY,…
* `price_amount_micros` - Price of the product at this store. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `price_currency_code` - Price of the product at this store. Three-letter ISO 4217 currency code
* `quantity` - Quantity of the product available at this store. Must be greater than or equal to zero
* `sale_price_amount_micros` - Sale price of the product at this store. Mandatory if salePriceEffectiveDate is defined. Amount in micros (1,000,000 micros = 1 unit of the currency)
* `sale_price_currency_code` - Sale price of the product at this store. Mandatory if salePriceEffectiveDate is defined. Three-letter ISO 4217 currency code
* `sale_price_effective_date` - The TimePeriod of the sale at this store. Serialized as a JSON object string
* `account_id` - ID of the Merchant Center account the product belongs to
* `product_name` - Resource name of the product this inventory belongs to, in the form `accounts/{account}/products/{product}`
* `product_id` - Identifier of the product within the account, in the form content\_language~~feed\_label~~offer\_id
# Google My Business as a data source
Source: https://docs.nekt.com/sources/google-my-business
Bring data from Google Business Profile to Nekt.
Google Business Profile (formerly Google My Business) is Google's platform for managing how businesses appear on Google Search and Maps. It allows business owners to manage their listings, respond to reviews, publish posts, and track performance metrics like search impressions, direction requests, and website clicks.
![]()
## Configuring Google My Business as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google My Business option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Google Business Profile data. Using the UI, you can click on the `Google Authorization` button and log in with the Google account associated with your business locations. Alternatively, you can configure your own OAuth app credentials. Once authenticated, all Business Profiles associated with your account will be available for extraction, unless you restrict them with the Account IDs or Location IDs settings below.
The following configurations are available:
* **Start date**: (Optional) Records created or updated after the start date will be extracted from the source. Used for incremental streams like reviews and performance metrics.
* **Account IDs**: (Optional) Comma-separated list of Business Profile account IDs to extract (e.g., `123456789, 987654321`). Leave empty to extract every account the authenticated user has access to. Use the number from the 'name' column of the accounts table (`accounts/123456789`), one per client business group. Locations, reviews, posts, and performance metrics are only extracted for the selected accounts. *Note: Providing an organization ID includes every business group under it. To restrict the sync to one client, use the ID of that client's business group (LOCATION\_GROUP) instead.*
* **Location IDs**: (Optional) Comma-separated list of Business Profile location IDs to extract (e.g., `123456789, 987654321`). Leave empty to extract every location of the extracted accounts. Use the number from the 'name' column of the locations table (`locations/123456789`). Reviews, posts and performance metrics are only extracted for the selected locations, so you can create one connection per location. Can be combined with Account IDs.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. The following streams are available:
* **accounts**: Business Profile accounts accessible to the authenticated user.
* **locations**: Business locations under each account, including address, hours, categories, and metadata.
* **reviews**: Customer reviews for each location, including star rating, comment, and owner replies.
* **posts**: Local posts published on each location (standard, event, offer, and alert types).
* **daily\_performance\_metrics**: Daily performance data per location, including search impressions, direction requests, call clicks, and website clicks.
* **search\_keywords**: Monthly search keyword impressions showing which terms led users to find each location.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is useful when you want to keep historical records.
* Full table: every time the extraction happens, we'll get the current state of the data, which is useful when you do not want deleted records in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how frequently you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Google My Business and their corresponding fields:
Stream for retrieving Google Business Profile accounts accessible to the authenticated user.
**Sync behavior:**
* Primary key: `name`
* Replication: Full table
| Field | Type | Description |
| :- | :- | :- |
| `name` | String | Resource name of the account. |
| `account_name` | String | Display name of the account. |
| `type` | String | Type of account (PERSONAL, LOCATION\_GROUP, etc). |
| `role` | String | Role of the authenticated user for this account. |
| `verification_state` | String | Verification state of the account. |
| `vetted_state` | String | Vetted state of the account. |
| `account_number` | String | Account reference number. |
| `permission_level` | String | Permission level of the caller. |
| `primary_owner` | String | Resource name of the primary owner. |
| `organization_info` | Object | Organization info including registered domain, address, and phone number. |
Stream for retrieving business locations under each account, including address, operating hours, categories, and metadata.
**Sync behavior:**
* Primary key: `name`
* Replication: Full table
* Parent stream: `accounts`
| Field | Type | Description |
| :- | :- | :- |
| `name` | String | Resource name of the location. |
| `language_code` | String | Language of the location. |
| `store_code` | String | External identifier for the location. |
| `title` | String | Location name (business name). |
| `phone_numbers` | Object | Primary and additional phone numbers. |
| `categories` | Object | Primary and additional business categories. |
| `storefront_address` | Object | Physical address of the location. |
| `website_uri` | String | Website URL. |
| `regular_hours` | Object | Standard operating hours. |
| `special_hours` | Object | Special hours for holidays and other exceptions. |
| `labels` | Array | User-provided labels for the location. |
| `latlng` | Object | Latitude and longitude coordinates. |
| `open_info` | Object | Open/closed status and opening date. |
| `metadata` | Object | Additional metadata including Maps URI, place ID, and capabilities. |
| `metadata.has_voice_of_merchant` | Boolean | Whether the location is verified and in good standing with Google. Locations without Voice of Merchant are not served by the Performance API. |
| `profile` | Object | Profile description of the location. |
| `relationship_data` | Object | Parent/children location relationships and chain info. |
Stream for retrieving customer reviews for each location, including star rating, comment, and owner reply.
**Sync behavior:**
* Primary key: `review_id`
* Replication key: `update_time` (incremental sync supported)
* Parent stream: `locations`
| Field | Type | Description |
| :- | :- | :- |
| `name` | String | Resource name of the review. |
| `review_id` | String | Encrypted unique identifier. |
| `reviewer` | Object | Author info including display name and profile photo URL. |
| `star_rating` | String | Star rating (ONE through FIVE). |
| `comment` | String | Body of the review. |
| `create_time` | Datetime | When the review was written. |
| `update_time` | Datetime | When the review was last modified. |
| `review_reply` | Object | Owner reply including comment and update time. |
| `account_id` | String | ID of the account. |
| `location_id` | String | ID of the location. |
Stream for retrieving local posts published on each location, including standard posts, events, offers, and alerts.
**Sync behavior:**
* Primary key: `name`
* Replication key: `update_time` (incremental sync supported)
* Parent stream: `locations`
| Field | Type | Description |
| :- | :- | :- |
| `name` | String | Resource name of the post. |
| `language_code` | String | Language of the post. |
| `summary` | String | Body of the post. |
| `call_to_action` | Object | Call-to-action including action type and URL. |
| `create_time` | Datetime | When the post was created. |
| `update_time` | Datetime | When the post was last modified. |
| `state` | String | Lifecycle state of the post. |
| `search_url` | String | Link to the post in Google Search. |
| `topic_type` | String | Type of post (STANDARD, EVENT, OFFER, ALERT). |
| `alert_type` | String | Alert type (for ALERT posts only). |
| `event` | Object | Event info including title and schedule (for EVENT/OFFER posts). |
| `offer` | Object | Offer details including coupon code and terms (for OFFER posts). |
| `media` | Array | Media items attached to the post. |
| `account_id` | String | ID of the account. |
| `location_id` | String | ID of the location. |
Stream for retrieving daily performance data per location, including impressions, direction requests, call clicks, and website clicks.
**Sync behavior:**
* Primary keys: `location_id`, `date`, `metric`
* Replication key: `date` (incremental sync supported)
* Parent stream: `locations`
**Available metrics:**
* `BUSINESS_IMPRESSIONS_DESKTOP_MAPS` / `BUSINESS_IMPRESSIONS_DESKTOP_SEARCH`
* `BUSINESS_IMPRESSIONS_MOBILE_MAPS` / `BUSINESS_IMPRESSIONS_MOBILE_SEARCH`
* `BUSINESS_CONVERSATIONS`, `BUSINESS_DIRECTION_REQUESTS`
* `CALL_CLICKS`, `WEBSITE_CLICKS`
* `BUSINESS_BOOKINGS`, `BUSINESS_FOOD_ORDERS`, `BUSINESS_FOOD_MENU_CLICKS`
| Field | Type | Description |
| :- | :- | :- |
| `location_id` | String | ID of the location. |
| `date` | Date | Date of the metric. |
| `metric` | String | Name of the metric (e.g. WEBSITE\_CLICKS). |
| `value` | Integer | Metric value for the date. |
Stream for retrieving monthly search keyword impressions, showing which search terms led users to find each location on Google.
**Sync behavior:**
* Primary keys: `location_id`, `month`, `search_keyword`
* Replication key: `month` (incremental sync supported)
* Parent stream: `locations`
| Field | Type | Description |
| :- | :- | :- |
| `location_id` | String | ID of the location. |
| `month` | String | Month in YYYY-MM format. |
| `search_keyword` | String | The search keyword. |
| `value` | Integer | Number of impressions for the keyword. |
# Data Model
The following diagram illustrates the parent-child relationships between the core data streams in the Google My Business API. Extractions happen hierarchically by first pulling accounts, then discovering locations within those accounts, and finally retrieving all related data for each location.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Accounts("Accounts");
Locations("Locations");
end
subgraph "Location Data"
Reviews("Reviews");
Posts("Posts");
DailyPerformanceMetrics("Daily Performance Metrics");
SearchKeywords("Search Keywords");
end
Locations -- "belongs to" --> Accounts;
Reviews -- "location_id" --> Locations;
Posts -- "location_id" --> Locations;
DailyPerformanceMetrics -- "location_id" --> Locations;
SearchKeywords -- "location_id" --> Locations;
```
## Implementation Notes
### Data behavior considerations
* The connector uses a parent-child stream hierarchy: accounts are fetched first, then locations under each account, and finally reviews, posts, and metrics for each location. If you specify `Account IDs` or `Location IDs` during the configuration, only those accounts or locations and their respective children will be extracted.
* Reviews are sorted by `updateTime desc` from the API but there is no server-side date filter. The connector paginates through all reviews and uses client-side filtering for incremental sync.
* Posts similarly lack a server-side date filter. All posts are fetched and filtered client-side based on `update_time`.
* Performance metrics support true date-range filtering. The API allows a maximum of 18 months of historical data.
* Search keywords are reported on a monthly basis. The current (incomplete) month is excluded.
* Performance data (`daily_performance_metrics` and `search_keywords`) is served only for locations that have Voice of Merchant (verified and in good standing with Google). Unverified, suspended, or duplicate listings are denied by the Performance API. The connector gracefully skips these locations for performance streams without failing the run. You can check the `has_voice_of_merchant` flag inside the location's `metadata` to see if a location is eligible.
### API limits & performance
* The connector accesses multiple Google sub-APIs (Account Management, Business Information, Performance). Ensure all necessary APIs are enabled in your Google Cloud project if providing custom credentials.
* If your account has many locations, extraction times will increase proportionally since child streams (reviews, posts, metrics, keywords) run per location. You can configure **Account IDs** or **Location IDs** in your source settings to restrict the extraction just to the specific clients or locations you need.
## Skills for agents
Google My Business connector documentation as plain markdown, for use in AI agent contexts.
# Google PageSpeed Insights as a data source
Source: https://docs.nekt.com/sources/google-pagespeed-insights
Bring data from Google PageSpeed Insights to your Lakehouse.
Google PageSpeed Insights runs a live [Lighthouse](https://developer.chrome.com/docs/lighthouse/overview) analysis of a web page and reports how well it performs. For each page it returns the performance, accessibility, best practices and SEO scores, the lab measurements behind those scores, and the Core Web Vitals that real Chrome users actually experienced on that page, taken from the [Chrome UX Report](https://developer.chrome.com/docs/crux).
Unlike most sources, PageSpeed Insights has nothing to list and no history to backfill: it analyzes the pages you point it at, as they are right now. You choose which URLs to monitor, and every run of this source records a fresh measurement of each one. Over time those measurements build into a performance history you can trend, alert on, and join with the rest of your data.
## Configuring Google PageSpeed Insights as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google PageSpeed Insights option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
This source needs no credentials. PageSpeed Insights measures publicly reachable pages and Nekt provides the API access, so you only have to say which pages to watch.
The following configurations are available:
* **URLs**: The full URLs of the pages you want to monitor, including the scheme (for example `https://www.example.com/pricing`). Add one entry per page. PageSpeed Insights analyzes a single page at a time, so it measures exactly the URLs you list here and nothing else. To follow a whole site, list the pages that matter most: the homepage, the main landing pages, and your highest-traffic templates.
* **Form Factors**: (Default: `MOBILE` and `DESKTOP`) The device profiles to analyze each URL for. Scores differ substantially between the two, because Lighthouse applies stricter thresholds and simulates a slower CPU and network on mobile. Comparing mobile against desktop is rarely meaningful; comparing the same form factor over time is.
* **Lighthouse Categories**: (Default: `PERFORMANCE`, `ACCESSIBILITY`, `BEST_PRACTICES`, `SEO`) Which categories to run. Also accepts `AGENTIC_BROWSING` and `PWA` (deprecated by Lighthouse 12.0). Requesting fewer categories makes each analysis faster, but the scores and audits for the categories you leave out will be empty.
* **Locale**: The language used for audit titles, descriptions and recommendations, for example `en`, `pt-BR` or `es`. This only affects wording, never the scores or measurements.
Once you're done, click **Next**.
Each analysis takes roughly 10 to 30 seconds, and one is run per URL per form factor. Twenty URLs on both form factors is around 40 analyses, so expect a run of several minutes. All three streams share the same analyses, so selecting all of them costs no extra time.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Use **INCREMENTAL** for this source. Each run measures the pages as they are at that moment, so incremental keeps every measurement and lets you trend performance over time. FULL\_TABLE would replace the previous run's rows and leave you with only the latest snapshot.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
A daily trigger suits most sites. The Chrome UX Report field data is a 28-day rolling aggregate, so it moves slowly and there is little value in analyzing more often than once a day. The lab measurements do vary between runs even on an unchanged page, so read trends rather than single data points.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Understanding the data
A few things are worth knowing before you query these tables.
**Lab data versus field data.** Lighthouse measures the page in a controlled environment: those are the *lab* metrics (`largest_contentful_paint_ms`, `total_blocking_time_ms`, and so on). The Chrome UX Report reports what real Chrome users experienced over the previous 28 days: those are the *field* metrics (the `field_` and `origin_field_` columns, and the whole `crux_metrics` stream). Lab data is reproducible and available for any page; field data is the ground truth but only exists for pages with enough real traffic. They will not agree, and that is expected.
**Scores are 0 to 1.** Multiply by 100 to get the familiar 0-100 Lighthouse score. A score is `null` when that category was not run.
**Cumulative Layout Shift is scaled in field data.** The Chrome UX Report returns CLS multiplied by 100, so `field_cls = 10` means a CLS of 0.10. The lab `cumulative_layout_shift` column is not scaled.
**Pages with little traffic fall back to the origin.** When a specific page has too few real-user samples, `field_origin_fallback` is `true` and the field metrics describe the whole origin rather than that page.
**A page that fails to load is skipped, not fatal.** If PageSpeed Insights cannot analyze a URL (the page is unreachable, times out, or sits behind a login), that URL is reported in the run log and skipped for that run. The other URLs are unaffected.
# Streams and Fields
All three streams join on `url`, `strategy` and `analysis_utc_timestamp`.
Below you'll find all available data streams from Google PageSpeed Insights and their corresponding fields:
One row per URL per form factor, per run. The headline table: category scores, the lab metrics behind them, and the Core Web Vitals for both the page and its origin.
**Key Fields:**
* `url` - URL that was submitted for analysis, as configured in the connector
* `strategy` - Form factor the page was analyzed for: MOBILE or DESKTOP
* `analysis_utc_timestamp` - UTC timestamp of when this analysis ran, used as the incremental replication key
* `id` - Canonicalized and final URL of the analyzed document, after following any redirects
* `kind` - Kind of result returned by the API
* `captcha_result` - Result of the CAPTCHA verification, if one was required
**Category Scores:**
* `performance_score` - Lighthouse performance score between 0 and 1
* `accessibility_score` - Lighthouse accessibility score between 0 and 1
* `best_practices_score` - Lighthouse best practices score between 0 and 1
* `seo_score` - Lighthouse SEO score between 0 and 1
* `pwa_score` - Progressive Web App score, deprecated by Lighthouse 12.0
* `agentic_browsing_score` - How well the page can be rendered by an automated browsing agent
**Lab Metrics (measured by Lighthouse):**
* `first_contentful_paint_ms` - When the first text or image was painted
* `largest_contentful_paint_ms` - When the largest content element became visible. A Core Web Vital
* `speed_index_ms` - How quickly the page contents are visibly populated
* `total_blocking_time_ms` - Total time the main thread was blocked long enough to prevent input responsiveness
* `cumulative_layout_shift` - Unitless score of how much visible content shifted unexpectedly. A Core Web Vital
* `time_to_interactive_ms` - When the page became reliably able to respond to user input
* `server_response_time_ms` - Server response time for the main document request
* `max_potential_fid_ms` - Maximum potential First Input Delay, based on the longest main-thread task
* `first_meaningful_paint_ms` - When the page's primary content became visible
**Field Data for this page (real Chrome users, 28-day window):**
* `field_data_available` - Whether the Chrome UX Report had enough traffic to report data for this page
* `field_id` - URL or URL pattern the page-level metrics were aggregated over
* `field_overall_category` - Overall real-user speed rating of this page: FAST, AVERAGE or SLOW
* `field_origin_fallback` - True when the page had too little traffic and the data falls back to the whole origin
* `field_lcp_ms` - 75th percentile Largest Contentful Paint. A Core Web Vital
* `field_inp_ms` - 75th percentile Interaction to Next Paint. A Core Web Vital
* `field_cls` - 75th percentile Cumulative Layout Shift, returned multiplied by 100. A Core Web Vital
* `field_fcp_ms` - 75th percentile First Contentful Paint
* `field_ttfb_ms` - 75th percentile Time to First Byte
* `field_fid_ms` - 75th percentile First Input Delay, superseded by Interaction to Next Paint
**Field Data for the whole origin:**
* `origin_field_data_available` - Whether field data was reported across the whole origin
* `origin_field_id` - Origin the metrics were aggregated over
* `origin_field_overall_category` - Overall real-user speed rating of the origin: FAST, AVERAGE or SLOW
* `origin_field_lcp_ms` - 75th percentile Largest Contentful Paint across the origin
* `origin_field_inp_ms` - 75th percentile Interaction to Next Paint across the origin
* `origin_field_cls` - 75th percentile Cumulative Layout Shift across the origin, multiplied by 100
* `origin_field_fcp_ms` - 75th percentile First Contentful Paint across the origin
* `origin_field_ttfb_ms` - 75th percentile Time to First Byte across the origin
* `origin_field_fid_ms` - 75th percentile First Input Delay across the origin
**Run Metadata:**
* `lighthouse_version` - Version of Lighthouse that generated this result
* `requested_url` - URL Lighthouse was originally asked to audit
* `final_url` - Final resolved URL that was actually audited, after redirects
* `main_document_url` - URL of the main document request of the final navigation
* `final_displayed_url` - URL displayed in the browser once Lighthouse finished
* `fetch_time` - Time at which Lighthouse fetched the page
* `user_agent` - User agent string used to run the analysis
* `environment_host_user_agent` - User agent of the Chrome version that ran the analysis
* `environment_network_user_agent` - User agent sent over the network during the analysis
* `environment_benchmark_index` - Benchmark index of the analysis machine, indicating its rough device class
* `config_locale` - Locale the results were formatted in
* `config_form_factor` - Form factor Lighthouse scored the run as
* `config_channel` - Channel Lighthouse was run from
* `timing_total_ms` - Total time Lighthouse took to run the analysis
* `runtime_error_code` - Lighthouse error code when the run hit a problem serious enough to make the result unreliable
* `runtime_error_message` - Human readable explanation of the runtime error
* `run_warnings` - Warnings Lighthouse raised during the run, which may explain unusual scores (JSON array)
* `stack_packs` - Platform-specific advice, for example WordPress or React, matched to this page (JSON array)
* `pagespeed_version_major` - Major version of PageSpeed that generated this result
* `pagespeed_version_minor` - Minor version of PageSpeed that generated this result
One row per individual Lighthouse audit, roughly 150 per analysis. These are the specific checks behind each category score, such as "Eliminate render-blocking resources" or "Background and foreground colors have a sufficient contrast ratio". This is where the actionable detail lives: what is wrong, how much it costs, and which resources are responsible.
**Key Fields:**
* `url` - URL that was submitted for analysis, as configured in the connector
* `strategy` - Form factor the page was analyzed for: MOBILE or DESKTOP
* `analysis_utc_timestamp` - UTC timestamp of the analysis this audit belongs to, used as the incremental replication key
* `audit_id` - Lighthouse identifier of the audit, for example `render-blocking-resources`
**Result:**
* `title` - Human readable name of the audit as shown in the Lighthouse report
* `description` - Explanation of what the audit checks and why it matters
* `score` - Score of the audit between 0 and 1. Null for informative or manual audits
* `score_display_mode` - How to interpret the score: binary, numeric, informative, notApplicable, manual or error
* `numeric_value` - Measured value behind the audit, with a meaning specific to it
* `numeric_unit` - Unit of the numeric value, for example millisecond, byte or element
* `display_value` - Preformatted summary as shown in the report, for example "Potential savings of 120 KiB"
* `explanation` - Explanation of the failures found by the audit, when it failed
* `error_message` - Error thrown inside the audit, when it could not be evaluated
**Classification:**
* `categories` - Lighthouse categories that reference this audit, for example `['performance', 'seo']`
* `acronym` - Conventional acronym when the audit is a headline metric, for example LCP or CLS
* `group` - Category group the audit is displayed under, for example metrics or diagnostics
* `weight` - Weight this audit's score carries in its category score. Zero means it is reported but does not affect the score
**Estimated Savings:**
* `metric_savings_lcp_ms` - Estimated Largest Contentful Paint improvement if this audit were fixed
* `metric_savings_fcp_ms` - Estimated First Contentful Paint improvement if this audit were fixed
* `metric_savings_tbt_ms` - Estimated Total Blocking Time improvement if this audit were fixed
* `metric_savings_cls` - Estimated Cumulative Layout Shift improvement if this audit were fixed
* `metric_savings_inp_ms` - Estimated Interaction to Next Paint improvement if this audit were fixed
**Details:**
* `warnings` - Warnings raised while running the audit (JSON array)
* `details` - Freeform details, whose shape differs per audit type. Typically the table of offending resources or elements with the potential savings for each (JSON object)
One row per Core Web Vitals field metric per scope, from the Chrome UX Report. Where `pagespeed_results` carries the headline percentiles, this stream keeps the full distribution: for each metric, the share of real Chrome users who experienced it as good, needing improvement, or poor. That breakdown is what shows whether a bad 75th percentile is a broad problem or a long tail affecting a minority of visitors.
Reported twice, once for the specific page and once for the whole origin, distinguished by `scope`. Only available for pages and origins with enough real-user traffic.
**Key Fields:**
* `url` - URL that was submitted for analysis, as configured in the connector
* `strategy` - Form factor the page was analyzed for: MOBILE or DESKTOP
* `analysis_utc_timestamp` - UTC timestamp of the analysis this metric belongs to, used as the incremental replication key
* `scope` - Whether the metric is aggregated for this specific page (`page`) or across the whole origin (`origin`)
* `metric_id` - Chrome UX Report metric identifier, for example `LARGEST_CONTENTFUL_PAINT_MS`, `INTERACTION_TO_NEXT_PAINT`, `CUMULATIVE_LAYOUT_SHIFT_SCORE` or `EXPERIMENTAL_TIME_TO_FIRST_BYTE`
**Measurements:**
* `percentile` - 75th percentile value across real Chrome users. In milliseconds for timing metrics; for `CUMULATIVE_LAYOUT_SHIFT_SCORE` it is the score multiplied by 100
* `median` - Median value of the metric across real Chrome users
* `category` - Rating of this individual metric: FAST, AVERAGE or SLOW
* `form_factor` - Device class the metric was collected on, when reported
**Distribution:**
* `distribution_good_proportion` - Share of real-user page loads in the "good" bucket, between 0 and 1
* `distribution_needs_improvement_proportion` - Share of page loads in the "needs improvement" bucket
* `distribution_poor_proportion` - Share of page loads in the "poor" bucket
* `distributions` - Full bucket breakdown with each bucket's min and max bounds and its proportion (JSON array)
**Context:**
* `experience_id` - URL, URL pattern or origin the metric was aggregated over
* `initial_url` - URL originally requested, which can differ from the resolved experience id
* `origin_fallback` - True when the page had too little traffic and these figures fall back to the whole origin
* `overall_category` - Overall real-user speed rating of the page or origin: FAST, AVERAGE or SLOW
# Google Play as a data source
Source: https://docs.nekt.com/sources/google-play
Bring data from the Google Play Console to Nekt.
Google Play is Google's distribution platform for Android apps. The Play Console publishes everything behind your app's performance (installs and uninstalls, ratings and reviews, store listing conversion, subscriptions, estimated sales, and the earnings you are actually paid) as monthly report files in a private Cloud Storage bucket belonging to your developer account. This source reads those reports.
## Configuring Google Play as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Play option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Google Play does not expose an API for these numbers: they are only published as report files, and access to them is granted to a Google Cloud service account that you invite into your Play Console developer account. Prepare it before starting the setup:
In the [Google Cloud console](https://console.cloud.google.com/iam-admin/serviceaccounts), create a service account and, under its **Keys** tab, add a key of type **JSON**. Save the file that is downloaded. Google does not let you download it again.
In [Play Console](https://play.google.com/console), open **Users and permissions**, click **Invite new users** and enter the service account's email address (it ends in `.iam.gserviceaccount.com`).
On the **Account permissions** tab of the invitation, turn on **View app information and download bulk reports**. Add **View financial data, orders, and cancellation survey responses** as well if you want the sales and earnings tables; without it, those two tables stay empty.
In Play Console, open **Download reports** and use the **Copy Cloud Storage URI** button. It looks like `gs://pubsite_prod_rev_0000000000000000000`.
The following configurations are available:
* **Service account credentials (JSON file)**: The JSON key file you downloaded from Google Cloud. The credential is used read-only: the source only lists and downloads report files.
* **Cloud Storage URI**: The location you copied from the Download reports page.
* **Application IDs**: (Optional) Restrict the sync to specific apps, one application id per entry, such as `com.example.app`. Leave it empty to load every app in the developer account.
* **Start Date**: (Optional) The earliest month to load. Google publishes one report file per month, so any date inside a month loads that whole month. Leave it empty to load all the history kept in the bucket.
Once you're done, click **Next**.
Google captures report data daily but publishes it three to seven days later, and the earnings report only once a month, typically by the fifth of the following month. A source created today will show the most recent days as missing until Google publishes them.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Each of the installs, ratings and crashes reports comes in two versions: an **overview**, with one row per day, and a **by dimension** version, with the same metrics split by country, device, language, app version, carrier or OS version. The overview is not the sum of the breakdown (Google reports values it could not attribute only in the overview), so selecting both and adding them together counts the same activity twice.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Google rewrites the current month's report files once a day, so a daily trigger matches how often new data actually exists.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How Google Play publishes this data
These are Google's own rules, not Nekt's, and they shape what you can expect to see:
* **One file per app, per month.** A month's file is rewritten in place as days are added, and Google restates days that were already published when late data arrives. The source tracks when each file was last rewritten and re-reads exactly the files that changed, so corrections update the existing rows instead of duplicating them.
* **Data appears three to seven days after it is captured.** The earnings report is generated once a month, typically by the fifth of the following month.
* **Sales and earnings never match exactly.** Estimated sales is what buyers were charged; earnings is the reconciled statement of what Google pays you, after fees, taxes and currency conversion. Only earnings carries the payout figure.
* **Store performance starts in March 2021.** The reports it replaced, Retained installers and Buyers 7-day, hold no data after August 2021 and are not read by this source.
* **Permissions decide which tables fill.** Sales and earnings need the financial data permission on the service account; without it those tables stay empty while every other table loads normally.
* **Search terms are thresholded.** Google only names a search term in the store performance report once it is above its own reporting threshold; everything below is grouped.
Every row also carries the report month, the report file it was read from, its position inside that file and when Google last rewrote it, so any number in your catalog can be traced back to its source file.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Installs overview | `installs_overview` | Daily installs, uninstalls and upgrades per app, by device and by user account. |
| Installs by dimension | `installs_by_dimension` | The same install metrics split by country, device, language, app version, carrier or OS version. |
| Ratings overview | `ratings_overview` | Daily average rating and the cumulative rating shown on the store listing. |
| Ratings by dimension | `ratings_by_dimension` | Average rating split by country, device, language, app version, carrier or OS version. |
| Crashes overview | `crashes_overview` | Daily crash and ANR counts per app. |
| Crashes by dimension | `crashes_by_dimension` | Crash and ANR counts split by device, app version or OS version. |
| Store performance by country | `store_performance_country` | Store listing visitors, acquisitions and conversion rate per country. |
| Store performance by traffic source | `store_performance_traffic_source` | The same funnel split by traffic source, search term and UTM tags. |
| Subscriptions by country | `subscriptions_country` | New, cancelled and active subscriptions per product, base plan, offer and country. |
| Reviews | `reviews` | Full review history: rating, title, text and your replies. |
| Sales | `sales` | Estimated sales: one row per charge, refund or chargeback. |
| Earnings | `earnings` | The reconciled monthly statement behind your payout, including fees and currency conversion. |
Every stream replicates incrementally on `source_file_modified_at`, the moment Google last rewrote the report file the row came from.
## Fields by stream
Below you'll find the main data streams from Google Play and their corresponding fields:
Daily install activity for each app, one row per day.
**Key Fields:**
* `date` - Day the metrics were measured
* `package_name` - Application id of the app, such as com.example.app
* `daily_device_installs` - Devices that newly installed the app on this day
* `daily_device_uninstalls` - Devices that uninstalled the app on this day
* `daily_device_upgrades` - Devices that upgraded to a new version on this day
* `current_device_installs` - Devices that had the app installed at the end of the day
* `active_device_installs` - Devices with the app installed that were active in the last 30 days
* `daily_user_installs` - User accounts that installed the app for the first time on this day
* `daily_user_uninstalls` - User accounts that removed the app from every device they own
* `current_user_installs` - User accounts with the app installed on at least one device
* `total_user_installs` - User accounts that have ever installed the app
The install metrics above, split by one dimension per row.
**Key Fields:**
* `date` - Day the metrics were measured
* `package_name` - Application id of the app
* `dimension` - Which breakdown the row belongs to: country, device, language, app\_version, carrier or os\_version
* `dimension_value` - Value of that breakdown, such as the country or the device model
* `daily_device_installs` - Devices that newly installed the app on this day
* `active_device_installs` - Devices with the app installed that were active in the last 30 days
* `current_user_installs` - User accounts with the app installed on at least one device
How your app is rated, per day and optionally per breakdown.
**Key Fields:**
* `date` - Day the ratings were measured
* `package_name` - Application id of the app
* `daily_average_rating` - Average of the ratings submitted on this day, empty when nobody rated
* `total_average_rating` - Average of every rating accumulated so far, the score shown on the store listing
* `dimension` / `dimension_value` - Breakdown and its value, on the by dimension stream only
Stability as reported by users who opted in to sharing crash data.
**Key Fields:**
* `date` - Day the errors were reported
* `package_name` - Application id of the app
* `daily_crashes` - Crashes reported on this day
* `daily_anrs` - Application Not Responding errors reported on this day
* `dimension` / `dimension_value` - Breakdown and its value, on the by dimension stream only
The store listing funnel, per country.
**Key Fields:**
* `date` - Day the visits were measured
* `package_name` - Application id of the app
* `country_region` - Country or region the visitors were in
* `store_listing_visitors` - Users who viewed the store listing, counted once per user
* `store_listing_acquisitions` - Users who first installed the app after viewing the listing
* `store_listing_conversion_rate` - Acquisitions divided by visitors, as a fraction between 0 and 1
The same funnel, attributed to what brought the visitor in.
**Key Fields:**
* `date` - Day the visits were measured
* `package_name` - Application id of the app
* `traffic_source` - Channel the visitor arrived through, such as Play Store search or third-party referrers
* `search_term` - Search term that led to the listing, when it is above Google's reporting threshold
* `utm_source` - utm\_source tag on the store listing link, when the link carried one
* `utm_campaign` - utm\_campaign tag on the store listing link, when the link carried one
* `store_listing_visitors` - Users who viewed the store listing
* `store_listing_acquisitions` - Users who first installed the app after viewing the listing
* `store_listing_conversion_rate` - Acquisitions divided by visitors
Subscription counts per product, base plan, offer and country.
**Key Fields:**
* `date` - Day the counts refer to
* `package_name` - Application id of the app
* `product_id` - Identifier of the subscription product, as declared in Play Console
* `country` - Country the subscribers are billed in
* `base_plan_id` - Base plan the counts refer to, empty for older subscriptions
* `offer_id` - Offer the subscriber signed up through, when one applied
* `new_subscriptions` - Subscriptions started on this day
* `canceled_subscriptions` - Subscriptions cancelled on this day, which stay active until the paid period ends
* `active_subscriptions` - Subscriptions active at the end of the day
Every review left on your apps, with your replies.
**Key Fields:**
* `review_id` - Identifier of the review, taken from the review link
* `package_name` - Application id of the reviewed app
* `star_rating` - Rating the user gave, 1 to 5
* `review_title` - Title of the review, empty for reviews left after Play dropped titles
* `review_text` - Body of the review, empty when the user rated without writing
* `reviewer_language` - Language the review was written in, such as pt-BR
* `device` - Device the reviewer used
* `app_version_code` - Version of the app the reviewer was running
* `review_submit_millis_since_epoch` - When the review was submitted, in milliseconds since the Unix epoch
* `review_last_update_millis_since_epoch` - When the review was last edited
* `developer_reply_text` - Text of your reply, when the review was answered
* `developer_reply_millis_since_epoch` - When the reply was posted
* `review_link` - Play Console link that opens the review
Estimated sales: one row per charge, refund or chargeback Google recorded.
**Key Fields:**
* `order_number` - Google order identifier, repeated across the rows of the same order
* `order_charged_date` - Date the order was charged
* `order_charged_timestamp` - Moment the order was charged, in milliseconds since the Unix epoch
* `financial_status` - Charged, Refunded, Charged-back or Partially refunded
* `package_id` - Application id the purchase belongs to
* `sku_id` - Identifier of the in-app product or subscription that was bought
* `product_title` - Display name of the product
* `currency_of_sale` - Currency the buyer was charged in
* `item_price` - List price in the currency of sale, before taxes
* `taxes_collected` - Tax collected on the transaction
* `charged_amount` - Total the buyer was charged, negative on refunds and chargebacks
* `country_of_buyer` - Country the buyer was in
* `base_plan_id` / `offer_id` - Base plan and offer, for subscription purchases
* `device_model` - Device the purchase was made on
The reconciled statement behind your payout, one row per charge, fee, tax or refund.
**Key Fields:**
* `description` - What the row settles, usually the order number of the transaction
* `transaction_date` - Date of the transaction, as exported by Google
* `transaction_type` - Nature of the row: Charge, Google fee, Tax, Refund and so on
* `package_id` - Application id the row belongs to
* `sku_id` - Identifier of the in-app product or subscription
* `buyer_currency` - Currency the buyer was charged in
* `amount_buyer_currency` - Amount of the row in the buyer's currency
* `currency_conversion_rate` - Rate used to convert into the merchant currency
* `merchant_currency` - Currency you are paid in
* `amount_merchant_currency` - Amount of the row in your currency, the figure that adds up to the payout
* `service_fee` - Service fee percentage Google applied
* `buyer_country` - Country of the buyer
# Google Search Console as a data source
Source: https://docs.nekt.com/sources/google-search-console
Bring data from Google Search Console to Nekt.
Google Search Console is a free web service offered by Google that helps you monitor, maintain, and troubleshoot your site's presence in Google Search results. It provides insights into how your website performs in search results, including data on clicks, impressions, click-through rates, and search queries.
![]()
## Configuring Google Search Console as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Search Console option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Google Search Console data.
The following configurations are available:
* **Site domain**: The site domain or URL to extract data from. Supports both domain properties (e.g., `sc-domain:example.com` or `example.com`) and URL-prefix properties (e.g., `https://example.com/`). The connector automatically normalizes and matches the provided value against available properties in your account.
* **Start date**: The earliest date from which records will be synced. Used for the first run or when no previous state exists. Format: `YYYY-MM-DD`.
* **Lookback window (days)**: (Default: 3) Fixed window to reprocess data to ensure recent data from past days is processed in its entirety.
The authenticated Google account must have access to the configured site domain.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select all streams or only specific ones.
> Tip: You can quickly find streams by typing their names.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
You need at least one successful source run to see the table in your [Catalog](https://app.nekt.ai/catalog).
# Streams and Fields
Below you'll find all available data streams from Google Search Console and their corresponding fields:
Daily Search Console performance aggregated by date.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `clicks` | Integer | Number of clicks from Google Search results. |
| `impressions` | Integer | Number of times the site appeared in Google Search results. |
| `ctr` | Number | Click-through rate for the record. |
| `position` | Number | Average position in Google Search results. |
| `date` | Date | Date for which performance metrics are reported. |
| `site_url` | String | URL of the Search Console property. |
Daily performance segmented by user country.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `clicks` | Integer | Number of clicks from Google Search results. |
| `impressions` | Integer | Number of times the site appeared in Google Search results. |
| `ctr` | Number | Click-through rate for the record. |
| `position` | Number | Average position in Google Search results. |
| `date` | Date | Date for which performance metrics are reported. |
| `country` | String | Country code associated with the search user location. |
| `site_url` | String | URL of the Search Console property. |
Daily performance segmented by device type.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `clicks` | Integer | Number of clicks from Google Search results. |
| `impressions` | Integer | Number of times the site appeared in Google Search results. |
| `ctr` | Number | Click-through rate for the record. |
| `position` | Number | Average position in Google Search results. |
| `date` | Date | Date for which performance metrics are reported. |
| `device` | String | Device type used for the search interaction. |
| `site_url` | String | URL of the Search Console property. |
Daily performance segmented by page URL.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `clicks` | Integer | Number of clicks from Google Search results. |
| `impressions` | Integer | Number of times the site appeared in Google Search results. |
| `ctr` | Number | Click-through rate for the record. |
| `position` | Number | Average position in Google Search results. |
| `date` | Date | Date for which performance metrics are reported. |
| `page` | String | Canonical URL of the page in the report. |
| `site_url` | String | URL of the Search Console property. |
Daily performance segmented by search query.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `clicks` | Integer | Number of clicks from Google Search results. |
| `impressions` | Integer | Number of times the site appeared in Google Search results. |
| `ctr` | Number | Click-through rate for the record. |
| `position` | Number | Average position in Google Search results. |
| `date` | Date | Date for which performance metrics are reported. |
| `query` | String | Search query that triggered the impression or click. |
| `site_url` | String | URL of the Search Console property. |
## Implementation Notes
* All streams use incremental replication by `date`. On the first run (or when no replication state exists), the connector automatically falls back to the configured **Start date** to extract data.
* Primary keys include stream dimensions plus `site_url`.
* The connector validates that the configured domain or URL exists in the authenticated Google Search Console account before running. It automatically resolves the input by matching it across available domain properties and URL-prefix properties, stripping out protocols and prefixes when validating.
* The connector applies the configured lookback window on every run to backfill recent days.
## Skills for agents
Google Search Console connector documentation as plain markdown, for use in AI agent contexts.
# Google Sheets as a data source
Source: https://docs.nekt.com/sources/google-sheets
Bring data from Google Sheets to Nekt.
Google Sheets is a cloud-based spreadsheet application that allows users to create, edit, and collaborate on spreadsheets in real-time. It's part of Google Workspace and provides tools for data organization, analysis, and sharing, making it popular for business data management and team collaboration.
![]()
## Configuring Google Sheets as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Google Sheets option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Authorize Nekt to access your Google Sheets data by clicking on the **Google Authorization** button and logging in with your Google account. Grant the necessary permissions for Nekt to fetch your spreadsheets.
The following configurations are available:
* **Spreadsheet Link**: The URL of the Google Sheets document you want to extract data from. Paste the complete link here.
Ensure that all columns you want to extract contain at least one alphanumeric character in their names. A column named `_`, for example, will cause an error.
* **Tab names** (Advanced Configs): You can filter specific tab names to extract. This is useful when you have a spreadsheet with multiple tabs, but you just want to extract a small subset of them. This config is optional. If not provided, all tabs will be available for extraction.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. The connector dynamically reads the tabs of your spreadsheet and lists them as streams (which will later become your tables).
You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data).
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE sync. Read more about Sync Types [here](https://docs.nekt.com/core-concepts/sync-types).
* **Cell range**: If your table contains cells that are not relevant, please make sure to specify the cell range that should be considered for extraction.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
The Google Sheets connector dynamically generates streams based on the tabs in your spreadsheet. Each selected tab becomes a distinct stream (and subsequently, a table in your catalog).
The fields (columns) are defined by the headers in the first row of your selected sheet, or of the cell range you configured, if any. All values are extracted as text.
**Column Naming Behavior**
Column names are automatically sanitized to keep only letters and numbers (e.g., spaces and punctuation are replaced by underscores).
If a header cell contains no letters or numbers (for example, just symbols or emojis like `-` or `✅`), the connector will attempt to use the symbol's Unicode name to generate a column name (e.g., `white_heavy_check_mark`). If that isn't possible, it will fall back to naming the column by its position (e.g., `unnamed_column_3`). Completely blank header cells will be skipped entirely.
# Implementation Notes
### Rate Limiting and Retries
The connector handles Google Sheets API rate limits and transient errors (such as `503 Service Unavailable`, `429 Too Many Requests`, and quota limits) using an exponential backoff strategy. If the API becomes temporarily unavailable, either during the extraction of records or during the initial discovery of spreadsheet tabs and columns, the connector automatically retries requests, waiting progressively longer between attempts. It will pause and retry for up to approximately 24 minutes in total before giving up, ensuring that transient network or Google-side issues do not immediately fail your sync.
## Skills for agents
Google Sheets connector documentation as plain markdown, for use in AI agent contexts.
# GoTo Connect as a data source
Source: https://docs.nekt.com/sources/goto-connect
Bring data from GoTo Connect to Nekt.
GoTo Connect is a cloud-based phone system and unified communications platform for business telephony. This connector extracts call history and call analytics, the organization's phone-system configuration, and voicemail, fax, and messaging data through the [GoTo Connect APIs](https://developer.goto.com/GoToConnect), landing it all in your Catalog.
## Configuring GoTo Connect as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the GoTo Connect option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
GoTo Connect uses managed OAuth2, so you do not need to create an app in GoTo or copy any credentials. Click the **Authorization** button and sign in with your GoTo account, then approve the permissions Nekt requests. Nekt requests read-only scopes, so the connector can only read your data: it cannot place calls, send messages, or change any setting in GoTo Connect.
The GoTo user you authorize with must hold **Super Admin** or **View Reports** permission in the organization. Without it, GoTo returns no call reports and the corresponding streams stay empty. See [how to grant Super Admin permissions](https://support.goto.com/connect/help/how-do-i-give-a-user-user-group-super-admin-permissions-gotoconnect-give-super-admin-permissions).
The permissions Nekt requests, and what each one unlocks:
| Permission | Streams it enables |
| :- | :- |
| Access call history for phone lines in the PBX | `call_history` and the three activity report streams |
| Access voice entities like phone numbers, devices and extensions | `extensions`, `devices`, `phone_numbers`, `call_queues`, `locations`, `billing_groups`, `phone_number_orders` |
| Retrieve various GoTo Connect settings of the authenticated user | `users` |
| Retrieve your phone line information | Line details inside `users` |
| Retrieve call events | `call_events_report_summaries` |
| Read your voicemails | `voicemailboxes`, `voicemails` |
| Access faxes | `faxes` |
| Access your messages and media | `conversations`, `messages` |
The following configurations are available:
* **Start date**: The earliest date from which records will be synced.
Optionally, under advanced settings you can define:
* **Account Key**: The GoTo Connect account (organization) key to extract. Leave this empty and the connector resolves it automatically, using the first account the authorized user belongs to.
* **Organization ID**: Only needed when the authorized user belongs to more than one organization. Without it, GoTo rejects the call report requests.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How authentication works
You only ever click **Authorization** and sign in. Everything below happens on Nekt's side, and is documented here so you know what to expect when something needs your attention.
### The authorization
Nekt operates a single registered GoTo application, shared by all Nekt customers. When you click **Authorization**, GoTo asks you to sign in and approve the read-only permissions listed above. Approving them issues Nekt a credential scoped to **your** organization: it grants no access to anyone else's data, and Nekt never sees your GoTo password.
The connection represents the user who authorized it. Everything the connector can read is exactly what that user can read in GoTo Connect, which is why the Super Admin or View Reports permission matters.
### Staying connected
GoTo issues short-lived access tokens that expire after **one hour**. The connector renews them automatically during a run, so a long extraction never fails halfway through for an expired token.
Renewal relies on a longer-lived credential that is valid for **30 days**. GoTo replaces it periodically, but only once 60% of its lifetime has passed, so most renewals return nothing new and the connector keeps the credential it already has. Whenever GoTo does issue a replacement, the connector stores it immediately and uses it from then on.
The renewal credential expires after **30 days without use**. If a source is paused, disabled, or fails continuously for longer than that, the connection lapses and you have to click **Authorization** again to reconnect. Keeping the source on any regular schedule is enough to prevent this.
If the credential is ever rejected, the run fails with a message telling you to reconnect the source. Re-authorizing restores it; no data is lost, and the next run resumes from where the last successful one stopped.
### Revoking access
Removing the source in Nekt stops all extraction. To revoke Nekt's access from the GoTo side, remove the authorization for the Nekt application in your GoTo account settings.
## API Behaviors and Limitations
### Call reports are aggregated per day
The `user_activity_reports`, `caller_activity_reports`, and `phone_number_activity_reports` streams are **aggregates**, not individual events. GoTo returns them as totals over a time window, with no timestamp on each row, so the connector requests **one day at a time**. Every row therefore represents one full day of activity for one user, caller, or phone number, identified by the `report_date` column.
Because a day is only requested once it is complete, activity for the current day appears in your Catalog on the following run after midnight UTC.
GoTo retains call report data for **13 months**. Setting a start date earlier than that returns no data for the missing period, and the connector automatically starts from the oldest date still available.
### Two views of the same calls
Two streams describe calls, at different grains:
* `call_history` has one record **per call leg**. A single call involving a transfer produces several rows, tied together by `originator_id`.
* `call_events_report_summaries` has one record **per conversation**, with every participant, the outcome, and the identifiers of any recordings and transcripts.
Use `call_history` for raw telephony detail and `call_events_report_summaries` for call-level analysis. The AI analysis fields in the latter (sentiment, topics, summary) are only populated on accounts licensed for Advanced reporting; on other accounts the field is empty.
### Unanswered calls
`call_history` has no dedicated flag for a missed call. A call was not answered when `answer_time` is empty, which is how GoTo itself documents the distinction.
### Messaging requires per-number access
`conversations` and `messages` are scoped to individual phone numbers rather than to the organization. The connector walks the numbers it can see and reads the conversations on each. A user without Super Admin privileges only sees their own numbers, so these streams cover the whole organization only when the authorized user is a Super Admin.
### Rate limits
GoTo allows **10 requests per second per API**. The connector spaces its requests to stay below that and, if GoTo asks it to slow down anyway, waits and retries automatically. This can make a run take longer but never loses data. Using the GoTo APIs carries no additional charge beyond your GoTo Connect subscription.
### Streams your plan or permissions do not cover
Fax, messaging, and voicemail depend on features that not every GoTo Connect plan includes, and on permissions the authorized user may not hold. When GoTo denies access to one of them, the connector **skips that stream with a warning** and continues, rather than failing the whole run. The stream's table simply is not updated.
Fields holding nested structures (such as `lines`, `caller`, `callee`, `participants`, `ai_analysis`, `address`, `route_to`, `media`, and `delivery_statuses`) are delivered as **JSON-encoded strings** rather than native nested objects. To work with their contents, parse the string downstream (for example with `JSON_PARSE`/`JSON_EXTRACT` on AWS or `JSON_VALUE`/`JSON_QUERY` on GCP).
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Users | `users` | Users of the account and the lines assigned to each of them. |
| Extensions | `extensions` | Extensions configured on the account. |
| Devices | `devices` | Desk phones and softphones registered on the account. |
| Phone Numbers | `phone_numbers` | Phone numbers (DIDs) owned by the organization. |
| Call Queues | `call_queues` | Call queues configured on the account. |
| Locations | `locations` | Physical locations registered for the account. |
| Billing Groups | `billing_groups` | Billing groups that phone numbers are charged against. |
| Phone Number Orders | `phone_number_orders` | Orders placed for new phone numbers. |
| Call History | `call_history` | Individual call legs handled by the platform. |
| Call Events Report Summaries | `call_events_report_summaries` | Completed call summaries, one record per conversation. |
| User Activity Reports | `user_activity_reports` | Daily call activity aggregated per user. |
| Caller Activity Reports | `caller_activity_reports` | Daily call activity aggregated per external caller number. |
| Phone Number Activity Reports | `phone_number_activity_reports` | Daily call activity aggregated per organization phone number. |
| Voicemail Boxes | `voicemailboxes` | Voicemail boxes the authorized user has access to. |
| Voicemails | `voicemails` | Voicemails left in each voicemail box. |
| Faxes | `faxes` | Incoming and outgoing faxes stored on the account. |
| Conversations | `conversations` | SMS and MMS conversations held on the organization's phone numbers. |
| Messages | `messages` | Individual SMS and MMS messages exchanged in each conversation. |
## Fields by stream
Below you'll find all available data streams from GoTo Connect and their corresponding fields.
Users of the account and the lines assigned to each of them.
**Primary key:** `user_key`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `user_key` | String | Identity key of the user, stable across organizations and used to correlate the user in call and messaging data. |
| `user_id` | String | Identifier of the user within the account. The same person has a different user\_id in each organization they belong to. |
| `lines` | String | Lines assigned to the user, as a JSON array of objects with id, name, number, primary, region, account\_key and organization. |
Extensions configured on the account.
**Primary key:** `id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the extension. |
| `account_key` | String | Identifier of the GoTo Connect account the extension belongs to. |
| `name` | String | Display name of the extension. |
| `number` | String | Extension number, unique within the organization. |
| `type` | String | Kind of extension, such as DIRECT\_EXTENSION, CALL\_QUEUE, CONFERENCE\_BRIDGE, DIAL\_PLAN or VIRTUAL\_FAX\_MACHINE. |
| `organization_id` | String | Unique identifier of the organization owning the extension. |
| `extension_region` | String | Two-letter country code of the extension's region, such as US or BR. |
| `language` | String | Extension language as an RFC-1766 tag, such as en-US or pt-BR. |
| `shared_appearance` | Boolean | Whether the extension may be shared across multiple phones. |
| `omit_from_directories` | Boolean | Whether the extension is hidden from the organization's directories. |
| `omit_from_auto_attendants` | Boolean | Whether the extension is excluded from auto-attendant menus. |
Desk phones and softphones registered on the account.
**Primary key:** `id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the device. |
| `name` | String | Display name of the device. |
| `model_name` | String | Model name of the device. |
| `assigned_user_key` | String | Key of the user the device is assigned to. |
| `mac_address` | String | MAC address of the device, set once at creation and read-only after. |
Phone numbers (DIDs) owned by the organization.
**Primary key:** `id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the phone number. |
| `organization_id` | String | Unique identifier of the organization owning the phone number. |
| `account_key` | String | Identifier of the GoTo Connect account the phone number belongs to. |
| `number` | String | The phone number in E.164 format. |
| `name` | String | Display name given to the phone number. |
| `caller_id_name` | String | Name portion of the outbound caller ID for this number. |
| `route_to` | String | Destination this number routes inbound calls to, as a JSON object identifying the target extension. |
| `billing_group_id` | String | Unique identifier of the billing group the phone number belongs to. |
| `status` | String | Current state of the phone number: ACTIVE or DEACTIVATED. |
Call queues configured on the account.
**Primary key:** `id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the call queue. |
| `name` | String | Display name of the call queue. |
| `extension_number` | String | Extension number callers dial to reach the queue. |
| `type` | String | Queue type: BASIC for a standard queue, ICR for an intelligent call-routing queue. |
Physical locations registered for the account.
**Primary key:** `id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the location. |
| `name` | String | Display name of the location. |
| `address` | String | Postal address of the location, as a JSON object with line1, line2, city, state, postal\_code and country. |
| `callback_phone_number` | String | Number emergency services call back on, as a JSON object holding the E.164 number. |
| `used_for_emergency_services` | Boolean | Whether this location is the one reported to emergency services. |
Billing groups that phone numbers are charged against.
**Primary key:** `id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the billing group. |
| `name` | String | Display name of the billing group. |
Orders placed for new phone numbers.
**Primary key:** `id`\
**Sync:** Incremental on `order_date`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the phone number order. |
| `status` | String | Order status: PROCESSING while still being fulfilled, then COMPLETED or CANCELED. |
| `order_date` | Datetime | Date the order was placed, used as the incremental replication key. |
Individual call legs handled by the platform.
**Primary key:** `leg_id`\
**Sync:** Incremental on `start_time`
| Field | Type | Description |
| :- | :- | :- |
| `leg_id` | String | Globally unique id of this call leg. |
| `originator_id` | String | Common id grouping every leg that belongs to a single call. |
| `caller` | String | Party that placed the call, as a JSON object with name and number. Either may be absent when the caller is anonymous. |
| `callee` | String | Party that was called, as a JSON object with name and number. Either may be absent. |
| `direction` | String | Direction of this leg from the platform's perspective. |
| `start_time` | Datetime | Timestamp at which this leg was created, used as the incremental replication key. |
| `answer_time` | Datetime | Timestamp at which this leg was answered. Absent when the call was never answered, which is how unanswered calls are identified. |
| `duration` | Integer | Total duration of this leg, in milliseconds. |
| `hangup_cause` | Integer | Q.850 disposition code describing how the leg ended. |
| `user_key` | String | Key of the user this leg belongs to. |
| `account_key` | String | Identifier of the GoTo Connect account the leg belongs to. |
| `owner_phone_number` | String | The user's own phone number, in E.164 format. |
Completed call summaries, one record per conversation.
**Primary key:** `conversation_space_id`\
**Sync:** Incremental on `call_created`
| Field | Type | Description |
| :- | :- | :- |
| `conversation_space_id` | String | Unique identifier of the conversation this summary describes. |
| `account_key` | String | Identifier of the GoTo Connect account the conversation belongs to. |
| `call_created` | Datetime | Timestamp at which the call was created, used as the incremental replication key. |
| `call_answered` | Datetime | Timestamp at which the call was answered, absent if never answered. |
| `call_ended` | Datetime | Timestamp at which the call ended. |
| `call_initiator` | String | How the call was triggered, for example DIRECT. |
| `direction` | String | Direction of the call: INBOUND or OUTBOUND. |
| `caller` | String | Participant that started the call, as a JSON object with name, number, leg\_id, recording\_id, transcript\_ids and the resolved type. |
| `caller_outcome` | String | How the call ended for the original caller, such as NORMAL, MISSED, VOICEMAIL or LEFT\_QUEUE. |
| `participants` | String | Other parties involved after the call was answered, as a JSON array of participant objects. |
| `ai_analysis` | String | AI analysis of the call as a JSON object with sentiment, topics, summary and flags. Only populated on accounts with Advanced reporting. |
Daily call activity aggregated per user.
**Primary key:** `report_date`, `user_id`\
**Sync:** Incremental on `report_date`
| Field | Type | Description |
| :- | :- | :- |
| `report_date` | Date | Day the activity was aggregated over, in UTC. Used as the incremental replication key. |
| `user_id` | String | Identifier of the user in the organization. |
| `user_name` | String | Full name of the user. |
| `user_alternative_names` | String | Every name recorded for this user during the reporting period, as a JSON array of strings. |
| `inbound_volume` | Integer | Number of inbound calls in the reporting day. |
| `inbound_duration` | Integer | Total duration of inbound calls in the reporting day, in milliseconds. |
| `outbound_volume` | Integer | Number of outbound calls in the reporting day. |
| `outbound_duration` | Integer | Total duration of outbound calls in the reporting day, in milliseconds. |
| `average_duration` | Integer | Average call duration in the reporting day, in milliseconds. |
| `volume` | Integer | Total number of calls in the reporting day, inbound plus outbound. |
| `total_duration` | Integer | Total duration of all calls in the reporting day, in milliseconds. |
Daily call activity aggregated per external caller number.
**Primary key:** `report_date`, `number`\
**Sync:** Incremental on `report_date`
| Field | Type | Description |
| :- | :- | :- |
| `report_date` | Date | Day the activity was aggregated over, in UTC. Used as the incremental replication key. |
| `number` | String | The external number as seen by the platform, not always E.164. |
| `name` | String | Most recent name recorded for this number. |
| `alternative_phone_number_names` | String | Every name recorded for this number during the reporting period, as a JSON array of strings. |
| `inbound_volume` | Integer | Number of inbound calls in the reporting day. |
| `inbound_duration` | Integer | Total duration of inbound calls in the reporting day, in milliseconds. |
| `outbound_volume` | Integer | Number of outbound calls in the reporting day. |
| `outbound_duration` | Integer | Total duration of outbound calls in the reporting day, in milliseconds. |
| `average_duration` | Integer | Average call duration in the reporting day, in milliseconds. |
| `volume` | Integer | Total number of calls in the reporting day, inbound plus outbound. |
| `total_duration` | Integer | Total duration of all calls in the reporting day, in milliseconds. |
Daily call activity aggregated per organization phone number.
**Primary key:** `report_date`, `phone_number_id`\
**Sync:** Incremental on `report_date`
| Field | Type | Description |
| :- | :- | :- |
| `report_date` | Date | Day the activity was aggregated over, in UTC. Used as the incremental replication key. |
| `phone_number_id` | String | Globally unique identifier of the organization's phone number. |
| `phone_number` | String | The phone number as seen by the platform. |
| `phone_number_name` | String | Most recent name recorded for this phone number. |
| `alternative_phone_number_names` | String | Every name recorded for this number during the reporting period, as a JSON array of strings. |
| `inbound_volume` | Integer | Number of inbound calls in the reporting day. |
| `inbound_duration` | Integer | Total duration of inbound calls in the reporting day, in milliseconds. |
| `outbound_volume` | Integer | Number of outbound calls in the reporting day. |
| `outbound_duration` | Integer | Total duration of outbound calls in the reporting day, in milliseconds. |
| `average_duration` | Integer | Average call duration in the reporting day, in milliseconds. |
| `volume` | Integer | Total number of calls in the reporting day, inbound plus outbound. |
| `total_duration` | Integer | Total duration of all calls in the reporting day, in milliseconds. |
Voicemail boxes the authorized user has access to.
**Primary key:** `voicemailbox_id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `voicemailbox_id` | String | Unique identifier of the voicemail box. |
| `organization_id` | String | Identifier of the organization the voicemail box belongs to. |
| `account_key` | String | Identifier of the GoTo Connect account the voicemail box belongs to. |
| `extension_number` | String | Extension number the voicemail box is attached to. |
| `last_updated_timestamp` | Datetime | Timestamp the voicemail box was last modified. |
| `new_message_count` | Integer | Number of unread messages currently in the box. |
| `read_message_count` | Integer | Number of read messages currently in the box. |
Voicemails left in each voicemail box.
**Primary key:** `voicemail_id`\
**Sync:** Incremental on `timestamp`
| Field | Type | Description |
| :- | :- | :- |
| `voicemail_id` | String | Unique identifier of the voicemail. |
| `voicemailbox_id` | String | Identifier of the voicemail box holding this voicemail. |
| `organization_id` | String | Identifier of the organization the voicemail belongs to. |
| `account_key` | String | Identifier of the GoTo Connect account the voicemail belongs to. |
| `external_user_key` | String | Key of the user associated with the voicemail box. |
| `extension_name` | String | Name of the extension the voicemail was left on. |
| `extension_number` | String | Number of the extension the voicemail was left on. |
| `called_number` | String | Number that was called, either a phone number or an extension. |
| `caller_name` | String | Name of the caller who left it. |
| `caller_number` | String | Number of the caller who left it. |
| `leg_id` | String | Identifier of the call leg during which the voicemail was left. |
| `duration_ms` | Integer | Length of the voicemail recording, in milliseconds. |
| `timestamp` | Datetime | Timestamp the voicemail was received in this box, used as the incremental replication key. |
| `original_timestamp` | Datetime | Timestamp the voicemail was originally left. Differs from timestamp when the voicemail was forwarded from another box. |
| `status` | String | Current status of the voicemail: NEW, READ or DELETED. |
| `transcription_status` | String | Status of the voicemail transcription, such as SUCCESS, IN\_PROGRESS, FAILED, NO\_AUDIO or NOT\_FOUND. |
| `forward_from` | String | Chain of extensions that forwarded this voicemail, as a JSON array of objects with extension\_name, extension\_number, forwarder\_user\_key, timestamp and notes. |
| `forward_to` | String | Extensions this voicemail was forwarded to, as a JSON array of objects with extension\_name, extension\_number, forwarder\_user\_key and timestamp. |
Incoming and outgoing faxes stored on the account.
**Primary key:** `fax_id`\
**Sync:** Full table
| Field | Type | Description |
| :- | :- | :- |
| `fax_id` | String | Unique identifier of the fax. |
| `type` | String | Whether the fax is INCOMING or OUTGOING. |
| `virtual_fax_machine_id` | String | Identifier of the virtual fax machine that handled the fax. |
| `caller_id_name` | String | Caller ID name associated with the fax. |
| `caller_id_number` | String | Caller ID number associated with the fax, in E.164 format. |
| `received_time` | Datetime | Timestamp the fax was received. Incoming faxes only. |
| `creation_time` | Datetime | Timestamp the fax was created. Outgoing faxes only. |
| `last_updated_time` | Datetime | Timestamp the fax was last updated. Outgoing faxes only. |
| `status` | String | Delivery status of the fax. Outgoing faxes only. |
| `error_code` | String | Error code explaining a failed delivery. Outgoing faxes only. |
| `page_count` | Integer | Total number of pages in the fax. |
| `pages_sent` | Integer | Number of pages successfully sent. Outgoing faxes only. |
| `file_size` | Integer | Size of the fax document, in bytes. Incoming faxes only. |
| `read` | Boolean | Whether the fax has been read. Incoming faxes only. |
| `user_key` | String | Key of the user who sent the fax. Outgoing faxes only. |
SMS and MMS conversations held on the organization's phone numbers.
**Primary key:** `owner_phone_number`, `contact_phone_numbers`\
**Sync:** Incremental on `last_message_timestamp`
| Field | Type | Description |
| :- | :- | :- |
| `owner_phone_number` | String | The organization's own number in the conversation, in E.164 format. |
| `contact_phone_numbers` | String | The external parties in the conversation, as a JSON array of E.164 numbers. |
| `unread_messages_count` | Integer | Number of unread messages in the conversation. |
| `last_message_id` | String | Identifier of the most recent message in the conversation. |
| `last_message_timestamp` | Datetime | Timestamp of the most recent message, used as the incremental replication key. |
| `last_read_message_id` | String | Identifier of the last message the owner has read. |
| `last_read_message_timestamp` | Datetime | Timestamp of the last message the owner has read. |
| `last_message` | String | The most recent message itself, as a JSON object. |
Individual SMS and MMS messages exchanged in each conversation.
**Primary key:** `id`\
**Sync:** Incremental on `timestamp`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the message. |
| `owner_phone_number` | String | The organization's own number in the conversation, in E.164 format. |
| `contact_phone_numbers` | String | The external parties, as a JSON array of E.164 numbers. |
| `author_phone_number` | String | Number the message was sent from, in E.164 format. |
| `author_external_user_key` | String | Key of the user who sent it; null for an incoming message. |
| `timestamp` | Datetime | Timestamp the message was sent or received, used as the incremental replication key. |
| `direction` | String | Whether the message is IN or OUT relative to the owner's number. |
| `deleted_timestamp` | Datetime | Timestamp the message was deleted, if it was. |
| `body` | String | Text body of the message; absent for a media-only message. |
| `delivery_statuses` | String | Delivery outcome per recipient, as a JSON array of objects with message\_id, contact\_phone\_number, timestamp, delivery\_status\_code, delivery\_status\_description and retryable. Outgoing messages only. |
| `media` | String | Attachments on the message, as a JSON array of objects with id, filename, content\_type and size. |
# Gov.br MCMV as a data source
Source: https://docs.nekt.com/sources/govbr-mcmv
Bring the Minha Casa Minha Vida open datasets to your Lakehouse.
Minha Casa Minha Vida (MCMV) is the Brazilian federal housing program. The Ministry of
Cities publishes open datasets about the program (subsidized housing developments and
FGTS-financed contracts) as public CSV files, refreshed roughly every quarter. Nekt
extracts these files directly from the official gov.br open-data page: no account or
credentials are required.
The published file names change on every release, so the connector automatically
discovers the current files from the government landing page on each run. Each release
is a full snapshot of the program data.
## Configuring Gov.br MCMV as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Gov.br MCMV option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
No credentials are required: the data is public. All settings are optional and only
exist as workarounds in case the government restructures its website:
* **Landing page URL**: (Optional, advanced) The gov.br page that lists the current MCMV data files. Defaults to the official page.
* **Subsidized developments file URL / FGTS synthetic file URL / FGTS analytical file URL**: (Optional, advanced) Direct file URLs that bypass the automatic discovery for each stream. Leave empty under normal circumstances.
* **User agent**: (Optional, advanced) The User-Agent header sent to gov.br. The default mimics a browser, which the government servers require.
* **Request timeout**: (Optional, advanced) Read timeout in seconds for each HTTP request. Defaults to 300.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
The `fgts_detalhado` stream is a large file (\~1.4 GB of CSV data, millions of rows, one per financing contract). Select it only if you need contract-level detail; the `fgts_sintetico` stream covers the same universe aggregated by municipality, year and month.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: use FULL\_TABLE for all streams: every release is a complete snapshot of the program data, so each extraction replaces the previous state.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. The government refreshes these datasets roughly every quarter, so a weekly or monthly trigger is more than enough.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Gov.br MCMV and their corresponding fields. Field names follow the official SNH data dictionary, normalized to ASCII snake\_case.
Housing developments contracted under the subsidized (OGU) MCMV track, one row per development. Full table.
**Key Fields:**
* `data_referencia` - Reference date of the data snapshot
* `cod_ibge` - 6-digit IBGE code of the municipality
* `txt_nome_municipio` / `txt_sigla_uf` / `txt_regiao` - Municipality, state and macro-region
* `dt_assinatura` - Contract signature date of the development
* `cod_operacao` - Code of the operation associated with the development
* `txt_nome_empreendimento` - Name of the housing development
* `txt_nome_agente_financeiro` - Financial agent responsible for the development (e.g. CAIXA)
* `txt_modalidade` - MCMV management modality (FAR, Entidades, Oferta Pública, Rural)
* `txt_situacao_empreendimento` - Status (Concluído, Não Concluído, Distratado/Cancelado)
* `qtd_uh` / `qtd_uh_entregues` / `qtd_uh_vigentes` / `qtd_uh_distratadas` - Housing units contracted, delivered, pending and terminated
* `val_contratado_total` / `val_desembolsado` - Contracted and disbursed amounts in BRL
* `txt_cnpj_construtora_entidade` / `txt_nome_construtora_entidade` - Construction company or social entity
* `txt_endereco` / `txt_cep` - Address and postal code of the development
FGTS-financed MCMV contracts aggregated by municipality, year and month. Full table.
**Key Fields:**
* `data_referencia` - Reference date of the data snapshot
* `cod_ibge` - 6-digit IBGE code of the municipality
* `txt_municipio` / `txt_uf` - Municipality and state
* `num_ano_financiamento` - Year in which the financing contracts were signed
* `num_mes_financiamento` - Month (1-12) in which the financing contracts were signed; empty for releases published before 2026-07-24
* `qtd_uh_financiadas` - Number of housing units contracted
* `vlr_financiamento` - Total amount in BRL of the credit operations
* `vlr_subsidio` - Total amount in BRL of subsidies granted by the OGU or FGTS
* `txt_compatibilidade_faixa_renda` - Income bracket of the beneficiaries under the MCMV program rules (numeric code 1 to 4 since the 2026-07-24 release)
FGTS/FS-financed MCMV contracts, one row per financing contract. Large stream (millions of rows). Full table.
**Key Fields:**
* `data_referencia` - Reference date of the data snapshot
* `cod_ibge` / `txt_municipio` / `mcmv_fgts_txt_uf` / `txt_regiao` - Municipality, state and macro-region
* `data_assinatura_financiamento` - Signature date of the financing contract
* `qtd_uh_financiadas` - Number of housing units contracted
* `vlr_financiamento` - Amount in BRL of the credit operation
* `vlr_subsidio_desconto_fgts` / `vlr_subsidio_desconto_ogu` - Subsidies that reduce the financed amount
* `vlr_subsidio_equilibrio_fgts` / `vlr_subsidio_equilibrio_ogu` - Subsidies that lower the interest rate
* `vlr_compra` - Asking price in BRL of the property
* `vlr_renda_familiar` - Family income in BRL of the borrower
* `txt_programa_fgts` - FGTS program under which the funds were applied
* `num_taxa_juros` - Initial interest rate of the loan (percentage)
* `txt_tipo_imovel` - Whether the property is new or used
* `bln_cotista` - Whether the borrower holds an FGTS account (S/N)
* `txt_sistema_amortizacao` - Amortization system (price, sac, sacre)
* `dte_nascimento` - Birth date of the beneficiary
* `txt_compatibilidade_faixa_renda` - MCMV income bracket of the beneficiary
* `txt_nome_empreendimento` - Name of the housing development, when available
# Gupy as a data source
Source: https://docs.nekt.com/sources/gupy
Bring data from Gupy to your Lakehouse.
Gupy is a Brazilian recruitment platform (ATS) used to open jobs, receive applications, evaluate candidates and register hires. The connector extracts data from the Gupy Public API (api.gupy.io) so you can analyze your whole hiring funnel in your Lakehouse: the jobs you open, the candidatures they receive with the candidate's profile, the offers you send and what happens to each hire afterwards.
## Configuring Gupy as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Gupy option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a Gupy access token so Nekt can read your data. Sign in to the Gupy panel as an administrator, open **Settings > Tokens Generation** and create a new token with read permission for the data you want to extract. The token belongs to the company, so a single token covers every job and candidature of your account.
The Gupy public API is only available on the **Premium** and **Enterprise** plans. If the Tokens Generation screen is not available in your panel, talk to your Customer Success contact at Gupy (or open a ticket with their support) to have the API enabled before setting up this source.
The following configurations are available:
* **Access token**: your Gupy access token. It is sent as a Bearer token on every request. This field is required.
* **Initial sync date**: (Optional) the oldest candidature to extract, by creation date. Gupy offers no date filter on its endpoints, so this cut-off is applied while reading the data: it limits how much data is written to your Lakehouse, not how long the extraction takes. Leave it empty to bring your whole history.
* **Job statuses**: (Optional) a comma-separated list of job statuses to read, for example `published, closed`. Only jobs in these statuses are extracted, along with their candidatures, a good way to speed up the extraction on companies with a long hiring history. Leave it empty to read every job. Allowed values: `draft`, `waiting_approval`, `approved`, `disapproved`, `published`, `frozen`, `closed`, `canceled`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
**Application comments**, **Application rating criteria** and **Job offers** are read one candidature at a time, because that is the only way Gupy exposes them. On companies with many candidatures they are by far the slowest part of the extraction; select them only when you need that data.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: no Gupy endpoint accepts a "last updated" filter, so every stream of this connector is extracted as **FULL\_TABLE**: each run brings the current state of your data, including status changes on candidatures created long ago.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Good to know
* **Candidate data comes with the candidature.** Gupy has no candidate endpoint of its own, so a candidate's profile (name, contact, address, schooling, work experience) is returned attached to their application and is extracted in the **Applications** stream, in the `candidate_` columns. The same person applying to three jobs appears in three rows, one per candidature.
* **Extraction cost grows with the number of jobs.** Candidatures and the other per-job streams are read one job at a time, so narrowing the **Job statuses** setting is the most effective way to make a large account sync faster.
* **Permissions follow the token.** If a module is not enabled for your company, or the token was created without permission for it, the corresponding stream is skipped with a warning in the run log instead of failing the whole extraction.
* **Nested data is stored as JSON text.** Fields that hold a list of objects (a candidate's work experience and academic background, the answers to a job's additional questions, a job's custom fields and position codes, the delivery history of an offer) are stored as JSON strings, so you can parse them in a Query without your table schema changing between runs.
* **Rate limits are respected automatically.** Gupy allows 500 requests per minute per IP address. The connector backs off and retries when it is throttled, and pauses when Gupy reports the request window is nearly spent, so a large extraction slows down instead of failing.
## Streams and Fields
Below you'll find all available data streams from Gupy and their corresponding fields:
Job openings (vagas) of the company. This is the entry point of the connector: the per-job and per-candidature streams are expanded from the jobs listed here.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the job in Gupy. |
| `code` | String | Job posting code shown in the platform (e.g. 999-99999). |
| `name` | String | Name of the job opening (e.g. Programador Node.js). |
| `status` | String | Current status of the job: draft, waiting\_approval, approved, disapproved, published, frozen, closed or canceled. |
| `type` | String | Contract type of the opening (e.g. vacancy\_type\_effective, vacancy\_type\_internship, vacancy\_type\_apprentice, vacancy\_type\_talent\_pool). |
| `publication_type` | String | Whether the job is published externally or only internally (external, internal). |
| `num_vacancies` | Integer | Number of positions available in this job. |
| `department_id` | Integer | Identifier of the department that owns the job. |
| `department_name` | String | Name of the department that owns the job. |
| `role_id` | Integer | Identifier of the role (cargo) of the job. |
| `role_name` | String | Name of the role (cargo) of the job. |
| `branch_id` | Integer | Identifier of the company branch of the job. |
| `branch_name` | String | Name of the company branch of the job. |
| `description` | String | Public description of the job. |
| `responsibilities` | String | Main responsibilities and attributions of the job. |
| `prerequisites` | String | Requirements and skills expected from candidates. |
| `additional_information` | String | Additional information shown in the posting. |
| `notes` | String | Internal notes about the job, not shown to candidates. |
| `disabilities` | Boolean | Whether the job is available for people with disabilities (PcD). |
| `address_country` | String | Country of the job's work location. |
| `address_country_short_name` | String | Country abbreviation of the work location. |
| `address_state` | String | State of the job's work location. |
| `address_state_short_name` | String | State abbreviation of the work location. |
| `address_city` | String | City of the job's work location. |
| `address_street` | String | Street of the job's work location. |
| `address_district` | String | District of the job's work location. |
| `address_complements` | String | Address complement of the work location. |
| `address_number` | String | Street number of the work location. |
| `address_zip_code` | String | Postal code (CEP) of the work location. |
| `address_latitude` | Number | Latitude of the work location. |
| `address_longitude` | Number | Longitude of the work location. |
| `remote_working` | Boolean | Whether the job is remote. Being deprecated by Gupy in favour of workplace\_type. |
| `workplace_type` | String | Workplace model of the job: hybrid, on-site or remote. |
| `reason` | String | Why the position was opened: staff\_increase or staff\_replacement. |
| `image` | String | URL of the image used on the job page. |
| `social_media_image` | String | URL of the image used when sharing on social media. |
| `application_deadline` | Date | Last date on which candidates can apply. |
| `career_page_id` | Integer | Identifier of the career page the job is published on. |
| `career_page_name` | String | Name of the career page the job is published on. |
| `vacancy_codes` | String | Position codes associated with the job (JSON array of objects with id, code, status and error\_message). |
| `manager_id` | Integer | User identifier of the hiring manager. |
| `manager_name` | String | Name of the hiring manager. |
| `manager_email` | String | Email address of the hiring manager. |
| `recruiter_id` | Integer | User identifier of the recruiter in charge of the job. |
| `recruiter_name` | String | Name of the recruiter in charge of the job. |
| `recruiter_email` | String | Email address of the recruiter in charge of the job. |
| `hiring_deadline` | Date | Deadline for hiring a candidate for this job. |
| `salary_currency` | String | Currency of the salary range offered (e.g. R\$). |
| `salary_starts_at` | Number | Lower bound of the salary range offered. |
| `salary_ends_at` | Number | Upper bound of the salary range offered. |
| `custom_fields` | String | Values of the company's custom job fields (JSON array of objects with id, label, value, type, order, required and enum). |
| `job_rating_criterias` | String | Rating criteria defined for the job (JSON array of objects with job\_id and name). |
| `cancel_reason` | String | Reason the job was cancelled (e.g. budget\_review, organizational\_restructuring, other). |
| `cancel_reason_notes` | String | Free-text note explaining the cancellation when cancel\_reason is 'other'. |
| `canceled_by_email` | String | Email address of the user who cancelled the job. |
| `cancel_at` | DateTime | Timestamp when the job was cancelled. |
| `created_at` | DateTime | Timestamp when the job was created. |
| `updated_at` | DateTime | Timestamp when the job was last updated. |
| `published_at` | DateTime | Timestamp when the job was published. |
| `approved_at` | DateTime | Timestamp when the job was approved. |
| `disapproved_at` | DateTime | Timestamp when the job was disapproved. |
| `closed_at` | DateTime | Timestamp when the job was closed. |
| `last_freeze_date` | DateTime | Timestamp when the job was last frozen. |
| `last_un_freeze_date` | DateTime | Timestamp when the job was last unfrozen. |
| `creator_id` | Integer | User identifier of whoever created the job. |
| `creator_name` | String | Name of whoever created the job. |
| `creator_email` | String | Email address of whoever created the job. |
| `video_url` | String | URL of a video presenting the job. |
| `approvers` | Array | User identifiers that must approve the job before it can be published. |
| `workflow_request_method` | String | How approvals are requested: all\_at\_once or in\_order. |
Candidatures (candidaturas) of every job, with the candidate's profile unpacked into `candidate_` columns. Gupy has no candidate endpoint of its own, so this stream is where candidate data comes from.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the application. |
| `job_id` | Integer | Identifier of the job the candidate applied to. |
| `job_name` | String | Name of the job the candidate applied to. |
| `job_status` | String | Status of the candidate's current step in the job. |
| `score` | Number | Affinity score Gupy calculated between the candidate and the job. |
| `matching` | Number | How well the candidate matches the company's culture, as calculated by Gupy. |
| `source` | String | Channel the candidate came from (e.g. google, linkedin, indeed, referral, company\_website, other). |
| `source_other` | String | Free-text description of the source when it is 'other'. |
| `status` | String | Current status of the application: in\_process, give\_up, reproved or hired. |
| `current_step_id` | Integer | Identifier of the step the candidate is currently in. |
| `current_step_name` | String | Name of the step the candidate is currently in. |
| `is_company_employee` | Boolean | Whether the candidate is already an employee (internal recruitment). |
| `employee_email` | String | Corporate email of the candidate when they are an employee. |
| `registration_complete` | Boolean | Whether the candidate finished the registration step of the application. |
| `referred` | Boolean | Whether the candidate was referred by someone. |
| `referred_by` | String | Name of the person who referred the candidate. |
| `referral_status` | String | Status of the referral: pending, confirmed, rejected or unknown. |
| `feedback_sent` | Boolean | Whether a feedback message was sent to the candidate. |
| `disapproval_reason` | String | Coded reason the candidate was rejected (e.g. insufficient\_knowledge, fulfilled\_vacancy). |
| `disapproval_reason_notes` | String | Free-text note explaining the rejection when the reason is 'other\_reason'. |
| `hiring_type` | String | How the candidate was hired: employee\_admission, employee\_readmission, employee\_reintegration, unrelated\_worker\_hiring or internal\_transfer. |
| `hiring_date` | DateTime | Date the candidate was hired. |
| `partner_name` | String | Job board or partner that originated the application (e.g. linkedin, indeed, catho). |
| `tags` | Array | Tags recruiters added to this application (e.g. Proativo, Comunicativo). |
| `additional_questions` | String | Answers the candidate gave to the job's additional questions (JSON array of objects with question and answer). |
| `created_at` | DateTime | Timestamp when the application was created. |
| `updated_at` | DateTime | Timestamp when the application was last updated. |
| `ended_at` | DateTime | Timestamp the application ended: rejection date when reproved, withdrawal date when give\_up, hiring date when hired. |
| `candidate_id` | Integer | Unique identifier of the candidate in Gupy. |
| `candidate_name` | String | First name of the candidate. |
| `candidate_last_name` | String | Last name of the candidate. |
| `candidate_email` | String | Email address of the candidate. |
| `candidate_identification_document` | String | Identification document of the candidate (CPF or equivalent). |
| `candidate_disabilities` | Boolean | Whether the candidate declared a disability. |
| `candidate_disability_types` | Array | Types of disability declared by the candidate. |
| `candidate_birthdate` | Date | Date of birth of the candidate. |
| `candidate_gender` | String | Gender declared by the candidate: male, female, other or notInformed. |
| `candidate_address_zip_code` | String | Postal code (CEP) of the candidate's address. |
| `candidate_address_number` | String | Street number of the candidate's address. |
| `candidate_address_street` | String | Street of the candidate's address. |
| `candidate_address_city` | String | City the candidate lives in. |
| `candidate_address_state` | String | State the candidate lives in. |
| `candidate_address_state_short_name` | String | Abbreviation of the candidate's state. |
| `candidate_address_country` | String | Country the candidate lives in. |
| `candidate_address_country_short_name` | String | Abbreviation of the candidate's country. |
| `candidate_mobile_number` | String | Mobile phone number of the candidate. |
| `candidate_phone_number` | String | Landline phone number of the candidate. |
| `candidate_areas_of_interest` | Array | Professional areas the candidate declared interest in. |
| `candidate_schooling` | String | Highest education level of the candidate (e.g. elementary\_school, high\_school). |
| `candidate_schooling_status` | String | Status of that education level: complete, in\_progress or incomplete. |
| `candidate_academic_qualification` | String | Academic background of the candidate (JSON array of objects with id, formation, institution, course, status, start\_month, start\_year, end\_month and end\_year). |
| `candidate_work_experience` | String | Professional experience of the candidate (JSON array of objects with id, role, company\_name, activities\_performed, start\_month, start\_year, end\_month and end\_year). |
| `candidate_languages` | String | Languages the candidate speaks (JSON array of objects with language and level). |
| `manual_candidate_id` | Integer | Identifier of the candidate when they were added manually by a recruiter. |
| `manual_candidate_name` | String | First name of the manually added candidate. |
| `manual_candidate_last_name` | String | Last name of the manually added candidate. |
| `manual_candidate_email` | String | Email of the manually added candidate. |
| `manual_candidate_mobile_number` | String | Mobile phone number of the manually added candidate. |
| `manual_candidate_linkedin_profile_url` | String | LinkedIn profile URL of the manually added candidate. |
| `manual_candidate_insertion_source` | String | How the recruiter sourced the candidate: candidate\_active\_contact, events, hunting, market\_referral\_consulting or employee\_referral. |
Selection steps (etapas) configured for each job. Read one job per request.
**Primary key:** `job_id` · `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the step. |
| `job_id` | Integer | Identifier of the job the step belongs to. |
| `name` | String | Name of the step as shown in the platform (e.g. Cadastro). |
| `description` | String | Description of what happens in this step. |
| `type` | String | Kind of step: registration, online, offline, pre\_hiring or hiring. |
| `category` | String | Category of the step: evaluation, hiring, interview, offer, pre-hiring, pre-interview, registration or screening. |
| `order` | Integer | Position of the step in the selection flow. |
| `end_date` | Date | Last date on which applications are accepted in this step. |
| `online_type` | String | For online steps, what it consists of: customAssessment, providerAssessment or video. |
| `created_at` | DateTime | Timestamp when the step was created. |
| `updated_at` | DateTime | Timestamp when the step was last updated. |
Position codes (códigos de vaga) attached to each job. Read one job per request. The same codes are also in the `vacancy_codes` column of Jobs.
**Primary key:** `job_id` · `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the position code. |
| `job_id` | Integer | Identifier of the job the position code belongs to. |
| `code` | String | Position code as registered by the company (e.g. COD123). |
| `status` | String | Validation status of the code: valid, invalid, error or waiting. |
| `error_message` | String | Message explaining the problem when status is 'error'. |
| `created_at` | DateTime | Timestamp when the position code was created. |
| `updated_at` | DateTime | Timestamp when the position code was last updated. |
Users that approve or qualify candidates in each job. Read one job per request.
**Primary key:** `job_id` · `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the classifier assignment. |
| `job_id` | Integer | Identifier of the job the classifier belongs to. |
| `type` | String | Role of the classifier in the job: approver or qualifier. |
| `user_id` | Integer | Identifier of the company user acting as classifier. |
| `user_name` | String | Name of the company user acting as classifier. |
Value each job filled in for the company's custom job fields, one row per field. Read one job per request. The same values are also in the `custom_fields` column of Jobs.
**Primary key:** `job_id` · `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the custom field. |
| `job_id` | Integer | Identifier of the job the value belongs to. |
| `label` | String | Label of the custom field as shown when editing the job. |
| `value` | String | Value filled in for this job, serialized as JSON because its type varies by field. |
| `type` | String | Data type of the field: select, multiSelect, text, boolean, number, date or time. |
| `order` | Integer | Position of the field in the job edit form. |
| `required` | Boolean | Whether filling this field is mandatory. |
| `enum` | Array | Options available for select and multiSelect fields. |
Comments recruiters left on a candidate's timeline. Read one candidature per request.
**Primary key:** `application_id` · `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the comment. |
| `application_id` | Integer | Identifier of the application the comment belongs to. |
| `job_id` | Integer | Identifier of the job the application belongs to. |
| `user_id` | Integer | Identifier of the company user who wrote the comment. |
| `text` | String | Text of the comment. |
| `hidden` | Boolean | Whether the comment is private to its author and the API. |
| `created_at` | DateTime | Timestamp when the comment was created. |
| `updated_at` | DateTime | Timestamp when the comment was last updated. |
Ratings each recruiter gave a candidate on the job's evaluation criteria, one row per rating. Read one candidature per request.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the rating given by a user. |
| `rating_criteria_id` | Integer | Identifier of the job criteria that was rated. |
| `rating_criteria_name` | String | Name of the job criteria that was rated. |
| `application_id` | Integer | Identifier of the application that was rated. |
| `candidate_id` | Integer | Identifier of the candidate that was rated. |
| `job_id` | Integer | Identifier of the job the application belongs to. |
| `user_id` | Integer | Identifier of the company user who gave the rating. |
| `rating` | Integer | Score the user gave the candidate on this criteria. |
| `created_at` | DateTime | Timestamp when the rating was given. |
| `updated_at` | DateTime | Timestamp when the rating was last updated. |
Hiring details recorded when a candidate is hired: salary, admission type and the position code filled. Only requested for candidatures whose status is `hired`.
**Primary key:** `application_id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `application_id` | Integer | Identifier of the hired candidate's application. |
| `job_id` | Integer | Identifier of the job the candidate was hired for. |
| `hiring_type` | String | How the candidate was hired: employee\_admission, employee\_readmission, employee\_reintegration, unrelated\_worker\_hiring or internal\_transfer. |
| `hiring_date` | DateTime | Date the candidate was hired. |
| `job_vacancy_code_id` | Integer | Identifier of the position code (código de vaga) filled by this hiring. |
| `salary` | Number | Salary agreed for the hiring. |
| `salary_currency_type` | String | Currency of the agreed salary (R$, $, € or £). |
Job offers (propostas) sent to a candidate, with their delivery history. Read one candidature per request.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the job offer. |
| `job_id` | Integer | Identifier of the job the offer belongs to. |
| `application_id` | Integer | Identifier of the application the offer was sent to. |
| `status` | String | Status of the offer: NOT\_SENT, SENT, VIEWED, ACCEPTED, REJECTED or CANCELED. |
| `sent_at` | DateTime | Timestamp when the offer was sent to the candidate. |
| `answered_at` | DateTime | Timestamp when the candidate answered the offer. |
| `reject_reason` | String | Reason the candidate gave for rejecting the offer. |
| `salary_amount` | Number | Salary proposed in the offer. |
| `salary_currency` | String | Currency symbol of the proposed salary. |
| `admission_date` | DateTime | Planned admission date stated in the offer. |
| `admission_type` | String | Hiring type stated in the offer (e.g. employee\_admission, internal\_transfer). |
| `template_id` | Integer | Identifier of the email template used to write the offer. |
| `body` | String | Body of the offer with its placeholders already resolved. |
| `created_by` | String | Name of the company user who created the offer. |
| `candidate_portal_url` | String | URL where the candidate reads the offer. Empty while the offer is NOT\_SENT or CANCELED. |
| `deliveries` | String | Delivery history of the offer, oldest first (JSON array of objects with id, channel, status, sent\_at and created\_at). |
| `created_at` | DateTime | Timestamp when the offer was created. |
| `updated_at` | DateTime | Timestamp when the offer was last updated. |
Departments (departamentos) registered by the company.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the department. |
| `name` | String | Name of the department (e.g. Recursos Humanos, Logística). |
| `code` | String | Code of the department in the company's own systems. |
| `similar_to` | String | Standard department Gupy maps this one to, used for market benchmarking. |
| `created_at` | DateTime | Timestamp when the department was created. |
| `updated_at` | DateTime | Timestamp when the department was last updated. |
Roles (cargos) registered by the company.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the role. |
| `name` | String | Name of the role (e.g. Desenvolvedor, Estagiário). |
| `code` | String | Code of the role in the company's own systems. |
| `similar_to` | String | Standard role Gupy maps this one to (e.g. analyst, coordinator, manager). |
| `created_at` | DateTime | Timestamp when the role was created. |
| `updated_at` | DateTime | Timestamp when the role was last updated. |
Branches (filiais) of the company, with their position in the organizational structure.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the branch. |
| `code` | String | Code of the branch in the company's own systems. |
| `name` | String | Name of the branch. |
| `path` | Array | Position of the branch in the organizational structure, from the root down. |
| `address_country` | String | Country of the branch address. |
| `address_country_short_name` | String | Country abbreviation of the branch address. |
| `address_state` | String | State of the branch address. |
| `address_state_short_name` | String | State abbreviation of the branch address. |
| `address_city` | String | City of the branch address. |
| `address_street` | String | Street of the branch address. |
| `address_number` | String | Street number of the branch address. |
| `address_zip_code` | String | Postal code (CEP) of the branch address. |
| `address_latitude` | Number | Latitude of the branch address. |
| `address_longitude` | Number | Longitude of the branch address. |
| `created_at` | DateTime | Timestamp when the branch was created. |
| `updated_at` | DateTime | Timestamp when the branch was last updated. |
Career pages (páginas de carreira) where the company publishes its jobs.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the career page. |
| `name` | String | Name of the career page. |
| `subdomain` | String | Subdomain the career page is served on. |
| `job_boards_name` | String | Company name used when the page's jobs are published on external job boards. |
| `company_section_title` | String | Title of the 'about the company' section. |
| `company_section_description` | String | Text of the 'about the company' section. |
| `jobs_section_title` | String | Title of the jobs section. |
| `jobs_section_description` | String | Text of the jobs section. |
| `social_media_section_title` | String | Title of the contacts section. |
| `social_media_section_website_url` | String | URL of the company's website. |
| `social_media_section_facebook_url` | String | URL of the company's Facebook page. |
| `social_media_section_instagram_url` | String | URL of the company's Instagram profile. |
| `social_media_section_linkedin_url` | String | URL of the company's LinkedIn page. |
| `social_media_section_lovemondays_url` | String | URL of the company's Love Mondays profile. |
| `videos_section_title` | String | Title of the videos section. |
| `videos_section_description` | String | Text of the videos section. |
| `videos_section_videos` | Array | URLs of the videos shown in the videos section. |
| `candidate_testimonials_section_title` | String | Title of the candidate testimonials section. |
| `candidate_testimonials_section_description` | String | Text of the candidate testimonials section. |
| `employee_testimonials_section_title` | String | Title of the employee testimonials section. |
| `employee_testimonials_section_description` | String | Text of the employee testimonials section. |
| `created_at` | DateTime | Timestamp when the career page was created. |
| `updated_at` | DateTime | Timestamp when the career page was last updated. |
Company users: recruiters, hiring managers and everyone else with access to the Gupy panel.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the user. |
| `name` | String | Full name of the user. |
| `email` | String | Email address the user signs in with. |
| `department_id` | Integer | Identifier of the department the user belongs to. |
| `role_id` | Integer | Identifier of the role the user holds. |
| `profile_test_enabled` | Boolean | Whether the user is required to take the behavioural profile test. |
| `type` | String | Kind of user: root, owner or guest. |
| `code` | String | Identifier of the user in the company's own systems. |
| `taxpayer_registry` | String | Tax document of the user (CPF or equivalent). |
| `country_of_origin` | String | Country abbreviation of the user (e.g. BR). |
| `access_profile_id` | Integer | Identifier of the access profile assigned to the user. |
| `branch_ids` | Array | Identifiers of the company branches the user has access to. |
| `created_at` | DateTime | Timestamp when the user was created. |
| `updated_at` | DateTime | Timestamp when the user was last updated. |
Access profiles that define what each company user is allowed to do.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the access profile. |
| `name` | String | Name of the access profile. |
| `permissions` | Array | Permissions granted to the users that hold this profile. |
| `created_at` | DateTime | Timestamp when the access profile was created. |
| `updated_at` | DateTime | Timestamp when the access profile was last updated. |
Employees registered for internal recruitment.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the employee record. |
| `name` | String | Name of the employee. |
| `country_of_origin` | String | Country abbreviation of the employee (e.g. BR). |
| `identification_document` | String | Identification document of the employee (CPF or equivalent). |
| `company_id` | Integer | Identifier of the company the employee belongs to. |
Email templates used to communicate with candidates.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the email template. |
| `name` | String | Name of the template. |
| `subject` | String | Subject line of the emails sent with this template. |
| `body` | String | Body of the template, with its placeholders unresolved. |
| `type` | String | What the template is used for (e.g. interview\_scheduling, negative\_feedback, hiring\_feedback, test\_or\_challenge). |
| `public` | Boolean | Whether the template is shared with the whole company or private to its owner. |
| `user_id` | Integer | Identifier of the company user who owns the template. |
| `created_at` | DateTime | Timestamp when the template was created. |
| `updated_at` | DateTime | Timestamp when the template was last updated. |
Job templates (modelos de vaga) used to create new openings.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the job template. |
| `name` | String | Name of the template (e.g. Programador Node.js). |
| `type` | String | Contract type the template creates (e.g. vacancy\_type\_effective, vacancy\_type\_internship). |
| `department_id` | Integer | Identifier of the department of the template. |
| `department_name` | String | Name of the department of the template. |
| `role_id` | Integer | Identifier of the role of the template. |
| `role_name` | String | Name of the role of the template. |
| `branch_id` | Integer | Identifier of the branch of the template. |
| `branch_name` | String | Name of the branch of the template. |
| `description` | String | Default job description filled in by the template. |
| `responsibilities` | String | Default responsibilities filled in by the template. |
| `prerequisites` | String | Default requirements filled in by the template. |
| `additional_information` | String | Default additional information of the template. |
| `notes` | String | Default internal notes of the template. |
| `disabilities` | Boolean | Whether jobs created from this template are open to people with disabilities (PcD). |
| `address_country` | String | Country of the default work location. |
| `address_country_short_name` | String | Country abbreviation of the default work location. |
| `address_state` | String | State of the default work location. |
| `address_state_short_name` | String | State abbreviation of the default work location. |
| `address_city` | String | City of the default work location. |
| `address_street` | String | Street of the default work location. |
| `address_district` | String | District of the default work location. |
| `address_complements` | String | Address complement of the default work location. |
| `address_number` | String | Street number of the default work location. |
| `address_zip_code` | String | Postal code (CEP) of the default work location. |
| `address_latitude` | Number | Latitude of the default work location. |
| `address_longitude` | Number | Longitude of the default work location. |
| `remote_working` | Boolean | Whether the template creates remote jobs. Being deprecated in favour of workplace\_type. |
| `workplace_type` | String | Default workplace model: hybrid, on-site or remote. |
| `reason` | String | Default opening reason: staff\_increase or staff\_replacement. |
| `image` | String | URL of the default image used on the job page. |
| `social_media_image` | String | URL of the default image used on social media. |
| `career_page_id` | Integer | Identifier of the default career page of the template. |
| `career_page_name` | String | Name of the default career page of the template. |
| `custom_fields` | String | Custom fields defined for this template (JSON array of objects with id, label, value, type, order, required and enum). |
Definitions of the company's custom job fields: label, data type and available options.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the custom field. |
| `label` | String | Label of the field as shown when editing a job. |
| `type` | String | Data type of the field: select, multiSelect, text, boolean, number, date or time. |
| `order` | Integer | Position of the field in the job edit form. |
| `required` | Boolean | Whether filling this field is mandatory. |
| `enum` | Array | Options available for select and multiSelect fields. |
Dismissals (desligamentos) reported back to Gupy for hired candidates.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the dismissal record. |
| `application_id` | Integer | Identifier of the application of the candidate who was dismissed. |
| `date` | Date | Date the employee was dismissed. |
| `reason` | String | Reason for the dismissal: no\_cause\_employer\_dismissal, contract\_termination\_dismissal, early\_contract\_termination\_by\_employer\_dismissal, voluntary\_dismissal or other. |
| `is_rehired` | Boolean | Whether the person was rehired by the company. |
| `rehiring_reason` | String | Reason for the rehiring: hired\_intern, partnership or other. |
| `created_at` | DateTime | Timestamp when the dismissal record was created. |
| `updated_at` | DateTime | Timestamp when the dismissal record was last updated. |
Performance evaluations reported back to Gupy for hired candidates.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the performance evaluation. |
| `application_id` | Integer | Identifier of the application of the candidate who was evaluated. |
| `date` | Date | Date of the performance evaluation. |
| `value` | String | Result of the evaluation: high, medium or low. |
| `created_at` | DateTime | Timestamp when the evaluation was created. |
| `updated_at` | DateTime | Timestamp when the evaluation was last updated. |
# Guru Digital Manager as a data source
Source: https://docs.nekt.com/sources/guru-digital-manager
Bring data from Guru Digital Manager to Nekt.
Guru Digital Manager is a digital marketing and campaign management platform that helps businesses plan, execute, and track their digital marketing strategies. It provides tools for campaign optimization, performance analytics, and marketing automation to improve ROI and customer engagement.
![]()
## Configuring Guru Digital Manager as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "add source" button located on the top right of your screen. Then, select the Guru Digital Manager option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You will need to provide your authentication details to allow Nekt to sync your data.
The following configurations are available:
* **User token**: The user token required to authenticate against the API service. You can find the User Token on the Guru platform.
* **Start date**: Records created or updated after the start date will be extracted from the source. Format: `YYYY-MM-DD`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data streams from Guru Digital Manager and their corresponding fields:
Stream for managing affiliation data.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `name` | String | Name of the affiliation |
| `commission` | Object | Commission details including `type` and `value` |
| `contact` | Object | Contact details including `id` and `name` |
| `deletable` | Integer | Indicates if it can be deleted |
| `editable` | Integer | Indicates if it can be edited |
| `is_affiliate` | Boolean | Affiliate status |
| `marketplace_id` | String | Marketplace identifier |
| `marketplace_name` | String | Name of the marketplace |
| `product` | Object | Product details including `id` and `name` |
| `type` | String | Type of affiliation |
| `updated_at` | Integer | Last update timestamp |
Stream containing contact details and information.
Stream containing subscription data and lifecycle events.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier |
| `subscription_code` | String | Code of the subscription |
| `contact` | Object | Contact details (`doc`, `email`, `id`, `name`, `phone_local_code`, `phone_number`) |
| `product` | Object | Product details (`id`, `marketplace_id`, `marketplace_name`, `name`) |
| `cancelled_at` | String | Cancellation date |
| `charged_every_days` | Integer | Billing cycle in days |
| `charged_times` | Integer | Number of times charged |
| `contracts` | String | Contract details |
| `created_at` | Integer | Creation timestamp |
| `marketplace_id` | String | Marketplace identifier |
| `marketplace_name` | String | Name of the marketplace |
| `next_cycle_at` | String | Date of next billing cycle |
| `own_engine` | Boolean | Indicates if using own engine |
| `payment_method` | String | Payment method used |
| `started_at` | Integer | Start timestamp |
| `trial_finished_at` | String | Trial finish date |
| `trial_started_at` | String | Trial start date |
| `updated_at` | Integer | Last update timestamp |
Stream containing transaction records and tracking data.
**Key Fields:**
* `pptc` - Object containing detailed tracking information:
* `tracking_group` - Object containing tracking group `id` and `name`
* `tracking_id` - Tracking identifier
* `tracking_name` - Tracking name
* `tracking_type` - Type of tracking
* `tracking_publisher` - Tracking publisher
* `user_name` - User name
* `checkout_id` - Checkout identifier
* `checkout_name` - Checkout name
* `utm_campaign` - UTM campaign
* `utm_medium` - UTM medium
* `utm_term` - UTM term
* `utm_content` - UTM content
## Skills for agents
Guru Digital Manager connector documentation as plain markdown, for use in AI agent contexts.
# HelenaCRM as a data source
Source: https://docs.nekt.com/sources/helena-crm
Bring data from HelenaCRM to Nekt.
HelenaCRM is a Brazilian CRM and customer engagement platform that combines contact management, sales pipelines, and multi-channel messaging (WhatsApp, chat) for businesses managing customer relationships.
## Configuring HelenaCRM as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the HelenaCRM option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your HelenaCRM Bearer token. To generate it, go to **Ajustes > Integrações > Integração via API** in your HelenaCRM account.
The following configurations are available:
* **API Key**: Your HelenaCRM Bearer token (required).
* **Start Date**: The earliest date from which records will be synced (advanced setting).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables.
* **Table name**: a name is suggested, but feel free to customize it. You can add a **prefix** to all tables at once.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: only new or updated data is fetched each run.
* Full table: the current state of the data is fetched each run.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Agents | `agents` | CRM agents/operators, including their profile and department membership. |
| Channels | `channels` | Messaging channels (e.g. WhatsApp) connected to the account. |
| Contacts | `contacts` | Contacts, with tags, portfolios, custom fields and origin data. |
| Custom fields | `custom_fields` | Definitions of custom fields configured in the account. |
| Departments | `departments` | Departments, including their agents, channels and distribution config. |
| Office hours | `office_hours` | Business/office hours configuration and offline responses. |
| Portfolios | `portfolios` | Contact portfolios (groupings). |
| Tags | `tags` | Tags available for labelling contacts. |
| Templates | `templates` | Message templates (e.g. WhatsApp templates) and their status. |
## Fields by stream
Below you'll find all available data streams from HelenaCRM and their corresponding fields:
| Field | Type |
| - | - |
| id | string |
| createdAt | datetime |
| updatedAt | datetime |
| companyId | string |
| userId | string |
| name | string |
| email | string |
| phoneNumber | string |
| phoneNumberFormatted | string |
| profile | string |
| isOwner | boolean |
| availability | string |
| departments | array of objects |
| Field | Type |
| - | - |
| id | string |
| createdAt | datetime |
| updatedAt | datetime |
| active | boolean |
| type | string |
| number | string |
| numberFormatted | string |
| identity | object |
| Field | Type |
| - | - |
| id | string |
| createdAt | datetime |
| updatedAt | datetime |
| companyId | string |
| name | string |
| nameWhatsapp | string |
| nameInstagram | string |
| nameMessenger | string |
| phoneNumber | string |
| phoneNumberFormatted | string |
| email | string |
| instagram | string |
| messengerId | string |
| annotation | string |
| tagIds | array of strings |
| tagNames | array of strings |
| status | string |
| origin | string |
| importedAt | datetime |
| utm | string (JSON) |
| customFields | string (JSON) |
| portfolioIds | array of strings |
| portfolioNames | array of strings |
| metadata | string (JSON) |
| Field | Type |
| - | - |
| id | string |
| createdAt | datetime |
| updatedAt | datetime |
| parentId | string |
| type | string |
| entityType | string |
| scopeId | string |
| companyId | string |
| name | string |
| key | string |
| position | integer |
| required | boolean |
| visible | boolean |
| isValueRange | boolean |
| options | string (JSON) |
| children | string (JSON) |
| Field | Type |
| - | - |
| id | string |
| companyId | string |
| createdAt | datetime |
| updatedAt | datetime |
| name | string |
| isDefault | boolean |
| distribuitionEnabled | boolean |
| restrictionType | string |
| distributionConfig | string (JSON) |
| directLinks | string (JSON) |
| agents | array of objects |
| channels | array of objects |
| Field | Type |
| - | - |
| limitHours | boolean |
| offlineResponse | string |
| timezoneDiff | number |
| days | array of objects |
| Field | Type |
| - | - |
| id | string |
| name | string |
| description | string |
| createdAt | datetime |
| updatedAt | datetime |
| Field | Type |
| - | - |
| id | string |
| createdAt | datetime |
| updatedAt | datetime |
| name | string |
| bgColor | string |
| textColor | string |
| Field | Type |
| - | - |
| id | string |
| createdAt | datetime |
| updatedAt | datetime |
| active | boolean |
| archived | boolean |
| channelId | string |
| name | string |
| text | string |
| footerText | string |
| status | string |
| channelType | string |
| type | string |
| params | string (JSON) |
| internalName | string |
| fileType | string |
| quickReplyAlias | string |
| rejectedDescription | string |
| categoryId | string |
| categoryName | string |
# Hi Platform as a data source
Source: https://docs.nekt.com/sources/hi-platform
Bring data from Hi Platform to Nekt.
![]()
Hi Platform is a customer experience management platform that centralizes customer interactions from various channels into a single view. It provides tools for monitoring social media, managing support tickets, and analyzing customer feedback to help businesses improve their customer service and engagement.
## Configuring Hi Platform as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Hi Platform option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Hi Platform API credentials for this connection. Check [this documentation](https://help.hiplatform.com/docs/api-de-integracao-para-o-hi-socialmonitoramentos-ocorrencias-publicacoes-chamados-relatorios-engajamento-outros#chaves-de-acesso) to see how to generate them.
* **API key**: The API key used to authenticate against the API service.
* **API secret**: The API secret used to authenticate against the API service.
* **Project ID**: The project ID you want to extract data from.
* **Initial sync date**: Starting point for data extraction. Only records created or updated after this date will be synced.
Click **Next**.
### 2. Select streams
Choose which data streams you want to sync. The Hi Platform connector currently has only one available stream called `search_results`.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find the available data stream from Hi Platform and its corresponding fields:
Stream for managing search results, which can include social media mentions, support tickets, and other customer interactions.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | The search result's system ID |
| `type` | String | The type of search result |
| `title` | String | The title of the content |
| `text` | String | The content text |
| `url` | String | URL of the content |
| `media_id` | String | Media identifier |
| `published_on` | Timestamp | Timestamp of publication |
| `created_at` | Timestamp | Timestamp of creation |
| `user` | String | Username |
| `user_id` | String | User ID |
| `user_image` | String | User profile image URL |
| `attached_image_url` | String | URL of attached image |
| `attached_image_thumb_url` | String | URL of attached image thumbnail |
| `social_media` | String | Social media platform |
| `search_term` | String | Search term used |
| `polarization` | Integer | Polarization score |
| `reach` | Integer | Reach metric |
| `favorite` | Boolean | Whether result is favorited |
| `latitude` | Number | Location latitude |
| `longitude` | Number | Location longitude |
| `country` | String | Country name |
| `country_short` | String | Country code |
| `state` | String | State name |
| `state_short` | String | State code |
| `city` | String | City name |
| `gender` | String | User gender |
| `crm_user_id` | Integer | CRM user ID |
| `can_hide` | Boolean | Whether content can be hidden |
| `hidden_on_media` | Boolean | Whether hidden on media platform |
| `likes` | String | Number of likes |
| `shares` | String | Number of shares |
| `comments` | String | Number of comments |
| `tags` | String | Content tags |
## Skills for agents
Hi Platform connector documentation as plain markdown, for use in AI agent contexts.
# Hotmart as a data source
Source: https://docs.nekt.com/sources/hotmart
Bring data from Hotmart to Nekt.
Hotmart is a digital platform that enables creators to sell digital products and online courses. It provides a comprehensive ecosystem for digital product creators, including tools for hosting, payment processing, and managing student access.
## Configuring Hotmart as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Hotmart option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Hotmart API credentials for this connection. You can obtain these from the Hotmart Developer Portal (check [official docs](https://developers.hotmart.com/docs/pt-BR/start/app-auth/)). Once you have them, add the following credentials:
* **Client ID**: The client ID to authenticate against the API service.
* **Client secret**: The client secret to authenticate against the API service.
* **Token**: The basic token to authenticate against the API service. You should pass only the token value without the `Basic` prefix.
* **Members area subdomain**: The subdomain of your member's area. It can be extracted from your `hotmart.com/club/{subdomain}`. The `Students` stream is only available when this field is filled in.
* **Start date**: Records created or updated after the start date will be extracted from the source.
Under **Advanced settings** you can also tune how the connector slices date-filtered requests. Each option defaults to `365` days and only needs to be reduced (e.g. `30` or `15`) if you keep hitting timeout errors on the corresponding streams:
* **Sales window (days)**: Number of days per request window for the `Sales History`, `Sales Commission` and `Sales Participants` streams.
* **Price details window (days)**: Number of days per request window for the `Price Details` stream.
* **Subscription window (days)**: Number of days per request window for the `Subscriptions` and `Subscription Transactions` streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. The available streams include sales history, subscriptions, products, and more.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Hotmart and their corresponding fields. Date fields are delivered as the Hotmart API returns them: integer epoch timestamps in milliseconds.
Stream containing historical sales data from your Hotmart products.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the sale (transaction ID) |
| `product` | Object | Product information |
| `product.id` | Integer | Product identifier |
| `product.name` | String | Name of the product |
| `buyer` | Object | Buyer information |
| `buyer.name` | String | Name of the buyer |
| `buyer.ucode` | String | Unique code for the buyer |
| `buyer.email` | String | Email of the buyer |
| `producer` | Object | Producer information |
| `producer.name` | String | Name of the producer |
| `producer.ucode` | String | Unique code for the producer |
| `purchase` | Object | Purchase details |
| `purchase.transaction` | String | Transaction identifier |
| `purchase.order_date` | Integer | Order date (epoch milliseconds) |
| `purchase.approved_date` | Integer | Approval date (epoch milliseconds) |
| `purchase.status` | String | Current status of the sale |
| `purchase.recurrency_number` | Integer | Recurrence number for subscriptions |
| `purchase.is_subscription` | Boolean | Whether this is a subscription purchase |
| `purchase.commission_as` | String | Commission type |
| `purchase.price.value` | Number | Price value |
| `purchase.price.currency_code` | String | Currency code |
| `purchase.payment.method` | String | Payment method |
| `purchase.payment.installments_number` | Integer | Number of installments |
| `purchase.payment.type` | String | Payment type |
| `purchase.tracking.source_sck` | String | Source tracking code (SCK) |
| `purchase.tracking.source` | String | Traffic source |
| `purchase.tracking.external_code` | String | External tracking code |
| `purchase.warranty_expire_date` | Integer | Warranty expiration date (epoch milliseconds) |
| `purchase.offer.payment_mode` | String | Payment mode |
| `purchase.offer.code` | String | Offer code |
| `purchase.hotmart_fee.total` | Number | Total Hotmart fee |
| `purchase.hotmart_fee.fixed` | Number | Fixed fee amount |
| `purchase.hotmart_fee.currency_code` | String | Currency code of the fee |
| `purchase.hotmart_fee.base` | Number | Base fee amount |
| `purchase.hotmart_fee.percentage` | Number | Fee percentage |
Stream containing information about recurring subscriptions.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `subscriber_code` | String | Code identifying the subscriber |
| `subscription_id` | Integer | Unique identifier for the subscription |
| `status` | String | Current status of the subscription |
| `accession_date` | Integer | Date when the subscription started (epoch milliseconds) |
| `end_accession_date` | Integer | Date when the subscription ended (epoch milliseconds) |
| `request_date` | Integer | Date when the subscription was requested (epoch milliseconds) |
| `date_next_charge` | Integer | Date of the next charge (epoch milliseconds) |
| `trial` | Boolean | Whether this is a trial subscription |
| `transaction` | String | Transaction identifier |
| `plan` | Object | Subscription plan details |
| `plan.id` | Integer | Plan identifier |
| `plan.name` | String | Name of the plan |
| `plan.recurrency_period` | Integer | Period between charges |
| `plan.max_charge_cycles` | Integer | Maximum number of charge cycles |
| `product` | Object | Product information |
| `product.id` | Integer | Product identifier |
| `product.name` | String | Product name |
| `product.ucode` | String | Unique product code |
| `price` | Object | Price information |
| `price.value` | Number | Price value |
| `price.currency_code` | String | Currency code |
| `subscriber` | Object | Subscriber information |
| `subscriber.name` | String | Subscriber's name |
| `subscriber.email` | String | Subscriber's email |
| `subscriber.ucode` | String | Unique subscriber code |
Stream containing transaction history for subscriptions.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the transaction |
| `last_recurrency_start_date` | Integer | Start date of the last recurrence (epoch milliseconds) |
| `has_unpaid_recurrency` | Boolean | Whether there are unpaid recurrences |
| `product` | Object | Product information |
| `product.id` | Integer | Product identifier |
| `product.name` | String | Product name |
| `subscriber` | Object | Subscriber information |
| `subscriber.id` | Integer | Subscriber identifier |
| `subscriber.name` | String | Subscriber's name |
| `subscriber.email` | String | Subscriber's email |
| `subscriber.phone` | String | Phone number |
| `subscriber.phone_ddd` | String | Phone area code |
| `recurrency` | Object | Recurrence information |
| `recurrency.number` | Integer | Recurrence number |
| `recurrency.scheduled_retry` | Integer | Scheduled retry timestamp (epoch milliseconds) |
| `recurrency.number_list` | String | List of recurrence numbers |
| `recurrency.transaction_sequence` | Integer | Transaction sequence number |
| `recurrency.start_datetime` | Integer | Start date and time (epoch milliseconds) |
| `recurrency.payment_delays_days` | Integer | Days of payment delay |
| `recurrency.transaction_type` | String | Type of transaction |
| `recurrency.is_current_purchase` | Boolean | Whether this is the current purchase |
| `recurrency.has_retry` | Boolean | Whether retries are available |
| `recurrency.status` | String | Status of the recurrence |
| `last_recurrency_number` | Integer | Last recurrence number |
| `trial_info` | Object | Trial information |
| `trial_info.trial` | Boolean | Whether this is a trial |
| `trial_info.trial_end` | Integer | Trial end date (epoch milliseconds) |
| `trial_info.trial_period` | Integer | Trial period duration |
| `purchase` | Object | Purchase information |
| `purchase.transaction` | String | Transaction identifier |
| `purchase.status` | String | Transaction status |
| `purchase.order_date` | Integer | Order date (epoch milliseconds) |
| `purchase.approved_date` | Integer | Approval date (epoch milliseconds) |
| `purchase.price.value` | Number | Base price value |
| `purchase.price.total_value` | Number | Total price value |
| `purchase.price.currency` | String | Currency code |
| `purchase.installment.installment_number` | Integer | Number of installments |
| `purchase.installment.installment_type` | String | Type of installment |
| `purchase.payment.payment_type` | String | Type of payment |
| `purchase.payment.credit_card_flag` | String | Credit card brand |
| `purchase.payment.refusal_message` | String | Payment refusal message |
| `purchase.payment.refund_chargeback_date` | Integer | Refund or chargeback date (epoch milliseconds) |
| `purchase.payment.billet_expiration_date` | Integer | Bank slip expiration date (epoch milliseconds) |
| `purchase.payment.billet_recovery_type` | String | Bank slip recovery type |
| `purchase.payment.billet_reprint_code` | String | Bank slip reprint code |
| `purchase.payment.pix_expiration_date` | Integer | PIX expiration date (epoch milliseconds) |
| `purchase.commission.original_value` | Number | Original commission value |
| `purchase.commission.original_paid_value` | Number | Original paid value |
| `purchase.commission.producer_value` | Number | Producer's commission value |
| `purchase.commission.producer_paid_value` | Number | Amount paid to the producer |
| `purchase.commission.currency` | String | Currency code |
| `purchase.commission.conversion_rate` | Number | Conversion rate |
| `cancellation_date` | Integer | Cancellation date (epoch milliseconds) |
| `is_paid_anticipation` | Boolean | Whether payment was anticipated |
| `max_cycles` | Integer | Maximum number of cycles |
| `adoption_date` | Integer | Adoption date (epoch milliseconds) |
| `subscriber_code` | String | Subscriber's code |
| `date_next_charge` | Integer | Next charge date (epoch milliseconds) |
| `is_paid_negotiation` | Boolean | Whether payment was negotiated |
| `last_update` | Integer | Last update timestamp (epoch milliseconds) |
| `billing_type` | String | Type of billing |
| `producer` | Object | Producer information |
| `producer.name` | String | Producer's name |
| `subscription_id` | Integer | Subscription identifier |
| `has_credit_card_change` | Boolean | Whether credit card was changed |
| `plan` | Object | Plan information |
| `plan.name` | String | Plan name |
| `plan.recurrency_period` | Integer | Period between recurrences |
| `plan.recurrency_type` | String | Type of recurrence |
| `plan.coupon_code` | String | Applied coupon code |
| `plan.offer.code` | String | Offer code |
| `plan.offer.key` | String | Offer key |
| `plan.offer.description` | String | Offer description |
| `status` | String | Overall status |
Stream containing commission information for sales.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `transaction` | String | Unique identifier for the transaction |
| `product` | Object | Product information |
| `product.id` | Integer | Product identifier |
| `product.name` | String | Name of the product |
| `exchange_rate_currency_payout` | Number | Exchange rate for currency payout |
| `commissions` | Array | Array of commission information |
| `commissions[].commission.value` | Number | Commission amount |
| `commissions[].commission.currency_value` | String | Currency of the commission |
| `commissions[].user.ucode` | String | Unique code of the commissioned user |
| `commissions[].user.name` | String | Name of the commissioned user |
| `commissions[].source` | String | Source of the commission |
Stream containing information about participants in sales transactions.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `transaction` | String | Unique identifier for the transaction |
| `product` | Object | Product information |
| `product.id` | Integer | Product identifier |
| `product.name` | String | Name of the product |
| `users` | Array | Array of user information |
| `users[].role` | String | Role of the user in the transaction |
| `users[].user.ucode` | String | Unique user code |
| `users[].user.name` | String | User's name |
| `users[].user.trade_name` | String | User's trade name |
| `users[].user.email` | String | Email address |
| `users[].user.phone` | String | Phone number |
| `users[].user.cellphone` | String | Cellphone number |
| `users[].user.locale` | String | User's locale |
| `users[].user.documents` | Array | Array of document information |
| `users[].user.documents[].type` | String | Document type |
| `users[].user.documents[].value` | String | Document value |
| `users[].user.address.address` | String | Street address |
| `users[].user.address.number` | String | Street number |
| `users[].user.address.complement` | String | Address complement |
| `users[].user.address.neighborhood` | String | Neighborhood |
| `users[].user.address.city` | String | City |
| `users[].user.address.state` | String | State |
| `users[].user.address.country` | String | Country |
| `users[].user.address.zip_code` | String | ZIP code |
Stream containing detailed pricing information for products.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `transaction` | String | Unique identifier for the transaction |
| `product` | Object | Product information |
| `product.id` | Integer | Product identifier |
| `product.name` | String | Product name |
| `base` | Object | Base price information |
| `base.value` | Number | Base price value |
| `base.currency_code` | String | Currency code |
| `total` | Object | Total price information |
| `total.value` | Number | Total price value |
| `total.currency_code` | String | Currency code |
| `vat` | Object | VAT information |
| `vat.value` | Number | VAT amount |
| `vat.currency_code` | String | Currency code |
| `fee` | Object | Fee information |
| `fee.value` | Number | Fee amount |
| `fee.currency_code` | String | Currency code |
| `coupon` | Object | Coupon information |
| `coupon.code` | String | Coupon code |
| `coupon.value` | Number | Coupon value |
| `real_conversion_rate` | Number | Real currency conversion rate |
Stream containing information about your Hotmart products.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the product |
| `name` | String | Name of the product |
| `ucode` | String | Unique product code |
| `status` | String | Current status of the product |
| `created_at` | Integer | Creation date (epoch milliseconds) |
| `format` | String | Product format |
| `is_subscription` | Boolean | Whether the product is a subscription |
| `warranty_period` | Integer | Warranty period in days |
Stream containing the discount coupons configured for each of your products. Coupons are fetched per product, so this stream depends on the `Products` stream.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the coupon |
| `coupon_code` | String | Coupon code |
| `active` | Boolean | Whether the coupon is active |
| `status` | String | Current status of the coupon |
| `discount` | Number | Discount applied by the coupon |
| `start_date` | Integer | Date from which the coupon is valid (epoch milliseconds) |
| `time_zone` | Object | Time zone in which the coupon dates are defined |
| `time_zone.id` | String | Time zone identifier |
| `time_zone.name` | String | Time zone name |
| `time_zone.offset` | String | Time zone offset |
| `time_zone.description` | String | Time zone description |
Stream containing information about students enrolled in your products. This stream is only available when the **Members area subdomain** is configured.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `user_id` | String | Unique identifier for the student |
| `engagement` | String | Student's engagement level |
| `name` | String | Student's name |
| `email` | String | Student's email |
| `last_access_date` | Integer | Last access timestamp (epoch milliseconds) |
| `role` | String | Student's role |
| `first_access_date` | Integer | First access timestamp (epoch milliseconds) |
| `locale` | String | Student's locale |
| `plus_access` | String | Plus access status |
| `progress` | Object | Progress information |
| `progress.completed_percentage` | Integer | Percentage of completion |
| `progress.total` | Integer | Total number of items |
| `progress.completed` | Integer | Number of completed items |
| `status` | String | Current status |
| `access_count` | Integer | Number of accesses |
| `is_deletable` | Boolean | Whether the student can be deleted |
| `class_id` | String | Class identifier |
| `type` | String | Student type |
## Implementation Notes
### API Pagination & Limits
* **Windowed Pagination**: To prevent timeouts and improve reliability when extracting large amounts of historical data, the date-filtered streams (`Sales History`, `Sales Commission`, `Sales Participants`, `Subscriptions`, `Subscription Transactions` and `Price Details`) utilize windowed pagination. The connector requests data in yearly (365-day blocks) starting from your configured `Start Date` up to the current date. The block size of each stream group can be changed through the **Sales window**, **Price details window** and **Subscription window** advanced settings.
* **Automatic Window Shrinking (504 Gateway Timeout handling)**: The Hotmart API can return `504 Gateway Timeout` errors when a single window holds too much data. When that happens on any of the windowed streams above, the connector automatically halves the current extraction window (e.g. 365 → 182 → 91 → 45 → 30 days), rebuilds the request for the same period and retries. The reduced window is kept for the rest of the run. Only when the window is already at the minimum of 30 days and the API still times out does the stream fail, with a message pointing to the corresponding window setting.
* **Automatic Token Refresh (401 Unauthorized handling)**: The Hotmart API occasionally invalidates live access tokens. If a `401 Unauthorized` error (`invalid_token` or `token_expired`) is received in the middle of an extraction, the connector automatically requests a new access token and retries the failed request.
* **Rate Limiting**: Requests are throttled to the Hotmart API limit and, when a `429 Too Many Requests` response is received, the connector waits for the period indicated by the API before retrying automatically.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `504 Gateway Timeout` while reading price details | The configured query window is too large for the volume on the account. | Lower **Price details window** in the source advanced settings. This does not lose data: it only splits the query into smaller slices. |
| `429 Too Many Requests` | You hit Hotmart's rate limit. | Reduce concurrency by scheduling sources that share the account at different times. |
## Skills for agents
Hotmart connector documentation as plain markdown, for use in AI agent contexts.
# HubSpot as a data source
Source: https://docs.nekt.com/sources/hubspot
Bring your data from HubSpot to Nekt.
HubSpot is a comprehensive customer relationship management (CRM) and marketing automation platform that helps businesses attract, engage, and delight customers. It provides tools for marketing, sales, customer service, and content management, all integrated into a single platform.
![]()
For more info on the streams and permissions, check the [HubSpot API Documentation](https://developers.hubspot.com/docs/api/overview).
## Configuring HubSpot as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the HubSpot option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Configure the following settings:
* **Authentication Mode**: The authentication mode to use. You can select **Sign in with HubSpot** (OAuth) or use a HubSpot private app token. If you choose to **Sign in with HubSpot**, you'll be prompted to log in with an account that has **Super Admin** permissions in order to install the Nekt app into the workspace.
* **Start Date**: The date of the first record you want to sync with.
* **Enable Leads Stream** (Optional): Enable to extract leads data (requires HubSpot Pro, Enterprise, or Enterprise Plus).
* **Associations** (Optional): Configure which associations to extract per object type. Each entry specifies a **HubSpot entity name** (e.g. `deals`, `contacts`, `companies`, `tickets`, `leads`, `calls`, `emails`, `meetings`, `notes`, `tasks`, `postal_mail`, `communication`, or a custom object name) and a **comma-separated list of associated entities** to retrieve for that object. Custom object IDs (e.g. `2-12345`) are also supported in the association list. Enable only if necessary, since it significantly increases the extraction time.
* **Extract Contact Property History** (Optional): Enable to track property changes for contacts. Enable only if necessary, since it significantly increases the extraction time.
* **Contact Property History Fields** (Optional): If enabled above, specify which properties to track (comma-separated). Default: `lifecyclestage, lead_status`.
* **Extract Deal Property History** (Optional): Enable to track property changes for deals. Enable only if necessary, since it significantly increases the extraction time.
* **Deal Property History Fields** (Optional): If enabled above, specify which properties to track (comma-separated). Default: `dealstage`.
* **Extract Company Property History** (Optional): Enable to track property changes for companies. Enable only if necessary, since it significantly increases the extraction time.
* **Company Property History Fields** (Optional): If enabled above, specify which properties to track (comma-separated). Default: `name`.
* **Extract Ticket Property History** (Optional): Enable to track property changes for tickets. Enable only if necessary, since it significantly increases the extraction time.
* **Ticket Property History Fields** (Optional): If enabled above, specify which properties to track (comma-separated). Default: `hs_pipeline_stage, hs_ticket_priority`.
* **Property objects** (Optional): Comma-separated list of HubSpot object types to fetch property definitions for. Default: `calls, communication, company, contacts, deals, email, line_item, meeting, notes, postal_mail, product, task, tickets`.
* **Properties as JSON string** (Optional): When enabled, the `properties` column in CRM object streams (contacts, companies, deals, tickets, etc.) is emitted as a JSON string instead of a structured object. Enable this if your HubSpot account has a very large number of custom properties and you are hitting destination limits on nested fields (e.g. BigQuery's 10,000 field limit). Default: `false`.
Enabling associations and property history options will increase extraction time. Enable only the object types and properties you actually need.
Click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked *optional* are only discovered when the corresponding setting is enabled.
| Stream | Slug | Description |
| - | - | - |
| Contacts | `contacts` | CRM contact records and their properties. |
| Companies | `companies` | CRM company records and their properties. |
| Deals | `deals` | CRM deal records and their properties. |
| Deals (Archived) | `deals_archived` | Archived (deleted) deal records. |
| Tickets | `tickets` | Support ticket records and their properties. |
| Line Items | `line_items` | Line items associated with deals and quotes. |
| Products | `products` | Products defined in the HubSpot product library. |
| Quotes | `quotes` | Sales quotes and their properties. |
| Services | `services` | Service object records. |
| Calls | `calls` | Logged call engagements. |
| Communications | `communications` | Communication engagements (e.g. WhatsApp, SMS, LinkedIn). |
| Emails | `emails` | Logged email engagements. |
| Meetings | `meetings` | Logged meeting engagements. |
| Notes | `notes` | Note engagements. |
| Postal Mail | `postal_mail` | Postal mail engagements. |
| Tasks | `tasks` | Task engagements. |
| Tasks (Archived) | `tasks_archived` | Archived (deleted) task records. |
| Owners | `owners` | HubSpot owners (users who can be assigned to records). |
| Users | `users` | HubSpot account users. |
| Users Properties | `users_properties` | CRM Users object with its full property set. Requires the sensitive `crm.objects.users.read` scope; skipped when not granted. |
| Deal Pipelines | `deal_pipelines` | Deal pipelines and their stages. |
| Lead Pipelines | `lead_pipelines` | *Optional.* Lead pipelines and their stages; only synced when `enable_leads_stream` is enabled. |
| Ticket Pipelines | `ticket_pipelines` | Ticket pipelines and their stages. |
| Properties | `properties` | Property (field) definitions for the configured object types. |
| Forms | `forms` | Marketing/CRM form definitions. |
| Form Submissions | `form_submissions` | Submissions received through forms. |
| Feedback Submissions | `feedback_submissions` | Read-only CRM object with NPS, CSAT, CES and custom survey responses. Requires the `crm.objects.feedback_submissions.read` scope; skipped when not granted. |
| Marketing Emails | `marketing_emails` | Marketing email assets. |
| Audit Logs | `audit_logs` | Account activity audit log entries. |
| Leads | `leads` | *Optional.* Lead records; only synced when `enable_leads_stream` is enabled (requires HubSpot Pro/Enterprise). |
| Custom Objects | *(per object name)* | *Optional.* One stream per custom object defined in the account; the slug is the custom object's name, discovered dynamically from the HubSpot schemas API. |
## Fields by stream
Below you'll find all available data streams from HubSpot and their corresponding fields:
Alongside all the fields listed here, most of the streams have additional/custom fields from HubSpot, specific to each workspace and account type. These fields are dynamically included based on your HubSpot configuration.
Additionally, if your HubSpot Private App Token lacks the required scopes to access a specific stream or object (such as Custom Objects), the connector will gracefully skip it during extraction instead of failing the sync. If you are missing expected streams, verify your token permissions in HubSpot.
HubSpot accounts with a very large number of custom properties may exceed destination limits on nested fields. For example, BigQuery enforces a maximum of 10,000 fields per table, and the `properties` column in CRM object streams (contacts, companies, deals, tickets, etc.) can exceed this limit. If you run into this issue, enable the **Properties as JSON string** option in your source configuration to emit the `properties` column as a flat JSON string instead of a structured object.
Archived sales deals and opportunities. Fetches archived deals via the list endpoint.
**Note:** This stream is full-table replication only, as HubSpot's search API does not return archived records.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the deal |
| `properties.dealname` | String | Deal name |
| `properties.amount` | String | Deal amount |
| `properties.dealstage` | String | Current stage |
| `properties.pipeline` | String | Pipeline ID |
| `properties.closedate` | String | Expected close date |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.createdate` | String | Creation date |
| `properties.hs_lastmodifieddate` | String | Last modification date |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `archivedAt` | DateTime | Timestamp when the record was archived |
| `url` | String | API URL of the record |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Archived task assignments and to-dos. Fetches archived tasks via the list endpoint.
**Note:** This stream is full-table replication only, as HubSpot's search API does not return archived records.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the task |
| `properties.hs_task_subject` | String | Task subject |
| `properties.hs_task_status` | String | Task status |
| `properties.hs_task_priority` | String | Priority level |
| `properties.hs_task_type` | String | Task type |
| `properties.hs_timestamp` | String | Creation timestamp |
| `properties.hubspot_owner_id` | String | Task owner |
| `properties.hs_task_body` | String | Task description |
| `properties.hs_task_due_date` | String | Due date |
| `archivedAt` | DateTime | Timestamp when the record was archived |
| `url` | String | API URL of the record |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Account activity audit logs. Tracks user actions such as CRM object creation, property updates, and security activity.
**Note:** Requires a HubSpot Enterprise account and the `account-info.security.read` scope.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the audit log entry |
| `category` | String | Category of the audit event (e.g. PIPELINE, PROPERTY\_VALUE) |
| `subCategory` | String | Sub-category of the audit event (e.g. DEAL, CONTACT) |
| `action` | String | Action performed (e.g. UPDATE, PERFORM, CREATE) |
| `targetObjectId` | String | ID of the object that was acted upon |
| `occurredAt` | DateTime | Timestamp when the audit event occurred |
| `actingUser` | Object | User who performed the audited action (contains `userId` and `userEmail`) |
Call activity and engagement data.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the call |
| `properties.hs_call_direction` | String | Call direction (inbound/outbound) |
| `properties.hs_call_duration` | String | Duration of the call |
| `properties.hs_call_status` | String | Call status |
| `properties.hs_timestamp` | String | When the call occurred |
| `properties.hubspot_owner_id` | String | Call owner |
| `properties.hs_call_title` | String | Call title or subject |
| `properties.hs_call_body` | String | Call notes or description |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Communication activities and engagements (SMS, WhatsApp, LinkedIn messages, etc.).
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the communication |
| `properties.hs_communication_channel_type` | String | Channel type (SMS, WhatsApp, etc.) |
| `properties.hs_communication_body` | String | Communication content |
| `properties.hs_timestamp` | String | When the communication occurred |
| `properties.hubspot_owner_id` | String | Communication owner |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Company and organization records.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the company |
| `properties.name` | String | Company name |
| `properties.domain` | String | Company website domain |
| `properties.industry` | String | Industry classification |
| `properties.phone` | String | Company phone number |
| `properties.address` | String | Company address |
| `properties.city` | String | Company city |
| `properties.state` | String | Company state/region |
| `properties.country` | String | Company country |
| `properties.zip` | String | Postal code |
| `properties.numberofemployees` | String | Employee count |
| `properties.annualrevenue` | String | Annual revenue |
| `properties.lifecyclestage` | String | Company lifecycle stage |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.createdate` | String | Creation date |
| `properties.hs_lastmodifieddate` | String | Last modification date |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Individual contact records and their properties.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the contact |
| `properties.email` | String | Email address |
| `properties.firstname` | String | First name |
| `properties.lastname` | String | Last name |
| `properties.phone` | String | Phone number |
| `properties.company` | String | Company name |
| `properties.jobtitle` | String | Job title |
| `properties.lifecyclestage` | String | Contact lifecycle stage |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.createdate` | String | Creation date |
| `properties.lastmodifieddate` | String | Last modification date |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Custom objects defined in your HubSpot account. These are automatically discovered and extracted.
**Note:** Each custom object will appear as a separate stream with its own name based on the object definition in HubSpot. The fields are dynamic and depend on the custom object schema configured in your HubSpot account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the custom object record |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
| `archived` | Boolean | Whether the record is archived |
Sales deals and opportunities.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the deal |
| `properties.dealname` | String | Deal name |
| `properties.amount` | String | Deal amount |
| `properties.dealstage` | String | Current stage |
| `properties.pipeline` | String | Pipeline ID |
| `properties.closedate` | String | Expected close date |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.createdate` | String | Creation date |
| `properties.hs_lastmodifieddate` | String | Last modification date |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Deal pipeline configurations and stages for sales workflows.
| Field | Type | Description |
| :- | :- | :- |
| `label` | String | Display name of the pipeline |
| `displayOrder` | Integer | Order in which the pipeline is displayed |
| `active` | Boolean | Whether the pipeline is active |
| `stages` | Array | Array of pipeline stages with their configurations |
| `objectType` | String | Type of object this pipeline applies to |
| `objectTypeId` | String | ID of the object type |
| `pipelineId` | String | Unique identifier for the pipeline |
| `createdAt` | String | Date when pipeline was created |
| `updatedAt` | String | Last modification timestamp |
| `default` | Boolean | Whether this is the default pipeline |
Email communications and messaging data.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the email |
| `properties.hs_email_subject` | String | Email subject line |
| `properties.hs_email_text` | String | Email body content |
| `properties.hs_email_direction` | String | Email direction (inbound/outbound) |
| `properties.hs_email_status` | String | Email status |
| `properties.hs_email_sender_email` | String | Sender email address |
| `properties.hs_email_sender_firstname` | String | Sender first name |
| `properties.hs_email_sender_lastname` | String | Sender last name |
| `properties.hs_email_to_email` | String | Recipient email address |
| `properties.hs_email_to_firstname` | String | Recipient first name |
| `properties.hs_email_to_lastname` | String | Recipient last name |
| `properties.hs_timestamp` | String | Email timestamp |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.createdate` | String | Date when email was created |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
| `archived` | Boolean | Whether the email is archived |
Email subscription type definitions (e.g., marketing, legal basis) used for consent and preferences.
This stream is currently disabled and will not appear in the stream selection list.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the subscription type |
| `portalId` | Integer | Associated portal ID |
| `name` | String | Name of the subscription type |
| `description` | String | Description of the email subscription |
| `active` | Boolean | Whether the subscription type is active |
| `internal` | Boolean | Whether the subscription is for internal use only |
| `category` | String | Category classification of the subscription |
| `channel` | String | Channel associated with the subscription |
| `internalName` | String | Internal name of the subscription |
| `businessUnitId` | Integer | Associated business unit ID |
Form definitions and configurations.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the form |
| `name` | String | Form name |
| `submitText` | String | Submit button text |
| `formType` | String | Type of form |
| `createdAt` | String | Creation timestamp |
| `updatedAt` | String | Last update timestamp |
| `fields` | Array | Array of form fields and their properties |
Submitted form data and responses.
| Field | Type | Description |
| :- | :- | :- |
| `conversionId` | String | Unique identifier of the form submission |
| `formId` | String | Associated form ID |
| `submittedAt` | Integer | Submission timestamp (epoch milliseconds) |
| `values` | Array | List of submitted form field values, each with `objectTypeId`, `name` and `value` |
| `pageUrl` | String | URL of the page where the form was submitted |
Read-only CRM object containing NPS, CSAT, CES, and custom survey responses.
**Note:** Requires the `crm.objects.feedback_submissions.read` scope. Accounts without this scope will automatically skip this stream during discovery.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the record |
| `properties` | Object | Object containing the feedback submission's properties (dynamically discovered from your account) |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `createdAt` | String | Timestamp when the record was created |
| `updatedAt` | String | Timestamp when the record was last updated |
| `archived` | Boolean | Whether the record is archived |
Goal targets and tracking for sales and marketing performance.
This stream is currently disabled and will not appear in the stream selection list.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the goal |
| `properties.hs_goal_name` | String | Name of the goal |
| `properties.hs_target_amount` | String | Target value or amount |
| `properties.hs_start_datetime` | String | Start date and time of the goal period |
| `properties.hs_end_datetime` | String | End date and time of the goal period |
| `properties.hs_created_by_user_id` | String | User who created the goal |
| `properties.createdate` | String | When the record was created |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
| `archived` | Boolean | Whether the record is archived |
Lead records for sales prospecting.
**Note:** This stream is only available for accounts with HubSpot Pro, Enterprise, or Enterprise Plus. Enable via "Enable Leads Stream" in the source configuration.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the lead |
| `properties.hs_lead_name` | String | Lead name |
| `properties.hs_lead_status` | String | Lead status |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
| `archived` | Boolean | Whether the lead is archived |
Lead pipeline configurations and stages for sales workflows.
**Note:** This stream is only available for accounts with HubSpot Pro, Enterprise, or Enterprise Plus. Enable via "Enable Leads Stream" in the source configuration.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the pipeline. |
| `label` | String | Name of the pipeline. |
| `displayOrder` | Integer | Order in which the pipeline is displayed. |
| `archived` | Boolean | Whether the pipeline is archived. |
| `createdAt` | DateTime | Timestamp when the pipeline was created. |
| `updatedAt` | DateTime | Timestamp when the pipeline was last updated. |
| `stages` | Array | List of pipeline stages containing ID, label, displayOrder, metadata, archived status, writePermissions, and timestamps. |
Individual items within deals and quotes.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the line item |
| `properties.name` | String | Line item name |
| `properties.price` | String | Unit price |
| `properties.quantity` | String | Quantity ordered |
| `properties.hs_product_id` | String | Associated product ID |
| `properties.hs_recurring_billing_period` | String | Billing period |
| `properties.createdate` | String | Date when line item was created |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `properties.recurringbillingfrequency` | String | Recurring billing frequency |
| `createdAt` | String | Date when record was created |
| `updatedAt` | String | Last modification timestamp |
| `archived` | Boolean | Whether the line item is archived |
Marketing email campaigns and their performance statistics.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the marketing email |
| `name` | String | Email name |
| `subject` | String | Email subject line |
| `state` | String | Email state (draft, published, etc.) |
| `type` | String | Email type |
| `isPublished` | Boolean | Whether email is published |
| `publishedAt` | String | Publication timestamp |
| `publishDate` | String | Scheduled publish date |
| `campaign` | String | Associated campaign ID |
| `campaignName` | String | Campaign name |
| `from` | Object | Sender information (fromName, replyTo, customReplyTo) |
| `to` | Object | Recipient configuration (contactIds, contactLists, suppressGraymail) |
| `subscriptionDetails` | Object | Subscription information (subscriptionId, subscriptionName, officeLocationId) |
| `content` | String | Email content (JSON string) |
| `emailTemplateMode` | String | Template mode (e.g., DRAG\_AND\_DROP) |
| `isAb` | Boolean | Whether it's an A/B test |
| `isTransactional` | Boolean | Whether it's a transactional email |
| `language` | String | Email language |
| `archived` | Boolean | Whether the email is archived |
| `createdAt` | String | Creation timestamp |
| `updatedAt` | String | Last update timestamp |
| `createdById` | String | Creator user ID |
| `updatedById` | String | Last updater user ID |
| `publishedById` | String | Publisher user ID |
| `publishedByName` | String | Publisher name |
| `publishedByEmail` | String | Publisher email |
| `stats.counters` | Object | Email performance counters: `sent`, `delivered`, `open`, `click`, `bounce`, `hardbounced`, `softbounced`, `unsubscribed`, `spamreport`, `reply`, `dropped`, `pending`, `suppressed`, `contactslost`, `notsent`, `selected` |
| `stats.ratios` | Object | Performance ratios (openratio, clickratio, bounceratio, etc.) |
| `stats.deviceBreakdown` | Object | Opens/clicks by device type (computer, mobile, unknown) |
| `webversion.enabled` | Boolean | Whether web version is enabled |
| `webversion.url` | String | Web version URL |
| `webversion.domain` | String | Web version domain |
| `webversion.slug` | String | Web version URL slug |
Scheduled meetings and appointments.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the meeting |
| `properties.hs_meeting_title` | String | Meeting title |
| `properties.hs_meeting_body` | String | Meeting description |
| `properties.hs_meeting_start_time` | String | Start time |
| `properties.hs_meeting_end_time` | String | End time |
| `properties.hs_meeting_outcome` | String | Meeting outcome |
| `properties.hubspot_owner_id` | String | Meeting owner |
| `createdAt` | String | Creation timestamp |
| `updatedAt` | String | Last update timestamp |
Notes and annotations on records.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the note |
| `properties.hs_note_body` | String | Note content |
| `properties.hubspot_owner_id` | String | Note creator |
| `properties.hs_timestamp` | String | Creation timestamp |
| `properties.hs_attachment_ids` | String | Attached file IDs |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Custom properties and field definitions for all HubSpot objects. This stream merges all individual property streams into a single comprehensive properties table. The `hubspot_object` field indicates which object the property belongs to.
| Field | Type | Description |
| :- | :- | :- |
| `updatedAt` | String | Timestamp when the record was last updated |
| `createdAt` | String | Timestamp when the record was created |
| `name` | String | Internal name of the property |
| `label` | String | Display label of the property |
| `type` | String | Type classification of the record |
| `fieldType` | String | Field type used in forms and UI |
| `description` | String | Description of the property |
| `groupName` | String | Name of the property group |
| `options` | Array | List of options for the property |
| `displayOrder` | Integer | Order in which the property is displayed |
| `calculated` | Boolean | Whether the property is calculated |
| `externalOptions` | Boolean | Whether the property uses external options |
| `hasUniqueValue` | Boolean | Whether the property must have a unique value |
| `hidden` | Boolean | Whether the property is hidden |
| `hubspotDefined` | Boolean | Whether the property is defined by HubSpot |
| `modificationMetadata` | Object | Metadata about property modification permissions |
| `formField` | Boolean | Whether the property is used in forms |
| `hubspot_object` | String | HubSpot object type the property belongs to |
| `createdUserId` | String | ID of the user who created the property |
| `updatedUserId` | String | ID of the user who last updated the property |
| `referencedObjectType` | String | Object type referenced by this property (e.g. OWNER) |
| `calculationFormula` | String | Formula used for calculated properties |
| `archived` | Boolean | Whether the property is archived |
| `dataSensitivity` | String | Data sensitivity level of the property |
HubSpot user and owner information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the owner |
| `email` | String | Owner's email address |
| `firstName` | String | Owner's first name |
| `lastName` | String | Owner's last name |
| `userId` | String | Associated user ID |
| `createdAt` | String | Date when owner was created |
| `updatedAt` | String | Last modification timestamp |
| `archived` | Boolean | Whether the owner is archived |
Postal mail activities and engagements for direct mail campaigns.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the postal mail |
| `properties.hs_postal_mail_body` | String | Postal mail content |
| `properties.hs_timestamp` | String | When the postal mail was sent |
| `properties.hubspot_owner_id` | String | Owner |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Product catalog and inventory information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the product |
| `properties.name` | String | Product name |
| `properties.description` | String | Product description |
| `properties.price` | String | Product price |
| `properties.hs_sku` | String | Stock keeping unit |
| `properties.hs_cost_of_goods_sold` | String | Cost of goods sold |
| `properties.createdate` | String | Date when product was created |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `properties.hs_recurring_billing_period` | String | Recurring billing period |
| `createdAt` | String | Date when record was created |
| `updatedAt` | String | Last modification timestamp |
| `archived` | Boolean | Whether the product is archived |
Sales quotes and proposals.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the quote |
| `properties.hs_title` | String | Quote title |
| `properties.hs_quote_amount` | String | Total quote amount |
| `properties.hs_quote_number` | String | Quote number |
| `properties.hs_status` | String | Quote status |
| `properties.hs_expiration_date` | String | Quote expiration date |
| `properties.hs_terms` | String | Quote terms |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.hs_createdate` | String | Date when quote was created |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
| `archived` | Boolean | Whether the quote is archived |
HubSpot "Services" standard object records (objectTypeId 0-162).
**Note:** This stream must be activated in your HubSpot account and granted via token scope. If the necessary permissions are missing, the stream will be automatically skipped without breaking the extraction.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the service |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
| `archived` | Boolean | Whether the service is archived |
Task assignments and to-dos.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the task |
| `properties.hs_task_subject` | String | Task subject |
| `properties.hs_task_status` | String | Task status |
| `properties.hs_task_priority` | String | Priority level |
| `properties.hs_task_type` | String | Task type |
| `properties.hs_timestamp` | String | Creation timestamp |
| `properties.hubspot_owner_id` | String | Task owner |
| `properties.hs_task_body` | String | Task description |
| `properties.hs_task_due_date` | String | Due date |
| `createdAt` | String | Record creation timestamp |
| `updatedAt` | String | Record update timestamp |
Customer support tickets and service requests.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the ticket |
| `properties.subject` | String | Ticket subject |
| `properties.hs_ticket_priority` | String | Priority level |
| `properties.hs_pipeline` | String | Associated pipeline |
| `properties.hs_pipeline_stage` | String | Current stage |
| `properties.hubspot_owner_id` | String | Assigned owner |
| `properties.createdate` | String | Date when ticket was created |
| `properties.hs_lastmodifieddate` | String | Last modification timestamp |
| `associations` | Object | Related records (if enabled). Custom object keys are prefixed with `custom_`. |
| `propertiesWithHistory` | Object | Property change history (if enabled) |
| `createdAt` | String | Date when record was created |
| `updatedAt` | String | Last modification timestamp |
| `archived` | Boolean | Whether the ticket is archived |
Ticket pipeline configurations and stages for customer support workflows.
| Field | Type | Description |
| :- | :- | :- |
| `label` | String | Display name of the pipeline |
| `displayOrder` | Integer | Order in which the pipeline is displayed |
| `active` | Boolean | Whether the pipeline is active |
| `stages` | Array | Array of pipeline stages with their configurations |
| `objectType` | String | Type of object this pipeline applies to |
| `objectTypeId` | String | ID of the object type |
| `pipelineId` | String | Unique identifier for the pipeline |
| `createdAt` | String | Date when pipeline was created |
| `updatedAt` | String | Last modification timestamp |
| `default` | Boolean | Whether this is the default pipeline |
HubSpot portal user information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the user |
| `email` | String | User's email address |
| `roleIds` | Array | Array of role IDs assigned to the user |
| `primaryteamid` | String | ID of the user's primary team |
CRM Users object (HubSpot object type "users"). Distinct from the standard `Users` stream which only exposes basic settings API fields (id, email, roles). This stream reads the CRM Users object and exposes its full property set.
Requires the `crm.objects.users.read` scope on the HubSpot private app, which is sensitive and must be granted explicitly by a Super Admin. Accounts/tokens without it will skip extracting this stream.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the user |
| `properties.hs_given_name` | String | User's given (first) name |
| `properties.hs_family_name` | String | User's family (last) name |
| `properties.hs_email` | String | User's email address |
| `properties.hs_job_title` | String | User's job title |
| `properties.hubspot_owner_id` | String | Associated HubSpot owner ID |
| `properties.hs_deactivated` | Boolean | Whether the user is deactivated |
| `properties.hs_searchable_calculated_name` | String | Searchable calculated full name |
| `properties.hs_lastmodifieddate` | Timestamp | Last modification timestamp |
# Data Model
The following diagram illustrates the relationships between the core data streams in HubSpot. The arrows indicate the common links between different CRM entities.
```mermaid theme={null}
graph TD;
subgraph "Core CRM Objects"
Companies("Companies");
Contacts("Contacts");
Deals("Deals");
Tickets("Tickets");
Leads("Leads");
end
subgraph "Engagements & Activities"
Calls("Calls");
Meetings("Meetings");
Notes("Notes");
Emails("Emails");
Tasks("Tasks");
end
subgraph "Sales & Products"
Products("Products");
LineItems("Line Items");
Quotes("Quotes");
end
subgraph "Configuration & Owners"
Owners("Owners");
Users("Users");
end
Contacts -- "associated" --> Companies;
Deals -- "associated" --> Companies;
Deals -- "associated" --> Contacts;
Tickets -- "associated" --> Contacts;
Tickets -- "associated" --> Companies;
Leads -- "associated" --> Contacts;
Leads -- "associated" --> Companies;
Calls -- "associated" --> Contacts;
Meetings -- "associated" --> Contacts;
Notes -- "associated" --> Deals;
Emails -- "associated" --> Contacts;
Tasks -- "associated" --> Owners;
Deals -- "has" --> LineItems;
LineItems -- "references" --> Products;
Quotes -- "associated" --> Deals;
Companies -- "hubspot_owner_id" --> Owners;
Contacts -- "hubspot_owner_id" --> Owners;
Deals -- "hubspot_owner_id" --> Owners;
Tickets -- "hubspot_owner_id" --> Owners;
```
# Common data transformations
Many streams from HubSpot send all the relevant data in a single column, called `properties`. For you to remove them from this single array and transform each one in a new and individual column, use the following template:
```sql theme={null}
SELECT
-- list the regular columns you want to see in the final table
id,
createdat,
updatedat,
archived,
_nekt_sync_at,
-- list the fields that come inside 'properties' column and you want to have a specific column for
properties.name,
properties.price,
properties.description
FROM
-- example: "nekt.raw"."hubspot_products"
"nekt_layer"."table_name"
```
## Use Cases for Data Analysis
HubSpot data can be leveraged for comprehensive business intelligence and analytics. Here are some common use cases with SQL transformations:
### 1. Sales Performance Analytics
**Objective:** Track sales team performance, deal velocity, and revenue forecasting.
```sql Sales Performance Dashboard theme={null}
-- Sales Performance Dashboard
WITH
deal_metrics AS (
SELECT
d.properties.hubspot_owner_id AS hubspot_owner_id,
o.firstname || ' ' || o.lastname AS owner_name,
COUNT(*) AS total_deals,
SUM(
CAST(NULLIF(d.properties.amount, '') AS DECIMAL(15, 2))
) AS total_value,
AVG(
CAST(NULLIF(d.properties.amount, '') AS DECIMAL(15, 2))
) AS avg_deal_size,
COUNT(
CASE
WHEN d.properties.dealstage = 'closedwon' THEN 1
END
) AS won_deals,
COUNT(
CASE
WHEN d.properties.dealstage = 'closedlost' THEN 1
END
) AS lost_deals
FROM
"nekt_raw"."hubspot_deals" d
LEFT JOIN "nekt_raw"."hubspot_owners" o ON d.properties.hubspot_owner_id = o.id
WHERE
CAST(SUBSTRING(d.properties.createdate, 1, 10) AS DATE) >= DATE_ADD ('month', -6, CURRENT_DATE)
GROUP BY
d.properties.hubspot_owner_id,
o.firstname,
o.lastname
)
SELECT
owner_name,
total_deals,
total_value,
avg_deal_size,
won_deals,
lost_deals,
ROUND((won_deals * 100.0 / total_deals), 2) AS win_rate,
ROUND((total_value / NULLIF(total_deals, 0)), 2) AS avg_deal_value
FROM
deal_metrics
ORDER BY
total_value DESC
```
### 2. Customer Journey Analysis
**Objective:** Understand customer lifecycle stages and conversion rates.
```sql theme={null}
-- Customer Journey Funnel
WITH
contact_stages AS (
SELECT
properties.lifecyclestage,
COUNT(*) AS contact_count,
COUNT(
CASE
WHEN properties.lifecyclestage IN (
'lead',
'marketingqualifiedlead',
'salesqualifiedlead'
) THEN 1
END
) AS qualified_leads,
COUNT(
CASE
WHEN properties.lifecyclestage = 'opportunity' THEN 1
END
) AS opportunities,
COUNT(
CASE
WHEN properties.lifecyclestage = 'customer' THEN 1
END
) AS customers
FROM
"nekt_raw"."hubspot_contacts"
WHERE
createdat >= DATE_ADD ('month', -12, CURRENT_DATE)
GROUP BY
properties.lifecyclestage
),
funnel_metrics AS (
SELECT
'Total Contacts' AS stage,
SUM(contact_count) AS COUNT,
100.0 AS conversion_rate
FROM
contact_stages
UNION ALL
SELECT
'Qualified Leads' AS stage,
SUM(qualified_leads) AS COUNT,
ROUND(
(SUM(qualified_leads) * 100.0 / SUM(contact_count)),
2
) AS conversion_rate
FROM
contact_stages
UNION ALL
SELECT
'Opportunities' AS stage,
SUM(opportunities) AS COUNT,
ROUND(
(SUM(opportunities) * 100.0 / SUM(qualified_leads)),
2
) AS conversion_rate
FROM
contact_stages
UNION ALL
SELECT
'Customers' AS stage,
SUM(customers) AS COUNT,
ROUND((SUM(customers) * 100.0 / SUM(opportunities)), 2) AS conversion_rate
FROM
contact_stages
)
SELECT
*
FROM
funnel_metrics
ORDER BY
COUNT DESC
```
These use cases demonstrate how HubSpot data can be transformed into actionable business intelligence, enabling data-driven decision making across sales, marketing, and customer service operations.
## Implementation Notes
* **Stream Permissions:** During the discovery phase, the connector verifies access to dynamic streams (such as custom objects). If the provided Private App Token lacks the necessary scopes for an object, the connector will gracefully skip that stream rather than failing the entire sync. If you notice missing streams, check your token's scopes in HubSpot.
* **API Limits & Performance:** Extracting associations and property history involves an additional API call per record. The connector uses connection pooling to keep this efficient, but enabling associations for multiple object types simultaneously will still significantly increase sync duration. Enable these features only when necessary.
* **Unified Associations Config:** Associations are configured through a single `associations` array. Each entry targets one object type (e.g. `deals`, `contacts`) and lists the associated entities to retrieve as a comma-separated string (e.g. `contacts, companies, tickets`). Custom object IDs such as `2-12345` are supported in the association list.
* **Custom Object Association Keys:** To ensure compatibility with downstream systems, custom object association keys are sanitized. Keys that are custom object type IDs (e.g., `2-12345`) or fully qualified names containing portal IDs are normalized to a stable format prefixed with `custom_` (e.g., `custom_2_12345`). Standard association names (like `contacts` or `companies`) remain unchanged.
* **Stream Discovery and Permissions:** If the authenticated HubSpot account lacks the necessary scopes or subscription tier to access certain objects (e.g., Services, Leads), the connector will gracefully skip those streams during discovery and log a warning, rather than failing the entire setup.
* **API Limits & Large Property Lists:** HubSpot imposes a hard limit of 3,000 characters on `/search` API request bodies. For heavily customized objects (such as Contacts with hundreds of custom properties), the list of properties can exceed this limit. The connector dynamically evaluates the request size: if the limit would be exceeded, it falls back to a lean search strategy: first requesting only the replication keys via `/search`, then calling the `/batch/read` endpoint to hydrate the full property values in chunks of 100 records. This increases the number of API calls but guarantees all custom properties are extracted without failing due to HubSpot's character constraints.
* **Audit Logs limit**: The HubSpot API enforces a strict hard limit of 365 days for retrieving historical audit logs. To prevent extraction failures, if your source configuration has a Start Date older than 365 days, the connector will automatically adjust it to 364 days ago to safely fetch the maximum history allowed. Additionally, if the API rejects a payload for requesting data out of this limit bounds, the connector will skip the stream without failing the entire pipeline.
* **Search API and `object_coordinates` properties:** HubSpot's CRM Search API (which is used for **Incremental** syncs) does not support querying properties of type `object_coordinates` (such as `hs_origin_object_coordinates`). Due to this restriction, properties of this type are excluded from the extraction payload to prevent API errors during Incremental syncs. To extract these properties, you must perform an **Additional Full Sync** or set your Sync Type to **Full table**.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `Too many total leaf fields` | The destination table exceeded the data warehouse field limit because of the number of custom properties. | Enable **Properties as JSON string** in the source advanced settings and run a full sync of the affected streams. |
| A property added to the history does not appear on older records | Incremental extraction only brings new or modified records. | Run a full sync of the affected streams. |
| A custom object does not appear in the stream list | Custom objects require specific scopes on the authorization. | Grant `crm.objects.custom.read` and `crm.schemas.custom.read`, then add the streams again. |
| Extraction fails with a permission error | The authorization needs a Super Admin user, or a private app token carrying the required scopes. | Re-authorize with a Super Admin user, or regenerate the private app token with the missing scopes. |
### Reading properties from the JSON column
With **Properties as JSON string** enabled, the `properties` column holds every property as JSON. Read individual properties in a Query:
```sql theme={null}
SELECT
id,
JSON_VALUE(properties, '$.lifecyclestage') AS lifecycle_stage,
JSON_VALUE(properties, '$.my_custom_property') AS my_custom_property
FROM bronze.hubspot_contacts
```
## Skills for agents
HubSpot connector documentation as plain markdown, for use in AI agent contexts.
# Huggy as a data source
Source: https://docs.nekt.com/sources/huggy
Bring data from Huggy to Nekt.
Huggy is a customer service platform that helps businesses manage customer interactions across WhatsApp and other messaging channels. It provides tools for conversation management, automation, and analytics to improve customer support and engagement.
## 1. Add your Huggy access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Huggy option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **Access Token:** The token to authenticate against the API service. For detailed instructions, [check Huggy docs](https://spotlight.huggy.io/publications/autentique-a-api-da-huggy?utm_source=developers\&utm_medium=API_v3\&utm_campaign=SPOT_Developers\&utm_term=hyperlink\&utm_content=exemplos_praticos).
* **Company ID:** Your company identifier in the Huggy platform. It uses to be a 6-digit number.
* **Chat report variables:** List of variables to be passed to the chat export. Please separate them with comma, if more than one variable.
* **Lookback window:** How many days in the past should every extraction look at for fetching the reports.
* **Start Date:** The date of the first record you want to sync with. We will extract every record created or updated after this.
3. Click **Next**.
## 2. Select your Huggy streams
1. The next step is letting us know which streams you want to bring. A stream is an entity and you can select the whole entity (meaning all available attributes will come in each table) or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Huggy data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Huggy data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Huggy connector documentation as plain markdown, for use in AI agent contexts.
# Humanitix as a data source
Source: https://docs.nekt.com/sources/humanitrix
Bring ticketing and event data from Humanitix to Nekt.
Humanitix is a ticketing platform for events. This connector extracts events, orders, tickets, and organizers' tags from the Humanitix API into Nekt for analysis.
## Configuring Humanitix as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Humanitix** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Humanitix API key. Add your credentials:
* **API Key**: Humanitix API key. Found under `Account > Advanced > Public API key`.
* **Start Date**: The earliest record date to sync (ISO 8601 format). Records created or updated after this date will be extracted from the source.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync:
* `events`
* `orders`
* `tickets`
* `tags`
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables once the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once to make this process faster!
* **Sync Type**: you can choose between `INCREMENTAL` and `FULL_TABLE`.
* Incremental: every run fetches only new/updated records since the last replicated timestamp (supported for `events`, `orders`, and `tickets` via `updatedAt`).
* Full table: every run fetches the current state of the stream. Use this for `tags` (replication is not incremental).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Humanitix and their corresponding fields.
Master data about events and their attributes.
**Key Fields:**
* `_id`: Unique event identifier (primary key)
* `userId`, `organiserId`
* `currency`
* `name`, `description`, `sharingDescription`, `slug`, `url`
* `tagIds`
* `category`
* `classification.category`, `classification.subcategory`
* `artists[]`: `origin`, `name`, `externalId`, `spotifyId`
* `public`, `published`, `suspendSales`, `markedAsSoldOut`
* `startDate`, `endDate`, `timezone`, `totalCapacity`
* `ticketTypes[]`:
* `_id`, `name`, `price`, `quantity`
* `priceRange.enabled`, `priceRange.min`, `priceRange.max`
* `priceOptions.enabled`, `priceOptions.options[]` (`value`)
* `description`, `disabled`, `deleted`, `isDonation`
* `tickets[]`: `ticketTypeId`, `quantity`
* `pricing.minimumPrice`, `pricing.maximumPrice`
* `paymentOptions.refundSettings.refundPolicy`, `paymentOptions.refundSettings.customRefundPolicy`
* `publishedAt`
* `additionalQuestions[]`: `_id`, `inputType`, `question`, `required`, `description`, `perOrder`, `disabled`, `createdAt`, `updatedAt`
* `bannerImage.url`, `featureImage.url`, `socialImage.url`
* `eventLocation`: `type`, `venueName`, `address`, `latLng[]`, `instructions`, `placeId`, `onlineUrl`, `mapUrl`, `city`, `region`, `country`
* `dates[]`: `_id`, `startDate`, `endDate`, `scheduleId`, `disabled`, `deleted`
* `packagedTickets[]`: `_id`, `name`, `price`, `quantity`, `description`, `disabled`, `deleted`
* `packagedTickets[].tickets[]`: `ticketTypeId`, `quantity`
* `accessibility`:
* `contactName`, `contactNumber`, `travelInstructions`, `entryInstructions`, `afterEntryInstructions`
* `hazards`, `toiletLocation`, `disabledParking`
* `features.access`, `features.wheelchairAccessibility`, `features.audioDescription`, `features.telephoneTypewriter`, `features.volumeControlTelephone`, `features.assistiveListeningSystems`, `features.signLanguageInterpretation`, `features.accessiblePrint`, `features.closedCaptioning`, `features.openedCaptioning`, `features.brailleSymbol`
* `affiliateCode.code`
* `keywords[]`
* `location`
* `createdAt`, `updatedAt` (replication)
Orders placed by users for event ticket purchases.
**Key Fields:**
* `_id`: Unique order identifier (primary key)
* `eventId`: References `events._id`
* `userId`, `currency`, `eventDateId`
* `status`, `financialStatus`
* `firstName`, `lastName`, `organisation`
* `mobile`, `email`, `accessCode`
* `discounts.autoDiscount.discountAmount`
* `discounts.discountCode.code`, `discounts.discountCode.discountAmount`
* `businessPurpose`, `businessTaxId`, `businessName`
* `paymentType`, `paymentGateway`, `manualOrder`, `tipFees`
* `clientDonation`, `notes`, `organiserMailListOptIn`
* `incompleteAt`, `completedAt`, `waitlistOfferId`
* `isInternationalTransaction`
* `totals`: `subtotal`, `amexFee`, `zipFee`, `humanitixFee`, `bookingFee`, `passedOnFee`, `clientDonation`, `netClientDonation`, `donation`, `dgrDonation`, `giftCardCredit`, `credit`, `outstandingAmount`, `feesIncluded`, `bookingTaxes`, `passedOnTaxes`, `taxes`, `totalTaxes`, `discounts`, `refunds`, `netSales`, `grossSales`, `referralAmount`, `total`
* `purchaseTotals`: same shape as `totals`
* `additionalFields[]`: `questionId`, `value`, `details.street`, `details.suburb`, `details.postalCode`, `details.city`, `details.state`, `details.country`
* `salesChannel`, `location`
* `createdAt`, `updatedAt` (replication)
Individual tickets issued for event purchases.
**Key Fields:**
* `_id`: Unique ticket identifier (primary key)
* `eventId`: References `events._id`
* `orderId`: References `orders._id`
* `eventDateId`, `userId`, `currency`
* `status`
* `firstName`, `lastName`, `email`, `mobile`, `organisation`
* `ticketTypeName`, `ticketTypeId`
* `price`
* `accessCode`
* `checkedIn`, `checkedInAt`, `checkedInBy`
* `checkedOut`, `checkedOutAt`
* `refunded`, `refundedAt`
* `cancelled`, `cancelledAt`
* `transferred`, `transferredAt`
* `qrCode`
* `seatLabel`, `tableLabel`
* `additionalFields[]`: `questionId`, `value`, `details.street`, `details.suburb`, `details.postalCode`, `details.city`, `details.state`, `details.country`
* `salesChannel`, `location`
* `createdAt`, `updatedAt` (replication)
Tags from Humanitix used to categorize organizers/events.
**Key Fields:**
* `_id`: Unique tag identifier (primary key)
* `name`
* `colour`
* `userId`, `organiserId`
* `createdAt`, `updatedAt`
# Data Model
The following diagram illustrates the relationships between Humanitix entities. The arrows indicate the join keys that link the streams.
```mermaid theme={null}
graph TD;
Events("Events");
Orders("Orders");
Tickets("Tickets");
Events -- "events._id = orders.eventId" --> Orders;
Events -- "events._id = tickets.eventId" --> Tickets;
Orders -- "orders._id = tickets.orderId" --> Tickets;
```
# Use Cases for Data Analysis
This guide outlines common analysis patterns using Humanitix data.
### 1. Ticket sales by event (last 30 days)
Track how many tickets were sold per event and the total ticket price (using the `tickets` stream).
```sql theme={null}
-- Replace `nekt_raw.humanitix_events` / `nekt_raw.humanitix_tickets`
-- with your configured layer + table names.
SELECT
e.name AS event_name,
COUNT(*) AS tickets_sold,
SUM(t.price) AS total_ticket_price
FROM
nekt_raw.humanitix_tickets t
JOIN
nekt_raw.humanitix_events e
ON e._id = t.eventId
WHERE
CAST(t.updatedAt AS TIMESTAMP) >= current_timestamp - INTERVAL '30' DAY
GROUP BY
1
ORDER BY
total_ticket_price DESC
```
```sql theme={null}
-- Replace `nekt_raw.humanitix_events` / `nekt_raw.humanitix_tickets`
-- with your configured dataset + table names.
SELECT
e.name AS event_name,
COUNT(*) AS tickets_sold,
SUM(t.price) AS total_ticket_price
FROM
`nekt_raw.humanitix_tickets` t
JOIN
`nekt_raw.humanitix_events` e
ON e._id = t.eventId
WHERE
TIMESTAMP(t.updatedAt) >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY
1
ORDER BY
total_ticket_price DESC
```
## Implementation Notes
### Data quality considerations
* `events`, `orders`, and `tickets` support incremental sync using `updatedAt` as the replication key.
* `tags` does not provide an incremental replication key (`replication_key = None`), so `FULL_TABLE` is the appropriate sync type.
* Nested objects and arrays (for example `ticketTypes[]` and `additionalQuestions[]`) are available as structured columns in Nekt; inspect your table schema in the Catalog or use Explorer to validate how they are represented in SQL.
### Performance considerations
* `orders` and `tickets` are fetched per event (`/v1/events/{event_id}/...`). Syncing many events can increase extraction time.
* Use `start_date` to control how far back incremental history begins, and consider running incremental syncs regularly.
## Skills for agents
Humanitix connector documentation as plain markdown, for use in AI agent contexts.
# IBM DB2 as a data source
Source: https://docs.nekt.com/sources/ibm-db2
Bring data from IBM DB2 to Nekt.
IBM DB2 is a relational database management system developed by IBM, widely used in enterprise environments for transactional workloads and data warehousing. It is available on multiple platforms including Linux, Unix, Windows (LUW), z/OS (mainframe), and IBM i.
![]()
## Required pre work
In order to connect Nekt to a database, you have to do some pre work to ensure access is granted in a secure way. So, make sure to check [this guide](/sources/pre-work-db-connection) before proceeding.
## 1. Add account access
1. Once you have done the pre work defined in section 0, you can inform your database accesses. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the IBM DB2 option from the list of connectors.
2. Click **Next** and you'll be prompted to add your database access.
You must ensure your database accepts incoming connections from Nekt, and the database user has `SELECT` permissions in order to fetch data. The default DB2 port is **50000**.
3. Click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. We recommend selecting only the tables that are useful for you. You can add new tables at anytime after the source is created.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Skills for agents
IBM DB2 connector documentation as plain markdown, for use in AI agent contexts.
# iFood as a data source
Source: https://docs.nekt.com/sources/ifood
Bring data from iFood to your Lakehouse.
iFood is the largest food and grocery delivery marketplace in Latin America. Through the iFood Merchant API, partner stores can read their sales, financial settlements, payment anticipations, and customer reviews: the data behind how much each store sold, what iFood charged, when it paid out, and how customers rated the orders.
## Before you start
Access to the iFood Merchant API is granted to an **application** registered in the [iFood Developer Portal](https://developer.ifood.com.br), not to a store directly. Two things must be in place before this source can extract anything:
1. **The application must be homologated by iFood.** Homologation is a review carried out by iFood's integration team and is required before an application can request access to production stores. It is only available for Professional (CNPJ) accounts.
2. **Each store must approve the application.** Once homologated, the application requests access to a store by ID or CNPJ, and the store owner approves it in the iFood Partner Portal.
After both steps, the application's **Client ID** and **Client Secret** are what you provide to Nekt.
Do not enable the Order or Events modules on the application used with Nekt. iFood expects integrations subscribed to those modules to poll every 30 seconds, and a store whose integration stops polling can be marked offline and stop receiving orders. This source never uses them; order data is read from the Financial module instead.
## 1. Add your iFood access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the iFood option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **Client ID:** The Client ID of your application in the iFood Developer Portal. You'll find it in the credentials tab of the application.
* **Client Secret:** The Client Secret of the same application.
* **Start Date:** The date of the first record you want to sync. Sales, financial entries and reviews from this date onwards will be extracted.
* **Store IDs:** *(optional)* Restrict the extraction to specific iFood store IDs. Leave it empty to extract every store the application has access to.
* **Lookback days:** *(optional, default 7)* How many days before the last synced record to re-read on every run. Sales and financial entries keep changing after they are created (status changes, commissions, refunds, and renegotiations land later), so this window is what keeps records that were already extracted up to date.
3. Click **Next**.
## 2. Select your iFood streams
1. The next step is letting us know which streams you want to bring. A stream is an entity and you can select the whole entity (meaning all available attributes will come in each table) or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your iFood data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your iFood data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Good to know
* **Orders come through the `Sales` stream.** iFood's order module is built for point-of-sale integrations: it delivers orders through a queue of events that must be consumed and acknowledged, and keeps order details for only 7 days. Reading it would interfere with the store's own POS integration, so this source reads orders from the Financial module's Sales API, which can be queried by date range as many times as needed.
* **Order items are not available.** No iFood bulk-read endpoint exposes the item-by-item content of an order. The `Sales` stream carries each order with its aggregate values (bag, delivery fee, service fee), payments, benefits, billing entries, and status history, but not the individual products.
* **Anticipations only exist for some stores.** The `Anticipations` stream returns data only for stores with a payment anticipation plan (D+1 or D+7) through the iFood digital account. For every other store the table stays empty; the extraction is not affected.
* **Money fields follow the API.** iFood returns most monetary values as text in the Sales and Financial Events endpoints, so those columns land as strings and should be cast in your queries.
# Streams and Fields
Below you'll find all available data streams from iFood and their corresponding fields:
All stores the application has access to. Useful as the reference list of store IDs used by every other stream.
**Key Fields:**
* `id` - Unique identifier (UUID) of the store on iFood
* `name` - Public name of the store, as shown to customers
* `corporate_name` - Legal (corporate) name of the company that owns the store
Full profile of each store: registration data, address and enabled operations.
**Key Fields:**
* `id` - Unique identifier (UUID) of the store on iFood
* `name` - Public name of the store
* `corporate_name` - Legal name of the company that owns the store
* `description` - Free-text description registered on iFood
* `average_ticket` - Average order value of the store, as calculated by iFood
* `exclusive` - Whether the store sells exclusively through iFood
* `type` - Type of the store (RESTAURANT, STORE, GROUP)
* `status` - Registration status (AVAILABLE, UNAVAILABLE, DISABLED)
* `created_at` - When the store was created on iFood
**Address:**
* `address_country`, `address_state`, `address_city` - Location of the store
* `address_postal_code`, `address_district`, `address_street`, `address_number` - Full address
* `address_latitude`, `address_longitude` - Geographic coordinates
**Operations:**
* `operations` - Operations enabled for the store (delivery, indoor) and their sales channels, as a JSON object
Every sale made by each store, with the values, payments and status history behind it. This is the order-level stream.
**Key Fields:**
* `id` - Unique identifier (UUID) of the sale
* `short_id` - Short order number shown to the store and the customer
* `created_at` - When the sale was created (incremental replication key)
* `type` - Type of the sale
* `category` - Business category of the sale (for example FOOD or GROCERY)
* `sales_channel` - Channel the sale came through (for example IFOOD or POS)
* `current_status` - Current status of the sale (for example CONCLUDED or CANCELLED)
**Store:**
* `merchant_id`, `merchant_short_id`, `merchant_name`, `merchant_type` - Store that made the sale
* `merchant_timezone` - Timezone the sale timestamps refer to
* `merchant_documents` - Legal documents (CNPJ and others) of the store, as a JSON array
**Values:**
* `sale_gross_value_bag` - Gross value of the items in the order bag, before discounts
* `sale_gross_value_delivery_fee` - Delivery fee charged on the sale
* `sale_gross_value_service_fee` - Service fee charged on the sale
* `benefits_total_value` - Total value of all discounts and benefits applied
* `benefits` - Discounts and benefits with their target and sponsors, as a JSON array
**Delivery:**
* `delivery_type` - Type of delivery (for example DELIVERY or TAKEOUT)
* `delivery_information_provider` - Party that provided the delivery information
* `delivery_logistic_provider` - Logistics provider responsible for the delivery
* `delivery_product` - Delivery product used by iFood
* `delivery_code` - Internal code of the delivery arrangement
* `delivery_scheduling_type` - Whether the delivery was immediate or scheduled
* `delivery_gross_value`, `delivery_discount`, `delivery_net_value` - Delivery pricing
**Payments and billing:**
* `payments` - Payment methods used, with brand, wallet, liability and installments, as a JSON array
* `billing_summary_sale_balance` - Net balance of the sale for the store
* `billing_entries` - Commissions, fees and credits that compose the sale balance, as a JSON array
* `order_status_history` - Chronological history of the order status changes, as a JSON array
* `order_events` - Financial and operational events generated by the order, as a JSON array
The individual credits and debits registered against each store: the ledger behind every payout. Entries are recorded in the week they happen, not the week of the original sale, so a single order can produce entries across several settlement periods.
**Key Fields:**
* `merchant_id` - Store the entry belongs to
* `name` - Type of the financial event (for example ORDER\_COMMISSION or IFOOD\_SUBSIDY)
* `description` - Human readable description of the event
* `product` - iFood product the event relates to
* `trigger` - What triggered the event (for example SALE\_CONCLUDED)
* `date_time` - When the event was registered (incremental replication key)
* `competence` - Accounting month the event belongs to, in yyyy-MM format
* `has_transfer_impact` - Whether the entry changes the amount iFood transfers to the store; entries without impact are informational only
**Values:**
* `amount_value` - Monetary value of the entry, negative when it is a debit
* `billing_base_value` - Base value the fee was calculated on, when applicable
* `billing_fee_percentage` - Percentage applied over the base value, when applicable
* `settlement_expected_date` - Date the entry is expected to be settled with the store
**References:**
* `period_begin_date`, `period_end_date` - Calculation period the event falls into
* `reference_type` - Type of the record the event refers to, usually an order
* `reference_id` - Identifier of the record the event refers to; empty when the event is not tied to one
* `reference_date` - Date of the record the event refers to
* `receiver_business_id`, `receiver_business_type`, `receiver_business_document` - Business that receives the amount
* `payment_method`, `payment_brand`, `payment_liability` - Payment associated with the entry
The weekly settlement periods and the transfers that closed them: what iFood actually paid the store. One record per calculation period per store.
**Key Fields:**
* `merchant_id` - Store the settlement belongs to
* `start_date_calculation` - First day of the period the settlement was calculated over
* `end_date_calculation` - Last day of that period (incremental replication key)
* `balance` - Net amount received by the store across the queried window
* `query_begin_date`, `query_end_date` - Window that was queried to obtain the record
* `consolidated_merchants` - Stores consolidated into this settlement when the payout is grouped, as a JSON array
* `closing_items` - Individual closing items of the period (transfer, bank slip, renegotiation or receivable record) with amount, status, payment date and bank account details, as a JSON array
Payment anticipations (D+1 or D+7) contracted through the iFood digital account. Only returns data for stores that have an anticipation plan.
**Key Fields:**
* `merchant_id` - Store the anticipation belongs to
* `start_date_calculation` - First day of the period the anticipation was calculated over
* `end_date_calculation` - Last day of that period (incremental replication key)
* `balance` - Total amount anticipated to the store across the queried window
* `query_begin_date`, `query_end_date` - Window that was queried to obtain the record
* `consolidated_merchants` - Stores consolidated into this anticipation when the payout is grouped, as a JSON array
* `closing_items` - Individual anticipation items with the original amount, the fee percentage and value, the anticipated amount, the status and the original and anticipated payment dates, as a JSON array
Reviews customers left for each store, together with the replies the store posted.
**Key Fields:**
* `id` - Unique identifier (UUID) of the review
* `merchant_id` - Store the review refers to
* `score` - Rating the customer gave the order
* `previous_score` - Rating the customer had given before changing the review
* `has_score_changed` - Whether the customer changed the rating after submitting it
* `status` - Status of the review (CREATED, NOT\_REPLIED, REPLIED, PUBLISHED, INVALID, DISCARDED, UNKNOWN)
* `comment` - Free-text comment the customer wrote about the order
* `visibility` - Whether the review is public or private
* `version` - Version of the review record
* `survey_id` - Survey version the review answered
* `created_at` - When the review was created (incremental replication key)
**Order:**
* `order_id` - Order the review refers to
* `order_short_id` - Short order number the review refers to
* `order_created_at` - When the reviewed order was created
**Replies:**
* `replies` - Replies posted to the review, with their author and timestamp, as a JSON array
# Imobzi as a data source
Source: https://docs.nekt.com/sources/imobzi
Bring data from Imobzi to Nekt.
Imobzi is a real estate management platform designed for property professionals. It provides tools for managing properties, clients, sales processes, and marketing campaigns to help real estate agents and agencies streamline their operations and grow their business.
## Configuring Imobzi as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Imobzi option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your API credentials to connect to Imobzi. Check [Imobzi documentation](https://developer.imobzi.com/#section/Authentication/Creating-your-Secret-Key) to discover where you can find the API Key for the connection.
The following configuration is available:
* **API Key**: Your Imobzi API key for authentication and access to the platform's data.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Imobzi and their corresponding fields:
Stream for managing bank information and configurations.
| Field | Type | Description |
| :- | :- | :- |
| `db_id` | Integer | ID intern of the bank in the database. Required for create |
| `code` | String | Code of the bank. Required for create |
| `name` | String | Bank name. Required for create |
| `logo_url` | String | URL of the bank logo |
| `id` | Integer | ID intern or alternative ID of the bank in the database |
Stream for managing contact bank account information.
| Field | Type | Description |
| :- | :- | :- |
| `contact_id` | String | Contact ID from parent stream |
| `bank_id` | String | Unique identifier of the bank |
| `action` | String | Action to be performed, such as create, update, or delete |
| `id` | String | Unique identifier of the record |
| `bank_name` | String | Name of the bank |
| `organization_id` | String | Unique identifier of the organization associated with the bank account |
| `person_id` | String | Unique identifier of the person associated with the bank account |
| `account` | String | Bank account number |
| `agency` | String | Bank branch (agency) number |
| `account_type` | String | Type of the bank account. Can be 'checking\_account' or 'savings\_account' |
| `notes` | String | Additional notes or remarks related to the bank account |
Stream for managing real estate contracts and their details.
| Field | Type | Description |
| :- | :- | :- |
| `code` | String | Contract code |
| `contract_type` | String | Contract type |
| `key` | String | Contract key |
| `db_id` | Integer | ID intern of the contract in database |
| `value` | Number | Contract value |
| `status` | String | Contract status. Enum: active, ended, suspended, in\_review, in\_progress, traveling |
| `start_at` | String | Contract start at date, in format ISO |
| `contacts` | Array | Contact list related to the contract |
| `source` | String | Source of the contract |
| `in_progress` | Boolean | Indicates if the contract is in progress |
| `insurances` | Array | Insurances related to the contract |
| `invoice_items_value` | Number | Value of invoice items related to the contract |
Stream for managing credit and financing analysis processes.
| Field | Type | Description |
| :- | :- | :- |
| `db_id` | Integer | ID intern of the analysis in database |
| `status` | String | Status of the analysis |
| `type` | String | Type of analysis (e.g., 'fc\_analise') |
| `lease_value` | Number | Lease value |
| `condominium_value` | Number | Condominium value |
| `iptu_value` | Number | IPTU value |
| `total_value` | Number | Total value |
| `suitors` | Array | List of suitors related to the analysis |
| `lease` | Object | Lease information related to the analysis |
Stream for managing deal filtering configurations and criteria.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the deal filter |
| `name` | String | Name of the deal filter |
| `description` | String | Description of the filter criteria |
| `created_at` | String | Filter creation date |
| `updated_at` | String | Filter last update date |
| `active` | Boolean | Indicates if the filter is active |
Stream for managing deal rotation processes and assignments.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the deal rotation |
| `deal_id` | Integer | Associated deal identifier |
| `from_user_id` | Integer | User ID from whom the deal is being rotated |
| `to_user_id` | Integer | User ID to whom the deal is being rotated |
| `rotation_date` | String | Date when the rotation occurred |
| `reason` | String | Reason for the deal rotation |
| `status` | String | Status of the rotation |
Stream for managing document information and metadata.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the document |
| `name` | String | Document name |
| `type` | String | Document type |
| `url` | String | URL to access the document |
| `created_at` | String | Document creation date |
| `updated_at` | String | Document last update date |
| `size` | Integer | Document size in bytes |
| `mime_type` | String | MIME type of the document |
Stream for managing financial account information and configurations.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the financial account |
| `name` | String | Name of the financial account |
| `type` | String | Type of financial account |
| `balance` | Number | Current account balance |
| `currency` | String | Account currency |
| `created_at` | String | Account creation date |
| `updated_at` | String | Account last update date |
| `active` | Boolean | Indicates if the account is active |
Stream for managing contact tags and classifications.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the contact tag |
| `name` | String | Name of the tag |
| `color` | String | Color associated with the tag |
| `description` | String | Description of the tag |
| `created_at` | String | Tag creation date |
Stream containing detailed information about deals.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the deal |
| `title` | String | Deal title |
| `value` | Number | Deal value |
| `status` | String | Current deal status |
| `created_at` | String | Deal creation date |
| `updated_at` | String | Deal last update date |
| `expected_close_date` | String | Expected closing date |
Stream for reporting on completed deals.
| Field | Type | Description |
| :- | :- | :- |
| `deal_id` | Integer | Unique identifier for the completed deal |
| `deal_title` | String | Title of the completed deal |
| `completion_date` | String | Date when the deal was completed |
| `final_value` | Number | Final value of the completed deal |
| `commission` | Number | Commission earned from the deal |
Stream containing a list of all deals.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the deal |
| `title` | String | Deal title |
| `status` | String | Current deal status |
| `value` | Number | Deal value |
| `created_at` | String | Deal creation date |
| `db_id` | String | Internal identifier of the deal in the database |
| `send_property_match` | Boolean | Indicates if property match is enabled for the deal |
| `send_property_match_by_whatsapp` | Boolean | Indicates if property match via WhatsApp is enabled for the deal |
| `contact` | Object | Contact information related to the deal |
Stream for managing reasons why deals are lost.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the lost reason |
| `name` | String | Name of the lost reason |
| `description` | String | Description of the lost reason |
| `active` | Boolean | Indicates if the reason is active |
Stream for managing lease agreements and information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the lease |
| `property_id` | Integer | Associated property identifier |
| `tenant_id` | Integer | Associated tenant identifier |
| `start_date` | String | Lease start date |
| `end_date` | String | Lease end date |
| `monthly_rent` | Number | Monthly rent amount |
Stream for managing media source information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the media source |
| `name` | String | Name of the media source |
| `type` | String | Type of media source |
| `url` | String | URL of the media source |
| `active` | Boolean | Indicates if the media source is active |
Stream for managing pipeline group configurations.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the pipeline group |
| `name` | String | Name of the pipeline group |
| `description` | String | Description of the pipeline group |
| `active` | Boolean | Indicates if the pipeline group is active |
Stream for managing sales pipeline information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the pipeline |
| `name` | String | Name of the pipeline |
| `stages` | Array | Pipeline stages configuration |
| `active` | Boolean | Indicates if the pipeline is active |
Stream containing contact information and details.
| Field | Type | Description |
| :- | :- | :- |
| `contact_id` | String | Unique identifier for the contact |
| `name` | String | Contact name |
| `email` | String | Contact email address |
| `phone` | String | Contact phone number |
| `contact_type` | String | Type of contact |
| `created_at` | String | Contact creation date |
Stream containing property listings and information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the property |
| `title` | String | Property title |
| `address` | String | Property address |
| `price` | Number | Property price |
| `type` | String | Property type |
| `status` | String | Property status |
Stream for managing financial transaction records.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the transaction |
| `amount` | Number | Transaction amount |
| `type` | String | Transaction type |
| `date` | String | Transaction date |
| `description` | String | Transaction description |
| `account_id` | Integer | Associated account identifier |
Stream for managing property reservation information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the reservation |
| `property_id` | Integer | Associated property identifier |
| `contact_id` | Integer | Associated contact identifier |
| `reservation_date` | String | Date of reservation |
| `status` | String | Reservation status |
| `db_id` | String | Internal identifier of the property reserve in the database |
Stream containing detailed property information.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the property |
| `title` | String | Property title |
| `description` | String | Property description |
| `specifications` | Object | Detailed property specifications |
| `amenities` | Array | List of property amenities |
| `location` | Object | Property location details |
| `building_code` | String | Code of the building (condominium) the property belongs to. |
| `building_id` | Integer | Internal identifier of the building the property belongs to. |
| `cib_code` | String | CIB code (Cadastro Imobiliário Brasileiro) of the property. |
| `database` | String | Identifier of the Imobzi account (namespace) that owns the record. |
| `deal_stage` | String | Stage of the deal currently associated with the property, when any. |
| `disabled_network_proposal` | Boolean | Whether proposals from the Imobzi partner network are disabled for this property. |
| `fiscal_registration_number` | String | Municipal tax registration number of the property (inscrição imobiliária / IPTU). |
| `key_id` | String | Identifier of the physical key of the property in the key control. |
| `latitude` | Number | Latitude of the property's location. |
| `longitude` | Number | Longitude of the property's location. |
| `member_register_code` | String | Registration code of the network member that listed the property. |
| `photos_count` | Integer | Number of photos attached to the property. |
| `platform_listing_url` | String | URL of the property's listing on the Imobzi platform. |
| `show_address` | Boolean | Whether the full address is shown publicly on the website. |
| `site_publish_rental_price` | Boolean | Whether the rental price is published on the website. |
| `site_publish_sale_price` | Boolean | Whether the sale price is published on the website. |
| `vacation_min_value` | Number | Minimum nightly value for vacation rental. |
| `vacation_max_value` | Number | Maximum nightly value for vacation rental. |
| `nearby` | Array | Names of nearby places or landmarks registered for the property. |
| `cover_photo` | Object | Photo used as the property's public cover. |
| `cover_photo_private` | Object | Photo used as the property's cover for internal (private) use. |
| `listing_brokers` | Array | Brokers responsible for listing the property. |
| `quality` | Object | Listing quality assessment computed by Imobzi. |
| `photos` | Object | Gallery of the property. |
| `owners` | Array | Owners of the property and their ownership share. |
| `links` | Array | People linked to the property, such as the listing broker. |
| `fields` | String | JSON string with the account's custom fields for the property, grouped by field group. Keys are defined by each customer. |
| `features` | String | JSON string with the property's features grouped by category (e.g. common areas). Categories are defined by each customer. |
| `network` | String | JSON string with the Imobzi partner-network sharing data of the property, when shared. |
| `proposal` | String | JSON string with the proposal currently attached to the property, when any. |
| `property_calendar` | String | JSON string with the vacation-rental calendar entries of the property. |
| `building_parameters` | String | JSON string with the building parameters of the property, when returned by the API. |
# Data Model
The following diagram illustrates the relationships between the core data streams in Imobzi:
```mermaid theme={null}
graph TD;
subgraph "Contact Management"
ContactsList("Contacts List");
ContactBank("Contact Bank");
ContactsTags("Contacts Tags");
end
subgraph "Property Management"
PropertiesList("Properties List");
PropertyDetails("Property Details");
PropertyReserves("Property Reserves");
Leases("Leases");
end
subgraph "Deal Management"
DealsList("Deals List");
DealsDetails("Deals Details");
DealsRotations("Deals Rotations");
DealFilters("Deal Filters");
DealsLostReasons("Deals Lost Reasons");
ReportsDealsDone("Reports Deals Done");
end
subgraph "Financial Management"
FinancialAccounts("Financial Accounts");
FinancialTransactions("Financial Transactions");
Contracts("Contracts");
CreditFinancingAnalysis("Credit Financing Analysis");
end
subgraph "System Configuration"
Banks("Banks");
MediaSources("Media Sources");
PipelineGroups("Pipeline Groups");
Pipelines("Pipelines");
Documents("Documents");
end
ContactBank -- "contact_id" --> ContactsList;
PropertyReserves -- "property_id" --> PropertiesList;
PropertyReserves -- "contact_id" --> ContactsList;
Leases -- "property_id" --> PropertiesList;
DealsDetails -- "id" --> DealsList;
DealsRotations -- "deal_id" --> DealsList;
FinancialTransactions -- "account_id" --> FinancialAccounts;
ContactBank -- "bank_id" --> Banks;
```
## Implementation Notes
### Data Synchronization
* Most streams support full table synchronization
* Some streams like financial transactions may benefit from incremental sync to capture new transactions efficiently
* Contact bank data is retrieved as a child stream of contacts, ensuring data consistency
* The connector safely serializes fractional numbers (decimals) inside nested JSON-string fields for streams like Property Details, preventing type conversion errors and extraction failures
### API Considerations
* The connector handles pagination automatically for large datasets
* Rate limiting is managed internally to ensure stable data extraction
* Parent-child relationships (like contacts and contact bank data) are properly maintained during sync
## Skills for agents
Imobzi connector documentation as plain markdown, for use in AI agent contexts.
# Imoview as a data source
Source: https://docs.nekt.com/sources/imoview
Bring data from Imoview to Nekt.
Imoview is a property management and real estate CRM platform that supports rental and sale workflows, tenant and guarantor contracts, inspections, condominiums, and client (user) data. Data is extracted via the Imoview API using an API key (chave) and optional email/senha for app endpoints that require user context.
![]()
## Configuring Imoview as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Imoview** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your Imoview API credentials so Nekt can access your data.
The following configurations are available:
* **API key**: API key (chave) for the Imoview API. The access key is provided by **UNIVERSAL SOFTWARE**.
* **Email**: Imoview user email. When provided together with **Password**, Nekt uses it to obtain `codigoacesso` and `codigoUsuario` via the app validation endpoint. This is required for streams that use App\_ endpoints (e.g. **client\_companies**, **rental\_contracts**).
* **Password**: Imoview user password. Used with **Email** to authenticate and retrieve app credentials for App\_ endpoints.
* **Start date**: Records created or updated after the start date will be extracted from the source. Used for the first full sync and for incremental syncs when supported by the stream.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data), and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: Every time the extraction happens, we'll get only the new or updated data. Supported for **clients**, **rental\_contracts**, and **condominiums** where the API supports replication by date.
* **Full table**: Every time the extraction happens, we'll get the current state of the data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Clients | `clients` | Stream for clients (locador, locatário, proprietário, comprador, fiador). |
| Rental Contracts | `rental_contracts` | Stream for rental contracts. |
| Client Companies | `client_companies` | Stream for companies from Imoview CRM. |
| Condominiums | `condominiums` | Stream for available condominiums. |
| Properties Rent | `properties_rent` | Stream for available rental properties (finalidade=1). |
| Properties Sale | `properties_sale` | Stream for available properties for sale (finalidade=2). |
## Fields by stream
Below you'll find all available data streams from Imoview and their corresponding fields:
Stream for clients (locador, locatário, proprietário, comprador, fiador). Source: RetornarTipo3.
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | Integer | Código do cliente |
| `nome` | String | Nome do cliente |
| `cpfoucnpj` | String | CPF ou CNPJ do cliente |
| `telefones` | String | Telefones do cliente |
| `email` | String | E-mail do cliente |
| `endereco` | String | Endereço do cliente |
| `locatario` | Boolean | Indica se o cliente é locatário |
| `comprador` | Boolean | Indica se o cliente é comprador |
| `locador` | Boolean | Indica se o cliente é locador |
| `vendedor` | Boolean | Indica se o cliente é vendedor |
| `fiador` | Boolean | Indica se o cliente é fiador |
| `parceiro` | Boolean | Indica se o cliente é parceiro |
| `quantidadeimoveislocador` | Integer | Quantidade de imóveis do locador |
| `quantidadeimoveisvendedor` | Integer | Quantidade de imóveis do vendedor |
| `quantidadecontratoslocatario` | Integer | Quantidade de contratos do locatário |
| `quantidadeboletosativoslocatario` | Integer | Quantidade de boletos ativos do locatário |
| `senhacadastrada` | Boolean | Indica se o cliente possui senha cadastrada |
| `emailcadastrado` | Boolean | Indica se o cliente possui e-mail cadastrado |
| `temvistoria` | Boolean | Indica se o cliente possui vistoria |
| `temesteira` | Boolean | Indica se o cliente possui esteira |
| `naoexigircpf` | Boolean | Indica se não exige CPF |
| `anosdisponiveis` | Array | Anos disponíveis para o cliente |
| `diasminimoavisodesocupacao` | Integer | Dias mínimos de aviso de desocupação |
| `temcashgo` | Boolean | Indica se o cliente possui CashGo |
| `naoexibiravisodesocupacao` | Boolean | Indica se não exibe aviso de desocupação |
| `whatsappbotiniciarConversa` | Integer | Configuração do bot WhatsApp para iniciar conversa |
| `datainclusao` | Date | Data de inclusão do cliente |
| `dataultimaalteracao` | DateTime | Data da última alteração do cliente |
Stream for rental contracts. Source: Locacao/RetornarContratos.
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | String | Código do contrato |
| `codigoauxiliar` | String | Código auxiliar |
| `unidadecodigo` | String | Código da unidade |
| `unidade` | String | Nome da unidade |
| `locatariocodigo` | String | Código do locatário |
| `locatarionome` | String | Nome do locatário |
| `locatariotelefone1` | String | Telefone 1 do locatário |
| `locatariotelefone2` | String | Telefone 2 do locatário |
| `locatarioemail` | String | E-mail do locatário |
| `locatariocpf` | String | CPF do locatário |
| `situacao` | String | Situação do contrato |
| `status` | String | Status do contrato |
| `motivostatus` | String | Motivo do status |
| `valoraluguel` | Number | Valor do aluguel |
| `taxaadministracao` | Number | Taxa de administração |
| `tipotaxaadministracao` | String | Tipo da taxa de administração |
| `taxaintermediacao` | Number | Taxa de intermediação |
| `tipotaxaintermediacao` | String | Tipo da taxa de intermediação |
| `datainclusao` | DateTime | Data de inclusão |
| `dataultimaalteracao` | DateTime | Data da última alteração |
| `dataativacao` | DateTime | Data de ativação |
| `datainicio` | Date | Data de início |
| `datafim` | Date | Data de fim |
| `dataproximoreajuste` | Date | Data do próximo reajuste |
| `dataproximoaluguel` | Date | Data do próximo aluguel |
| `dataproximorepasse` | Date | Data do próximo repasse |
| `diavencimento` | Integer | Dia de vencimento |
| `diarepasse` | Integer | Dia de repasse |
| `diavencimentoaluguel` | Integer | Dia de vencimento do aluguel |
| `diavencimentorepasse` | Integer | Dia de vencimento do repasse |
| `dataavisodesocupacao` | Date | Data de aviso de desocupação |
| `dataprevisaorescisao` | Date | Data de previsão de rescisão |
| `datarescisao` | Date | Data de rescisão |
| `motivorescisao` | String | Motivo da rescisão |
| `complementomotivo` | String | Complemento do motivo |
| `formacobranca` | String | Forma de cobrança |
| `formarecebimento` | String | Forma de recebimento |
| `comofazerrepasse` | String | Como fazer repasse |
| `destinacao` | String | Destinação |
| `ramoatividade` | String | Ramo de atividade |
| `indicereajuste` | String | Índice de reajuste |
| `garantido` | Boolean | Indica se é garantido |
| `tipogarantido` | String | Tipo de garantia |
| `quantosmesesgarantido` | Integer | Quantos meses garantido |
| `responsavel` | String | Responsável |
| `advogadoresponsavel` | String | Advogado responsável |
| `resumo` | String | Resumo |
| `contratosassinado` | Array | Contratos assinados |
| `vistoriasassinado` | Array | Vistorias assinadas |
| `imoveis` | Array | Array de imóveis |
| `seguroincendio` | Object | Seguro incêndio |
| `garantialocaticia` | Object | Garantia locatícia |
Stream for companies from Imoview CRM. Source: Cliente/App\_RetornarEmpresas. Requires email and senha for app credentials.
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | Integer | Código da empresa |
| `nome` | String | Nome da empresa |
| `telefone` | String | Telefone |
| `email1` | String | E-mail principal |
Stream for available condominiums. Source: Imovel/RetornarCondominiosDisponiveis (POST).
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | String | Código do condomínio |
| `nome` | String | Nome do condomínio |
| `codigodestinacao` | String | Código da destinação |
| `destinacao` | String | Destinação |
| `tipo` | String | Tipo |
| `formacobranca` | String | Forma de cobrança |
| `construtora` | String | Construtora |
| `anoconstrucao` | Integer | Ano de construção |
| `arealote` | Number | Área do lote |
| `datahoracadastro` | DateTime | Data e hora de cadastro |
| `datahoraultimaalteracao` | DateTime | Data e hora da última alteração |
| `descricao` | String | Descrição |
| `codigobairro` | String | Código do bairro |
| `bairro` | String | Bairro |
| `codigocidade` | String | Código da cidade |
| `cidade` | String | Cidade |
| `estado` | String | Estado |
| `codigoregiao` | String | Código da região |
| `regiao` | String | Região |
| `codigosubregiao` | String | Código da sub-região |
| `subregiao` | String | Sub-região |
| `cep` | String | CEP |
| `endereco` | String | Endereço |
| `numero` | String | Número |
| `complemento` | String | Complemento |
| `pontoreferencia` | String | Ponto de referência |
| `melhoracesso` | String | Melhor acesso |
| `latitude` | Number | Latitude |
| `longitude` | Number | Longitude |
| `numerovagavisitante` | Integer | Número de vagas para visitante |
| `tipovagavisitante` | String | Tipo de vaga para visitante |
| `numeroelevador` | Integer | Número de elevadores |
| `numerotorres` | Integer | Número de torres |
| `numeroandares` | Integer | Número de andares |
| `unidadesporandar` | Integer | Unidades por andar |
| `totalunidades` | Integer | Total de unidades |
| `cartorio` | String | Cartório |
| `matriculacartorio` | String | Matrícula do cartório |
| `livrocartorio` | String | Livro do cartório |
| `folhacartorio` | String | Folha do cartório |
| `valorcondominio` | Number | Valor do condomínio |
| `valorfundoreserva` | Number | Valor do fundo de reserva |
| `valoraluguel` | Number | Valor do aluguel |
| `valoralugado` | Number | Valor alugado |
| `valorvenda` | Number | Valor de venda |
| `valorvendido` | Number | Valor vendido |
| `areainterna` | Number | Área interna |
| `areaexterna` | Number | Área externa |
| `numeroquartos` | Integer | Número de quartos |
| `numerobanhos` | Integer | Número de banheiros |
| `numerosuites` | Integer | Número de suítes |
| `numerovagas` | Integer | Número de vagas |
| `primeiroquartilaluguel` | Number | Primeiro quartil de aluguel |
| `medianaquartilaluguel` | Number | Mediana do quartil de aluguel |
| `terceiroquartilaluguel` | Number | Terceiro quartil de aluguel |
| `primeiroquartilvenda` | Number | Primeiro quartil de venda |
| `medianaquartilvenda` | Number | Mediana do quartil de venda |
| `terceiroquartilvenda` | Number | Terceiro quartil de venda |
| `aguaindividual` | Boolean | Água individual |
| `alarme` | Boolean | Alarme |
| `piscina` | Boolean | Piscina |
| `academia` | Boolean | Academia |
| `churrasqueira` | Boolean | Churrasqueira |
| `portaria24horas` | Boolean | Portaria 24 horas |
| `urlfotoprincipal` | String | URL da foto principal |
| `urlvideo` | String | URL do vídeo |
| `fotos` | Array | Fotos |
| `fotos360` | Array | Fotos 360 |
| `extras` | Object | Extras |
| `extras2` | Object | Extras 2 |
| `unidadesaluguel` | Array | Unidades de aluguel |
| `unidadesvenda` | Array | Unidades de venda |
Stream for available rental properties (finalidade=1). Source: Imovel/RetornarImoveisDisponiveis (POST).
| Field | Type | Description |
| :- | :- | :- |
| `codigo` | String | Código do imóvel |
| `publicado` | Boolean | Imóvel publicado (aparece na listagem pública da imobiliária); false para imóveis disponíveis no sistema que não estão publicados |
| `codigomae` | String | Código mãe |
| `emdesocupacao` | Boolean | Em desocupação |
| `empreendimento` | String | Empreendimento |
| `empreendimentofilho` | String | Empreendimento filho |
| `nomeempreendimento` | String | Nome do empreendimento |
| `siteempreendimento` | String | Site do empreendimento |
| `codigocondominio` | String | Código do condomínio |
| `nomecondominio` | String | Nome do condomínio |
| `titulo` | String | Título |
| `metadescription` | String | Meta descrição |
| `codigoauxiliar` | String | Código auxiliar |
| `unidade` | String | Unidade |
| `nomeunidade` | String | Nome da unidade |
| `telefoneunidade` | String | Telefone da unidade |
| `emaileunidade` | String | E-mail da unidade |
| `finalidade` | String | Finalidade |
| `codigodestinacao` | String | Código da destinação |
| `destinacao` | String | Destinação |
| `codigotipo` | String | Código do tipo |
| `tipo` | String | Tipo |
| `codigotipo2` | String | Código do tipo 2 |
| `tipo2` | String | Tipo 2 |
| `situacao` | String | Situação |
| `valor` | Number | Valor |
| `valoranterior` | Number | Valor anterior |
| `valorm2` | Number | Valor por m² |
| `valorcondominio` | Number | Valor do condomínio |
| `valoriptu` | Number | Valor do IPTU |
| `codigobairro` | String | Código do bairro |
| `bairro` | String | Bairro |
| `codigocidade` | String | Código da cidade |
| `cidade` | String | Cidade |
| `estado` | String | Estado |
| `codigoregiao` | String | Código da região |
| `regiao` | String | Região |
| `cep` | String | CEP |
| `endereco` | String | Endereço |
| `numero` | String | Número |
| `complemento` | String | Complemento |
| `numeroquartos` | Integer | Número de quartos |
| `numerobanhos` | Integer | Número de banheiros |
| `numerovagas` | Integer | Número de vagas |
| `numerosuites` | Integer | Número de suítes |
| `areaprincipal` | Number | Área principal |
| `areainterna` | Number | Área interna |
| `areaexterna` | Number | Área externa |
| `latitude` | Number | Latitude |
| `longitude` | Number | Longitude |
| `descricao` | String | Descrição |
| `aceitafinanciamento` | Boolean | Aceita financiamento |
| `aceitapermuta` | Boolean | Aceita permuta |
| `datahoracadastro` | DateTime | Data e hora de cadastro |
| `datahoraultimaalteracao` | DateTime | Data e hora da última alteração |
| `datahoraultimavalidacao` | DateTime | Data e hora da última validação |
| `urlfotoprincipal` | String | URL da foto principal |
| `urlpublica` | String | URL pública |
| `anexos` | Array | Anexos |
| `proprietarios` | Array | Proprietários |
| `fotos` | Array | Fotos |
Stream for available properties for sale (finalidade=2). Same endpoint as Properties Rent with finalidade=2.
**Key Fields:** Same structure as Properties Rent (codigo, condomínio, título, tipo, situação, valor, endereço, quartos, áreas, datas, fotos, etc.).
# Data Model
The following diagram illustrates the relationships between the data streams in Imoview. **Clients** is the main entity; **rental\_contracts** are filtered by client role (locatário). **Client companies**, **condominiums**, **properties\_rent**, and **properties\_sale** are independent streams.
```mermaid theme={null}
graph TD;
Clients("Clients");
RentalContracts("Rental Contracts");
ClientCompanies("Client Companies");
Condominiums("Condominiums");
PropertiesRent("Properties Rent");
PropertiesSale("Properties Sale");
RentalContracts -- "codigoCliente / locatário" --> Clients;
```
# Implementation Notes
### Authentication
* For **client\_companies** and **rental\_contracts**, the tap calls the app validation endpoint with **email** and **senha** (MD5) to obtain `codigoacesso` and `codigoUsuario`, which are then sent in headers or params as required by the API.
### Replication
* **Clients** supports incremental sync via `dataultimaalteracao` (filter `dataHoraInicialUltimaAlteracao`).
* **Rental contracts** and **condominiums** support replication by their respective date fields where the API allows.
### Pagination
* Streams use page-based pagination (`numeroPagina` / `numeroRegistros`). Page size is 100 for most streams, except for properties which use a page size of 20 (as the API returns an empty list above 20).
### Properties Listing
* The Imoview API filters property availability based on the `User-Agent` header (an undocumented behavior). The connector makes use of this to fetch the full catalog of available properties and performs a secondary pass to flag which ones are actually published (`publicado`).
### Resilience
* The connector implements retry logic for transient errors and unauthorized responses.
* App credentials are automatically refreshed when needed with a 30-minute TTL.
* Record cleansing is applied to ensure data consistency with day-first date format.
## Skills for agents
Imoview connector documentation as plain markdown, for use in AI agent contexts.
# Inhire as a data source
Source: https://docs.nekt.com/sources/inhire
Bring data from Inhire to Nekt.
Inhire is a recruitment and talent acquisition platform that helps businesses find, attract, and hire qualified candidates. It provides tools for job posting, candidate screening, interview management, and recruitment analytics to streamline the hiring process.
![]()
## Configuring Inhire as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Inhire** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Inhire account credentials for this connection.
The following configurations are available:
* **Email**: The email associated with your Inhire account.
* **Password**: The password associated with your Inhire account. This value is stored securely and is never displayed after saving.
* **Tenant ID**: The ID of the subdomain used in your Inhire URL. For example, if your URL is `https://my_app.inhire.app`, your tenant ID is `my_app`.
* **Filter Job Talent Status**: (Optional) A list of job talent statuses to filter for. When this is set, the `JobTalentScorecard` stream will only be extracted for talents matching these statuses. Values are compared against the candidate's current stage name and must match exactly. At the end of each run, the run logs summarize how many candidates were skipped per stage and warn you if any configured value matched no candidate, so no scorecards are silently dropped. This can significantly speed up extraction by skipping irrelevant scorecards (e.g., for rejected or hired candidates).
Please ensure you're using a user account with sufficient privileges to access the account data.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables once the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between **INCREMENTAL** and **FULL\_TABLE**.
* **Incremental**: Every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* **Full table**: Every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Inhire and their corresponding fields:
Stream containing the lean job listing used as the parent stream for job-related child streams.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the job |
| `name` | String | Name of the job |
| `status` | String | Current status of the job (e.g., active, archived) |
| `area` | String | Business area of the job |
| `userId` | String | Identifier of the user responsible for the job |
| `recruiterId` | String | Identifier of the recruiter assigned to the job |
| `managerId` | String | Identifier of the hiring manager |
| `createdAt` | String | When the job was created |
| `updatedAt` | String | When the job was last updated |
Stream containing full job details for each job from the `Jobs` stream.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the job |
| `name` | String | Name of the job |
| `status` | String | Current status of the job |
| `seniority` | String | Seniority level associated with the job |
| `salaryMin` | Number | Minimum salary for the position |
| `salaryTarget` | Number | Target salary for the position |
| `salaryMax` | Number | Maximum salary for the position |
| `talentsCount` | Integer | Total number of talents associated with the job |
| `customFields` | Array | Custom fields configured for the job |
| `stages` | Array | Hiring stages configured for the job |
| `statusHistory` | Array | Historical status changes of the job |
| `postingPage` | Object | Public posting page settings and metadata |
Stream containing candidates (talents) associated with each job.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the job-talent relationship |
| `talentId` | String | ID of the talent |
| `userId` | String | ID of the user responsible for the candidate in the process |
| `status` | String | Current status of the talent in the hiring process |
| `stage` | Object | Detailed information about the current stage of the talent |
| `timeInCurrentStage` | Integer | Time the talent has spent in the current stage |
| `source` | String | The source from where the talent was acquired |
| `targetSalary` | Number | Candidate salary expectations |
| `talent` | Object | Candidate profile information (email, phone, location, LinkedIn, and attributes) |
| `createdAt` | String | When the talent was associated with the job |
| `updatedAt` | String | When the job-talent relationship was last updated |
Stream containing evaluation scorecards for each job-talent relationship.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `jobTalentId` | String | ID of the associated job-talent relationship |
| `userId` | String | ID of the user who submitted the scorecard |
| `userName` | String | Name of the user who submitted the scorecard |
| `evaluationsFeedbacks` | Array | Interview feedback entries with comments and proceed recommendations |
| `skillComments` | Array | Qualitative comments grouped by skill |
| `skillsScore` | Array | Skill-level score breakdowns for each interview step |
Stream containing metadata definitions for custom fields in the `JOB_TALENTS` entity.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the custom field |
| `name` | String | Name of the custom field |
| `type` | String | Field type (e.g., text, select) |
| `entity` | String | Entity where the custom field applies (`JOB_TALENTS`) |
| `required` | Boolean | Whether the field is required |
| `options` | Array | Available options for option-based fields |
| `showIn` | Array | Contexts where the field is displayed |
| `requiredIn` | Array | Contexts where the field becomes mandatory |
Stream containing job position records linked to each job, including approval and hiring dates.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the position |
| `jobId` | String | Identifier of the associated job |
| `requisitionId` | String | Identifier of the requisition linked to the position |
| `reason` | String | Reason associated with creating the position |
| `approvedAt` | String | Timestamp when the position was approved |
| `hiredAt` | String | Timestamp when the position was filled |
| `createdAt` | String | Timestamp when the position was created |
| `updatedAt` | String | Timestamp when the position was last updated |
Stream containing Inhire user records (team members and recruiters).
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the user |
| `name` | String | Name of the user |
| `email` | String | Email of the user |
| `tenantId` | String | Identifier of the tenant |
| `roleId` | String | Identifier of the role |
| `roleName` | String | Name of the role |
| `position` | String | Position/job title of the user |
| `teams` | Array | Teams associated with the user |
| `active` | Boolean | Whether the user is active |
| `createdAt` | String | Timestamp when the user was created |
| `updatedAt` | String | Timestamp when the user was last updated |
## Implementation Notes
### Pagination & Extraction
The connector utilizes dynamic, cursor-based pagination driven directly by the API response envelope. The Inhire job talents endpoint ignores the requested page size and always caps a page at 200 records, so pagination is never driven by the number of records in a page. This ensures robust and reliable extraction of large datasets (such as high volumes of Job Talents), effectively preventing missed records or broken syncs that can occur with standard page-size limits.
## Skills for agents
Inhire connector documentation as plain markdown, for use in AI agent contexts.
# Instagram as a data source
Source: https://docs.nekt.com/sources/instagram
Bring data from Instagram to Nekt.
Instagram is a social media platform for sharing photos, videos, and stories. The Instagram Business API provides access to insights and analytics about your business account's performance and audience engagement.
![]()
## Requirements
Before setting up Instagram as a data source, ensure your account meets these requirements:
* The Instagram account must be a **Business** account (personal accounts are not supported)
* The Instagram profile must have at least **100 followers**
* The Instagram profile must be linked to a Facebook page
* The user connecting the account must have one of these roles on the Facebook page:
* Owner
* Administrator
* Editor
**Instagram Limitations:**
1. Media insights (such as views and engagement metrics) are only available for content that has received at least 5 views.
2. Stories data is only available for 24 hours after posting. If your pipeline runs during this window, the data will be captured and stored in your destination, but new pipelines won't be able to fetch stories older than 24 hours.
3. Business Discovery (competitor data) only provides public profile information and post-level metrics (likes, comments). Private insights such as reach, impressions, saves, and follower demographics are not available for competitor accounts.
## Configuring Instagram as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Instagram option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to add the following information to connect to Instagram:
* **Facebook Authorization**: Complete Facebook authorization with an account that has access to the page you want to get data from.
* **Instagram Page**: Select the Instagram Page you'd like to extract data from.
* **Start Date**: The earliest date from which records will be synced. Used for the first full sync and when no previous state exists.
* **Lookback Window**: The number of days to look in the past for new updates in Media (such as Comments and Likes). What the lookback does is check data from X days ago to ensure it captures all new updates. You can use the default value if you are not sure about it.
5. (Optional) **Competitor Usernames**: Add a list of Instagram usernames of competitor Business or Creator accounts you want to monitor. This uses the Instagram Business Discovery API to fetch public profile data and posts from those accounts using your own authentication. The competitor accounts must be public Business or Creator accounts.
Click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will contain the fetched data), and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables once the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental**: Every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* **Full table**: Every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Instagram and their corresponding fields:
Stream for basic account information about your Instagram Business account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Account ID (Primary Key) |
| `ig_id` | Integer | Instagram internal numeric ID |
| `username` | String | Account username |
| `name` | String | Account display name |
| `biography` | String | Account bio text |
| `website` | String | Account website URL |
| `profile_picture_url` | String | Profile picture URL |
| `follows_count` | Integer | Number of accounts being followed |
| `followers_count` | Integer | Number of followers |
| `media_count` | Integer | Total number of media posts |
Stream for posts, stories, and reels published on your account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Media ID (Primary Key) |
| `ig_id` | String | Instagram internal ID |
| `caption` | String | Post caption text |
| `media_type` | String | Type of media (IMAGE, VIDEO, CAROUSEL\_ALBUM) |
| `media_product_type` | String | Product type (FEED, STORY, REELS) |
| `timestamp` | String | When the media was posted (Replication Key) |
| `media_url` | String | URL to the media content |
| `permalink` | String | Permanent link to the post |
| `thumbnail_url` | String | Thumbnail image URL (for videos) |
| `shortcode` | String | Short code for the media |
| `comments_count` | Integer | Number of comments |
| `like_count` | Integer | Number of likes |
| `is_comment_enabled` | Boolean | Whether comments are enabled |
| `is_shared_to_feed` | Boolean | Whether shared to feed (for Reels) |
| `username` | String | Username who posted |
| `owner` | JSON | Owner object containing `id` (Owner account ID) |
| `boost_eligibility_info` | JSON | Boost eligibility object containing `eligible_to_boost` |
Stream for media performance metrics. Insights are only available for media with at least 5 views.
| Field | Type | Description |
| :- | :- | :- |
| `media_id` | String | Media ID (Primary Key) |
| `media_type` | String | Type of media (IMAGE, VIDEO, CAROUSEL\_ALBUM) |
| `media_product_type` | String | Product type (FEED, STORY, REELS) |
| `timestamp` | String | When the media was posted (Replication Key) |
| `comments` | Integer | Number of comments |
| `likes` | Integer | Number of likes |
| `shares` | Integer | Number of shares |
| `saved` | Integer | Number of saves |
| `total_interactions` | Integer | Total engagement count |
| `replies` | Integer | Number of replies (Stories) |
| `reach` | Integer | Number of unique accounts that saw the post |
| `views` | Integer | Number of views |
| `follows` | Integer | Number of new follows from this media |
| `profile_visits` | Integer | Profile visits from this media |
| `ig_reels_avg_watch_time` | Integer | Average watch time in milliseconds |
| `ig_reels_video_view_total_time` | Integer | Total view time in milliseconds |
Stream for accounts invited as collaborators on a media published by the connected account. This is a child stream of Media. Collaborator tagging is not supported on stories, so they are skipped.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | App-scoped identifier of the collaborator's Instagram account (Primary Key) |
| `media_id` | String | Identifier of the media the collaborator was invited to (Primary Key) |
| `username` | String | Username of the collaborator's Instagram profile |
| `invite_status` | String | Status of the collaboration invite, either Accepted or Pending |
| `media_timestamp` | Timestamp | Timestamp when the media was published, inherited from the parent media (Replication Key) |
Stream for comments on your posts. Comments are extracted as a child stream of Media.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Comment ID (Primary Key) |
| `media_id` | String | ID of the media being commented on |
| `text` | String | Comment text content |
| `timestamp` | String | When the comment was posted |
| `like_count` | Integer | Number of likes on the comment |
| `hidden` | Boolean | Whether the comment is hidden |
Stream for replies to comments. Replies are extracted as a child stream of Comments.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Reply ID (Primary Key) |
| `parent_comment_id` | String | ID of the parent comment |
| `text` | String | Reply text content |
| `timestamp` | String | When the reply was posted |
| `like_count` | Integer | Number of likes on the reply |
| `hidden` | Boolean | Whether the reply is hidden |
Daily account-level insights with a 1-day rolling window (`insights_rolling_day`).
| Field | Type | Description |
| :- | :- | :- |
| `date` | Timestamp | Last day of the rolling window the metrics are aggregated over. (Primary Key, Replication Key) |
| `reach` | Integer | Number of unique accounts that saw the account's content at least once during the window, including ads. Deduplicated by the API across the whole window; values from different rows must not be summed. |
| `views` | Integer | Total number of times the account's content was played or displayed during the window, including ads. Replaces the deprecated impressions metric. |
| `total_interactions` | Integer | Total number of interactions (likes, comments, shares, saves and replies) on the account's content during the window. |
| `accounts_engaged` | Integer | Number of unique accounts that interacted with the account's content during the window. Deduplicated by the API across the whole window; values from different rows must not be summed. |
| `likes` | Integer | Number of likes on the account's posts, reels and videos during the window. |
| `shares` | Integer | Number of shares of the account's posts, stories, reels and videos during the window. |
| `replies` | Integer | Number of replies received on the account's stories during the window. |
| `follows_and_unfollows` | Integer | Number of accounts that started following minus accounts that stopped following the account during the window. |
| `profile_links_taps` | Integer | Number of taps on the account's business address, call, email and text links during the window. |
| `website_clicks` | Integer | Number of taps on the website link in the account's profile during the window. |
| `profile_views` | Integer | Number of times the account's profile was visited during the window. |
Account-level insights with a 7-day rolling window (`insights_rolling_week`).
| Field | Type | Description |
| :- | :- | :- |
| `date` | Timestamp | Last day of the rolling window the metrics are aggregated over. (Primary Key, Replication Key) |
| `reach` | Integer | Number of unique accounts that saw the account's content at least once during the window, including ads. Deduplicated by the API across the whole window; values from different rows must not be summed. |
| `views` | Integer | Total number of times the account's content was played or displayed during the window, including ads. Replaces the deprecated impressions metric. |
| `total_interactions` | Integer | Total number of interactions (likes, comments, shares, saves and replies) on the account's content during the window. |
| `accounts_engaged` | Integer | Number of unique accounts that interacted with the account's content during the window. Deduplicated by the API across the whole window; values from different rows must not be summed. |
| `likes` | Integer | Number of likes on the account's posts, reels and videos during the window. |
| `shares` | Integer | Number of shares of the account's posts, stories, reels and videos during the window. |
| `replies` | Integer | Number of replies received on the account's stories during the window. |
| `follows_and_unfollows` | Integer | Number of accounts that started following minus accounts that stopped following the account during the window. |
| `profile_links_taps` | Integer | Number of taps on the account's business address, call, email and text links during the window. |
| `website_clicks` | Integer | Number of taps on the website link in the account's profile during the window. |
| `profile_views` | Integer | Number of times the account's profile was visited during the window. |
Account-level insights with a 28-day rolling window (`insights_rolling_days28`).
| Field | Type | Description |
| :- | :- | :- |
| `date` | Timestamp | Last day of the rolling window the metrics are aggregated over. (Primary Key, Replication Key) |
| `reach` | Integer | Number of unique accounts that saw the account's content at least once during the window, including ads. Deduplicated by the API across the whole window; values from different rows must not be summed. |
| `views` | Integer | Total number of times the account's content was played or displayed during the window, including ads. Replaces the deprecated impressions metric. |
| `total_interactions` | Integer | Total number of interactions (likes, comments, shares, saves and replies) on the account's content during the window. |
| `accounts_engaged` | Integer | Number of unique accounts that interacted with the account's content during the window. Deduplicated by the API across the whole window; values from different rows must not be summed. |
| `likes` | Integer | Number of likes on the account's posts, reels and videos during the window. |
| `shares` | Integer | Number of shares of the account's posts, stories, reels and videos during the window. |
| `replies` | Integer | Number of replies received on the account's stories during the window. |
| `follows_and_unfollows` | Integer | Number of accounts that started following minus accounts that stopped following the account during the window. |
| `profile_links_taps` | Integer | Number of taps on the account's business address, call, email and text links during the window. |
| `website_clicks` | Integer | Number of taps on the website link in the account's profile during the window. |
| `profile_views` | Integer | Number of times the account's profile was visited during the window. |
Lifetime follower demographics broken down by country (`follower_demographics_country_lifetime`).
| Field | Type | Description |
| :- | :- | :- |
| `dimension_value` | String | Country code (e.g., "US", "BR", "GB") (Primary Key) |
| `value` | Integer | Number of followers from this country |
Lifetime follower demographics broken down by age range (`follower_demographics_age_lifetime`).
| Field | Type | Description |
| :- | :- | :- |
| `dimension_value` | String | Age range (e.g., "18-24", "25-34", "35-44") (Primary Key) |
| `value` | Integer | Number of followers in this age range |
Lifetime follower demographics broken down by gender (`follower_demographics_gender_lifetime`).
| Field | Type | Description |
| :- | :- | :- |
| `dimension_value` | String | Gender (e.g., "M", "F", "U") (Primary Key) |
| `value` | Integer | Number of followers of this gender |
Lifetime follower demographics broken down by city (`follower_demographics_city_lifetime`).
| Field | Type | Description |
| :- | :- | :- |
| `dimension_value` | String | City name (e.g., "São Paulo, Brazil") (Primary Key) |
| `value` | Integer | Number of followers from this city |
Stream that tracks historical absolute follower counts (`follower_count_historical`).
On the first run (no bookmark), this stream reconstructs the last 30 days of follower counts by querying exact daily follows and unfollows from the Instagram Insights API and working backwards from the current total. On subsequent incremental runs, it simply records today's current follower count as a snapshot, optimizing API usage.
The historical reconstruction is limited to a maximum of **30 days** of data due to Instagram API constraints on the insights endpoint. Running regular incremental syncs allows you to build a continuous snapshot history beyond that limit.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Timestamp | Date of the follower count (Primary Key, Replication Key) |
| `followers_count` | Integer | Absolute number of followers on that date |
Stream for public profile data of competitor Instagram Business/Creator accounts (`business_discovery_profiles`). Requires `competitor_usernames` to be configured.
This stream uses the Instagram Business Discovery API to fetch publicly available profile information from other Business or Creator accounts, authenticating with your own account.
**Key Fields:**
* `id` - Competitor account ID (Primary Key)
* `ig_id` - Instagram internal numeric ID
* `username` - Competitor account username
* `name` - Competitor account display name
**Profile Information:**
* `biography` - Account bio text
* `website` - Account website URL
* `profile_picture_url` - Profile picture URL
**Metrics:**
* `follows_count` - Number of accounts being followed
* `followers_count` - Number of followers
* `media_count` - Total number of media posts
Stream for public media posts of competitor Instagram Business/Creator accounts (`business_discovery_media`). This is a child stream of Business Discovery Profiles. Requires `competitor_usernames` to be configured.
Paginates through competitor media from newest to oldest, stopping when reaching the configured `start_date`.
Only publicly available metrics are returned for competitor posts. Private insights such as reach, impressions, saves, shares, and follower demographics are not available through the Business Discovery API.
**Key Fields:**
* `id` - Media ID (Primary Key)
* `caption` - Post caption text
* `media_type` - Type of media (IMAGE, VIDEO, CAROUSEL\_ALBUM)
* `media_product_type` - Product type (FEED, REELS)
* `timestamp` - When the media was posted (Replication Key)
**URLs and References:**
* `media_url` - URL to the media content
* `permalink` - Permanent link to the post
**Engagement Metrics:**
* `comments_count` - Number of comments
* `like_count` - Number of likes
**Additional Information:**
* `username` - Username of the competitor account
## Data Model
The following diagram illustrates the relationships between the core data streams in Instagram. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
UserDetails("User Details");
Media("Media");
MediaCollaborators("Media Collaborators");
end
subgraph "Engagement"
Comments("Comments");
CommentReplies("Comment Replies");
end
subgraph "Analytics"
MediaInsights("Media Insights");
UserInsightsDay("User Insights - Day");
UserInsightsWeek("User Insights - Week");
UserInsights28("User Insights - 28 Days");
end
subgraph "Demographics"
DemoCountry("Demographics - Country");
DemoAge("Demographics - Age");
DemoGender("Demographics - Gender");
DemoCity("Demographics - City");
end
subgraph "Competitor Analysis"
CompProfiles("BD - Competitor Profiles");
CompMedia("BD - Competitor Media");
end
Media -- "owner.id" --> UserDetails;
MediaCollaborators -- "media_id" --> Media;
Comments -- "media_id" --> Media;
CommentReplies -- "parent_comment_id" --> Comments;
MediaInsights -- "media_id" --> Media;
CompMedia -- "username" --> CompProfiles;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Instagram data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
## Content Performance Analysis
### 1. Media Performance Overview
Track the overall performance of your posts, stories, and reels to identify top-performing content.
**Business Value:**
* Identify which content types drive the most engagement
* Understand what content resonates best with your audience
* Optimize your content strategy based on performance data
* Track reach and engagement trends over time
```sql theme={null}
WITH
media_performance AS (
SELECT
m.id AS media_id,
m.media_type,
m.media_product_type,
m.caption,
m.timestamp AS posted_at,
m.like_count,
m.comments_count,
mi.reach,
mi.shares,
mi.saved,
mi.views,
mi.total_interactions,
mi.ig_reels_avg_watch_time,
CAST(mi.total_interactions AS DOUBLE) * 100.0 / NULLIF(mi.reach, 0) AS engagement_rate
FROM
nekt_raw.instagram_media m
LEFT JOIN nekt_raw.instagram_media_insights mi ON m.id = mi.media_id
WHERE
DATE(m.timestamp) >= CURRENT_DATE - INTERVAL '30' DAY
)
SELECT
media_product_type,
COUNT(*) AS total_posts,
SUM(like_count) AS total_likes,
SUM(comments_count) AS total_comments,
SUM(reach) AS total_reach,
SUM(shares) AS total_shares,
SUM(saved) AS total_saves,
SUM(views) AS total_views,
SUM(total_interactions) AS total_interactions,
ROUND(AVG(engagement_rate), 2) AS avg_engagement_rate,
ROUND(AVG(ig_reels_avg_watch_time) / 1000.0, 2) AS avg_watch_time_seconds
FROM
media_performance
GROUP BY
media_product_type
ORDER BY
total_interactions DESC
```
```sql theme={null}
WITH
media_performance AS (
SELECT
m.id AS media_id,
m.media_type,
m.media_product_type,
m.caption,
m.timestamp AS posted_at,
m.like_count,
m.comments_count,
mi.reach,
mi.shares,
mi.saved,
mi.views,
mi.total_interactions,
mi.ig_reels_avg_watch_time,
SAFE_DIVIDE(CAST(mi.total_interactions AS FLOAT64) * 100.0, mi.reach) AS engagement_rate
FROM
`nekt_raw.instagram_media` m
LEFT JOIN `nekt_raw.instagram_media_insights` mi ON m.id = mi.media_id
WHERE
DATE(m.timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
)
SELECT
media_product_type,
COUNT(*) AS total_posts,
SUM(like_count) AS total_likes,
SUM(comments_count) AS total_comments,
SUM(reach) AS total_reach,
SUM(shares) AS total_shares,
SUM(saved) AS total_saves,
SUM(views) AS total_views,
SUM(total_interactions) AS total_interactions,
ROUND(AVG(engagement_rate), 2) AS avg_engagement_rate,
ROUND(SAFE_DIVIDE(AVG(ig_reels_avg_watch_time), 1000.0), 2) AS avg_watch_time_seconds
FROM
media_performance
GROUP BY
media_product_type
ORDER BY
total_interactions DESC
```
| media\_product\_type | total\_posts | total\_likes | total\_comments | total\_reach | total\_shares | total\_saves | total\_views | total\_interactions | avg\_engagement\_rate | avg\_watch\_time\_seconds |
| - | - | - | - | - | - | - | - | - | - | - |
| REELS | 12 | 8,420 | 342 | 156,890 | 1,245 | 890 | 234,560 | 10,897 | 6.94 | 8.45 |
| FEED | 18 | 5,230 | 198 | 89,450 | 456 | 623 | NULL | 6,507 | 7.27 | NULL |
| STORY | 45 | NULL | NULL | 45,230 | 234 | NULL | 67,890 | 234 | 0.52 | NULL |
### 2. Top Performing Content
Identify your best-performing posts based on engagement metrics to replicate successful content strategies.
**Business Value:**
* Discover content patterns that drive high engagement
* Identify optimal content formats and topics
* Inform future content creation decisions
* Benchmark new content against top performers
```sql theme={null}
WITH
ranked_content AS (
SELECT
m.id AS media_id,
m.media_product_type,
SUBSTRING(m.caption, 1, 100) AS caption_preview,
m.permalink,
DATE(m.timestamp) AS posted_date,
m.like_count,
m.comments_count,
mi.reach,
mi.shares,
mi.saved,
mi.total_interactions,
CAST(mi.total_interactions AS DOUBLE) * 100.0 / NULLIF(mi.reach, 0) AS engagement_rate,
ROW_NUMBER() OVER (
PARTITION BY m.media_product_type
ORDER BY mi.total_interactions DESC
) AS rank_by_type
FROM
nekt_raw.instagram_media m
LEFT JOIN nekt_raw.instagram_media_insights mi ON m.id = mi.media_id
WHERE
DATE(m.timestamp) >= CURRENT_DATE - INTERVAL '90' DAY
AND mi.reach > 0
)
SELECT
media_product_type,
caption_preview,
posted_date,
like_count,
comments_count,
reach,
shares,
saved,
total_interactions,
ROUND(engagement_rate, 2) AS engagement_rate_pct
FROM
ranked_content
WHERE
rank_by_type <= 5
ORDER BY
media_product_type,
total_interactions DESC
```
```sql theme={null}
WITH
ranked_content AS (
SELECT
m.id AS media_id,
m.media_product_type,
SUBSTR(m.caption, 1, 100) AS caption_preview,
m.permalink,
DATE(m.timestamp) AS posted_date,
m.like_count,
m.comments_count,
mi.reach,
mi.shares,
mi.saved,
mi.total_interactions,
SAFE_DIVIDE(CAST(mi.total_interactions AS FLOAT64) * 100.0, mi.reach) AS engagement_rate,
ROW_NUMBER() OVER (
PARTITION BY m.media_product_type
ORDER BY mi.total_interactions DESC
) AS rank_by_type
FROM
`nekt_raw.instagram_media` m
LEFT JOIN `nekt_raw.instagram_media_insights` mi ON m.id = mi.media_id
WHERE
DATE(m.timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY)
AND mi.reach > 0
)
SELECT
media_product_type,
caption_preview,
posted_date,
like_count,
comments_count,
reach,
shares,
saved,
total_interactions,
ROUND(engagement_rate, 2) AS engagement_rate_pct
FROM
ranked_content
WHERE
rank_by_type <= 5
ORDER BY
media_product_type,
total_interactions DESC
```
| media\_product\_type | caption\_preview | posted\_date | like\_count | comments\_count | reach | shares | saved | total\_interactions | engagement\_rate\_pct |
| - | - | - | - | - | - | - | - | - | - |
| REELS | 🎬 Behind the scenes of our latest product launch! Watch how we... | 2024-10-15 | 2,340 | 89 | 45,670 | 456 | 234 | 3,119 | 6.83 |
| REELS | Quick tutorial: 5 tips to boost your productivity 💡 #tips... | 2024-10-22 | 1,890 | 67 | 38,920 | 312 | 189 | 2,458 | 6.31 |
| FEED | Introducing our newest collection ✨ Available now! Link in... | 2024-11-01 | 1,456 | 123 | 28,450 | 89 | 312 | 1,980 | 6.96 |
| FEED | Thank you for 100K followers! 🎉 We couldn't have done it... | 2024-09-28 | 3,210 | 234 | 42,180 | 156 | 178 | 3,778 | 8.96 |
| REELS | POV: When the coffee hits different on Monday ☕ #relatable... | 2024-10-08 | 1,234 | 45 | 32,890 | 234 | 145 | 1,658 | 5.04 |
## Audience Engagement Analysis
### 3. Account Growth and Engagement Trends
Track your account's growth and engagement metrics over time to monitor performance trends.
**Business Value:**
* Monitor account health and growth trajectory
* Identify periods of high or low engagement
* Correlate engagement with content posting patterns
* Track progress toward growth goals
```sql theme={null}
WITH
daily_insights AS (
SELECT
DATE(date) AS insight_date,
reach,
views,
total_interactions,
accounts_engaged,
likes,
shares,
replies,
follows_and_unfollows,
profile_views,
website_clicks
FROM
nekt_raw.instagram_insights_rolling_day
WHERE
DATE(date) >= CURRENT_DATE - INTERVAL '30' DAY
),
with_moving_avg AS (
SELECT
*,
AVG(reach) OVER (ORDER BY insight_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS reach_7day_avg,
AVG(total_interactions) OVER (ORDER BY insight_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS interactions_7day_avg,
SUM(follows_and_unfollows) OVER (ORDER BY insight_date) AS cumulative_net_followers
FROM
daily_insights
)
SELECT
insight_date,
reach,
views,
ROUND(reach_7day_avg, 0) AS reach_7day_avg,
total_interactions,
ROUND(interactions_7day_avg, 0) AS interactions_7day_avg,
accounts_engaged,
follows_and_unfollows AS net_followers,
cumulative_net_followers,
profile_views,
website_clicks
FROM
with_moving_avg
ORDER BY
insight_date DESC
```
```sql theme={null}
WITH
daily_insights AS (
SELECT
DATE(date) AS insight_date,
reach,
views,
total_interactions,
accounts_engaged,
likes,
shares,
replies,
follows_and_unfollows,
profile_views,
website_clicks
FROM
`nekt_raw.instagram_insights_rolling_day`
WHERE
DATE(date) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
),
with_moving_avg AS (
SELECT
*,
AVG(reach) OVER (ORDER BY insight_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS reach_7day_avg,
AVG(total_interactions) OVER (ORDER BY insight_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS interactions_7day_avg,
SUM(follows_and_unfollows) OVER (ORDER BY insight_date) AS cumulative_net_followers
FROM
daily_insights
)
SELECT
insight_date,
reach,
views,
ROUND(reach_7day_avg, 0) AS reach_7day_avg,
total_interactions,
ROUND(interactions_7day_avg, 0) AS interactions_7day_avg,
accounts_engaged,
follows_and_unfollows AS net_followers,
cumulative_net_followers,
profile_views,
website_clicks
FROM
with_moving_avg
ORDER BY
insight_date DESC
```
| insight\_date | reach | views | reach\_7day\_avg | total\_interactions | interactions\_7day\_avg | accounts\_engaged | net\_followers | cumulative\_net\_followers | profile\_views | website\_clicks |
| - | - | - | - | - | - | - | - | - | - | - |
| 2024-11-27 | 12,450 | 18,230 | 11,234 | 892 | 845 | 456 | 23 | 342 | 234 | 45 |
| 2024-11-26 | 11,890 | 17,450 | 10,980 | 823 | 812 | 412 | 18 | 319 | 198 | 38 |
| 2024-11-25 | 10,234 | 15,100 | 10,567 | 756 | 798 | 389 | 12 | 301 | 187 | 32 |
| 2024-11-24 | 8,920 | 12,650 | 10,123 | 654 | 776 | 312 | -5 | 289 | 145 | 28 |
| 2024-11-23 | 9,456 | 13,420 | 9,890 | 712 | 754 | 345 | 8 | 294 | 156 | 31 |
| 2024-11-22 | 14,230 | 21,800 | 9,678 | 1,023 | 732 | 523 | 45 | 286 | 312 | 67 |
### 4. Comment Engagement Analysis
Analyze comment activity on your posts to understand audience sentiment and engagement patterns.
**Business Value:**
* Measure audience engagement depth
* Identify posts that spark conversations
* Track comment response rates
* Monitor community health and sentiment
```sql theme={null}
WITH
comment_stats AS (
SELECT
m.id AS media_id,
m.media_product_type,
SUBSTRING(m.caption, 1, 80) AS caption_preview,
DATE(m.timestamp) AS posted_date,
m.comments_count,
COUNT(c.id) AS actual_comments,
SUM(c.like_count) AS total_comment_likes,
AVG(c.like_count) AS avg_likes_per_comment,
SUM(CASE WHEN c.hidden = false THEN 1 ELSE 0 END) AS visible_comments,
SUM(CASE WHEN c.hidden = true THEN 1 ELSE 0 END) AS hidden_comments
FROM
nekt_raw.instagram_media m
LEFT JOIN nekt_raw.instagram_comments c ON m.id = c.media_id
WHERE
DATE(m.timestamp) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
m.id,
m.media_product_type,
m.caption,
m.timestamp,
m.comments_count
)
SELECT
media_product_type,
COUNT(*) AS total_posts,
SUM(actual_comments) AS total_comments,
ROUND(AVG(actual_comments), 1) AS avg_comments_per_post,
SUM(total_comment_likes) AS total_comment_likes,
ROUND(AVG(avg_likes_per_comment), 2) AS avg_likes_per_comment,
SUM(hidden_comments) AS total_hidden_comments,
ROUND(
CAST(SUM(hidden_comments) AS DOUBLE) * 100.0 / NULLIF(SUM(actual_comments), 0),
2
) AS hidden_comment_rate_pct
FROM
comment_stats
GROUP BY
media_product_type
ORDER BY
total_comments DESC
```
```sql theme={null}
WITH
comment_stats AS (
SELECT
m.id AS media_id,
m.media_product_type,
SUBSTR(m.caption, 1, 80) AS caption_preview,
DATE(m.timestamp) AS posted_date,
m.comments_count,
COUNT(c.id) AS actual_comments,
SUM(c.like_count) AS total_comment_likes,
AVG(c.like_count) AS avg_likes_per_comment,
SUM(CASE WHEN c.hidden = false THEN 1 ELSE 0 END) AS visible_comments,
SUM(CASE WHEN c.hidden = true THEN 1 ELSE 0 END) AS hidden_comments
FROM
`nekt_raw.instagram_media` m
LEFT JOIN `nekt_raw.instagram_comments` c ON m.id = c.media_id
WHERE
DATE(m.timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
m.id,
m.media_product_type,
m.caption,
m.timestamp,
m.comments_count
)
SELECT
media_product_type,
COUNT(*) AS total_posts,
SUM(actual_comments) AS total_comments,
ROUND(AVG(actual_comments), 1) AS avg_comments_per_post,
SUM(total_comment_likes) AS total_comment_likes,
ROUND(AVG(avg_likes_per_comment), 2) AS avg_likes_per_comment,
SUM(hidden_comments) AS total_hidden_comments,
ROUND(
SAFE_DIVIDE(CAST(SUM(hidden_comments) AS FLOAT64) * 100.0, SUM(actual_comments)),
2
) AS hidden_comment_rate_pct
FROM
comment_stats
GROUP BY
media_product_type
ORDER BY
total_comments DESC
```
| media\_product\_type | total\_posts | total\_comments | avg\_comments\_per\_post | total\_comment\_likes | avg\_likes\_per\_comment | total\_hidden\_comments | hidden\_comment\_rate\_pct |
| - | - | - | - | - | - | - | - |
| REELS | 12 | 342 | 28.5 | 1,234 | 3.61 | 8 | 2.34 |
| FEED | 18 | 198 | 11.0 | 567 | 2.86 | 3 | 1.52 |
| STORY | 45 | 0 | 0.0 | 0 | 0.00 | 0 | 0.00 |
## Audience Demographics Analysis
### 5. Follower Geographic Distribution
Understand where your followers are located to optimize content timing and targeting.
**Business Value:**
* Identify key markets and geographic audiences
* Optimize posting times for primary time zones
* Inform localization and content strategies
* Support geo-targeted campaigns
```sql theme={null}
WITH
country_data AS (
SELECT
dimension_value AS country_code,
value AS follower_count
FROM
nekt_raw.instagram_follower_demographics_country_lifetime
),
total_followers AS (
SELECT SUM(follower_count) AS total FROM country_data
)
SELECT
cd.country_code,
cd.follower_count,
ROUND(
CAST(cd.follower_count AS DOUBLE) * 100.0 / tf.total,
2
) AS percentage,
SUM(cd.follower_count) OVER (ORDER BY cd.follower_count DESC) AS cumulative_followers,
ROUND(
CAST(SUM(cd.follower_count) OVER (ORDER BY cd.follower_count DESC) AS DOUBLE) * 100.0 / tf.total,
2
) AS cumulative_percentage
FROM
country_data cd
CROSS JOIN total_followers tf
ORDER BY
cd.follower_count DESC
LIMIT 20
```
```sql theme={null}
WITH
country_data AS (
SELECT
dimension_value AS country_code,
value AS follower_count
FROM
`nekt_raw.instagram_follower_demographics_country_lifetime`
),
total_followers AS (
SELECT SUM(follower_count) AS total FROM country_data
)
SELECT
cd.country_code,
cd.follower_count,
ROUND(
SAFE_DIVIDE(CAST(cd.follower_count AS FLOAT64) * 100.0, tf.total),
2
) AS percentage,
SUM(cd.follower_count) OVER (ORDER BY cd.follower_count DESC) AS cumulative_followers,
ROUND(
SAFE_DIVIDE(
CAST(SUM(cd.follower_count) OVER (ORDER BY cd.follower_count DESC) AS FLOAT64) * 100.0,
tf.total
),
2
) AS cumulative_percentage
FROM
country_data cd
CROSS JOIN total_followers tf
ORDER BY
cd.follower_count DESC
LIMIT 20
```
| country\_code | follower\_count | percentage | cumulative\_followers | cumulative\_percentage |
| - | - | - | - | - |
| BR | 45,230 | 42.35 | 45,230 | 42.35 |
| US | 18,920 | 17.72 | 64,150 | 60.07 |
| PT | 8,450 | 7.91 | 72,600 | 67.98 |
| AR | 6,230 | 5.83 | 78,830 | 73.81 |
| MX | 4,890 | 4.58 | 83,720 | 78.39 |
| GB | 3,450 | 3.23 | 87,170 | 81.62 |
### 6. Audience Demographics Summary
Get a complete view of your audience demographics including age, gender, and location.
**Business Value:**
* Build comprehensive audience personas
* Align content with audience preferences
* Support ad targeting decisions
* Identify growth opportunities in demographics
```sql theme={null}
WITH
age_data AS (
SELECT 'age' AS dimension_type, dimension_value, value AS count
FROM nekt_raw.instagram_follower_demographics_age_lifetime
),
gender_data AS (
SELECT 'gender' AS dimension_type, dimension_value, value AS count
FROM nekt_raw.instagram_follower_demographics_gender_lifetime
),
city_data AS (
SELECT 'city' AS dimension_type, dimension_value, value AS count
FROM nekt_raw.instagram_follower_demographics_city_lifetime
),
all_demographics AS (
SELECT * FROM age_data
UNION ALL
SELECT * FROM gender_data
UNION ALL
SELECT * FROM city_data
),
totals AS (
SELECT dimension_type, SUM(count) AS total
FROM all_demographics
GROUP BY dimension_type
)
SELECT
ad.dimension_type,
ad.dimension_value,
ad.count AS follower_count,
ROUND(
CAST(ad.count AS DOUBLE) * 100.0 / t.total,
2
) AS percentage,
RANK() OVER (
PARTITION BY ad.dimension_type
ORDER BY ad.count DESC
) AS rank_in_category
FROM
all_demographics ad
JOIN totals t ON ad.dimension_type = t.dimension_type
ORDER BY
ad.dimension_type,
ad.count DESC
```
```sql theme={null}
WITH
age_data AS (
SELECT 'age' AS dimension_type, dimension_value, value AS count
FROM `nekt_raw.instagram_follower_demographics_age_lifetime`
),
gender_data AS (
SELECT 'gender' AS dimension_type, dimension_value, value AS count
FROM `nekt_raw.instagram_follower_demographics_gender_lifetime`
),
city_data AS (
SELECT 'city' AS dimension_type, dimension_value, value AS count
FROM `nekt_raw.instagram_follower_demographics_city_lifetime`
),
all_demographics AS (
SELECT * FROM age_data
UNION ALL
SELECT * FROM gender_data
UNION ALL
SELECT * FROM city_data
),
totals AS (
SELECT dimension_type, SUM(count) AS total
FROM all_demographics
GROUP BY dimension_type
)
SELECT
ad.dimension_type,
ad.dimension_value,
ad.count AS follower_count,
ROUND(
SAFE_DIVIDE(CAST(ad.count AS FLOAT64) * 100.0, t.total),
2
) AS percentage,
RANK() OVER (
PARTITION BY ad.dimension_type
ORDER BY ad.count DESC
) AS rank_in_category
FROM
all_demographics ad
JOIN totals t ON ad.dimension_type = t.dimension_type
ORDER BY
ad.dimension_type,
ad.count DESC
```
| dimension\_type | dimension\_value | follower\_count | percentage | rank\_in\_category |
| - | - | - | - | - |
| age | 25-34 | 38,450 | 36.01 | 1 |
| age | 18-24 | 28,920 | 27.08 | 2 |
| age | 35-44 | 21,340 | 19.98 | 3 |
| age | 45-54 | 10,890 | 10.20 | 4 |
| gender | F | 62,340 | 58.37 | 1 |
| gender | M | 42,890 | 40.16 | 2 |
| gender | U | 1,570 | 1.47 | 3 |
| city | São Paulo, Brazil | 12,450 | 11.66 | 1 |
| city | Rio de Janeiro, Brazil | 8,920 | 8.35 | 2 |
| city | New York, United States | 4,560 | 4.27 | 3 |
## Time-Based Analysis
### 7. Best Days and Times for Posting
Identify the best days of the week for engagement to optimize your posting schedule.
**Business Value:**
* Optimize posting schedule for maximum reach
* Identify peak engagement periods
* Improve content planning and scheduling
* Maximize return on content creation efforts
```sql theme={null}
WITH
daily_performance AS (
SELECT
DATE(date) AS insight_date,
DATE_FORMAT(DATE(date), '%W') AS day_of_week,
reach,
total_interactions,
accounts_engaged,
profile_views
FROM
nekt_raw.instagram_insights_rolling_day
WHERE
DATE(date) >= CURRENT_DATE - INTERVAL '90' DAY
),
day_averages AS (
SELECT
day_of_week,
AVG(reach) AS avg_reach,
AVG(total_interactions) AS avg_interactions,
AVG(accounts_engaged) AS avg_engaged,
AVG(profile_views) AS avg_profile_views,
COUNT(*) AS days_counted
FROM
daily_performance
GROUP BY
day_of_week
)
SELECT
day_of_week,
ROUND(avg_reach, 0) AS avg_daily_reach,
ROUND(avg_interactions, 0) AS avg_daily_interactions,
ROUND(avg_engaged, 0) AS avg_accounts_engaged,
ROUND(avg_profile_views, 0) AS avg_profile_views,
days_counted,
CASE
WHEN avg_interactions >= (SELECT MAX(avg_interactions) * 0.8 FROM day_averages) THEN 'Peak Day'
WHEN avg_interactions <= (SELECT MIN(avg_interactions) * 1.2 FROM day_averages) THEN 'Low Day'
ELSE 'Average'
END AS day_classification
FROM
day_averages
ORDER BY
CASE day_of_week
WHEN 'Monday' THEN 1
WHEN 'Tuesday' THEN 2
WHEN 'Wednesday' THEN 3
WHEN 'Thursday' THEN 4
WHEN 'Friday' THEN 5
WHEN 'Saturday' THEN 6
WHEN 'Sunday' THEN 7
END
```
```sql theme={null}
WITH
daily_performance AS (
SELECT
DATE(date) AS insight_date,
FORMAT_DATE('%A', DATE(date)) AS day_of_week,
reach,
total_interactions,
accounts_engaged,
profile_views
FROM
`nekt_raw.instagram_insights_rolling_day`
WHERE
DATE(date) >= DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY)
),
day_averages AS (
SELECT
day_of_week,
AVG(reach) AS avg_reach,
AVG(total_interactions) AS avg_interactions,
AVG(accounts_engaged) AS avg_engaged,
AVG(profile_views) AS avg_profile_views,
COUNT(*) AS days_counted
FROM
daily_performance
GROUP BY
day_of_week
)
SELECT
day_of_week,
ROUND(avg_reach, 0) AS avg_daily_reach,
ROUND(avg_interactions, 0) AS avg_daily_interactions,
ROUND(avg_engaged, 0) AS avg_accounts_engaged,
ROUND(avg_profile_views, 0) AS avg_profile_views,
days_counted,
CASE
WHEN avg_interactions >= (SELECT MAX(avg_interactions) * 0.8 FROM day_averages) THEN 'Peak Day'
WHEN avg_interactions <= (SELECT MIN(avg_interactions) * 1.2 FROM day_averages) THEN 'Low Day'
ELSE 'Average'
END AS day_classification
FROM
day_averages
ORDER BY
CASE day_of_week
WHEN 'Monday' THEN 1
WHEN 'Tuesday' THEN 2
WHEN 'Wednesday' THEN 3
WHEN 'Thursday' THEN 4
WHEN 'Friday' THEN 5
WHEN 'Saturday' THEN 6
WHEN 'Sunday' THEN 7
END
```
| day\_of\_week | avg\_daily\_reach | avg\_daily\_interactions | avg\_accounts\_engaged | avg\_profile\_views | days\_counted | day\_classification |
| - | - | - | - | - | - | - |
| Monday | 10,234 | 756 | 389 | 187 | 13 | Average |
| Tuesday | 11,450 | 823 | 412 | 198 | 13 | Average |
| Wednesday | 12,890 | 892 | 456 | 234 | 13 | Peak Day |
| Thursday | 13,120 | 945 | 478 | 245 | 13 | Peak Day |
| Friday | 11,890 | 834 | 423 | 212 | 13 | Average |
| Saturday | 8,920 | 654 | 312 | 145 | 13 | Low Day |
| Sunday | 9,456 | 712 | 345 | 156 | 13 | Average |
## Competitor Analysis
### 8. Competitor Benchmarking
Compare your account's performance against competitors using Business Discovery data. This requires `competitor_usernames` to be configured.
**Business Value:**
* Benchmark engagement rates against industry competitors
* Identify competitor content strategies that drive high engagement
* Track competitor follower growth over time
* Discover gaps and opportunities in your content strategy
```sql theme={null}
WITH
competitor_stats AS (
SELECT
username,
media_product_type,
COUNT(*) AS total_posts,
SUM(like_count) AS total_likes,
SUM(comments_count) AS total_comments,
ROUND(AVG(like_count), 0) AS avg_likes_per_post,
ROUND(AVG(comments_count), 0) AS avg_comments_per_post
FROM
nekt_raw.instagram_business_discovery_media
WHERE
DATE(timestamp) >= CURRENT_DATE - INTERVAL '90' DAY
GROUP BY
username, media_product_type
),
own_stats AS (
SELECT
username,
media_product_type,
COUNT(*) AS total_posts,
SUM(like_count) AS total_likes,
SUM(comments_count) AS total_comments,
ROUND(AVG(like_count), 0) AS avg_likes_per_post,
ROUND(AVG(comments_count), 0) AS avg_comments_per_post
FROM
nekt_raw.instagram_media
WHERE
DATE(timestamp) >= CURRENT_DATE - INTERVAL '90' DAY
GROUP BY
username, media_product_type
)
SELECT * FROM own_stats
UNION ALL
SELECT * FROM competitor_stats
ORDER BY
media_product_type, avg_likes_per_post DESC
```
```sql theme={null}
WITH
competitor_stats AS (
SELECT
username,
media_product_type,
COUNT(*) AS total_posts,
SUM(like_count) AS total_likes,
SUM(comments_count) AS total_comments,
ROUND(AVG(like_count), 0) AS avg_likes_per_post,
ROUND(AVG(comments_count), 0) AS avg_comments_per_post
FROM
`nekt_raw.instagram_business_discovery_media`
WHERE
DATE(timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY)
GROUP BY
username, media_product_type
),
own_stats AS (
SELECT
username,
media_product_type,
COUNT(*) AS total_posts,
SUM(like_count) AS total_likes,
SUM(comments_count) AS total_comments,
ROUND(AVG(like_count), 0) AS avg_likes_per_post,
ROUND(AVG(comments_count), 0) AS avg_comments_per_post
FROM
`nekt_raw.instagram_media`
WHERE
DATE(timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY)
GROUP BY
username, media_product_type
)
SELECT * FROM own_stats
UNION ALL
SELECT * FROM competitor_stats
ORDER BY
media_product_type, avg_likes_per_post DESC
```
| username | media\_product\_type | total\_posts | total\_likes | total\_comments | avg\_likes\_per\_post | avg\_comments\_per\_post |
| - | - | - | - | - | - | - |
| competitor\_a | REELS | 24 | 156,000 | 4,800 | 6,500 | 200 |
| your\_account | REELS | 18 | 45,200 | 1,890 | 2,511 | 105 |
| competitor\_b | REELS | 12 | 28,400 | 960 | 2,367 | 80 |
| competitor\_a | FEED | 15 | 89,000 | 2,400 | 5,933 | 160 |
| your\_account | FEED | 22 | 52,300 | 2,640 | 2,377 | 120 |
| competitor\_b | FEED | 8 | 12,800 | 480 | 1,600 | 60 |
## Implementation Notes
### Data Quality Considerations
* Media insights are only available for content with at least 5 views
* Stories data is only available for 24 hours after posting: ensure your pipeline runs frequently to capture this data
* User insights have a maximum lookback of 2 years from the Instagram API
* Follower demographics require a minimum of 100 followers
* Historical follower counts are reconstructed up to 30 days initially due to API constraints, but running regular incremental syncs allows you to build a continuous daily snapshot history beyond that limit.
* User Insights rolling windows natively deduplicate unique-account metrics such as `reach` and `accounts_engaged` over the window; you should not attempt to sum these metrics manually from smaller daily datasets.
### API Limitations
* Uses Instagram Graph API v26.0
* The Instagram API has rate limits that may affect extraction times for accounts with large amounts of content
* The lookback window parameter helps re-fetch updated metrics for recent content
* Some metrics may not be available for all media types (e.g., Reels-specific metrics only apply to Reels)
* Business Discovery (competitor data) only works with public Business or Creator accounts. Personal or private accounts cannot be queried
* Competitor media only includes public engagement metrics (likes and comments). Private insights like reach, impressions, saves, and shares are not available
### Best Practices
* Schedule extractions at least daily to capture Stories data before it expires
* Use incremental sync for media and comments to efficiently track updates
* Monitor the `reach` metric alongside engagement to understand true content performance
* Consider the 28-day rolling insights for trend analysis over the daily insights for day-to-day fluctuations
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| The profile does not appear in the list | The asset must belong to the same Business Manager used for the authorization. | Add the asset to the Business Manager and authorize again. |
| Media metrics are empty on recent posts | The API only exposes metrics once a post reaches a minimum number of views. | Wait for views to accumulate. No setting works around it. |
| The token has been invalidated | The password of the authorizing user was changed, or access was revoked. | Re-authorize the source. |
## Skills for agents
Instagram connector documentation as plain markdown, for use in AI agent contexts.
# Intelbras as a data source
Source: https://docs.nekt.com/sources/intelbras
Bring data from Intelbras to Nekt.
Intelbras is a leading company in security, networks, and communication solutions. This connector allows you to bring data from Intelbras access control systems into your Lakehouse, enabling you to analyze access logs and enhance your security monitoring capabilities.
![]()
## Configuring Intelbras as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Intelbras option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide credentials to connect to your Intelbras controller.
The following configurations are available:
* **IP address**: The IP address of the controller (e.g. 192.168.1.100).
* **Port**: The port of the controller (e.g. 8080).
* **Username**: The username to authenticate against the controller (e.g. admin).
* **Password**: The password to authenticate against the controller.
* **Start Date**: The earliest date from which records will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Intelbras and their corresponding fields:
Stream for managing access logs from the controller.
**Key Fields:**
* `RecNo` - Unique identifier for the record
* `CreateTime` - Timestamp of when the log was created
* `UserID` - ID of the user
* `CardNo` - Card number used for access
* `Door` - Door or access point identifier
* `Method` - Access method used (e.g., card, password)
* `Status` - Status of the access attempt (e.g., success, failure)
* `Type` - Type of event
* `AttendanceState` - State of attendance
* `CardName` - Name associated with the card
* `CardType` - Type of the card
* `ErrorCode` - Error code if the access failed
* `Mask` - Mask status
* `Notes` - Additional notes
* `Password` - Password used for access
* `ReaderID` - ID of the reader device
* `RemainingTimes` - Remaining access times
* `ReservedInt` - Reserved integer field
* `ReservedString` - Reserved string field
* `RoomNumber` - Room number associated with the access
* `URL` - URL related to the event
* `UserType` - Type of the user
## Use Cases for Data Analysis
This guide outlines a valuable business intelligence use case when consolidating Intelbras data, along with a ready-to-use SQL query that you can run on [Explorer](https://app.nekt.ai/explorer).
## Security Monitoring
### 1. Monitoring Access Events
Track access events to monitor who is accessing your facilities, when, and where.
**Business Value:**
* Enhance security by monitoring access in real-time
* Identify unauthorized access attempts
* Keep a historical record of all access events for auditing purposes
```sql lines theme={null}
SELECT
createtime,
userid,
cardno,
door,
method,
status
FROM
nekt_raw.intelbras_access_logs
ORDER BY
createtime DESC
```
## Skills for agents
Intelbras connector documentation as plain markdown, for use in AI agent contexts.
# Intercom as a data source
Source: https://docs.nekt.com/sources/intercom
Bring data from Intercom to Nekt.
Intercom is a customer messaging platform that helps businesses communicate with their customers through live chat, email, and in-app messaging. It provides tools for customer support, lead generation, and user engagement, with features like automated messaging, customer data management, and integration with various business tools to enhance customer relationships and drive growth.
![]()
## Configuring Intercom as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Intercom option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Intercom data. Click on the **Intercom Authorization** button and log in with your Intercom account. Grant the necessary permissions for the data you want to extract.
The following configuration is available:
* **Start Date**: The earliest date from which records will be synced (e.g., `2010-01-01`). Used for the first full sync and when no previous state exists.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you. The available streams include companies, contacts, tickets, ticket types, conversations, and conversation parts.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## API Limitations
When syncing the **Conversation Parts** stream, be aware that Intercom's API enforces a limit on the number of parts returned per conversation. If a conversation exceeds this limit, the connector will only extract up to the API limit, and any parts beyond that boundary will not be extracted. A warning is logged during extraction if a conversation is affected by this limitation.
# Streams and Fields
Below you'll find all available data streams from Intercom and their corresponding fields:
Stream for managing company information within your Intercom workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the company |
| `company_id` | String | Company ID from your system |
| `name` | String | Name of the company |
| `created_at` | Integer | When the company was created (Unix timestamp) |
| `updated_at` | Integer | When the company was last updated (Unix timestamp) |
| `monthly_spend` | Number | Monthly spend amount |
| `session_count` | Integer | Number of sessions |
| `user_count` | Integer | Number of users in the company |
| `type` | String | Object type identifier |
| `app_id` | String | ID of the application |
| `tags` | Object | Tags associated with the company (contains `type` and an array of `tags`) |
| `segments` | Object | Segments the company belongs to (contains `type` and an array of `segments`) |
| `plan` | Object | Plan information (contains `type`, `id`, `url`) |
| `custom_attributes` | Array | Array of custom attributes (contains `key`, `value`) |
Stream for managing contact information and their interactions with your business.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the contact |
| `type` | String | Object type identifier |
| `workspace_id` | String | ID of the workspace |
| `external_id` | String | External identifier from your system |
| `role` | String | Role of the contact (user, lead, visitor) |
| `email` | String | Email address |
| `phone` | String | Phone number |
| `name` | String | Full name of the contact |
| `avatar` | String | Avatar URL |
| `owner_id` | String | ID of the admin who owns this contact |
| `created_at` | Integer | When the contact was created (Unix timestamp) |
| `updated_at` | Integer | When the contact was last updated (Unix timestamp) |
| `signed_up_at` | Integer | When the contact signed up (Unix timestamp) |
| `last_seen_at` | Integer | Last time the contact was seen (Unix timestamp) |
| `last_replied_at` | Integer | Last time the contact replied (Unix timestamp) |
| `last_contacted_at` | Integer | Last time the contact was contacted (Unix timestamp) |
| `last_email_opened_at` | Integer | Last time the contact opened an email (Unix timestamp) |
| `last_email_clicked_at` | Integer | Last time the contact clicked an email link (Unix timestamp) |
| `has_hard_bounced` | Boolean | Whether emails have hard bounced |
| `marked_email_as_spam` | Boolean | Whether contact marked emails as spam |
| `unsubscribed_from_emails` | Boolean | Whether contact unsubscribed from emails |
| `location` | Object | Geographic location information |
| `browser` | String | Browser name |
| `browser_version` | String | Browser version |
| `browser_language` | String | Browser language |
| `os` | String | Operating system |
| `language_override` | String | Language override setting |
| `android_app_name` | String | Android app name |
| `android_app_version` | String | Android app version |
| `android_device` | String | Android device model |
| `android_os_version` | String | Android OS version |
| `android_sdk_version` | String | Android SDK version |
| `android_last_seen_at` | Integer | Last seen on Android (Unix timestamp) |
| `ios_app_name` | String | iOS app name |
| `ios_app_version` | String | iOS app version |
| `ios_device` | String | iOS device model |
| `ios_os_version` | String | iOS OS version |
| `ios_sdk_version` | String | iOS SDK version |
| `ios_last_seen_at` | Integer | Last seen on iOS (Unix timestamp) |
| `social_profiles` | Object | Social media profiles |
| `utm_campaign` | String | UTM campaign parameter |
| `utm_content` | String | UTM content parameter |
| `utm_medium` | String | UTM medium parameter |
| `utm_source` | String | UTM source parameter |
| `utm_term` | String | UTM term parameter |
| `referrer` | String | Referrer URL |
| `sms_consent` | Boolean | Whether contact consented to SMS |
| `unsubscribed_from_sms` | Boolean | Whether contact unsubscribed from SMS |
| `enabled_push_messaging` | Boolean | Push messaging enablement status |
| `tags` | Object | Tags associated with the contact |
| `notes` | Object | Notes about the contact |
| `companies` | Object | Companies the contact belongs to |
| `custom_attributes` | Array | Array of custom attributes (contains `key`, `value`) |
Stream for managing the different types of tickets configured in your Intercom workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the ticket type |
| `type` | String | Object type identifier |
| `name` | String | Name of the ticket type |
| `description` | String | Description of the ticket type |
| `icon` | String | Icon for the ticket type |
| `workspace_id` | String | ID of the workspace |
| `category` | String | Category of the ticket type |
| `archived` | Boolean | Whether the ticket type is archived |
| `is_internal` | Boolean | Whether the ticket type is internal only |
| `created_at` | Integer | When the ticket type was created (Unix timestamp) |
| `updated_at` | Integer | When the ticket type was last updated (Unix timestamp) |
| `ticket_type_attributes` | Object | Attributes defined for this ticket type |
| `ticket_states` | Object | States available for this ticket type |
Stream for managing support tickets and their details.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the ticket |
| `type` | String | Object type identifier |
| `ticket_id` | String | Human-readable ticket ID |
| `company_id` | String | ID of the associated company |
| `created_at` | Integer | When the ticket was created (Unix timestamp) |
| `updated_at` | Integer | When the ticket was last updated (Unix timestamp) |
| `open` | Boolean | Whether the ticket is open |
| `category` | String | Category of the ticket |
| `is_shared` | Boolean | Whether the ticket is shared |
| `admin_assignee_id` | String | ID of the admin assigned to the ticket |
| `team_assignee_id` | String | ID of the team assigned to the ticket |
| `ticket_type` | Object | Type of the ticket |
| `ticket_state` | Object | Current state of the ticket |
| `ticket_attributes` | Object | Attributes and custom fields of the ticket |
| `contacts` | Object | Contacts associated with the ticket |
| `ticket_parts` | Object | Parts (messages, notes, state changes) of the ticket |
| `linked_objects` | Object | Objects linked to this ticket |
Stream for managing conversations between your team and customers.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the conversation |
| `type` | String | Object type identifier |
| `title` | String | Title of the conversation |
| `created_at` | Integer | When the conversation was created (Unix timestamp) |
| `updated_at` | Integer | When the conversation was last updated (Unix timestamp) |
| `open` | Boolean | Whether the conversation is open |
| `state` | String | Current state of the conversation |
| `read` | Boolean | Whether the conversation has been read |
| `priority` | String | Priority of the conversation |
| `waiting_since` | Integer | Timestamp when the conversation started waiting (Unix timestamp) |
| `snoozed_until` | Integer | When the conversation is snoozed until (Unix timestamp) |
| `admin_assignee_id` | String | ID of the admin assigned |
| `team_assignee_id` | String | ID of the team assigned |
| `company_id` | String | ID of the associated company |
| `tags` | Object | Tags associated with the conversation |
| `conversation_rating` | Object | Rating given for this conversation |
| `source` | Object | Source of the conversation (email, chat, etc.) |
| `contacts` | Object | Contacts involved in the conversation |
| `teammates` | Object | Teammates involved in the conversation |
| `custom_attributes` | Array | Array of custom attributes (contains `key`, `value`) |
| `first_contact_reply` | Object | First reply from the contact |
| `sla_applied` | Object | SLA information if applicable |
| `statistics` | Object | Detailed conversation statistics (`time_to_assignment`, `time_to_admin_reply`, `time_to_first_close`, `time_to_last_close`, `median_time_to_reply` (which supports decimal values), and more) |
| `linked_objects` | Object | Objects linked to this conversation |
| `ai_agent_participated` | Boolean | Whether an AI agent participated |
| `ai_agent` | Object | AI agent details if applicable |
Stream providing the individual parts (messages, comments, state changes, etc.) that make up a conversation's history.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the conversation part. |
| `conversation_id` | String | Identifier of the parent conversation. |
| `type` | String | Type classification of the record. |
| `part_type` | String | Type of the part (e.g., comment, note, assignment). |
| `body` | String | Body content of the part. |
| `created_at` | Integer | Timestamp when the part was created (Unix timestamp). |
| `updated_at` | Integer | Timestamp when the part was last updated (Unix timestamp). |
| `notified_at` | Integer | Timestamp when the part notification was sent (Unix timestamp). |
| `assigned_to` | Object | Assignment details containing `type` and `id`. |
| `author` | Object | Author details containing `id`, `type`, `name`, `email`, etc. |
| `attachments` | Array | List of attachments. |
| `external_id` | String | External identifier for the part. |
| `redacted` | Boolean | Indicates if the part content has been redacted. |
| `event_details` | Object | Event details associated with this part, containing `event` (`type`, `result`) and `workflow` (`name`, `group_title`). |
## Data Model
The following diagram illustrates the relationships between the core data streams in Intercom. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Companies("Companies");
Contacts("Contacts");
end
subgraph "Support & Engagement"
Tickets("Tickets");
TicketTypes("Ticket Types");
Conversations("Conversations");
ConversationParts("Conversation Parts");
end
Contacts -- "companies.data[].id" --> Companies;
Tickets -- "company_id" --> Companies;
Tickets -- "contacts.contacts[].id" --> Contacts;
Tickets -- "ticket_type.id" --> TicketTypes;
Conversations -- "company_id" --> Companies;
Conversations -- "contacts.contacts[].id" --> Contacts;
ConversationParts -- "conversation_id" --> Conversations;
```
## Implementation Notes
### API Limitations & Search Behavior
* **Search Payload**: Intercom's `/search` endpoints require a query hash in the request body. When executing a full sync or an initial incremental extraction (where no start date or prior state exists), the connector automatically applies a match-all filter (`> 0` timestamp) to ensure the API accepts the payload and fetches the complete historical data seamlessly.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| A conversations stream returns fewer records than expected | The API search has depth and window limits. | Split the extraction into smaller periods. See [API Limitations & Search Behavior](#api-limitations-search-behavior). |
| Permission error | The token lacks the scopes required by that resource. | Review the application's scopes in Intercom and re-authorize. |
## Skills for agents
Intercom connector documentation as plain markdown, for use in AI agent contexts.
# iRancho as a data source
Source: https://docs.nekt.com/sources/irancho
Bring data from iRancho to your Lakehouse.
iRancho is a Brazilian livestock management software for beef and dairy cattle operations. The connector extracts your herd (animals, weighings, lot and pasture movements, deaths, breeding exposures and handling executions), your farm structure and assets (farms, subdivisions, improvements, machinery and refuelings), animal sales and cattle purchase slips, your finances (accounts, transactions, categories, cost centers and cost allocations), and your inputs, stock and feed (products, stock movements, feed formulas, batches and nutrition). With it you can analyze herd performance, costs and results in your Lakehouse.
## Configuring iRancho as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the iRancho option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
iRancho authenticates with an integration token. To generate it:
1. Log in to iRancho with a user of the account.
2. Open **Configurações** (settings).
3. Under **Token de integração**, click **Gerar token** and confirm.
4. Copy the token.
iRancho warns that the integration token gives other systems access to read and change your account's data. The Nekt connector only reads data. Deleting the token in iRancho (Excluir token) stops the connector from working until a new token is set.
The following configurations are available:
* **Integration Token**: the token generated in iRancho. Required.
* **Start Date**: the earliest date from which dated records are synced: animal movements, deaths, breeding exposures, handling executions, stock movements, financial transactions and cost allocations. Required. Every sync re-reads from this date, because iRancho lets you back-date, edit and delete these records, so an earlier date makes each sync longer. Registers such as animals, farms, products and sales are always synced in full.
* **Requests Per Minute**: how fast the connector may call iRancho. The default is 120.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: every iRancho stream is FULL\_TABLE: each sync replaces the table with what iRancho currently holds. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from iRancho and their corresponding fields. API reference: [iRancho API](https://api.irancho.com.br/API-docs).
Column names follow iRancho's own field names (in Portuguese). Every table also has a `raw_payload` column with the whole record exactly as iRancho returned it, so information without a column of its own is still available. Lists and objects are stored as JSON text. Calendar dates are kept as the date shown in iRancho, and timestamps are converted to UTC.
Every animal of the account, active and inactive.
Primary key: `id_animal`
| Field | Type | Description |
| :- | :- | :- |
| `id_animal` | String | Unique identifier of the animal (UUID). |
| `id_animal_fazenda` | String | Identification of the animal on the farm (the tag number users see). Not unique: it can be reused after the animal is deactivated. |
| `no_animal` | String | Name of the animal. |
| `nome` | String | Name of the animal, as some endpoints spell it. |
| `id_brinco_eletronico` | String | Electronic ear tag (RFID). |
| `id_rastreabilidade` | String | SISBOV traceability identifier. |
| `id_rastreabilidade_manejo` | String | Handling identifier. |
| `id_animal_nascimento` | String | Birth registration number. |
| `id_animal_definitivo` | String | Definitive registration number. |
| `dt_nascimento` | Date | Birth date. |
| `idade_meses` | Number | Age in months. |
| `id_sexo` | Integer | Sex of the animal (1 male, 2 female). |
| `no_sexo` | String | Name of the sex. |
| `id_raca` | Integer | Identifier of the breed. |
| `no_raca` | String | Name of the breed. |
| `id_categoria_animal` | Integer | Identifier of the current animal category. |
| `no_categoria_animal` | String | Name of the current animal category. |
| `id_atividade` | Integer | Current activity (1 cria/breeding, 2 recria/rearing, 3 engorda/fattening). |
| `no_animal_atividade` | String | Name of the current activity. |
| `id_situacao_vaca` | Integer | Reproductive status of the cow. |
| `no_situacao_vaca` | String | Name of the cow's reproductive status. |
| `id_cor_pelagem` | Integer | Identifier of the coat colour. |
| `no_pelagem` | String | Name of the coat colour. |
| `id_padrao_racial` | Integer | Identifier of the breed standard. |
| `no_padrao_racial` | String | Name of the breed standard. |
| `id_tipo_parto` | Integer | Identifier of the type of birth. |
| `no_tipo_parto` | String | Name of the type of birth. |
| `id_tipo_chifre` | Integer | Identifier of the horn type. |
| `no_tipo_chifre` | String | Name of the horn type. |
| `ic_animal_ativo` | Boolean | Whether the animal is active in the herd. |
| `ic_animal_castrado` | Boolean | Whether the animal is castrated. |
| `publico` | Boolean | Whether the animal is flagged as public. |
| `peso` | Number | Last recorded weight, in kilograms. |
| `dt_ultima_pesagem` | Date | Date of the last weighing. |
| `id_fazenda` | Integer | Identifier of the farm the animal is on. |
| `no_fazenda` | String | Name of the farm the animal is on. |
| `id_fazenda_subdivisao` | Integer | Identifier of the subdivision (pasture or pen) the animal is in. |
| `no_subdivisao` | String | Name of the subdivision the animal is in. |
| `dt_ultima_movimentacao` | Datetime | Timestamp of the last move between subdivisions. |
| `id_lote` | Integer | Identifier of the lot the animal belongs to. |
| `no_lote` | String | Name of the lot the animal belongs to. |
| `id_pai` | String | Identifier of the sire, when registered. |
| `pai_id_animal_fazenda` | String | Farm identification of the sire. |
| `pai_no_animal` | String | Name of the sire. |
| `mae_id_animal_fazenda` | String | Farm identification of the dam. |
| `mae_no_animal` | String | Name of the dam. |
| `mae_receptora` | String | Recipient dam, for embryo transfer. |
| `id_venda_animal` | Integer | Identifier of the sale the animal left in, when sold. |
| `dt_morte_animal` | Datetime | Timestamp of the animal's death, when dead. |
| `no_morte_causa` | String | Cause of death, when dead. |
| `sr_sui_abcz` | String | ABCZ series. |
| `id_melhoramento_delta_gen` | String | DeltaGen genetic improvement identifier. |
| `ps_nascimento` | Number | Birth weight, in kilograms. |
| `ps_desmame` | Number | Weaning weight, in kilograms. |
| `dt_desmame` | Date | Weaning date. |
| `dt_compra_animal` | Date | Purchase date, when bought. |
| `vr_compra_animal` | Number | Purchase price, when bought. |
| `dt_venda_animal` | Date | Sale date, when sold. |
| `vr_venda_animal` | Number | Sale price, when sold. |
| `qt_vaca_partos` | Integer | Number of calvings of the cow. |
| `dt_vaca_primeiro_parto` | Date | Date of the cow's first calving. |
| `dt_vaca_ultimo_parto` | Date | Date of the cow's last calving. |
| `detalhes` | String (JSON) | Additional details of the animal: weights, purchase, sale, calvings, origin (JSON object). |
| `dt_criacao` | Datetime | Timestamp the animal was registered in iRancho. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Weighing history of every animal, one row per weighing.
Primary key: `record_key`
| Field | Type | Description |
| :- | :- | :- |
| `record_key` | String | Identifier of the record built by the connector from the fields that identify it (or a hash of the record when iRancho does not send them). |
| `id_pesagem` | String | Identifier of the weighing. |
| `id_animal` | String | Identifier of the weighed animal (UUID). |
| `id_animal_fazenda` | String | Farm identification of the weighed animal. |
| `dt_pesagem` | Datetime | Timestamp of the weighing. |
| `ps_animal` | Number | Weight of the animal, in kilograms. |
| `nu_gmdup` | Number | Average daily gain since the previous weighing, in kilograms per day. |
| `id_tipo_pesagem` | Integer | Identifier of the weighing type. |
| `no_tipo_pesagem` | String | Name of the weighing type. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Entries and exits of animals in lots, one row per movement. Read in date windows by movement date, from **Start Date**.
Primary key: `record_key`
| Field | Type | Description |
| :- | :- | :- |
| `record_key` | String | Identifier of the record built by the connector from the fields that identify it (or a hash of the record when iRancho does not send them). |
| `id_animal` | String | Identifier of the animal (UUID). |
| `id_animal_fazenda` | String | Farm identification of the animal. |
| `id_lote` | Integer | Identifier of the lot. |
| `lote` | String | Name of the lot. |
| `dt_animal_entrada_lote` | Date | Date the animal entered the lot. |
| `dt_animal_saida_lote` | Date | Date the animal left the lot (empty while it is still in it). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Moves of animals between subdivisions (pastures, pens), one row per move. Read in date windows by movement date, from **Start Date**.
Primary key: `record_key`
| Field | Type | Description |
| :- | :- | :- |
| `record_key` | String | Identifier of the record built by the connector from the fields that identify it (or a hash of the record when iRancho does not send them). |
| `id_animal` | String | Identifier of the animal (UUID). |
| `id_animal_fazenda` | String | Farm identification of the animal. |
| `id_fazenda_subdivisao` | Integer | Identifier of the subdivision the animal moved into. |
| `no_subdivisao` | String | Name of the subdivision the animal moved into. |
| `id_fazenda` | Integer | Identifier of the farm of the subdivision. |
| `no_fazenda` | String | Name of the farm of the subdivision. |
| `dt_movimentacao` | Datetime | Timestamp of the move. |
| `dt_saida_subdivisao` | Datetime | Timestamp the animal left the subdivision (empty while it is still in it). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Death records, read in date windows by death date, from **Start Date**.
Primary key: `record_key`
| Field | Type | Description |
| :- | :- | :- |
| `record_key` | String | Identifier of the record built by the connector from the fields that identify it (or a hash of the record when iRancho does not send them). |
| `id_morte` | String | Identifier of the death record. |
| `id_animal` | String | Identifier of the dead animal (UUID). |
| `id_animal_fazenda` | String | Farm identification of the dead animal. |
| `dt_morte_animal` | Datetime | Timestamp of the death. |
| `id_morte_causa` | Integer | Identifier of the cause of death. |
| `no_morte_causa` | String | Name of the cause of death. |
| `id_morte_tipo` | Integer | Identifier of the type of death. |
| `no_morte_tipo` | String | Name of the type of death. |
| `id_morte_destino` | Integer | Identifier of the carcass destination. |
| `id_responsavel` | String | Identifier of the person responsible for the record. |
| `ds_remocao_animal` | String | Description of the carcass removal. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Breeding exposures (natural mating, AI, FTAI, embryo transfer) of every animal whose exposure started on or after **Start Date**, with their pregnancy diagnosis and calving status.
Primary key: `id_exposicao_reprodutiva`
| Field | Type | Description |
| :- | :- | :- |
| `id_exposicao_reprodutiva` | String | Unique identifier of the breeding exposure. |
| `id_tipo_exposicao_reprodutiva` | Integer | Type of exposure (1 natural mating, 2 AI, 3 multiple sires, 4 FTAI, 5 ET, 6 IVF, 7 clone). |
| `no_tipo_exposicao_reprodutiva` | String | Name of the exposure type. |
| `id_animal_situacao_exposicao` | Integer | Status of the exposure (1 planned, 2 done, 3 confirmed, 4 calving due, 5 calving recorded, 6 unproductive). |
| `no_exposicao_situacao` | String | Name of the exposure status. |
| `id_vaca` | String | Identifier of the cow (UUID). |
| `vaca_id_animal_fazenda` | String | Farm identification of the cow. |
| `id_touro` | String | Identifier of the bull (UUID). |
| `touro_id_animal_fazenda` | String | Farm identification of the bull. |
| `doador_id_animal_fazenda` | String | Farm identification of the donor, for embryo transfer. |
| `id_time_touro` | Integer | Identifier of the bull team, for multiple-sire exposures. |
| `no_time_touro` | String | Name of the bull team. |
| `dt_inicio_exposicao` | Date | Start date of the exposure. |
| `dt_termino_exposicao` | Date | End date of the exposure. |
| `dt_inicio_previsao_parto` | Date | Start of the expected calving period. |
| `dt_termino_previsao_parto` | Date | End of the expected calving period. |
| `dt_inicio_janela_parto` | Date | Start of the calving window. |
| `dt_confirmacao_prenhes` | Datetime | Timestamp the pregnancy was confirmed. |
| `dt_realizacao_exame` | Date | Date of the pregnancy diagnosis exam. |
| `vr_resultado_exame` | String | Result of the pregnancy diagnosis exam. |
| `ic_parto_realizado` | Boolean | Whether the calving happened. |
| `ic_exame_diagnostico_realizado` | Boolean | Whether the pregnancy diagnosis was done. |
| `ic_prenhes_confirmada` | Boolean | Whether the pregnancy is confirmed. |
| `ic_apresentou_cio` | Boolean | Whether the cow showed heat. |
| `id_lote` | Integer | Identifier of the lot where the exposure started. |
| `id_safra` | Integer | Identifier of the breeding season. |
| `ds_safra` | String | Name of the breeding season. |
| `nascimentos_reproducao` | String (JSON) | Births resulting from the exposure (JSON array). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Handling (manejo) executions registered in iRancho, one row per lane execution, with the animals handled stored as JSON. Read week by week by execution date, from **Start Date**.
Primary key: `record_key`
| Field | Type | Description |
| :- | :- | :- |
| `record_key` | String | Identifier of the record built by the connector from the fields that identify it (or a hash of the record when iRancho does not send them). |
| `idExecucaoRaia` | Integer | Identifier of the lane execution. |
| `idExecucaoManejo` | Integer | Identifier of the handling execution (groups its lanes). |
| `idExecucaoManejoApp` | String | Identifier of the execution in the iRancho field app. |
| `idManejo` | Integer | Identifier of the handling protocol. |
| `nomeManejo` | String | Name of the handling protocol. |
| `dataExecucao` | Date | Reference date of the execution. |
| `dtInicial` | Date | Start date of the handling execution. |
| `status` | Integer | Status of the lane execution. |
| `qtAnimais` | Integer | Number of animals in the execution. |
| `raia_id` | Integer | Identifier of the lane (raia). |
| `raia_nu_dias` | Integer | Day number of the lane in the protocol. |
| `manejo` | String (JSON) | Execution details: date, animals handled with weights, medications, exams and exits (JSON object). |
| `dtCriacao` | Datetime | Timestamp the execution was created. |
| `dtAlteracao` | Datetime | Timestamp the execution was last changed. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Farms of the account.
Primary key: `id_fazenda`
| Field | Type | Description |
| :- | :- | :- |
| `id_fazenda` | Integer | Unique identifier of the farm. |
| `no_fazenda` | String | Name of the farm. |
| `qt_area_total` | Number | Total area of the farm, in the farm's unit of measure. |
| `qt_area_reserva` | Number | Legal reserve area of the farm, in the farm's unit of measure. |
| `no_unidade_medida` | String | Unit of measure of the areas (square metres or hectares). |
| `qt_subdivisoes` | Integer | Number of subdivisions (pastures, pens) of the farm. |
| `latitude` | Number | Latitude of the farm headquarters. |
| `longitude` | Number | Longitude of the farm headquarters. |
| `no_municipio` | String | Municipality of the farm. |
| `sg_uf` | String | State (UF) of the farm. |
| `ic_sisbov_fazenda` | Boolean | Whether the farm takes part in SISBOV traceability. |
| `ativo` | Boolean | Whether the farm is active. |
| `coordenadas` | String (JSON) | Boundary polygon of the farm drawn in iRancho (JSON object). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Subdivisions of every farm: pastures, pens and paddocks.
Primary key: `id_fazenda_subdivisao`
| Field | Type | Description |
| :- | :- | :- |
| `id_fazenda_subdivisao` | Integer | Unique identifier of the subdivision. |
| `id_fazenda` | Integer | Identifier of the farm the subdivision belongs to. |
| `no_subdivisao` | String | Name of the subdivision. |
| `sg_subdivisao` | String | Short code of the subdivision. |
| `qt_area_subdivisao` | Number | Area of the subdivision, in the farm's unit of measure. |
| `capacidade_max` | Integer | Maximum number of animals the subdivision holds. |
| `id_subdivisao_tipo` | Integer | Type of subdivision (5 = feedlot pen). |
| `id_subdivisao_pastagem` | Integer | Identifier of the pasture type. |
| `id_fazenda_setor` | Integer | Identifier of the farm sector the subdivision is in. |
| `no_fazenda_setor` | String | Name of the farm sector. |
| `id_fazenda_retiro` | Integer | Identifier of the retiro (sub-area) the subdivision is in. |
| `no_fazenda_retiro` | String | Name of the retiro. |
| `ic_ativo` | Boolean | Whether the subdivision is active. |
| `coordenadas` | String (JSON) | Boundary polygon of the subdivision drawn in iRancho (JSON object). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Farm improvements such as buildings, fences and corrals.
Primary key: `id_benfeitoria`
| Field | Type | Description |
| :- | :- | :- |
| `id_benfeitoria` | Integer | Unique identifier of the improvement. |
| `no_benfeitoria` | String | Name of the improvement. |
| `no_benfeitoria_tipo` | String | Type of improvement. |
| `id_fazenda` | Integer | Identifier of the farm the improvement is on. |
| `aa_benfeitoria` | Integer | Year the improvement was built. |
| `vida_util` | Integer | Useful life of the improvement, in years. |
| `vr_benfeitoria` | Number | Value of the improvement, in BRL. |
| `porcentagem_atividade` | Number | Share of the improvement used by the activity, in percent. |
| `observacao` | String | Notes about the improvement. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Machinery and vehicles, including deactivated ones.
Primary key: `id_maquinario`
| Field | Type | Description |
| :- | :- | :- |
| `id_maquinario` | Integer | Unique identifier of the machine. |
| `no_maquinario` | String | Name of the machine. |
| `id_maquinario_tipo` | Integer | Identifier of the machine type. |
| `no_maquinario_tipo` | String | Name of the machine type. |
| `id_marca` | Integer | Identifier of the brand. |
| `no_maquinario_marca` | String | Name of the brand. |
| `id_modelo` | String | Model of the machine (free text, despite the name). |
| `aa_fabricacao_maquinario` | Integer | Year of manufacture. |
| `aa_compra_maquinario` | Integer | Year of purchase. |
| `vr_compra_maquinario` | Number | Purchase value, in BRL. |
| `co_placa_maquinario` | String | Licence plate or internal code. |
| `id_fazenda` | Integer | Identifier of the farm the machine belongs to. |
| `ic_combustivel_tipo` | Integer | Identifier of the fuel type (product subcategory). |
| `no_combustivel` | String | Name of the fuel type. |
| `ic_unidade_trabalho` | String | Unit the usage is measured in (km, hora, nenhum). |
| `vida_util` | Integer | Useful life, in years. |
| `porcentagem_atividade` | Number | Share of the machine used by the activity, in percent. |
| `ic_estado_maquinario` | Boolean | Whether the machine is active. |
| `dt_desativacao` | Date | Date the machine was deactivated. |
| `observacao` | String | Notes about the machine. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Fuel fills of every machine.
Primary key: `id_maquinario_abastecimento`
| Field | Type | Description |
| :- | :- | :- |
| `id_maquinario_abastecimento` | Integer | Unique identifier of the refueling. |
| `id_maquinario` | Integer | Identifier of the machine refueled. |
| `no_maquinario` | String | Name of the machine refueled. |
| `dt_abastecimento` | Date | Date of the refueling. |
| `qt_abastecimento` | Number | Quantity of fuel, in litres. |
| `vr_unidade_abastecimento` | Number | Price per litre, in BRL. |
| `nu_odometro` | Number | Odometer or hour-meter reading at the refueling. |
| `ic_abasteceu_na_fazenda` | Boolean | Whether the fuel came from the farm's own stock. |
| `id_combustivel_tipo` | Integer | Identifier of the fuel type (product subcategory). |
| `no_combustivel` | String | Name of the fuel product. |
| `no_responsavel` | String | Person responsible for the refueling. |
| `no_estoque_origem` | String | Stock the fuel was taken from, when refueled at the farm. |
| `id_fazenda` | Integer | Identifier of the farm, when refueled at the farm. |
| `de_operacao_abastecimento` | String | Description of the operation. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Contacts: suppliers, buyers, producers and employees.
Primary key: `id_pessoa`
| Field | Type | Description |
| :- | :- | :- |
| `id_pessoa` | String | Unique identifier of the contact (UUID). |
| `nome` | String | Name of the contact. |
| `sobrenome` | String | Surname of the contact. |
| `no_completo` | String | Full name of the contact. |
| `email` | String | Email address of the contact. |
| `nu_cpf_cnpj` | String | CPF or CNPJ of the contact. |
| `tel_comercial` | String | Business phone. |
| `tel_residencial` | String | Home phone. |
| `tipo_pessoa` | String | Individual (pessoa\_fisica) or company (pessoa\_juridica). |
| `fk_id_municipio` | Integer | Identifier of the municipality (IBGE). |
| `fk_id_estado` | Integer | Identifier of the state (IBGE). |
| `fazendas` | String (JSON) | Third-party farms registered for the contact (JSON array). |
| `categorias` | String (JSON) | Categories of the contact: supplier, producer, customer, employee... (JSON array). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Categories contacts are grouped in.
Primary key: `id_contato_categoria`
| Field | Type | Description |
| :- | :- | :- |
| `id_contato_categoria` | Integer | Unique identifier of the category (1 supplier, 2 producer, 3 customer, 4 employee, 5 partner, 6 other, 7 manufacturer). |
| `de_categoria` | String | Name of the category. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Animal sales, all of them on every sync.
Primary key: `id_venda_animal`
| Field | Type | Description |
| :- | :- | :- |
| `id_venda_animal` | Integer | Unique identifier of the sale. |
| `dt_venda` | Date | Date of the sale. |
| `qt_animal` | Integer | Number of animals sold. |
| `peso_medio_animal` | Number | Average live weight of the animals sold, in kilograms. |
| `peso_total_animal` | Number | Total live weight of the animals sold, in kilograms. |
| `pct_rend_carcaca_total` | Number | Carcass yield, in percent. |
| `vr_venda_total` | Number | Total value of the sale, in BRL. |
| `fk_id_fazenda` | Integer | Identifier of the farm the animals left from. |
| `no_fazenda_origem` | String | Name of the farm the animals left from. |
| `id_pessoa` | String | Identifier of the buyer (a contact). |
| `no_destino` | String | Name of the buyer. |
| `no_responsavel` | String | Name of the user responsible for the sale. |
| `qt_macho` | Integer | Number of males sold. |
| `peso_medio_macho_kg` | Number | Average live weight of the males sold, in kilograms. |
| `peso_total_macho_kg` | Number | Total live weight of the males sold, in kilograms. |
| `vr_medio_macho` | Number | Average value per male sold, in BRL. |
| `qt_femea` | Integer | Number of females sold. |
| `peso_medio_femea_kg` | Number | Average live weight of the females sold, in kilograms. |
| `peso_total_femea_kg` | Number | Total live weight of the females sold, in kilograms. |
| `vr_medio_femea` | Number | Average value per female sold, in BRL. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Animals sold in each sale, one row per animal, read from each sale's detail.
Primary key: `id_venda_animal`, `id_animal`
| Field | Type | Description |
| :- | :- | :- |
| `id_venda_animal` | Integer | Identifier of the sale. |
| `id_animal` | String | Identifier of the animal sold (UUID). |
| `id_animal_fazenda` | String | Farm identification of the animal sold. |
| `dt_venda` | Date | Date of the sale. |
| `tipo_venda` | String | How the sale was priced (Peso Vivo = live weight, Unidade = per head). |
| `id_pessoa` | String | Identifier of the buyer (a contact). |
| `fk_id_fazenda` | Integer | Identifier of the farm the animal left from. |
| `id_produtor_fazenda` | Integer | Destination producer farm (empty when sold to a slaughterhouse). |
| `ps_morto_frigorifico` | Number | Carcass weight at the slaughterhouse for the sale, in kilograms. |
| `pct_rend_carcaca_total` | Number | Carcass yield of the sale, in percent. |
| `ic_status_venda_sisbov` | Boolean | Whether the sale is a SISBOV sale. |
| `vr_venda_animal` | Number | Sale value of this animal, in BRL. |
| `ps_frigorifico` | Number | Carcass weight of this animal at the slaughterhouse, in kilograms. |
| `pct_rend_carcaca` | Number | Carcass yield of this animal, in percent. |
| `no_lote` | String | Lot the animal was in. |
| `no_sexo` | String | Sex of the animal. |
| `no_raca` | String | Breed of the animal. |
| `id_venda_animal_gta` | Integer | Identifier of the transit document (GTA) the animal left with. |
| `nu_gta` | String | Number of the GTA. |
| `nu_serie_gta` | String | Series of the GTA. |
| `no_carreta` | String | Truck the animal left in. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Cattle purchase slips (borderôs de compra).
Primary key: `id_bordero_digital`
| Field | Type | Description |
| :- | :- | :- |
| `id_bordero_digital` | Integer | Unique identifier of the purchase slip. |
| `dt_compra` | Date | Purchase date. |
| `dt_pagamento` | Date | Payment date. |
| `situacao` | String | Status of the purchase slip. |
| `fazenda_origem` | String | Farm the animals were bought from. |
| `fazenda_destino` | String | Farm the animals were taken to. |
| `fornecedor` | String | Supplier of the animals. |
| `responsavel` | String | User responsible for the purchase slip. |
| `qt_animais` | Integer | Number of animals bought. |
| `ps_medio` | Number | Average weight of the animals, in kilograms. |
| `ps_total` | Number | Total weight of the animals, in kilograms. |
| `vr_medio` | Number | Average value per animal, in BRL. |
| `vr_arroba` | Number | Price per arroba (15 kg), in BRL. |
| `vr_total` | Number | Total value of the purchase, in BRL. |
| `observacoes` | String | Notes. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Financial accounts: cash, checking, savings, investment and credit card.
Primary key: `id_conta`
| Field | Type | Description |
| :- | :- | :- |
| `id_conta` | Integer | Unique identifier of the account. |
| `no_fin_conta` | String | Name of the account. |
| `nu_conta` | String | Bank account number. |
| `nu_conta_agencia` | String | Bank branch number. |
| `saldo_inicial` | Number | Opening balance of the account, in BRL. |
| `ic_conta_principal` | Boolean | Whether this is the main account. |
| `ic_lock_transacao` | Integer | Edit lock state of the account's transactions. |
| `id_fin_tipo_conta` | Integer | Account type (1 cash, 2 checking, 3 savings, 4 investment, 5 credit card). |
| `no_fin_tipo_conta` | String | Name of the account type. |
| `id_banco` | Integer | Identifier of the bank (empty for cash accounts). |
| `no_banco` | String | Name of the bank. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Financial transactions of every account, read month by month from **Start Date** to 12 months ahead (future installments already exist). One row per installment, recurrence occurrence and account side of a transfer. Forecasts of recurrences that were not recorded yet are not extracted.
Primary key: `id_conta`, `id_transacao`
| Field | Type | Description |
| :- | :- | :- |
| `id_conta` | Integer | Identifier of the account the transaction is in. |
| `id_transacao` | String | Identifier of the transaction (a new one is generated when the transaction is edited). |
| `dt_transacao` | Date | Date of the transaction or installment (its due date for future ones). |
| `no_transacao_apelido` | String | Name of the transaction. |
| `vr_transacao` | Number | Value of this transaction or installment, in BRL (always positive; see ic\_receita\_despesa). |
| `ic_receita_despesa` | String | R for revenue, D for expense. |
| `no_tipo_pagamento` | String | Payment type: A\_VISTA, PARCELADO, RECORRENTE or TRANSFERENCIA. |
| `ic_liquidado` | Boolean | Whether the transaction is settled (paid or received). |
| `id_transacao_conta_dia` | Integer | Order of the transaction within the account and day. |
| `id_categoria_transacao` | Integer | Identifier of the transaction category. |
| `no_categoria_transacao` | String | Name of the transaction category. |
| `id_subcategoria_transacao` | Integer | Identifier of the transaction subcategory. |
| `no_subcategoria_transacao` | String | Name of the transaction subcategory. |
| `id_centro_custo` | Integer | Identifier of the cost center. |
| `id_fin_atividade` | Integer | Identifier of the activity (within the cost center). |
| `id_fazenda` | Integer | Identifier of the farm the transaction is assigned to. |
| `id_pessoa` | String | Identifier of the contact (payer or payee). |
| `nu_documento_transacao` | String | Document or bank statement code of the transaction. |
| `fk_id_transacao` | String | Identifier of the originating transaction (first installment or origin side of a transfer). |
| `id_recorrencia` | Integer | Identifier of the recurrence rule, for recurring transactions. |
| `id_conta_destino` | Integer | Destination account, for transfers. |
| `vr_transacao_total` | Number | Total value of the transaction across installments, in BRL. |
| `ic_custo` | Boolean | Whether the transaction has costs allocated to animals. |
| `qtd_custos` | Integer | Number of animals the transaction's cost is allocated to. |
| `de_observacao` | String | Notes about the transaction. |
| `de_destinacao` | String | Destination (purpose) text of the transaction. |
| `tags` | String (JSON) | Tags of the transaction (JSON array). |
| `documentos_fiscais` | String (JSON) | Fiscal documents (invoices) of the transaction (JSON array). |
| `parcelas` | String (JSON) | Other installments of the transaction, on the first installment (JSON array). |
| `item` | String (JSON) | Line items of the transaction (JSON array). |
| `fin_recorrencia` | String (JSON) | Recurrence rule, for recurring transactions (JSON object). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Transaction categories.
Primary key: `id_categoria_transacao`
| Field | Type | Description |
| :- | :- | :- |
| `id_categoria_transacao` | Integer | Unique identifier of the category. |
| `no_categoria_transacao` | String | Name of the category (default categories come in Portuguese). |
| `ic_receita_despesa` | String | R for revenue categories, D for expense categories. |
| `id_assinatura` | Integer | Subscription that created the category (0 = iRancho default). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Transaction subcategories.
Primary key: `id_subcategoria_transacao`
| Field | Type | Description |
| :- | :- | :- |
| `id_subcategoria_transacao` | Integer | Unique identifier of the subcategory. |
| `no_subcategoria_transacao` | String | Name of the subcategory. |
| `de_subcategoria_transacao` | String | Description of the subcategory. |
| `id_categoria_transacao` | Integer | Identifier of the category the subcategory belongs to. |
| `id_assinatura` | Integer | Subscription that created the subcategory (0 = iRancho default). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Cost centers.
Primary key: `id_centro_custo`
| Field | Type | Description |
| :- | :- | :- |
| `id_centro_custo` | Integer | Unique identifier of the cost center. |
| `no_centro_custo` | String | Name of the cost center. |
| `de_centro_custo` | String | Description of the cost center. |
| `id_assinatura` | Integer | Subscription that created the cost center (0 = iRancho default). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Activities within cost centers.
Primary key: `id_fin_atividade`
| Field | Type | Description |
| :- | :- | :- |
| `id_fin_atividade` | Integer | Unique identifier of the activity. |
| `no_fin_atividade` | String | Name of the activity. |
| `de_fin_atividade` | String | Description of the activity. |
| `id_fin_centro_custo` | Integer | Identifier of the cost center the activity belongs to. |
| `id_assinatura` | Integer | Subscription that created the activity (0 = iRancho default). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Tags (markers) of financial transactions.
Primary key: `id_tag`
| Field | Type | Description |
| :- | :- | :- |
| `id_tag` | Integer | Unique identifier of the tag. |
| `no_tag` | String | Name of the tag. |
| `no_tag_cor` | String | Colour of the tag (hex code without '#'). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Types of cost allocation.
Primary key: `id_tp_custo`
| Field | Type | Description |
| :- | :- | :- |
| `id_tp_custo` | Integer | Unique identifier of the cost type. |
| `tp_custo` | String | Name of the cost type. |
| `id_assinatura` | Integer | Subscription that created the cost type (0 = iRancho default). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Costs allocated to animals by accrual month, read month by month from **Start Date**. A recurring allocation appears once per month.
Primary key: `id_fin_competencia`, `dt_alocacao_custo`
| Field | Type | Description |
| :- | :- | :- |
| `id_fin_competencia` | Integer | Identifier of the cost allocation. |
| `dt_alocacao_custo` | Date | Accrual date of the allocation. |
| `de_custo` | String | Description of the cost. |
| `vr_custo_a_ratear` | Number | Amount apportioned among the animals, in BRL. |
| `id_tp_custo` | Integer | Identifier of the cost type. |
| `tp_custo` | String | Name of the cost type. |
| `custo_recorrencia` | String (JSON) | Recurrence rule, for recurring allocations (JSON object). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Products and inputs, active and inactive, with their current stock.
Primary key: `id_produto`
| Field | Type | Description |
| :- | :- | :- |
| `id_produto` | String | Unique identifier of the product. |
| `nome` | String | Name of the product. |
| `id_produto_assinatura` | String | Product code defined by the user ('ID único' in iRancho). |
| `ic_produto_ativo` | Boolean | Whether the product is active. |
| `id_produto_categoria` | Integer | Identifier of the product category. |
| `no_produto_categoria` | String | Name of the product category. |
| `fk_id_produto_subcategoria` | Integer | Identifier of the product subcategory. |
| `fk_id_produto_fabricante` | String | Identifier of the manufacturer (a contact). |
| `no_fabricante` | String | Name of the manufacturer. |
| `id_unidade_base` | Integer | Identifier of the base unit of measure. |
| `no_unidade_base` | String | Name of the base unit of measure. |
| `sg_unidade_base` | String | Symbol of the base unit of measure. |
| `qt_estoque` | Number | Current stock across all locations, in the base unit. |
| `qt_minima_produto` | Number | Minimum stock alert level, in the base unit. |
| `qt_maxima_produto` | Number | Maximum stock alert level, in the base unit. |
| `pct_proteina_bruta` | Number | Crude protein, in percent (feed inputs). |
| `pct_nutrientes_digestiveis_totais` | Number | Total digestible nutrients, in percent (feed inputs). |
| `pct_materia_seca` | Number | Dry matter, in percent (feed inputs). |
| `qt_dia_carencia_abate` | Integer | Slaughter withdrawal period, in days (pharmacy products). |
| `ic_produzido_fazenda` | Boolean | Whether the product is produced on the farm. |
| `ic_usado_pasto` | Boolean | Whether the product is used on pasture. |
| `ic_monitorar_produto` | Boolean | Whether the product's stock is monitored. |
| `ic_especifico_sexo` | Boolean | Whether the product applies to one sex only. |
| `publico` | Boolean | Whether the product comes from iRancho's shared catalogue. |
| `embalagens` | String (JSON) | Packagings the product can be moved in (JSON array). |
| `saldos` | String (JSON) | Stock lots per location, with FIFO unit cost (JSON array). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Stock locations (warehouses) of each farm.
Primary key: `id_produto_estoque`
| Field | Type | Description |
| :- | :- | :- |
| `id_produto_estoque` | Integer | Unique identifier of the stock location. |
| `no_produto_estoque` | String | Name of the stock location. |
| `id_fazenda` | Integer | Identifier of the farm the stock location belongs to. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Stock movements of every product (entries, exits, internal consumption, losses, sales, transfers), read month by month by movement date, from **Start Date**.
Primary key: `id_movimentacao_produto`
| Field | Type | Description |
| :- | :- | :- |
| `id_movimentacao_produto` | Integer | Unique identifier of the stock movement. |
| `id_produto_sistema` | String | Identifier of the product moved (see products.id\_produto). |
| `id_produto` | String | Product code shown as 'ID único' in iRancho (not the product identifier). |
| `produto_nome` | String | Name of the product moved. |
| `categoria` | String | Category of the product. |
| `subcategoria` | String | Subcategory of the product. |
| `dt_movimentacao` | Date | Date of the movement. |
| `id_tipo_movimentacao_produto` | Integer | Movement type (1 exit, 2 entry, 3 internal consumption, 4 loan, 5 loss, 6 sale, 7 stock transfer, 8 purchase). |
| `ds_tipo_movimentacao` | String | Name of the movement type. |
| `qt_unidade` | Number | Quantity moved, in the unit used. |
| `unidade` | String | Unit the quantity was moved in. |
| `qt_movimentacao_un_base` | Number | Quantity moved, in the product's base unit. |
| `vr_produto_unidade_base` | Number | Unit value in the base unit, in BRL. |
| `vr_frete` | Number | Freight value, in BRL. |
| `nome_estoque_origem` | String | Stock location the product left from. |
| `destino` | String | Destination of the movement. |
| `no_pessoa` | String | Supplier or contact of the movement. |
| `nu_nota_fiscal` | String | Invoice number. |
| `dt_validade_produto` | Date | Expiry date of the product moved. |
| `manejo` | String | Handling (manejo) that generated the movement, when any. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Feed formulas (rations).
Primary key: `id_produto_formula`
| Field | Type | Description |
| :- | :- | :- |
| `id_produto_formula` | String | Unique identifier of the formula (the identifier of the formula's product). |
| `nome` | String | Name of the formula. |
| `id_unidade_base` | Integer | Identifier of the formula's base unit of measure. |
| `sg_unidade_base` | String | Symbol of the formula's base unit of measure. |
| `pct_materia_seca` | Number | Dry matter of the formula, in percent. |
| `pct_peso_vivo` | Number | Planned intake as a share of live weight, in percent. |
| `nu_gdp` | Number | Expected daily weight gain. |
| `ic_arredondar` | String | Rounding mode of quantities (N none, C up, B down). |
| `qt_casa_decimal` | Integer | Decimal places used when rounding. |
| `qt_batidas` | Integer | Number of batches mixed with the formula. |
| `composicao_formula` | String (JSON) | Ingredients of the formula and their share (JSON array). |
| `dt_criacao` | Datetime | Timestamp the formula was created. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Feed batches mixed from formulas.
Primary key: `id_batida`
| Field | Type | Description |
| :- | :- | :- |
| `id_batida` | Integer | Unique identifier of the batch. |
| `dt_batida` | Datetime | Timestamp the batch was mixed. |
| `id_produto_formula` | String | Identifier of the formula mixed. |
| `no_formula` | String | Name of the formula mixed. |
| `qt_planejada` | Number | Planned quantity. |
| `qt_produzida` | Number | Quantity actually produced. |
| `pct_materia_seca` | Number | Dry matter of the batch, in percent. |
| `pct_proteina_bruta` | Number | Crude protein of the batch, in percent. |
| `pct_nutrientes_digestiveis_totais` | Number | Total digestible nutrients of the batch, in percent. |
| `id_pessoa_planejamento` | String | Identifier of the person who planned the batch. |
| `id_pessoa_producao` | String | Identifier of the person who mixed the batch. |
| `no_responsavel` | String | Name of the person responsible. |
| `id_maquinario` | Integer | Identifier of the mixer used. |
| `movimentacoes_batida` | String (JSON) | Stock movements generated by the batch, with unit costs (JSON array). |
| `baixas_da_batida` | String (JSON) | Ingredients consumed by the batch (JSON array). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Feed supplied to each subdivision per day.
Primary key: `id_nutricao`
| Field | Type | Description |
| :- | :- | :- |
| `id_nutricao` | Integer | Unique identifier of the nutrition entry. |
| `dt_nutricao` | Date | Date the feed was supplied. |
| `id_fazenda_subdivisao` | Integer | Identifier of the subdivision (pasture or pen) fed. |
| `no_subdivisao` | String | Name of the subdivision fed. |
| `id_fazenda` | Integer | Identifier of the farm. |
| `id_produto` | String | Identifier of the product supplied to the adults. |
| `no_produto` | String | Name of the product supplied to the adults. |
| `qt_produto` | Number | Quantity supplied to the adults. |
| `id_unidade_medida_produto` | Integer | Unit of the quantity supplied. |
| `id_produto_creep` | String | Identifier of the creep-feed product supplied to the calves. |
| `no_produto_creep` | String | Name of the creep-feed product. |
| `qt_produto_creep` | Number | Quantity of creep feed supplied. |
| `id_unidade_medida_creep` | Integer | Unit of the creep-feed quantity. |
| `qt_animais` | Integer | Number of animals in the subdivision that day. |
| `qt_bezerros` | Integer | Number of calves in the subdivision that day. |
| `tp_sobra` | String | Leftover level (nada, pouco, medio, muito). |
| `qt_sobra` | Number | Leftover quantity. |
| `tp_sobra_creep` | String | Creep-feed leftover level. |
| `qt_sobra_creep` | Number | Creep-feed leftover quantity. |
| `id_unidade_medida_sobra` | Integer | Unit of the leftover quantity. |
| `id_salgador` | String | Identifier of the person who supplied the feed. |
| `no_salgador` | String | Name of the person who supplied the feed. |
| `ds_observacoes` | String | Notes. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Lots (batches) animals are grouped in.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the lot (lote) animals are grouped in. |
| `name` | String | Name of the lot (lote) animals are grouped in. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Cattle breeds.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the breed (raça). |
| `name` | String | Name of the breed (raça). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Animal categories (calf, steer, bull, heifer, cow).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the animal category (1 bezerro, 2 garrote, 3 touro, 4 boi, 5 bezerra, 6 novilha, 7 vaca). |
| `name` | String | Name of the animal category (1 bezerro, 2 garrote, 3 touro, 4 boi, 5 bezerra, 6 novilha, 7 vaca). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Animal sexes.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the sex (1 male, 2 female). |
| `name` | String | Name of the sex (1 male, 2 female). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Types of weighing.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the weighing type. |
| `name` | String | Name of the weighing type. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Product categories.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the product category (1 Farmácia, 2 Nutricionais, 3 Combustíveis, 4 Ferramentas, 5 Peças, 6 Agrícolas, 7 EPIs, 8 Outros Insumos, 9 Lubrificantes). |
| `name` | String | Name of the product category (1 Farmácia, 2 Nutricionais, 3 Combustíveis, 4 Ferramentas, 5 Peças, 6 Agrícolas, 7 EPIs, 8 Outros Insumos, 9 Lubrificantes). |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Product subcategories, with their category.
Primary key: `id`, `id_produto_categoria`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the product subcategory. |
| `name` | String | Name of the product subcategory. |
| `id_produto_categoria` | Integer | Identifier of the product category the subcategory belongs to. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
Units of measure.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the unit of measure. |
| `name` | String | Name of the unit of measure. |
| `raw_payload` | String (JSON) | The whole record exactly as iRancho returned it (JSON object), including fields that have no column of their own. |
# Irroba as a data source
Source: https://docs.nekt.com/sources/irroba
Bring data from your Irroba e-commerce store to your Lakehouse.
Irroba is a Brazilian e-commerce platform for building and running online stores, with catalog, checkout, payments, shipping and marketplace integrations in one place. Its REST API exposes the store's orders, customers, products, categories, coupons and invoices.
![]()
## Configuring Irroba as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Irroba option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Irroba authenticates integrations with the credentials of an **API user** of the store. To create one:
1. Log in to your Irroba store admin panel.
2. Open **APS**, then **Usuários**, then **Usuário API**, and click to add a new API user.
3. Save it and copy the generated **username** and **password**.
Paste both into the connector. The connector exchanges them for a short-lived access token on every run and renews it automatically; the password is never sent with data requests.
Create a dedicated API user for Nekt. Irroba keeps one active token per API user, so sharing a user with another integration makes both of them log in again and again.
The following configurations are also available:
* **Start date**: The earliest date from which orders, customers and invoices will be synced (store local time). Catalog tables such as products and categories are always read in full. If not provided, the connector starts at 2015-01-01.
* **Requests per minute**: The request quota of your Irroba plan: Bronze 30, Silver 60, Gold 90, Premium 120. The connector paces itself to never exceed it. The default of 30 works for every plan; raise it to match your plan and speed up the first sync.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
| Stream | Slug | Replication | Description |
| - | - | - | - |
| Orders | `orders` | Incremental | Orders with customer, payment, shipping, totals and current status; nested details as JSON columns. |
| Order Products | `order_products` | Incremental | One row per order line item (product, quantity, prices, options). |
| Order History | `order_history` | Incremental | One row per status change of an order. |
| Customers | `customers` | Incremental | Registered customers with group and addresses. |
| Invoices | `invoices` | Incremental | Invoices (NF-e) issued through Irroba. |
| Products | `products` | Full table | Product catalog with prices, descriptions, categories, promotions and stock options. |
| Categories | `categories` | Full table | Product categories and hierarchy. |
| Manufacturers | `manufacturers` | Full table | Manufacturers and brands. |
| Options | `options` | Full table | Stock option definitions (size, color) and their values. |
| Coupons | `coupons` | Full table | Discount coupons and their rules. |
| Banners | `banners` | Full table | Storefront banners. |
| Affiliates | `affiliates` | Full table | Affiliate partners. |
| Customer Groups | `customer_groups` | Full table | Customer groups of the store. |
| Abandoned Carts | `abandoned_carts` | Full table | Carts left behind and their products. |
| Order Statuses | `order_statuses` | Full table | Reference table of order status ids and names. |
## Fields by stream
Below you'll find all available data streams from Irroba and their corresponding fields:
One row per order with customer, payment, shipping, totals and the current status. Line items, status history, totals breakdown, coupon and invoices travel as JSON columns and are also exposed row by row in the Order Products and Order History streams.
**Slug:** `orders`\
**Primary key:** `order_id`\
**Replication:** Incremental (`date_modified`)
**Fields:**
* `order_id` - Unique identifier of the order
* `marketplace_order_id` - Identifier of the order inside the marketplace it came from, when applicable
* `marketplace` - Name of the marketplace the order came from (empty for the store's own site)
* `store_url` - URL of the store that received the order
* `customer_id` - Identifier of the customer who placed the order
* `customer_group_id` - Identifier of the customer's group (8 is the default group)
* `customer_group_name` - Name of the customer's group
* `return_id` - Identifier of the exchange/return request linked to the order, 0 when none
* `return_details` - Exchange/return details: return\_id, return\_reason, tracking code and link (JSON object)
* `nsu` - Unique sequential number (NSU) generated by the Rede card acquirer, when available
* `cv` - Transaction identifier generated by the payment acquirer
* `type_person` - Whether the buyer is an individual (F) or a company (J)
* `firstname` - Buyer's first name (full name or company name)
* `lastname` - Buyer's last name (trade name for companies)
* `email` - Buyer's e-mail address
* `telephone` - Buyer's landline phone, formatting may vary
* `cellphone` - Buyer's mobile phone, formatting may vary
* `cpf_cnpj` - Buyer's CPF or CNPJ (Brazilian tax id), formatting may vary
* `rg_ie` - Buyer's RG or state registration (IE), formatting may vary
* `customer_sex` - Buyer's gender as registered (M/F)
* `cnpj_contact` - Contact person name for company buyers
* `customer_custom_field` - Custom fields filled by the customer at checkout (JSON object)
* `number_parcel` - Number of installments of the payment
* `payment_firstname` - Name of the person responsible for the payment
* `payment_address_1` - Billing street address
* `payment_number_home` - Billing address number
* `payment_neighborhood` - Billing address neighborhood
* `payment_address_2` - Billing address complement
* `payment_city` - Billing city
* `payment_postcode` - Billing postal code (CEP), formatting may vary
* `payment_country` - Billing country name
* `payment_zone` - Billing state name
* `payment_zone_id` - Identifier of the billing state (Irroba zone id, 440-466 for Brazilian states)
* `payment_method` - Display name of the payment method (not a stable identifier)
* `payment_code` - Code of the payment method, the stable identifier (e.g. paypal, pagseguro\_billet)
* `payment_condition` - Payment condition, only for invoice/billing payments
* `payment_details` - Gateway details: NSU, transaction\_id, authorization\_code, boleto line/number/due date (JSON object)
* `shipping_firstname` - First name of the person receiving the order
* `shipping_lastname` - Last name of the person receiving the order
* `shipping_address_1` - Shipping street address
* `shipping_number_home` - Shipping address number
* `shipping_neighborhood` - Shipping address neighborhood
* `shipping_address_2` - Shipping address complement
* `shipping_city` - Shipping city
* `shipping_postcode` - Shipping postal code (CEP), formatting may vary
* `shipping_country` - Shipping country name
* `shipping_zone` - Shipping state name
* `shipping_zone_id` - Identifier of the shipping state (Irroba zone id)
* `shipping_custom_field` - Custom field of the shipping address (JSON string)
* `shipping_method` - Display name of the shipping method (see Irroba's shipping methods table)
* `shipping_code` - Code of the shipping method, the stable identifier (e.g. sedex.sedex)
* `shipping_delivery_time` - Estimated delivery time in days
* `melhor_envio_service` - Melhor Envio service id, only when shipped through Melhor Envio
* `smart_envios_service` - SmartEnvios service id, only when shipped through SmartEnvios
* `data_frete_id_table` - Data Frete table id, only when shipped through Data Frete
* `intelipost_delivery_id` - Intelipost delivery method id, only when shipped through Intelipost
* `comment` - Comment left by the customer on the order
* `total` - Total amount of the order
* `interest` - Interest amount charged on the order
* `order_status_id` - Identifier of the current status of the order (see the order\_statuses stream)
* `order_status_name` - Name of the current status, taken from the matching status history entry
* `tracking` - Shipping tracking code, when the order has been shipped
* `code_tracking` - Tracking code as returned by the details endpoint
* `date_added` - When the order was placed (store local time, no timezone)
* `date_modified` - When the order was last changed (store local time); falls back to date\_added when never modified. Used as the incremental replication key
* `attacked_status` - Whether the order was closed under wholesale (atacado) pricing
* `pickup_address` - Store pickup address, when the order is picked up in store
* `user_seller` - Name of the seller responsible for the order
* `user_seller_id` - Identifier of the seller responsible for the order
* `external_reference` - External reference, typically the order id in the store's ERP
* `gift` - Gift (brinde) attached to the order: product id, name, sku, price, quantity (JSON object)
* `is_mirrored` - Whether the order is mirrored from another store
* `affiliate_id` - Identifier of the affiliate responsible for the order, when any
* `affiliate_firstname` - First name of the affiliate responsible for the order
* `affiliate_lastname` - Last name of the affiliate responsible for the order
* `intermediary_name` - Name of the payment intermediary
* `intermediary_cnpj` - CNPJ of the payment intermediary
* `intermediary_operation_name` - Name of the operation intermediary
* `intermediary_operation_cnpj` - CNPJ of the operation intermediary
* `order_history` - Full status history of the order, last entry is the current status (JSON array); also available row by row in the order\_history stream
* `order_product` - Line items of the order with options, dimensions and attributes (JSON array); also available row by row in the order\_products stream
* `order_total` - Order totals breakdown: sub\_total, shipping, discounts, interest, total (JSON array of code/title/value)
* `order_coupon` - Coupon applied to the order: type, models, gift model, discount (JSON object)
* `invoices` - Invoices (NF-e) linked to the order: quantity and items with number, series, key, status, XML/DANFE links (JSON object)
* `nfe_details` - Legacy invoice structure kept by Irroba during the transition to 'invoices' (JSON); scheduled for removal on 2026-09-15
One row per line item of an order (product, quantity, prices, selected options, dimensions), derived from the Orders stream without extra requests.
**Slug:** `order_products`\
**Primary key:** `order_id`, `order_product_id`\
**Replication:** Incremental (`order_date_modified`)
**Fields:**
* `order_id` - Identifier of the order this line item belongs to
* `order_product_id` - Identifier of the line item inside the order
* `product_id` - Identifier of the product sold
* `name` - Product name at the time of the sale
* `model` - Product code (model/SKU) at the time of the sale
* `quantity` - Quantity purchased
* `price` - Unit price at the time of the sale
* `price_attacked` - Unit wholesale (atacado) price at the time of the sale
* `total` - Line total (unit price times quantity)
* `total_attacked` - Line total at wholesale price
* `present` - Whether the item was marked as a gift (1) or not (0)
* `present_value` - Gift-wrapping value charged for the item
* `shipping` - Shipping flag of the item as returned by Irroba
* `shipping_date` - Shipping date of the item, when shipped separately (store local time)
* `shipping_method` - Shipping method of the item, when shipped separately
* `shipping_value` - Shipping value of the item, when shipped separately
* `delivery_method_id` - Delivery method identifier of the item, when shipped separately
* `seller_id` - Identifier of the seller of the item (marketplace/multi-seller stores)
* `status_id` - Status identifier of the item, when tracked per item
* `manufacturer_id` - Identifier of the product's manufacturer/brand
* `date_modified` - When the line item was last changed (store local time)
* `link_xml_nfe` - Link to the NF-e XML of the item, when issued per item
* `chave_nota` - NF-e access key of the item, when issued per item
* `free_product_coupon` - Whether the item was given for free through a coupon
* `options_summary` - Selected product options as 'Option: value' pairs separated by '; ' (e.g. 'Tamanho: 42; Cor: Preto')
* `options` - Selected product options with the stock option values (JSON array)
* `weight_class_id` - Weight unit class of the product (1 = kilogram)
* `weight` - Product weight
* `length` - Product length
* `width` - Product width
* `height` - Product height
* `product_attribute` - Product attributes at the time of the sale (JSON array)
* `order_date_added` - When the parent order was placed (store local time)
* `order_date_modified` - When the parent order was last changed (store local time); used as the incremental replication key
One row per status change of an order, derived from the Orders stream without extra requests.
**Slug:** `order_history`\
**Primary key:** `order_id`, `order_history_id`\
**Replication:** Incremental (`order_date_modified`)
**Fields:**
* `order_id` - Identifier of the order
* `order_history_id` - Unique identifier of the status change
* `order_status_id` - Identifier of the status set by this change (see the order\_statuses stream)
* `order_status_name` - Name of the status set by this change
* `notify` - Whether the customer was notified of this change (1) or not (0)
* `comment` - Comment recorded with the status change
* `user_log` - User or integration that made the change (e.g. lojista, paypal)
* `date_added` - When the status change happened (store local time)
* `order_date_added` - When the parent order was placed (store local time)
* `order_date_modified` - When the parent order was last changed (store local time); used as the incremental replication key
Registered customers with their group and all addresses (as a JSON column).
**Slug:** `customers`\
**Primary key:** `customer_id`\
**Replication:** Incremental (`date_modified`)
**Fields:**
* `customer_id` - Unique identifier of the customer
* `type_person` - Whether the customer is an individual (F) or a company (J)
* `firstname` - First name (full name for individuals, company name for companies)
* `lastname` - Last name (trade name for companies; may be empty when included in firstname)
* `cnpj_contact` - Contact person name for company customers
* `email` - E-mail address
* `birthday` - Date of birth
* `sex` - Gender as registered (M/F)
* `telephone` - Landline phone, formatting may vary
* `cellphone` - Mobile phone, formatting may vary
* `mobile_operator` - Mobile carrier
* `cpf_cnpj` - CPF or CNPJ (Brazilian tax id), formatting may vary
* `rg_ie` - RG or state registration (IE), formatting may vary
* `limit_credit` - Credit limit granted to the customer
* `newsletter` - Whether the customer accepts the newsletter (1) or not (0)
* `address_id` - Identifier of the customer's default address
* `status` - Status of the customer in the store (1 = active, 0 = inactive)
* `approved` - Whether the customer registration is approved (1) or pending (0)
* `affiliate_tracking_code` - Tracking code of the affiliate linked to the customer's registration
* `customer_group_id` - Identifier of the customer's group (8 is the default group)
* `customer_group_name` - Name of the customer's group
* `date_added` - When the customer registered (store local time, no timezone)
* `date_modified` - When the customer was last changed (store local time); falls back to date\_added when never modified. Used as the incremental replication key
* `address` - All addresses registered by the customer: address\_id, names, street, number, complement, neighborhood, city, postcode, country\_id, zone\_id (JSON array)
* `custom_fields` - Custom registration fields configured by the store (JSON array)
* `upload` - Files uploaded by the customer during registration (JSON array)
Customer groups of the store (pricing and visibility tiers).
**Slug:** `customer_groups`\
**Primary key:** `customer_group_id`\
**Replication:** Full table
**Fields:**
* `customer_group_id` - Unique identifier of the customer group
* `name` - Name of the customer group
* `description` - Description of the customer group
* `sort_order` - Display order in the store (lower first)
* `buying_minimum_value` - Minimum order value required for customers of this group to buy
* `minimum_quantity_of_products` - Minimum quantity of products required for customers of this group to buy
* `minimum_quantity_per` - Whether the minimum quantity counts products (P) or items (I)
Carts left behind by customers and the products they contained.
**Slug:** `abandoned_carts`\
**Primary key:** `customer_cart_id`\
**Replication:** Full table
**Fields:**
* `customer_cart_id` - Unique identifier of the abandoned cart
* `firstname` - Customer's first name
* `lastname` - Customer's last name
* `email` - Customer's e-mail address
* `product_info` - Products left in the cart: product\_id, model, name, image and selected stock options with quantities (JSON array)
Product catalog with prices, descriptions, categories, promotions, quantity discounts and stock options (variants).
**Slug:** `products`\
**Primary key:** `product_id`\
**Replication:** Full table
**Fields:**
* `product_id` - Unique identifier of the product
* `model` - Product code (model), the store's own reference
* `name` - Product name (from the first product description entry)
* `external_reference` - External reference, such as the manufacturer's code or the ERP id
* `stock_status_id` - Identifier of the manufacturing/stock lead-time status
* `manufacturer_id` - Identifier of the manufacturer/brand (see the manufacturers stream)
* `image` - Main product image (internal path or URL)
* `price` - Regular selling price
* `cost_price` - Cost price
* `price_sale` - Sale price
* `price_pos` - Point-of-sale (POS) price
* `price_attacked` - Wholesale (atacado) price
* `weight` - Weight
* `length` - Length
* `width` - Width
* `height` - Height
* `subtract` - Whether stock is controlled for the product (1) or not (0)
* `sku` - SKU, usually the EAN/barcode
* `quantity` - Stock quantity of the product, when not tracked per option
* `minimum` - Minimum purchase quantity
* `multiple_sales_quantity` - Purchase quantity multiple
* `status` - Whether the product is enabled (1) or disabled (0)
* `criticized` - Whether the product was flagged (criticized) by the platform (1) or not (0)
* `criticized_message` - Message explaining why the product was flagged
* `approval_certificate` - Approval certificate (CA) for personal protective equipment products
* `url` - URL slug of the product page
* `user_log` - User who made the last change
* `date_added` - When the product was created (store local time, no timezone)
* `date_modified` - When the product was last changed (store local time)
* `ncm_name` - Name of the product's NCM (Mercosur tax classification)
* `ncm_number` - NCM code of the product
* `product_description` - Descriptions per language: name, description, tag, meta title/description/keywords, summary (JSON array)
* `product_image` - Secondary product images (JSON array)
* `product_to_category` - Categories the product belongs to: category\_id and name (JSON array)
* `product_special` - Promotions configured for the product: customer group, priority, quantity, price, dates (JSON array)
* `product_discount` - Quantity discounts configured for the product: customer group, quantity, priority, price, dates (JSON array)
* `product_option` - Stock options (variants) with per-option quantity and price/weight adjustments (JSON array)
* `product_like` - Identifiers of similar products (JSON array)
Product categories and their hierarchy.
**Slug:** `categories`\
**Primary key:** `category_id`\
**Replication:** Full table
**Fields:**
* `category_id` - Unique identifier of the category
* `name` - Category name (from the first description entry)
* `parent_id` - Identifier of the parent category, 0 for top-level categories
* `image` - Category image
* `image_icon` - Category icon image
* `sort_order` - Display order in the store (lower first)
* `status` - Whether the category is enabled (1) or disabled (0)
* `date_added` - When the category was created (store local time, no timezone)
* `category_description` - Descriptions per language: name, description (may contain HTML), meta title/description/keywords (JSON array)
Manufacturers and brands.
**Slug:** `manufacturers`\
**Primary key:** `manufacturer_id`\
**Replication:** Full table
**Fields:**
* `manufacturer_id` - Unique identifier of the manufacturer/brand
* `name` - Manufacturer name
* `image` - Manufacturer logo image URL
* `banner` - Manufacturer banner image URL
* `meta_title` - Meta title for search engines
* `meta_description` - Meta description for search engines
* `description` - Manufacturer description
* `sort_order` - Display order relative to other manufacturers (lower first)
* `date_added` - When the manufacturer was created (store local time, no timezone)
* `date_modified` - When the manufacturer was last changed (store local time)
* `user_log` - User who made the last change
Stock option definitions (size, color, and the like) and their values.
**Slug:** `options`\
**Primary key:** `option_id`\
**Replication:** Full table
**Fields:**
* `option_id` - Unique identifier of the option
* `name` - Option name (e.g. Tamanho, Cor)
* `type` - Option input type: select, select\_list, grid, radio, checkbox, text, textarea, datetime
* `sort_order` - Display order in the store (lower first)
* `status` - Whether the option is enabled (1) or disabled (0)
* `option_info` - Values of the option: option\_value\_id, name, sort\_order, allow\_delete (JSON array)
Discount coupons with validity, limits and applicability rules.
**Slug:** `coupons`\
**Primary key:** `coupon_id`\
**Replication:** Full table
**Fields:**
* `coupon_id` - Unique identifier of the coupon
* `name` - Coupon name
* `code` - Code typed by the customer at checkout
* `type` - Discount type: P = percentage, F = fixed value
* `product_id` - Product the coupon is bound to, 0 when not product-specific
* `discount` - Discount value (percentage or amount, according to type)
* `free_shipping` - Whether the coupon grants free shipping (1) or not (0)
* `free_shipping_methods` - Shipping methods covered by the free shipping
* `logged` - Whether the coupon is only visible to logged-in customers (1) or to everyone (0)
* `total` - Minimum order total required before the coupon is valid
* `discount_disabled` - Whether the cash-payment discount is disabled when this coupon is used (1) or not (0)
* `date_start` - First day the coupon is valid
* `date_end` - Last day the coupon is valid
* `uses_total` - Maximum number of uses of the coupon, 0 for unlimited
* `uses_customer` - Maximum number of uses per customer
* `status` - Whether the coupon is enabled (1) or disabled (0)
* `retail_or_attacked` - Whether the coupon applies to retail, wholesale or both
* `user_log` - User who made the last change
* `coupon_product` - Products the coupon applies to (JSON array of product\_id)
* `coupon_category` - Categories the coupon applies to (JSON array of category\_id)
* `coupon_manufacturer` - Manufacturers the coupon applies to (JSON array of manufacturer\_id)
Storefront banners and their display window.
**Slug:** `banners`\
**Primary key:** `banner_new_id`\
**Replication:** Full table
**Fields:**
* `banner_new_id` - Unique identifier of the banner
* `name_banner` - Banner name
* `link_banner` - URL the banner links to
* `link_image` - Banner image file
* `position` - Banner placement: banner\_full, banner\_full\_side, banner\_targe, banner\_side, banner\_mini
* `status` - Whether the banner is enabled (1) or disabled (0)
* `date_start` - First day the banner is shown
* `date_end` - Last day the banner is shown
* `sort_order` - Display order in the store (lower first)
* `date_added` - When the banner was created (store local time, no timezone)
Affiliate partners registered in the store.
**Slug:** `affiliates`\
**Primary key:** `affiliate_id`\
**Replication:** Full table
**Fields:**
* `affiliate_id` - Unique identifier of the affiliate
* `firstname` - Affiliate's first name
* `lastname` - Affiliate's last name
* `email` - Affiliate's e-mail address
* `telephone` - Affiliate's landline phone
* `cellphone` - Affiliate's mobile phone
* `address_1` - Street address
* `neighborhood` - Neighborhood
* `city` - City
* `postcode` - Postal code (CEP)
* `zone_id` - Identifier of the state (Irroba zone id, 440-466 for Brazilian states)
* `code` - Affiliate tracking code used in links
* `cpf_cnpj` - CPF or CNPJ (Brazilian tax id)
* `status` - Whether the affiliate is active (1) or inactive (0)
* `approved` - Whether the affiliate is approved (1) or pending (0)
Reference table with the id and name of every order status of the store.
**Slug:** `order_statuses`\
**Primary key:** `order_status_id`\
**Replication:** Full table
**Fields:**
* `order_status_id` - Unique identifier of the order status
* `name` - Status name as shown in the store (e.g. Pagamento Aprovado, Pedido Entregue)
Invoices (NF-e) issued through Irroba, with number, series, access key, status and document links.
**Slug:** `invoices`\
**Primary key:** `id`\
**Replication:** Incremental (`issue_date`)
**Fields:**
* `id` - Unique identifier of the invoice
* `series` - Invoice series
* `number` - Invoice number
* `type` - Operation type of the invoice (e.g. saida, entrada, sale, return)
* `order_id` - Identifier of the order the invoice belongs to
* `key` - NF-e access key (44 digits)
* `issue_date` - When the invoice was issued (store local time); used as the incremental replication key
* `status_code` - Numeric status code of the invoice
* `status` - Status description (e.g. Autorizada, Cancelada)
* `status_reason` - Reason attached to the status, when any
* `link_xml` - URL of the invoice XML file
* `link_danfe` - URL of the DANFE (printable invoice)
* `products` - Products covered by the invoice (JSON array)
## Implementation Notes
* **Orders and customers** are read through Irroba's date-range endpoints, which return records created *or* modified in the requested period. Each run starts from the day of the last synced record and walks to today in fixed 30-day windows, so orders whose status changed are picked up again and updated.
* **Invoices** are read from the invoice list filtered by issue date, starting at the last synced invoice.
* **Products, categories, manufacturers, options, coupons, banners, affiliates, customer groups, abandoned carts and order statuses** have no date filter in the API and are read in full on every run.
* **Order Products** and **Order History** are derived from each order's payload, so they cost no extra requests.
* **Timestamps** are returned by Irroba in the store's local time without a time zone. The connector reads them as America/Sao\_Paulo and stores them as UTC. Calendar dates (birthdays, coupon validity) are kept as dates.
* **Numbers** that Irroba serializes as text (prices, quantities, flags) are converted to numeric columns; empty values become null. Nested structures such as order line items, addresses and product options are stored as JSON text columns.
* **Rate limits**: the connector never exceeds the configured requests per minute and retries with a growing wait when Irroba answers "too many requests".
Irroba caches order queries for the same period for about 30 minutes, so an order placed a moment ago may only appear on the following run.
# IUGU as a data source
Source: https://docs.nekt.com/sources/iugu
Bring data from IUGU to Nekt.
IUGU is a payment processing platform designed for Brazilian businesses. It provides tools for accepting online payments, managing subscriptions, and handling financial transactions securely, helping companies streamline their payment operations and improve cash flow.
![]()
## Configuring IUGU as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the IUGU option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Connect using your IUGU API token. See [IUGU authentication](https://dev.iugu.com/reference/autentica%C3%A7%C3%A3o#criar-api_token) for where to create the token in your IUGU account.
The following configurations are available:
* **API token**: The token to authenticate against the Iugu API service. This field is required and stored securely.
The `financial_statement` stream requires a live API token; test tokens will return no balance movement.
* **Initial sync date**: Starting point for data extraction. Only records created or updated after this date will be synced. This field is required.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
All IUGU streams use `updated_at` or `entry_date` as the replication key where applicable, so incremental syncs advance based on last update time.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from IUGU and their corresponding fields:
Stream for customer profiles, contact data, addresses, and saved payment methods.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique customer identifier |
| `email` | String | Primary email |
| `name` | String | Customer name |
| `cpf_cnpj` | String | Tax ID (CPF or CNPJ) |
| `notes` | String | Internal notes |
| `created_at`, `updated_at` | DateTime | Timestamps (replication uses `updated_at`) |
| `cc_emails` | String | Additional emails copied on notifications |
| `phone`, `phone_prefix` | String | Phone number and prefix |
| `zip_code`, `street`, `number`, `complement`, `district`, `city`, `state` | String | Address fields |
| `payment_methods` | Array | Array of saved methods (`id`, `description`, `item_type`, `customer_id`, `data` with card metadata) |
| `default_payment_method_id` | String | Default method id |
| `proxy_payments_from_customer_id` | String | Customer id when payments are proxied from another customer |
| `custom_variables` | Array | Array of `{ name, value }` |
Stream for invoices (charges), including payer data, line items, Pix/boleto context, splits, and financial return schedule.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique invoice identifier |
| `status` | String | Invoice status |
| `currency` | String | Currency code |
| `due_date` | Date | Due date |
| `updated_at` | DateTime | Last update (replication key) |
| `subscription_id` | String | Related subscription when applicable |
| `customer_id`, `customer_name`, `customer_ref` | String | Customer link and display fields |
| `order_id`, `external_reference` | String | External system references |
| `total_cents`, `total_paid_cents`, `paid_cents`, `tax_cents`, `taxes_paid_cents`, `discount_cents`, `overpaid_cents`, `refunded_cents`, `remaining_captured_cents`, `commission_cents`, `advance_fee_cents` | Integer | Amounts in cents |
| `total`, `total_paid`, `paid`, `commission`, `discount`, `interest` | String | Formatted amount strings |
| `payer_name`, `payer_email`, `payer_cpf_cnpj`, `payer_phone`, `payer_phone_prefix` | String | Payer contact information |
| `payer_address_*` | String | Zip, street, district, city, state, number, complement, country |
| `payable_with` | String | Allowed payment methods |
| `payment_method` | String | Method used when paid |
| `notification_url`, `return_url`, `secure_id`, `secure_url` | String | Hosted pay link and callbacks |
| `ignore_due_email`, `ignore_canceled_email` | Boolean | Email suppression flags |
| `email`, `cc_emails` | String | Invoice notification emails |
| `credit_card_brand`, `credit_card_bin`, `credit_card_last_4`, `credit_card_tid`, `credit_card_captured_at`, `credit_card_transaction`, `transaction_number` | String | Card / acquirer information (when applicable) |
| `bank_slip`, `bank_slip_extra_due`, `bank_account_branch`, `bank_account_number`, `account_name`, `account_id` | String | Bank slip and account fields |
| `pix` | Object | Object with `qrcode`, `qrcode_text`, `status`, payer fields, `end_to_end_id`, `end_to_end_refund_id`, `account_number_last_digits` |
| `created_at` / `created_at_iso`, `paid_at`, `authorized_at` / `authorized_at_iso`, `expired_at` / `expired_at_iso`, `refunded_at` / `refunded_at_iso`, `canceled_at` / `canceled_at_iso`, `protested_at` / `protested_at_iso`, `chargeback_at` / `chargeback_at_iso`, `financial_return_date`, `occurrence_date` | String / DateTime | Lifecycle timestamps |
| `late_payment_fine`, `late_payment_fine_cents`, `per_day_interest`, `per_day_interest_cents`, `max_installments_value`, `installments`, `early_payment_discount`, `split_id`, `external_payment_id`, `payment_booklet_id`, `user_id` | Various | Fees, interest, installments |
| `variables` | Array | Template variables (`variable`, `value`) |
| `custom_variables` | Array | Custom metadata (`name`, `value`) |
| `items` | Array | Line items (`id`, `description`, `price_cents`, `quantity`, `price`, timestamps) |
| `financial_return_dates` | Array | Per-installment settlement rows (`installment`, `return_date`, `amount_cents`, `status`, fees, `advanced`, etc.) |
| `split_rules` | Array | Split configuration per recipient and payment method, including cents/percent fields |
| `logs` | Array | Event or status log entries |
| `refundable`, `duplicated_invoice_id`, `double_payment_id`, `original_payment_id` | Various | Other metadata flags and IDs |
Stream for subscription plan definitions, pricing per currency, features, and billing rules.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique plan identifier |
| `name` | String | Plan display name |
| `identifier` | String | Stable plan string identifier (referenced by subscriptions) |
| `interval` | Integer | Billing interval length |
| `interval_type` | String | Billing interval unit |
| `created_at`, `updated_at` | DateTime | Timestamps (replication uses `updated_at`) |
| `prices` | Array | Array of `{ id, currency, value_cents, created_at, updated_at }` |
| `features` | Array | Array of plan features (`id`, `name`, `identifier`, `value`, `important`, `position`, timestamps) |
| `payable_with` | String | Allowed payment methods |
| `max_cycles` | Integer | Maximum billing cycles |
| `billing_days` | Integer | Day of month for due dates |
| `invoice_max_installments` | Integer | Max card installments on plan invoices |
Stream for active and historical subscriptions, including plan reference, customer, add-ons, recent invoices, and audit logs.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique subscription identifier |
| `active` | Boolean | Whether the subscription is active |
| `suspended` | Boolean | Whether billing is suspended |
| `plan_identifier`, `plan_name`, `plan_ref` | String | Plan link and labels |
| `customer_id`, `customer_name`, `customer_email`, `customer_ref` | String | Customer link |
| `price_cents` | Integer | Price in cents |
| `currency` | String | Currency |
| `created_at`, `updated_at`, `cycled_at`, `expires_at` | DateTime | Timestamps (replication uses `updated_at`) |
| `payable_with` | String | Allowed payment methods |
| `max_cycles`, `cycles_count` | Integer | Cycle limits and count |
| `ignore_due_email` | Boolean | Due email suppression |
| `suspend_on_invoice_expired`, `two_step`, `in_trial` | Boolean | Workflow flags |
| `credits_based`, `credits`, `credits_min`, `credits_cycle` | Various | Credits configuration |
| `features` | Array | Same shape as on plans (active on this subscription) |
| `subitems` | Array | Add-on lines (`description`, `quantity`, `price_cents`, `recurrent`, `price`, `total`) |
| `recent_invoices` | Array | Recent invoices (`id`, `due_date`, `status`, `total`, `secure_url`) |
| `logs` | Array | Audit entries (`description`, `notes`, `subscription_changes`, `created_at`) |
| `custom_variables` | Array | Array of `{ name, value }` |
Account financial statement stream (Extrato Financeiro): every credit and debit on the Iugu balance.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Synthetic identifier of the entry, generated by Nekt. Hash of entry details and its position. |
| `entry_date` | Date | Date the entry was booked on the account balance. |
| `type` | String | Direction of the entry: credit or debit. |
| `transaction_type` | String | Category of the movement as classified by Iugu (e.g., misc). |
| `description` | String | Description of the entry, e.g., payment received for an invoice, fees or a transfer. |
| `amount_cents` | Integer | Signed amount of the entry in cents (negative for debits). |
| `amount` | String | Amount of the entry formatted as currency text. |
| `balance_cents` | Integer | Account balance in cents right after this entry. |
| `balance` | String | Account balance after this entry, formatted as currency text. |
| `reference` | String | Identifier of the object that generated the entry (e.g., the invoice or transfer id). |
| `reference_type` | String | Type of the object that generated the entry (e.g., Invoice, Transfer). |
| `account_id` | String | Identifier of the Iugu account related to the entry. |
| `invoice_email` | String | Email of the invoice related to the entry, when any. |
| `customer_name` | String | Name of the customer related to the entry, when any. |
| `customer_ref` | String | Customer reference of the related invoice (usually the email), when any. |
| `payer_name` | String | Name of the payer related to the entry, when any. |
# Data Model
The diagram below shows how the main IUGU streams relate. Join keys reflect typical analytics use; `plan_identifier` on subscriptions matches `identifier` on plans.
```mermaid theme={null}
graph TD;
subgraph "Core entities"
Customers("customers");
Plans("plans");
Subscriptions("subscriptions");
Invoices("invoices");
FinancialStatement("financial_statement");
end
Subscriptions -- "customer_id" --> Customers;
Subscriptions -- "plan_identifier matches plans.identifier" --> Plans;
Invoices -- "customer_id" --> Customers;
Invoices -- "subscription_id" --> Subscriptions;
FinancialStatement -- "reference (when reference_type='Invoice')" --> Invoices;
```
# Use Cases for Data Analysis
Examples below assume default table names in the raw layer (`iugu_customers`, `iugu_invoices`, `iugu_plans`, `iugu_subscriptions`). Adjust schema and table names to match your catalog. Run queries in [Explorer](https://app.nekt.ai/explorer).
### 1. Invoice volume and totals by status
Summarize how much was billed and collected, grouped by invoice status.
**Business value:**
* Monitor cash position and outstanding receivables
* Compare paid vs. open or canceled invoices over a period
```sql theme={null}
SELECT
i.status,
COUNT(*) AS invoice_count,
SUM(i.total_cents) / 100.0 AS total_billed,
SUM(i.total_paid_cents) / 100.0 AS total_paid
FROM
nekt_raw.iugu_invoices i
WHERE
CAST(i.updated_at AS TIMESTAMP) >= CURRENT_TIMESTAMP - INTERVAL '90' DAY
GROUP BY
i.status
ORDER BY
total_billed DESC
```
```sql theme={null}
SELECT
i.status,
COUNT(*) AS invoice_count,
SUM(i.total_cents) / 100.0 AS total_billed,
SUM(i.total_paid_cents) / 100.0 AS total_paid
FROM
`nekt_raw.iugu_invoices` i
WHERE
TIMESTAMP(i.updated_at) >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)
GROUP BY
i.status
ORDER BY
total_billed DESC
```
### 2. Active subscriptions with customer email
List active subscriptions joined to customers for outreach or churn analysis.
**Business value:**
* Segment subscribers by plan or contact
* Join subscription flags to customer master data
```sql theme={null}
SELECT
s.id AS subscription_id,
s.plan_name,
s.customer_email,
c.name AS customer_name,
s.price_cents / 100.0 AS price,
s.currency,
s.cycles_count,
s.updated_at
FROM
nekt_raw.iugu_subscriptions s
LEFT JOIN nekt_raw.iugu_customers c ON s.customer_id = c.id
WHERE
s.active = true
ORDER BY
s.updated_at DESC
```
```sql theme={null}
SELECT
s.id AS subscription_id,
s.plan_name,
s.customer_email,
c.name AS customer_name,
s.price_cents / 100.0 AS price,
s.currency,
s.cycles_count,
s.updated_at
FROM
`nekt_raw.iugu_subscriptions` s
LEFT JOIN `nekt_raw.iugu_customers` c ON s.customer_id = c.id
WHERE
s.active = TRUE
ORDER BY
s.updated_at DESC
```
## Implementation Notes
### Amounts and types
* Monetary amounts are often exposed both as integer **cents** fields and as **formatted strings** from the API; prefer `_cents` columns for aggregations when available.
* Some invoice timestamp fields exist as plain strings alongside `*_iso` datetime fields; use the ISO fields when you need reliable temporal filtering.
### Incremental sync
* Streams use `updated_at` (or `entry_date`) as the replication key; set **Start Date** to bound the initial backfill.
* Use **Additional Full Sync** if you need periodic reconciliation with IUGU’s current state.
### Financial Statement (Extrato Financeiro)
* **Live token required**: The `financial_statement` stream requires a live API token. Test tokens will not return any balance movement.
* **Synthetic IDs**: The statement entries do not have a native ID returned by the API. The connector generates a synthetic SHA-256 ID based on the entry's identifying fields and its position among identical entries on the same day.
* **Timezones and daily syncs**: The stream filters only by date, walking one day at a time using the `America/Sao_Paulo` timezone. Since entries do not have a time of day, hourly runs will re-read the current day and merge entries by their synthetic ID securely.
* **Cent conversions**: `amount_cents` and `balance_cents` arrive from the API as strings (`"100.0"`) and are automatically cast to integers by the connector.
### API and performance
* Selecting only the streams you need reduces extraction time and API load.
* Large nested arrays (for example `split_rules`, `items`, `payment_methods`) can make rows wide; consider flattening in a downstream Query or Notebook if you model them as separate tables.
## Skills for agents
IUGU connector documentation as plain markdown, for use in AI agent contexts.
# Jetimob as a data source
Source: https://docs.nekt.com/sources/jetimob
Bring data from Jetimob to Nekt.
Jetimob is a real estate portal and property listing platform that connects property buyers, sellers, and real estate professionals. It provides tools for property search, listing management, and market analysis to facilitate real estate transactions and market insights.
## Configuring Jetimob as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Jetimob option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You need to provide your Jetimob API credentials to authenticate and extract data. According to the documentation, API keys should be requested directly by getting in touch with Jetimob support.
The following configurations are available:
* **Web Service key**: The Web Service key to authenticate against the API service (used for the property/`imoveis` and `condominios` endpoints).
* **Private key**: The private key to authenticate against the API service for the leads endpoint.
* **Public key**: The public key to authenticate against the API service for the leads endpoint.
The `leads` stream is only available when both the **Public key** and the **Private key** are provided.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
* `imoveis`: the list of properties (`imoveis`) available for publish, obtained from the `/imoveis` endpoint.
* `imoveis_todos`: the list of all properties (`imoveis`) available in the account, obtained from the `/imoveis/todos` endpoint.
* `condominios`: the list of condominiums, obtained from the `/condominios` endpoint.
* `leads`: the list of leads, obtained from the leads endpoint (requires the Public and Private keys).
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Jetimob:
The list of properties (`imoveis`) available for publish, obtained from the `/imoveis` endpoint. Includes property details such as type, subtype, contract, rooms, areas, and location information.
The list of all properties (`imoveis`) available in the account, obtained from the `/imoveis/todos` endpoint. This stream supports incremental sync based on the property's last update date (`data_update`).
The list of condominiums, obtained from the `/condominios` endpoint. Includes condominium details such as type, structural system, construction progress, location, and address information.
The list of leads, obtained from the leads endpoint. Only available when both the Public key and the Private key are configured.
## Skills for agents
Jetimob connector documentation as plain markdown, for use in AI agent contexts.
# Jira as a data source
Source: https://docs.nekt.com/sources/jira
Bring data from Jira to Nekt.
Jira is Atlassian's project management and issue tracking platform designed for software development teams. It provides tools for agile project management, bug tracking, and workflow automation to help teams plan, track, and release software efficiently.
![Jira]()
## Configuring Jira as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Jira** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Use your Jira Cloud credentials and workspace information. If needed, check Jira's [REST API documentation](https://developer.atlassian.com/cloud/jira/platform/rest/v3/intro/#version).
The following configurations are available:
* **API token**: The token used to authenticate against Jira. For instructions on generating it, check [Atlassian's guide](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/).
* **User email**: The email address associated with the API token.
* **Workspace domain**: Your Jira domain, without `https://` (for example, `company.atlassian.net`).
* **Start date**: Optional. The earliest timestamp used for incremental extraction. If left empty, all available history will be fetched.
* **Project keys**: Optional list of project keys to scope extraction. When defined, project-dependent streams are filtered to those projects.
* **Issues stream JQL query**: Optional custom JQL filter for the `issues` stream. It is combined with the start date filter and project filter.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
When you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Jira and their corresponding key fields.
Base project entities used by other project-scoped streams.
**Primary key:**
* `id`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `key` | String | Project key used as Jira identifier |
| `name` | String | Project name |
| `project_type_key` | String | Project type |
| `is_private` | Boolean | Project visibility flag |
| `project_category` | Object | Project category metadata |
| `insight.total_issue_count` | Integer | Total number of issues |
Agile boards associated with projects.
**Primary key:**
* `id`
**Parent stream:**
* `projects` (filtered by project context)
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `name` | String | Board name |
| `type` | String | Board type (`scrum`, `kanban`, etc.) |
| `location.project_id` | String | Related project ID |
| `location.project_key` | String | Related project key |
| `location.project_name` | String | Related project name |
Sprint entities collected from agile boards.
**Replication key:**
* `id`
**Parent stream:**
* `boards` (only boards with type `scrum`)
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `state` | String | Sprint state |
| `name` | String | Sprint name |
| `start_date` | String | Sprint start date |
| `end_date` | String | Sprint end date |
| `complete_date` | String | Completion date |
| `board_id` | String | Parent board ID |
| `goal` | String | Sprint goal |
Issue type definitions available per project.
**Primary key:**
* `id`
**Parent stream:**
* `projects` (filtered by project context)
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `name` | String | Issue type name |
| `description` | String | Issue type description |
| `subtask` | Boolean | Whether the type is a subtask |
| `hierarchy_level` | Integer | Hierarchy level in Jira |
| `scope.project.id` | String | Project-scoped type project ID |
| `scope.project.key` | String | Project-scoped type project key |
Field definitions and metadata available for projects.
**Primary key:**
* `id`
**Parent stream:**
* `projects` (filtered by project context)
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `key` | String | Field key |
| `name` | String | Field name |
| `custom` | Boolean | Whether the field is custom |
| `orderable` | Boolean | Whether the field can be used in ordering |
| `searchable` | Boolean | Whether the field can be searched |
| `schema.type` | String | Data type for field values |
| `schema.custom` | String | Custom schema identifier |
Jira users available to the authenticated account.
**Primary key:**
* `account_id`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `display_name` | String | User display name |
| `email_address` | String | User email |
| `account_type` | String | Account type |
| `active` | Boolean | Whether the user is active |
| `time_zone` | String | User time zone |
| `locale` | String | User locale |
Main issue entities from Jira. The `fields` property is serialized as a single JSON string, containing all standard and custom issue fields.
**Primary key:**
* `id`
**Replication key:**
* `updated`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the record |
| `key` | String | Issue key |
| `self` | String | URL of the resource |
| `fields` | String | Full set of issue properties as a JSON string (contains summary, status, assignee, and custom fields) |
Change history for each issue.
**Primary key:**
* `id`
**Replication key:**
* `created`
**Parent stream:**
* `issues`
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `issue_id` | String | Parent issue ID |
| `author.account_id` | String | Author account ID |
| `created` | String | Changelog creation timestamp |
| `items.field` | String | Changed field |
| `items.from` | String | Previous value ID |
| `items.to` | String | New value ID |
| `items.from_string` | String | Previous value (display) |
| `items.to_string` | String | New value (display) |
# Data Model
The following diagram shows the relationship between the main Jira streams. Note that links originating from `Issues` require extracting values from the `fields` JSON string.
```mermaid theme={null}
graph TD;
Projects("Projects");
Boards("Boards");
Sprints("Sprints");
IssueTypes("Issue Types");
Fields("Fields");
Issues("Issues");
IssueChangelog("Issue Changelog");
Users("Users");
Boards -- "project_key" --> Projects;
Sprints -- "board_id" --> Boards;
IssueTypes -- "project_id" --> Projects;
Fields -- "project_id" --> Projects;
Issues -- "fields -> project.id" --> Projects;
Issues -- "fields -> assignee.account_id" --> Users;
Issues -- "fields -> reporter.account_id" --> Users;
IssueChangelog -- "issue_id" --> Issues;
```
# Transformation example: extracting fields from JSON
The `fields` column in the `issues` stream stores all standard and custom attributes as a single JSON string. To analyze dimensions like summary, status, project, or assignee in [Explorer](https://app.nekt.ai/explorer) or downstream models, parse the JSON and expose the keys as separate columns.
```sql theme={null}
SELECT
id,
key,
json_extract_scalar(fields, '$.summary') AS summary,
json_extract_scalar(fields, '$.status.name') AS status_name,
json_extract_scalar(fields, '$.project.key') AS project_key,
json_extract_scalar(fields, '$.assignee.account_id') AS assignee_account_id,
json_extract_scalar(fields, '$.issuetype.name') AS issue_type,
CAST(json_extract_scalar(fields, '$.created') AS TIMESTAMP) AS created_at,
CAST(json_extract_scalar(fields, '$.updated') AS TIMESTAMP) AS updated_at
FROM
nekt_raw.jira_issues
```
```sql theme={null}
SELECT
id,
key,
JSON_VALUE(fields, '$.summary') AS summary,
JSON_VALUE(fields, '$.status.name') AS status_name,
JSON_VALUE(fields, '$.project.key') AS project_key,
JSON_VALUE(fields, '$.assignee.account_id') AS assignee_account_id,
JSON_VALUE(fields, '$.issuetype.name') AS issue_type,
CAST(JSON_VALUE(fields, '$.created') AS TIMESTAMP) AS created_at,
CAST(JSON_VALUE(fields, '$.updated') AS TIMESTAMP) AS updated_at
FROM
`nekt_raw.jira_issues`
```
You can run this as an ad-hoc query in Explorer or turn it into a [transformation](https://docs.nekt.com/get-started/core-concepts/transformations) that writes to a new table so you have a flattened view of Jira issues for reporting and joins.
## Implementation Notes
### Filtering behavior
* `project_keys` limits project-scoped extraction to specific Jira projects.
* `issues` applies an `updated >= start_date` filter if a starting timestamp is available, and can additionally apply a custom JQL query.
* The final JQL for `issues` combines the start date (if present), optional project keys, and optional custom JQL with logical `and`.
* The Jira `/search/jql` API endpoint rejects queries that are entirely unbounded. To safely capture all historical records during a full table extraction or the initial run of an incremental extraction (before a state or bookmark is set), the connector automatically falls back to a minimal date bound (`updated >= '1900/01/01 00:00'`). This ensures all issues are properly synced while keeping the request bounded as required by Jira's servers.
### Stream hierarchy behavior
* `boards`, `fields`, and `issue_types` are project-scoped child streams.
* `sprints` is collected only from `scrum` boards.
* `issue_changelog` is collected per issue and includes `issue_id` as parent context.
## Skills for agents
Jira connector documentation as plain markdown, for use in AI agent contexts.
# JSON Content as a data source
Source: https://docs.nekt.com/sources/json-content
Bring data from JSON Content to Nekt.
JSON Content refers to data stored in JSON (JavaScript Object Notation) format, which is a lightweight data interchange format. It's commonly used for APIs, configuration files, and data storage, providing a human-readable way to represent structured data.
## Configuring JSON Content as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **JSON Content** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add connection details
You'll need to provide the connection details to access your JSON data.
The following configurations are available:
* **Website URL**: The URL of the website to extract data from. It should be a single-page URL containing a JSON payload. This field is required.
* **Records JSON Path**: The JSON path to identify where in the JSON payload the data should be extracted. Default is `$`. This field is required.
* **Number of samples to generate schema**: (Advanced setting) The maximum number of records to sample for schema generation. Defines how many items the connector reads to figure out the table columns. You should set a higher number of samples for highly variable payloads. If the records are consistent across the payload, the default (`100`) is usually sufficient.
**Understanding JSON Path:**
As an example to understand how JSON path works, consider the following payload obtained from the Website URL:
```json theme={null}
{
"total_listings": 2069,
"listings": [
{
"id": 26747,
"type": "House",
"purpose": "commercial",
"transaction": "rent",
"value": 1800
},
{
"id": 774,
"type": "Apartment",
"purpose": "residencial",
"transaction": "rent",
"value": 2400
},
{
"id": 1553,
"type": "Studio",
"purpose": "residencial",
"transaction": "rent",
"value": 1800
}
]
}
```
If you want to extract all `listings` from it, your JSON path should be `$.listings[*]`. This way, each listing inside the array would represent a row in the extracted table, such as:
| id | type | purpose | transaction | value |
| - | - | - | - | - |
| 26747 | House | commercial | rent | 1800 |
| 774 | Apartment | residencial | rent | 2400 |
| 1553 | Studio | residencial | rent | 1800 |
For more information about how JSON path works, please check the [JsonPath documentation](https://github.com/json-path/JsonPath).
Once you're done, click **Next**.
### 2. Select streams
The JSON Content connector exposes a single dynamic stream based on your configured JSON Path. Choose whether you want to sync it.
Select the stream and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for the table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
### 5. Check your new source
Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
The JSON Content connector extracts data dynamically based on the payload returned by your URL and the specific JSON path configured.
Because the JSON Content connector extracts data from an arbitrary URL, the resulting fields and data types depend entirely on the target JSON payload.
**How it works:**
* **Dynamic Schema Discovery**: The connector reads a sample of records up to the specified "Number of samples to generate schema" parameter (default 100). It automatically infers the column names and data types (e.g., String, Integer, Boolean) from the JSON keys it encounters.
* **JSON Path Filtering**: The fields exposed in the stream are the direct keys of the objects returned by your `Records JSON Path` query. For example, if your JSON Path targets an array of `user` objects, the resulting table will have columns for `id`, `name`, `email`, etc., exactly matching the keys inside each user object.
**Implementation Notes:**
* Ensure your JSON path extracts an array of objects or an object containing a consistent set of keys to correctly populate the table format.
* If your JSON payload is highly variable or contains sparse fields, consider increasing the **Number of samples to generate schema** parameter in the source configuration to ensure all possible fields are discovered.
## Skills for agents
JSON Content connector documentation as plain markdown, for use in AI agent contexts.
# Juridiq as a data source
Source: https://docs.nekt.com/sources/juridiq
Bring data from Juridiq to your Lakehouse.
Juridiq is a Brazilian legal practice management platform for law firms. It covers cases (processos), court publications and notices, deadlines and tasks, hearings, clients and parties, electronic signatures and time tracking. The connector extracts data from the Juridiq REST API (api.juridiq.com.br) so you can analyse your firm's operation in your Lakehouse.
## Configuring Juridiq as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Juridiq option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a Juridiq API key so Nekt can access your data. Sign in to the Juridiq panel, open **Settings > API** and generate a new key. The key belongs to the law office, so a single key covers all the data of your firm.
The Juridiq API is a paid add-on of the product. If the API section is not available in your panel, contact Juridiq to enable the API plan for your office before setting up this source.
The following configurations are available:
* **API key**: your Juridiq API key. It is sent in the `x-juridiq-api-key` header on every request. This field is required.
* **Initial sync date**: (Optional) the earliest date to read for the streams that accept a date filter: **Publications** and **Events**. All other streams always return your office's full history, regardless of this setting.
* **Fetch full details**: (Optional, enabled by default) reads the complete record of every case and person with one extra request each. Turn it off on very large offices to make the extraction faster; the trade-off is that the columns only the detail endpoint returns stay empty: case value, phase, parties, custom field values, document, address, and timestamps.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
The **Law suit movements** stream reads the docket entries of one case per request, so it is by far the slowest stream on offices with many cases. Select it only when you need the case timeline.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: the Juridiq API does not expose a "last updated" filter on its list endpoints, so every stream of this connector is extracted as **FULL\_TABLE**: each run brings the current state of the data.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Good to know
* **Permissions follow the API key.** If a module is not enabled for your office (electronic signatures, for example), the corresponding stream is skipped with a warning in the run log instead of failing the whole extraction.
* **Nested data is stored as JSON text.** Fields that hold an object or a list of objects (parties of a case, participants of a hearing, signatories of a document, custom field values) are stored as JSON strings, so you can parse them in a Query without your table schema changing between runs.
* **Date fields keep the API's own format.** Fields the Juridiq API documents as timestamps are typed as DateTime. Fields the API returns as plain text dates are kept exactly as received, so nothing is misread.
* **The gov.br password of a person is never extracted.** The Juridiq API returns it on the person record; the connector drops it so it never reaches your Lakehouse.
## Streams and Fields
Below you'll find all available data streams from Juridiq and their corresponding fields:
Cases handled by the office. Each listed case is completed with one extra request to the case detail endpoint, which adds the case value, phase, parties and custom field values.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the law suit in Juridiq. |
| `process_number` | String | Case number in the Brazilian CNJ format (e.g. 0012345-67.2026.8.26.0100). |
| `value_of_cause` | String | Monetary value assigned to the case, as returned by the API. |
| `status` | String | Current status of the case in the office's workflow. |
| `current_instance` | Integer | Court instance (grau) the case is currently in. |
| `court` | String | Court where the case is filed (e.g. TJSP). |
| `service_step` | String | Phase of the case within the office's service flow. |
| `is_private` | Boolean | Whether the case is restricted to specific users of the office. |
| `is_secret` | Boolean | Whether the case runs under judicial secrecy (segredo de justiça). |
| `office_id` | String | Identifier of the law office that owns the case. |
| `custom_fields` | String | Values of the office's custom case fields (JSON array of objects with field\_id, name, type, options, value and is\_archived). |
| `persons` | String | Parties of the case (JSON array of objects with id, name and person\_origin, where person\_origin is 'Cliente' for the represented party and 'Requerida' for the opposing one). |
Docket entries (movimentações) of every case. Read one case at a time, so this stream is the slowest one on offices with many cases.
**Primary key:** `law_suit_id`, `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Identifier of the docket entry. |
| `law_suit_id` | String | Identifier of the law suit the docket entry belongs to. |
| `conteudo` | String | Full text of the docket entry. |
| `data` | String | Date of the docket entry, kept exactly as returned by the API. |
| `instance` | Integer | Court instance (grau) the docket entry originated from. |
| `law_suit_instance` | String | Case instance the entry is attached to (JSON object with id, process\_number and instance). |
| `tags` | Array | Identifiers of the tags applied to the docket entry. |
| `tags_details` | String | Tags applied to the docket entry (JSON array of objects with id, name and color). |
Definitions of the custom case fields your office created, useful to interpret the `custom_fields` column of Law suits.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the custom field definition. |
| `name` | String | Label of the custom field as shown in Juridiq. |
| `type` | String | Data type of the field: text, longText, date, time, dateTime, number, select or multiSelect. |
| `options` | Array | Allowed values, for select and multiSelect fields. |
| `is_archived` | Boolean | Whether the field was archived and is no longer offered on new cases. |
| `created_at` | DateTime | Timestamp when the field was created. |
| `updated_at` | DateTime | Timestamp when the field was last updated. |
Clients, opposing parties and other contacts. Each listed person is completed with one extra request to the person detail endpoint, which adds the document, address and timestamps.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the person in Juridiq. |
| `name` | String | Full name of the person or company. |
| `email` | String | Primary email address. |
| `phone` | String | Primary phone number. |
| `document` | String | Brazilian tax document, CPF or CNPJ. |
| `rg` | String | Brazilian identity card number (RG). |
| `person_origin` | String | Role of the person in the office (e.g. cliente, parte). |
| `person_type` | String | Whether the record is an individual or a company (física / jurídica). |
| `marital_status` | String | Marital status of the person. |
| `nationality` | String | Nationality of the person. |
| `profession` | String | Occupation of the person. |
| `birth_date` | String | Date of birth, kept exactly as returned by the API. |
| `zip_code` | String | Postal code (CEP) of the address. |
| `state` | String | State of the address. |
| `city` | String | City of the address. |
| `neighborhood` | String | Neighborhood of the address. |
| `street_and_number` | String | Street and number of the address. |
| `address_complement` | String | Additional address information. |
| `image_url` | String | URL of the person's picture. |
| `client_discover_office` | String | How the client found the office, used for acquisition analysis. |
| `annotation` | String | Free-form notes about the person. |
| `is_private` | Boolean | Whether the record is restricted to specific users of the office. |
| `owner_id` | String | Identifier of the office user responsible for the person. |
| `legal_representant` | String | Legal representative of the person. |
| `corretor` | String | Broker associated with the person. |
| `code` | String | Internal code the office assigned to the person. |
| `office_id` | String | Identifier of the law office that owns the record. |
| `created_at` | DateTime | Timestamp when the person was created. |
| `updated_at` | DateTime | Timestamp when the person was last updated. |
Tasks and deadlines (prazos) tracked by the office.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the task. |
| `title` | String | Title of the task. |
| `priority` | String | Priority assigned to the task. |
| `status` | String | Current status of the task. |
| `has_sheet` | Boolean | Whether the task has time entries (apontamentos) logged against it. |
| `initial_date` | String | Start date of the task, kept exactly as returned by the API. |
| `final_date` | String | Due date of the task, kept exactly as returned by the API. |
| `is_archived` | Boolean | Whether the task was archived. |
| `is_private` | Boolean | Whether the task is restricted to specific users of the office. |
| `owner_id` | String | Identifier of the user who created the task. |
| `points` | Number | Effort points assigned to the task. |
Columns of the task kanban, with the board and folder they belong to.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the kanban column. |
| `name` | String | Name of the kanban column. |
| `index` | Integer | Position of the column inside its board. |
| `office_id` | String | Identifier of the law office that owns the board. |
| `board_id` | String | Identifier of the kanban board. |
| `board_name` | String | Name of the kanban board. |
| `folder_id` | String | Identifier of the folder holding the board. |
| `folder_name` | String | Name of the folder holding the board. |
| `hex_color` | String | Color of the column, in hexadecimal notation. |
| `is_done_column` | Boolean | Whether tasks moved to this column are considered completed. |
| `is_first_column` | Boolean | Whether this is the entry column of the board. |
| `created_at` | DateTime | Timestamp when the column was created. |
| `updated_at` | DateTime | Timestamp when the column was last updated. |
Court publications and notices captured for the office, including the AI-generated summary.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the publication. |
| `title` | String | Title of the publication. |
| `content` | String | Full text of the publication. |
| `description_small` | String | Short summary of the publication, generated by Juridiq's AI. |
| `official_diary` | String | Official gazette (diário oficial) the publication came from. |
| `availability_date` | String | Date the publication was made available in the gazette, kept exactly as returned by the API. |
| `publication_date` | String | Date the publication took legal effect, kept exactly as returned by the API. |
| `location` | String | Location the publication refers to. |
| `process_number` | String | Case number (CNJ format) the publication refers to. |
| `law_suit_id` | String | Identifier of the law suit the publication was matched to, when matched. |
| `is_handled` | Boolean | Whether the office already processed the publication. |
| `read_at` | DateTime | Timestamp when the publication was read. |
| `read_by` | String | User who read the publication (JSON object with id, name and image\_url). |
| `has_similarity` | Boolean | Whether Juridiq found other publications similar to this one. |
Court hearings scheduled by the office.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the hearing. |
| `title` | String | Title of the hearing. |
| `date` | String | Date the hearing takes place, kept exactly as returned by the API. |
| `initial_hour` | String | Time the hearing starts. |
| `final_hour` | String | Time the hearing ends. |
| `modality` | String | Whether the hearing is online or in person. |
| `type` | String | Type of the hearing, as classified by Juridiq. |
| `audience_link` | String | Link to the hearing, for online hearings. |
| `description` | String | Free-form description of the hearing. |
| `process_number` | String | Case number (CNJ format) the hearing belongs to. |
| `law_suit_id` | String | Identifier of the related law suit. |
| `person_id` | String | Identifier of the related person. |
| `completed_at` | DateTime | Timestamp when the hearing was marked as completed. |
| `is_private` | Boolean | Whether the hearing is restricted to specific users of the office. |
| `has_sheet` | Boolean | Whether the hearing has time entries (apontamentos) logged against it. |
| `location` | String | Place where the hearing takes place. |
| `participants` | String | People taking part in the hearing (JSON array of objects with id, name and email). |
| `owner_id` | String | Identifier of the office user responsible for the hearing. |
| `created_at` | DateTime | Timestamp when the hearing was created. |
| `updated_at` | DateTime | Timestamp when the hearing was last updated. |
Calendar events. Tasks and hearings are excluded here because they have their own streams.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the event. |
| `title` | String | Title of the event. |
| `start` | String | Start of the event, kept exactly as returned by the API. |
| `end` | String | End of the event, kept exactly as returned by the API. |
| `all_day` | Boolean | Whether the event spans the whole day. |
| `date` | String | Date the event happens, kept exactly as returned by the API. |
| `initial_hour` | String | Time the event starts. |
| `final_hour` | String | Time the event ends. |
| `type` | String | Type of the calendar item. |
| `status` | String | Current status of the event. |
| `priority` | String | Priority of the event. |
| `is_private` | Boolean | Whether the event is restricted to specific users of the office. |
| `tags` | String | Tags applied to the event (JSON array of objects with id, name and color). |
| `link` | String | External link of the event, for Google entries. |
| `text_color` | String | Text color used to render the event. |
| `background_color` | String | Background color used to render the event. |
| `border_color` | String | Border color used to render the event. |
| `class_names` | String | CSS classes used to render the event. |
Time logged by the office's users against tasks, subtasks and hearings.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the time entry. |
| `title` | String | Title of the time entry. |
| `module` | String | What the time was logged against: TASK, AUDIENCE, SUBTASK or GENERAL. |
| `is_done` | Boolean | Whether the time entry was completed. |
| `status` | String | Current status: PENDING, IN\_PROGRESS, PAUSED, COMPLETED or CANCELLED. |
| `office_id` | String | Identifier of the law office. |
| `user_id` | String | Identifier of the user who logged the time. |
| `task_id` | String | Identifier of the related task. |
| `audience_id` | String | Identifier of the related hearing. |
| `subtask_id` | String | Identifier of the related subtask. |
| `estimated_time_minutes` | Integer | Time estimated for the work, in minutes. |
| `time_spent_seconds` | Integer | Total time actually spent, in seconds, summed across the status history. |
| `cancel_reason` | String | Reason given when the entry was cancelled. |
| `completed_at` | DateTime | Timestamp when the entry was completed or cancelled. |
| `vinculate_item` | String | Task, subtask or hearing the time was logged against (JSON object with id, name and process\_number). |
| `user_detail` | String | User who logged the time (JSON object with name and url\_image). |
| `status_histories` | String | Status transitions of the entry (JSON array of objects with id, description, status, started\_at, ended\_at and time\_spent). |
| `this_user_can_edit` | Boolean | Whether the API key's user owns the entry and may edit it. |
| `created_at` | DateTime | Timestamp when the entry was created. |
| `updated_at` | DateTime | Timestamp when the entry was last updated. |
Documents sent for electronic signature through PlugSign.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the signature request. |
| `document_name` | String | File name of the document sent to sign. |
| `sign_name` | String | Display name of the signature request. |
| `document_key` | String | Key identifying the document in PlugSign. |
| `signing_key` | String | Key identifying the signing session in PlugSign. |
| `status` | String | Current status of the request (e.g. pending, signed, cancelled). |
| `signatories` | String | People asked to sign the document (JSON, as returned by the API). |
| `responsible_id` | String | Identifier of the office user responsible for the request. |
| `responsible` | String | Office user responsible for the request (JSON object with id, name and image\_url). |
| `person_on_signatures` | String | People from the office's CRM linked to the request (JSON, as returned by the API). |
| `is_private` | Boolean | Whether the document is restricted to specific users of the office. |
| `created_at` | DateTime | Timestamp when the request was created. |
| `updated_at` | DateTime | Timestamp when the request was last updated. |
Batches grouping several documents sent for signature at once.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the signature batch. |
| `name` | String | Name given to the batch. |
| `plugsign_lote_id` | String | Identifier of the batch in PlugSign. |
| `status` | String | Current status of the batch. |
| `signatories` | String | People asked to sign the batch (JSON, as returned by the API). |
| `office_id` | String | Identifier of the law office that owns the batch. |
| `responsible_id` | String | Identifier of the office user responsible for the batch. |
| `responsible` | String | Office user responsible for the batch (JSON object with id, name and image\_url). |
| `person_on_batches` | String | People from the office's CRM linked to the batch (JSON, as returned by the API). |
| `signatures` | String | Documents that make up the batch (JSON array of objects with id, status, document\_name, sign\_name and created\_at). |
| `is_private` | Boolean | Whether the batch is restricted to specific users of the office. |
| `expire_date` | DateTime | Timestamp when the batch expires. |
| `created_at` | DateTime | Timestamp when the batch was created. |
| `updated_at` | DateTime | Timestamp when the batch was last updated. |
Members of the law office, with their role and permission profile.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the user. |
| `name` | String | Full name of the user. |
| `email` | String | Email address used to sign in. |
| `document` | String | Brazilian tax document, CPF or CNPJ. |
| `contact` | String | Phone or other contact information. |
| `role` | String | Role of the user inside the office. |
| `profile` | String | Permission profile of the user (JSON object with id and name). |
| `oab_number` | String | Brazilian bar association number (OAB). |
| `ctps_number` | String | Brazilian work card number (CTPS). |
| `person_type` | String | Whether the user is registered as an individual or a company. |
| `salary` | String | Salary registered for the user. |
| `admission_date` | String | Date the user joined the office, kept exactly as returned by the API. |
| `resignation_date` | String | Date the user left the office, kept exactly as returned by the API. |
| `date_of_birth` | String | Date of birth, kept exactly as returned by the API. |
| `postal_code` | String | Postal code (CEP) of the user's address. |
| `address_number` | String | Street number of the user's address. |
| `image_url` | String | URL of the user's picture. |
| `office_id` | String | Identifier of the law office. |
| `first_login` | Boolean | Whether the user has not signed in for the first time yet. |
| `has_password` | Boolean | Whether the user has a password set. |
| `has_google_linked` | Boolean | Whether the user linked a Google account to Juridiq. |
| `google_email` | String | Email of the linked Google account. |
| `google_integration_type` | String | How the Google calendar is synchronized for this user. |
| `created_at` | DateTime | Timestamp when the user was created. |
| `updated_at` | DateTime | Timestamp when the user was last updated. |
Tags used to classify cases, tasks and other records.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the tag. |
| `name` | String | Label of the tag. |
| `color` | String | Color used to render the tag. |
| `modules` | Array | Modules of Juridiq where the tag can be applied. |
| `office_id` | String | Identifier of the law office that owns the tag. |
| `created_at` | DateTime | Timestamp when the tag was created. |
| `updated_at` | DateTime | Timestamp when the tag was last updated. |
Action types (tipos de ação) used to classify cases.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the action type. |
| `name` | String | Name of the action type. |
| `action_group_id` | String | Identifier of the action group the type belongs to. |
| `action_group_name` | String | Name of the action group. |
| `office_id` | String | Identifier of the law office. |
| `created_at` | DateTime | Timestamp when the action type was created. |
| `updated_at` | DateTime | Timestamp when the action type was last updated. |
Action groups that organize the action types.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the action group. |
| `name` | String | Name of the action group. |
| `office_id` | String | Identifier of the law office. |
| `created_at` | DateTime | Timestamp when the action group was created. |
| `updated_at` | DateTime | Timestamp when the action group was last updated. |
Risk levels the office assigns to its cases.
**Primary key:** `id` · **Sync type:** FULL\_TABLE
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the risk level. |
| `name` | String | Name of the risk level. |
| `office_id` | String | Identifier of the law office. |
# Kamino as a data source
Source: https://docs.nekt.com/sources/kamino
Bring data from Kamino to Nekt.
Kamino is a financial management platform that helps businesses manage their receivables, financial movements, and customer data.
![]()
## Configuring Kamino as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Kamino option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the following information from your Kamino account:
* **App Token**: The App Token to authenticate against the Kamino API service
* **Company Identifier**: The company identifier from your Kamino account (it should be a text such as `mycompany`)
* **User Identifier**: The user identifier associated with the generated credentials
* **User Token**: The user token associated with the generated credentials
* **Hash**: The hash associated with the API credentials
* **API URL**: The URL for the API service
These credentials can be found following the steps on [this page](https://kamino.readme.io/docs/getting-started).
Once you have all the required credentials, fill the required fields and click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
The `financial_movement` stream corresponds to the "Movimentação Financeira com rateio" endpoint by default. An advanced setting ("Extract financial movement with rateio") lets you switch to the plain endpoint that doesn't include allocation (rateio) information.
The `payables` stream corresponds to the accounts payable endpoint.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Streams and Fields
Below you'll find all available data streams from Kamino and their corresponding fields:
Stream for managing people and companies (customers, suppliers, contacts).
**Key Fields:**
* `ID` - Unique identifier for the person
* `Nome` - Name of the person or company
* `NomeExibicao` - Display name
* `CPFCNPJ` - Brazilian tax identification number
* `NomeFantasia` - Trade name (for companies)
* `Cliente` - Whether the person is a customer
* `Fornecedor` - Whether the person is a supplier
* `Ativo` - Whether the person is active
* `Email`, `Telefone`, `Celular` - Contact information
* `Logradouro`, `Nro`, `Bairro`, `CEP`, `Cidade`, `UF` - Address fields
* `AtribFlex1` - Custom flexible attribute
* `AlteradoEm`, `DataHoraInclusao` - Timestamps
Stream for receivables (accounts receivable) with payment details, boleto information, and bank reconciliation.
**Key Fields:**
* `ID` - Unique identifier for the receivable
* `Descri` - Description of the receivable
* `DtaVenc` - Due date
* `VlrVenc` - Amount due
* `VlrBruto` - Gross amount
* `IDPessoa` - Associated person identifier
* `Pessoa` - Associated person details (nested object)
* `DtaPagto`, `VlrPagto` - Payment date and amount
* `SitConta`, `DescricaoSituacao` - Status
* `NomeCentroCusto`, `NomeUnidadeNegocio` - Cost center and business unit
* `ContaPagamento` - Payment account details (nested object)
* `LinkBoleto`, `LinhaDigitavelBoleto` - Boleto payment information
Stream for receivables that have been excluded from the main receivables list. Includes exclusion reason and the original receivable data.
**Key Fields:**
* `ID` - Unique identifier for the exclusion record
* `IDConta` - Identifier of the excluded receivable
* `DataHoraExclusao` - When the record was excluded
* `Motivo` - Reason for exclusion
* `IDUsuario` - User who performed the exclusion
* `Usuario` - User details (nested object)
* `Recebimento` - Full excluded receivable data (nested object)
Stream for financial movements with optional cost allocation (rateio). Corresponds to the "Movimentação Financeira com rateio" endpoint when rateio is enabled (the default).
**Key Fields:**
* `ID` - Unique identifier for the movement
* `Tipo` - Type classification
* `NumeroFatia` - Sequential index (starting at 0) of this row within the financial movement, distinguishing allocation (rateio) slices and transfer legs that share the same `ID` and `Tipo`
* `Data`, `DataPagamento`, `DataVencimento`, `DataCompetencia` - Date fields
* `ValorTotal`, `ValorPrevisto`, `ValorRealizado` - Monetary values
* `PercentualRateio` - Allocation percentage for cost splitting
* `Realizado` - Whether the movement has been realized
* `IDContaOrigem`, `NomeContaOrigem` - Source account
* `IDContaDestino`, `NomeContaDestino` - Destination account
* `IDCentroCusto`, `NomeCentroCusto` - Cost center
* `IDPessoa`, `NomePessoa` - Associated person
* `Juros`, `Multa`, `Desconto`, `Taxas` - Interest, penalties, and fees
**Notes:**
* **Primary keys**: The stream uses a composite primary key consisting of `ID`, `Tipo`, and `NumeroFatia`. Kamino assigns movement IDs sequentially per `Tipo`, meaning different movements can share the same ID across types. Transfers (Tipo 3) are returned as two legs with the same ID and Tipo, and rateio mode returns one row per allocation slice with no natural key. `NumeroFatia` is a synthesized row index assigned during extraction to uniquely identify each slice.
* **Pagination and deduplication**: The Kamino paginated API can occasionally return the same record on two consecutive pages. Identical records re-read across page boundaries are automatically dropped before the `NumeroFatia` index is assigned to prevent creating phantom rows.
Stream for retrieving accounts payable.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `ID` | Integer | Unique identifier of the payable. |
| `Descricao` | String | Description of the payable. |
| `DataVencimento` | DateTime | Due date of the payable. |
| `DataCompetencia` | DateTime | Competence date for accounting. |
| `DataPagamento` | DateTime | Payment date; null while the payable is open. |
| `DataPrevisaoPagamento` | DateTime | Expected payment date. |
| `DataAgendaPagamento` | DateTime | Scheduled payment date. |
| `ValorVencimento` | Number | Amount due for the payable. |
| `ValorBruto` | Number | Gross amount of the payable. |
| `ValorPagamento` | Number | Amount actually paid; null while the payable is open. |
| `VlrDesc` | Number | Discount amount applied. |
| `VlrJuros` | Number | Interest amount applied. |
| `VlrMulta` | Number | Late payment penalty amount. |
| `TotalComposicaoValores` | Number | Total monetary value of the composition. |
| `Situacao` | Integer | Current status code of the payable. |
| `DescricaoSituacao` | String | Human-readable status description (e.g. Pago, Pendente). |
| `DataHoraSituacao` | DateTime | Timestamp of the last status change. |
| `IDUsuarioSituacao` | Integer | Identifier of the user who last changed the status. |
| `IDPessoa` | Integer | Identifier of the associated supplier. |
| `NomePessoa` | String | Name of the associated supplier. |
| `CPFCNPJPessoa` | String | Brazilian tax identification number of the supplier. |
| `Pessoa` | Object | Associated supplier details. |
| `IDCentroCusto` | Integer | Identifier of the associated cost center. |
| `NomeCentroCusto` | String | Cost center name. |
| `CentroCusto` | Object | Associated cost center details. |
| `IDUnidadeNegocio` | Integer | Identifier of the associated business unit. |
| `NomeUnidadeNegocio` | String | Business unit name. |
| `UnidadeNegocio` | Object | Associated business unit details. |
| `IDPlanoContaClassificacao` | String | Identifier of the classification chart of accounts. |
| `NomeContaClassificacao` | String | Classification chart of accounts name. |
| `ContaClassificacao` | Object | Classification chart of accounts details. |
| `IDPlanoContaPagamento` | String | Identifier of the payment chart of accounts. |
| `NomeContaPagamento` | String | Payment account name. |
| `ContaPagamento` | Object | Payment account details. |
| `IDContaPagamentoPreferencial` | Integer | Identifier of the preferred payment account. |
| `ContaPagamentoPreferencial` | Object | Preferred payment account details. |
| `FormaPagamento` | Integer | Payment method code. |
| `DescricaoFormaPagamento` | String | Human-readable payment method description. |
| `CanalInput` | Integer | Channel through which the payable was created. |
| `IDMoeda` | Integer | Identifier of the associated currency. |
| `SimbMoeda` | String | Currency symbol. |
| `SequenciaParcelas` | String | Installment sequence identifier. |
| `Obs` | String | Observations or notes for the record. |
| `NroBoleto` | String | Barcode digits for boleto payment. |
| `IDBoleto` | Integer | Identifier of the associated boleto. |
| `BoletoFileExtension` | String | File extension of the boleto attachment. |
| `NroNotaFiscal` | Integer | Invoice number. |
| `IDNotaFiscalEntrada` | Integer | Identifier of the inbound fiscal note. |
| `NotaFiscalFileExtension` | String | File extension of the invoice attachment. |
| `IsFromFiscalNote` | Boolean | Whether the payable was created from a fiscal note. |
| `NfseIdentificationCode` | String | Identification code of the municipal service invoice. |
| `ChavePixPessoa` | String | PIX key of the supplier. |
| `IDTipoChavePixPessoa` | Integer | Type code of the supplier's PIX key. |
| `NomeTipoChavePix` | String | Human-readable PIX key type. |
| `PixCopiaECola` | String | PIX copy-and-paste payment code. |
| `IDBancoPix` | Integer | Identifier of the PIX bank. |
| `NomeBancoPix` | String | Name of the PIX bank. |
| `NumeroAgenciaPix` | String | PIX branch number. |
| `NumeroContaPix` | String | PIX account number. |
| `IDTipoContaPix` | Integer | Type code of the PIX account. |
| `NomeBancoPessoa` | String | Name of the supplier's bank. |
| `CodigoBancoPessoa` | Integer | Bank code of the supplier's bank. |
| `AgenciaPessoa` | String | Branch number of the supplier's bank account. |
| `DigitoAgenciaPessoa` | String | Branch check digit of the supplier's bank account. |
| `ContaCorrentePessoa` | String | Supplier's bank account number. |
| `ContaCorrentePessoaSemDigito` | String | Supplier's bank account number without check digit. |
| `DigitoContaCorrentePessoa` | String | Check digit of the supplier's bank account. |
| `OperacaoContaPessoa` | String | Bank transfer operation type (e.g. TED). |
| `TipoContaBancaria` | Integer | Bank account type of the supplier. |
| `VincularEmailPessoa` | Boolean | Whether to link the supplier's email. |
| `EnviadoAoBanco` | Boolean | Whether the payment was sent to the bank. |
| `ArquivoGerado` | Boolean | Whether a remittance file was generated. |
| `DataArquivoGerado` | DateTime | Timestamp when the remittance file was generated. |
| `BaixaManual` | Boolean | Whether the payable was settled manually. |
| `BloqueiaEdicaoTotal` | Boolean | Whether full editing of the payable is blocked. |
| `ConciliacaoCartao` | Boolean | Whether the payable comes from credit card reconciliation. |
| `TransIDCartaoCredito` | String | Credit card transaction identifier. |
| `IDConciliacaoBancaria` | Integer | Identifier of the bank reconciliation. |
| `IDExtratoBanco` | Integer | Identifier of the associated bank statement. |
| `TransacaoExtrato` | Object | Bank statement transaction details. |
| `IDContaPagarOrigem` | Integer | Identifier of the source payable. |
| `IDContaPagarRelacaoImposto` | Integer | Identifier of the related tax payable. |
| `IDTabelaImposto` | Integer | Identifier of the tax table. |
| `IDTabelaImpostoRetencao` | Integer | Identifier of the tax withholding table. |
| `IDTipoImposto` | Integer | Identifier of the tax type. |
| `IDContrato` | Integer | Identifier of the associated contract. |
| `IDVenda` | Integer | Identifier of the associated sale. |
| `IDOrdemCompra` | Integer | Identifier of the associated purchase order. |
| `IDFaturaKamino` | Integer | Identifier of the associated Kamino invoice. |
| `IDPagamentoJuno` | String | Identifier of the Juno gateway payment. |
| `IDExterno` | String | External system reference identifier. |
| `CodigoExterno` | String | External system reference code. |
| `IDInbox` | Integer | Identifier of the associated inbox item. |
| `GuidDDA` | String | Globally unique identifier of the DDA record. |
| `GuidAzure` | String | Globally unique identifier of the Azure attachment. |
| `IDProjeto` | String | Identifier of the associated project. |
| `Projeto` | String | Project name. |
| `SplitEm` | DateTime | Timestamp when the payable was split. |
| `MesclagemAutomatica` | Boolean | Whether the payable was merged automatically. |
| `LancamentoDepartamentoPessoal` | Boolean | Whether the payable is a personnel department entry. |
| `PossuiAnexos` | Boolean | Whether the payable has attachments. |
| `QtdAnexos` | Integer | Number of attachments. |
| `ListaTipoAnexo` | String | Attachment types as a JSON string. |
| `IDUsuarioInclusao` | Integer | Identifier of the user who created the record. |
| `UsuarioInclusao` | Object | Details of the user who created the record. |
| `DataHoraInclusao` | DateTime | Timestamp when the record was created. |
| `UserIp` | String | IP address of the user who created the record. |
| `_DataVencimento` | DateTime | Internal representation of the due date. |
| `_DataCompetencia` | DateTime | Internal representation of the competence date. |
| `_DataPagamento` | DateTime | Internal representation of the payment date. |
## Skills for agents
Kamino connector documentation as plain markdown, for use in AI agent contexts.
# Kenlo as a data source
Source: https://docs.nekt.com/sources/kenlo
Bring data from Kenlo Imob to your Lakehouse.
Kenlo Imob is a Brazilian CRM for real estate agencies. The connector uses the Kenlo Open API v2 to extract your properties and the listings published on the channels, your customers, brokers and users, your leads with their proposals and history, your developments with their towers, units, attributes and combos, and the reference lists behind the property fields. With it you can analyze your portfolio, your funnel and your brokers' performance in your Lakehouse.
## Configuring Kenlo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Kenlo option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Kenlo gives API access through the **Kenlo Marketplace**, and only to agencies on the **K2 plan**. When the integration is registered in the Marketplace, Kenlo issues two values that are used together:
* the **API key**;
* the **user info**, a long Base64 text that identifies your agency.
In the Marketplace, the agency also chooses which data the integration may read (consents). To extract leads, their proposals and history, and the list of brokers and users, the lead-reading consent (`lead_get_status_consent`) must be enabled. Prices, owner data and full addresses also depend on their own consents; without them those fields come back empty.
The following configurations are available:
* **API Key**: the API key issued by Kenlo. Required.
* **User Info**: the user info issued together with the API key. Paste it exactly as you received it. Required.
* **Start Date**: the earliest last-update date of the properties and leads read on the first sync. Later syncs continue from where the previous one stopped. Leave it empty to read the whole history. Listings, customers, developments and the reference lists are always extracted in full.
* **Requests Per Minute** (advanced): how fast the connector may call Kenlo. The default is 60. The actual limit is set in your Kenlo contract; when Kenlo asks the connector to slow down, it waits and continues automatically.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Lead proposals and lead history take one request per lead each. On the first sync of an agency with many leads, they make the extraction noticeably longer; later syncs only visit the leads that changed.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: properties, leads, lead proposals and lead history are INCREMENTAL: each sync brings what changed since the previous one. The other streams are FULL\_TABLE: each sync replaces the table with what Kenlo currently holds. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Kenlo and their corresponding fields. API reference: [Kenlo Open API v2](https://kenlo-open-api-prd.kenlo.io/docs/v2.html).
Every table has a `raw_payload` column with the whole record exactly as Kenlo returned it, so information without a column of its own is still available. Lists and objects are stored as JSON text, and timestamps are converted to UTC. During Kenlo's free period, Kenlo masks personal data (such as phone numbers and e-mails) and limits the number of records.
Properties as registered in the agency's Kenlo CRM. Incremental: each sync reads the properties updated since the previous one (from one day before, as Kenlo filters by date).
Table: `properties` · Primary key: `id` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the property in the Kenlo CRM. Not the listing id, which is text such as OPEN\_PROPERTY\_84499613. |
| `property_code` | String | Business code of the property at the agency (e.g. AP-100). |
| `title` | String | Title of the property. |
| `description` | String | Description of the property. |
| `property_type` | String | Property type by name (e.g. APARTMENT), when Kenlo sends it. |
| `type_id` | Integer | Property type; see the 'type' catalog in property\_catalogs. |
| `purpose` | String | Purpose by name (e.g. SALE), when Kenlo sends it. |
| `purpose_id` | Integer | Purpose (sale, rent, seasonal); see the 'purpose' catalog. |
| `status_id` | Integer | Status of the property (e.g. active, inactive); see the 'status' catalog. |
| `owner_id` | Integer | Identifier of the owner (a customer). |
| `sale_price` | Number | Sale price, in BRL. |
| `rental_price` | Number | Monthly rent, in BRL. |
| `condo_fee` | Number | Monthly condominium fee, in BRL. |
| `iptu_value` | Number | IPTU (property tax), in BRL. |
| `total_area` | Number | Total area, in square meters. |
| `usable_area` | Number | Usable area, in square meters. |
| `common_area` | Number | Common area, in square meters. |
| `bedrooms` | Integer | Number of bedrooms. |
| `suites` | Integer | Number of suites. |
| `bathrooms` | Integer | Number of bathrooms. |
| `garages` | Integer | Number of parking spaces. |
| `full_address` | String | Full address in a single line (RESO unparsedAddress). |
| `street_name` | String | Street name. |
| `street_number` | String | Street number. |
| `unit_number` | String | Unit or complement (apartment, suite). |
| `neighborhood` | String | Neighborhood. |
| `city` | String | City. |
| `state` | String | State abbreviation (e.g. SP). |
| `postal_code` | String | Postal code (CEP). |
| `country` | String | Country code (e.g. BR). |
| `latitude` | Number | Latitude of the address. |
| `longitude` | Number | Longitude of the address. |
| `features` | Array of strings | Features of the property (e.g. GYM, SWIMMING\_POOL). |
| `photos` | String (JSON) | Photos of the property with their URLs (JSON array). |
| `owners` | String (JSON) | Owners of the property (JSON array). Empty unless the agency granted the owner data consent. |
| `captured_by` | String (JSON) | Broker who captured the property (JSON object). |
| `created_at` | Datetime | When the property was registered in the CRM. |
| `updated_at` | Datetime | When the property was last updated in the CRM, used as the incremental replication key. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Listings published on the channels (portals and the agency website). A property can have zero or many listings. Extracted in full on every sync.
Table: `listings` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Identifier of the listing (text such as OPEN\_PROPERTY\_84499613). Not the property id of the CRM. |
| `agency_id` | Integer | Identifier of the agency. |
| `property_code` | String | Business code of the property at the agency (e.g. AP-100). |
| `title` | String | Title of the listing. |
| `description` | String | Description of the listing. |
| `full_address` | String | Full address in a single line (RESO unparsedAddress). |
| `street_name` | String | Street name. |
| `street_number` | String | Street number. |
| `unit_number` | String | Unit or complement (apartment, suite). |
| `neighborhood` | String | Neighborhood. |
| `city` | String | City. |
| `state` | String | State abbreviation (e.g. SP). |
| `postal_code` | String | Postal code (CEP). |
| `country` | String | Country code (e.g. BR). |
| `latitude` | Number | Latitude of the address. |
| `longitude` | Number | Longitude of the address. |
| `sale_price` | Number | Advertised sale price, in BRL. |
| `rental_price` | Number | Advertised monthly rent, in BRL. |
| `media` | String (JSON) | Media of the listing (JSON array). |
| `created_at` | Datetime | When the listing was created. |
| `updated_at` | Datetime | When the listing was last updated. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Reference lists of the property fields, one row per option. Join `catalog = 'type'` and `value = properties.type_id` to get the name of a property type; the same works for usage, purpose, status and the other lists.
Table: `property_catalogs` · Primary key: `catalog`, `value` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `catalog` | String | Which list the option belongs to: usage, type, purpose, status, property\_standard, location\_standard, commission\_type, rate\_type, face, position or iptu\_payment\_condition. |
| `value` | Integer | Identifier of the option, as used in the property fields. |
| `label` | String | Readable name of the option. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Combinations of property type and purpose the Kenlo CRM accepts.
Table: `property_type_purposes` · Primary key: `type_id`, `purpose_id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `type_id` | Integer | Property type; see the 'type' catalog in property\_catalogs. |
| `purpose_id` | Integer | Purpose accepted for that type; see the 'purpose' catalog. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
People relevant to the agency: owners, clients, leads and brokers (see `role`). Extracted in full on every sync.
Table: `customers` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Identifier of the customer in the Kenlo CRM (digits, sent as text by this endpoint; the same number as client\_id in leads). |
| `name` | String | Name of the customer. |
| `person_type` | Integer | Type of person: 1 = individual (PF), 2 = company (PJ). |
| `cpf` | String | CPF of an individual customer. |
| `cnpj` | String | CNPJ of a company customer. |
| `media_id` | Integer | Identifier of the media (origin) the customer came from. |
| `email` | String | E-mail address of the customer. |
| `phone` | String | Phone number of the customer. |
| `role` | String | Role of the person for the agency: owner, client, lead or broker. |
| `created_at` | Datetime | When the customer was registered in the CRM. |
| `updated_at` | Datetime | When the customer was last updated in the CRM. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Brokers and users of the agency, active and inactive. Requires the agency's lead-reading consent.
Table: `agents` · Primary key: `user_id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `user_id` | Integer | Identifier of the user in the Kenlo CRM (responsible\_user\_id in leads). |
| `name` | String | Full name of the user. |
| `email` | String | E-mail address of the user. |
| `profile_id` | Integer | Profile of the user: 1 = admin, 2 = manager, 3 = broker, 4 = assistant, 5 = photographer. |
| `profile_name` | String | Readable name of the profile. |
| `active` | Boolean | Whether the user is active. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Leads (atendimentos) of the agency's funnel. Incremental: each sync reads the leads updated since the previous one. Requires the agency's lead-reading consent.
Table: `leads` · Primary key: `id` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the lead (clientAttendId), used by the lead routes. |
| `agency_id` | Integer | Identifier of the agency. |
| `client_id` | Integer | Identifier of the person behind the lead (a customer). |
| `client_name` | String | Name of the person. |
| `client_email` | String | E-mail address of the person (masked during Kenlo's free period). |
| `client_phone` | String | Phone number of the person (masked during Kenlo's free period). |
| `client_document` | String | CPF/CNPJ of the person. |
| `client_external_id` | String | Identifier of the person in an external system, when sent. |
| `stage` | String | Funnel stage: WITHOUT\_ATTENDANCE, ATTENDING, VISIT, PROPOSAL, GUARANTEE, CLOSURE or CONCLUDED. |
| `stage_id` | Integer | Numeric id of the funnel stage. |
| `operation` | String | Operation of interest: RENT, SALE or UNDEFINED. |
| `operation_id` | Integer | Numeric id of the operation of interest. |
| `responsible_user_id` | Integer | User (broker) responsible for the lead; see agents. |
| `brokers` | String (JSON) | Brokers assigned to the lead (JSON array). |
| `tags` | Array of strings | Tags of the lead (e.g. WhatsApp, Indicação). |
| `origin_media` | String | Where the lead came from (e.g. portal-kenlo). |
| `lead_scoring` | String | Qualification score of the lead, when scored. |
| `active` | Boolean | Whether the lead is active. |
| `is_archived` | Boolean | Whether the lead is archived. |
| `archiving_reason` | String | Reason chosen by the broker when archiving the lead. |
| `finished` | Boolean | Whether the lead's attendance is finished. |
| `last_history_type` | String | Type of the latest history entry of the lead. |
| `last_history_at` | Datetime | When the latest history entry of the lead was recorded. |
| `interested_properties` | String (JSON) | Properties of interest linked to the lead (JSON array). Best effort on Kenlo's side: missing when its property lookup fails. |
| `last_interaction_at` | Datetime | When the last interaction with the lead happened. |
| `created_at` | Datetime | When the lead was created. |
| `updated_at` | Datetime | When the lead was last updated, used as the incremental replication key. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Proposals of each lead, read again whenever the lead changes. Kenlo does not document the fields of this list, so the columns follow the proposal fields Kenlo accepts and everything else is in `raw_payload`.
Table: `lead_proposals` · Primary key: `lead_id`, `proposal_id` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `lead_id` | Integer | Identifier of the lead. |
| `lead_updated_at` | Datetime | Last update of the lead when its proposals were read, used as the incremental replication key. |
| `proposal_id` | String | Identifier of the proposal (a hash of the record when Kenlo sends no id). |
| `attend_property_id` | Integer | Identifier of the lead-property link the proposal is for. |
| `property_id` | Integer | Identifier of the property the proposal is for. |
| `status` | String | Status of the proposal. |
| `price` | Number | Value of the proposal, in BRL. |
| `operation` | String | Operation of the proposal (0 = sale, 1 = rent, 2 = both). |
| `payment_type` | String | Payment method of the proposal. |
| `tenant_id` | Integer | Identifier of the tenant or buyer (a customer). |
| `site_origin` | String | Where the proposal was made (CMS, INGAIASITES or APP). |
| `notes` | String | Notes of the proposal. |
| `expire_at` | Datetime | When the proposal expires. |
| `created_at` | Datetime | When the proposal was created. |
| `updated_at` | Datetime | When the proposal was last updated. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
History entries of each lead (stage changes, contacts, comments), read again whenever the lead changes. Kenlo does not document the fields of this list, so the columns follow the documented entry types and everything else is in `raw_payload`.
Table: `lead_history` · Primary key: `lead_id`, `history_id` · Sync: Incremental
| Field | Type | Description |
| :- | :- | :- |
| `lead_id` | Integer | Identifier of the lead. |
| `lead_updated_at` | Datetime | Last update of the lead when its history was read, used as the incremental replication key. |
| `history_id` | String | Identifier of the history entry (a hash of the record when Kenlo sends no id). |
| `type` | String | Type of the entry (e.g. ATTEND\_CREATED, STEP\_CHANGED, CONTACT\_WHATSAPP, COMMENT\_ADDED). |
| `description` | String | Text of the entry. |
| `user_id` | Integer | User who recorded the entry, when sent; see agents. |
| `operation_id` | Integer | Operation the entry refers to (1 = rent, 2 = sale). |
| `created_at` | Datetime | When the entry was recorded. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Real estate developments (empreendimentos). Extracted in full on every sync.
Table: `developments` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the development. |
| `name` | String | Name of the development. |
| `description` | String | Description of the development. |
| `full_address` | String | Full address in a single line (RESO unparsedAddress). |
| `street_name` | String | Street name. |
| `street_number` | String | Street number. |
| `unit_number` | String | Unit or complement (apartment, suite). |
| `neighborhood` | String | Neighborhood. |
| `city` | String | City. |
| `state` | String | State abbreviation (e.g. SP). |
| `postal_code` | String | Postal code (CEP). |
| `country` | String | Country code (e.g. BR). |
| `latitude` | Number | Latitude of the address. |
| `longitude` | Number | Longitude of the address. |
| `status_id` | Integer | Status of the development. |
| `phase_id` | Integer | Construction phase of the development. |
| `facade_id` | Integer | Facade type of the development. |
| `property_standard_id` | Integer | Construction standard of the development. |
| `partner_id` | Integer | Partner (developer or builder) of the development. |
| `created_at` | Datetime | When the development was registered. |
| `updated_at` | Datetime | When the development was last updated. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Towers of each development.
Table: `development_towers` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the tower. |
| `development_id` | Integer | Identifier of the development the tower belongs to. |
| `name` | String | Name of the tower. |
| `floors_count` | Integer | Number of floors. |
| `units_per_floor` | Integer | Number of units per floor. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Units of each tower.
Table: `development_units` · Primary key: `development_id`, `tower_id`, `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Identifier of the unit. |
| `development_id` | Integer | Identifier of the development. |
| `tower_id` | Integer | Identifier of the tower. |
| `unit_type_id` | Integer | Unit type; see development\_unit\_types. |
| `numbering` | String | Number of the unit (e.g. 1201). |
| `floor` | Integer | Floor of the unit. |
| `private_area` | Number | Private (usable) area, in square meters. |
| `total_area` | Number | Total area, in square meters. |
| `bedrooms` | Integer | Number of bedrooms. |
| `suites` | Integer | Number of suites. |
| `bathrooms` | Integer | Number of bathrooms. |
| `parking_spots` | Integer | Number of parking spots. |
| `sale_price` | Number | Sale price of the unit, in BRL. |
| `rental_price` | Number | Monthly rent of the unit, in BRL. |
| `is_sale` | Boolean | Whether the unit is for sale. |
| `is_rental` | Boolean | Whether the unit is for rent. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Technical attributes of each development (parking, leisure, security and so on). Each value is either a boolean or a number, depending on `variable_type`.
Table: `development_details` · Primary key: `development_id`, `detail_id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `development_id` | Integer | Identifier of the development. |
| `detail_id` | Integer | Identifier of the attribute. |
| `name` | String | Name of the attribute. |
| `variable_type` | String | Type of the value: Boolean, Decimal or Int32. |
| `value` | String | Value of the attribute, as text. |
| `value_boolean` | Boolean | Value of a Boolean attribute (empty for the other types). |
| `value_number` | Number | Value of a Decimal or Int32 attribute (empty for Boolean ones). |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Combos (predefined sets of units) of each development.
Table: `development_combos` · Primary key: `development_id`, `id`, `combo_item_id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `development_id` | Integer | Identifier of the development. |
| `id` | String | Identifier of the combo. |
| `combo_item_id` | Integer | Identifier of the combo item, when Kenlo lists combos item by item. |
| `name` | String | Name of the combo. |
| `units_count` | Integer | Number of units in the combo. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Sales systems available for developments.
Table: `development_sales_systems` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Identifier of the sales system. |
| `name` | String | Name of the sales system. |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
Unit types available for development units.
Table: `development_unit_types` · Primary key: `id` · Sync: Full table
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Identifier of the unit type. |
| `unit_type` | String | Name of the unit type. |
| `is_vertical` | Boolean | Whether the type is vertical (in a building). |
| `raw_payload` | String (JSON) | The whole record exactly as Kenlo returned it (JSON object), including fields that have no column of their own. |
# Kiwify as a data source
Source: https://docs.nekt.com/sources/kiwify
Bring data from Kiwify to your Lakehouse.
Kiwify is a Brazilian platform for selling digital products (online courses, memberships, ebooks and events) covering the checkout, payments, affiliate program and payouts. Nekt extracts this data through the Kiwify public API.
## Configuring Kiwify as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Kiwify option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
API access is created inside Kiwify, under **Apps > API**. Create a credential there and copy its values before leaving the screen: the client secret is displayed only once, at creation time.
The following configurations are required:
* **Client ID**: The client ID of the API credential.
* **Client Secret**: The client secret of the API credential. It is shown only once, when the credential is created.
* **Account ID**: The Kiwify account (store) ID shown next to the API credential. It identifies which store the extraction reads from.
Optionally, under advanced settings:
* **Start Date**: The earliest date from which records will be synced. Sales are swept from this date in 85-day windows, so an unnecessarily old date makes every sync slower. Defaults to two years ago.
* **Full Sale Details**: Whether sales are fetched with the payment breakdown, UTM tracking, revenue splits and affiliate commission. Enabled by default; turn it off to fetch only the summary fields.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
Monetary values are returned by Kiwify as integers in the smallest unit of the currency. An amount of `8853` means R\$ 88,53.
# Streams and Fields
Below you'll find all available data streams from Kiwify and their corresponding fields. Sales, daily statistics and event participants sync incrementally; the remaining streams are extracted as full tables. Product details, event participants and balances are fetched per parent record, so they cost one extra request per product or legal entity.
Orders placed on the store's checkout, incrementally synced by their last-change date. Because Kiwify filters orders by creation date in windows of at most 90 days, the connector sweeps the whole configured history on every run and asks only for the orders changed since the last sync, so refunds and chargebacks of older orders are picked up as well.
**Key Fields:**
* `id` - Unique identifier of the sale
* `reference` - Short order reference shown to the buyer
* `status` - Current status (paid, waiting\_payment, refunded, chargedback, refused, and so on)
* `payment_method` - How the order was paid: credit\_card, boleto or pix
* `net_amount` - Amount credited to the producer after fees, in cents
* `currency` - Currency the order was charged in
* `sale_type` - Role of this account in the sale (producer, affiliate or co-producer)
* `created_at` / `updated_at` - Order placement and last-change timestamps
* `approved_date` / `refunded_at` - Payment approval and refund timestamps
* `product_id` / `product_name` - Product that was sold
* `customer_id`, `customer_name`, `customer_email`, `customer_cpf`, `customer_cnpj`, `customer_mobile`, `customer_instagram`, `customer_country` - Buyer identification
* `customer_address_street`, `customer_address_number`, `customer_address_city`, `customer_address_state`, `customer_address_zipcode` - Buyer address
* `installments`, `card_type`, `card_last_digits`, `card_rejection_reason`, `two_cards`, `is_one_click` - Payment attempt details
* `payment_charge_amount`, `payment_net_amount`, `payment_fee`, `payment_product_base_price`, `payment_settlement_amount`, `payment_sale_tax_rate`, `payment_sale_tax_amount` - Full payment breakdown, with the matching currency fields
* `tracking_utm_source`, `tracking_utm_medium`, `tracking_utm_campaign`, `tracking_utm_content`, `tracking_utm_term`, `tracking_src`, `tracking_sck`, `tracking_s1`, `tracking_s2`, `tracking_s3` - Attribution parameters captured at the checkout
* `affiliate_commission_name`, `affiliate_commission_email`, `affiliate_commission_document`, `affiliate_commission_amount` - Affiliate credited for the sale
* `shipping_id`, `shipping_name`, `shipping_price` - Shipping option chosen for physical products
* `revenue_partners` - Co-producers who shared the revenue of the order (JSON)
* `parent_order_id` - Originating order, for renewals and upsells
Checkout performance metrics aggregated by day, incrementally synced. Useful for tracking approval and refund quality over time without recomputing it from the orders.
**Key Fields:**
* `date` - Day the metrics refer to
* `total_sales` - Number of sales approved on the day
* `total_net_amount` - Net revenue credited on the day, in cents
* `credit_card_approval_rate` - Share of credit card attempts approved, as a percentage
* `refund_rate` - Share of the day's sales that were refunded, as a percentage
* `chargeback_rate` - Share of the day's sales that were charged back, as a percentage
* `total_boleto_generated` / `total_boleto_paid` - Boletos issued and paid on the day
* `boleto_rate` - Share of the issued boletos that were paid, as a percentage
Products sold by the store, extracted as a full table.
**Key Fields:**
* `id` - Unique identifier of the product
* `name` - Name of the product as shown on the checkout
* `type` - Kind of product being sold (for example membership, course or event)
* `status` - Current status of the product
* `payment_type` - Whether the product is a one-off charge or a recurring subscription
* `price` - Price in cents, when the product has a base price
* `currency` - Currency the product is priced in
* `affiliate_enabled` - Whether the product accepts affiliates
* `created_at` - Product creation timestamp
Full checkout configuration of each product, extracted as a full table. Complements the Products stream with the data returned only by the per-product endpoint, so it costs one request per product on every run.
**Key Fields:**
* `id` - Unique identifier of the product
* `store_id` - Kiwify store that owns the product
* `offers` - Alternative price offers of the product (JSON)
* `links` - Checkout links of the product (JSON)
* `bumps` - Order bumps configured on the checkout (JSON)
* `subscriptions` - Subscription plans, for recurring products (JSON)
* `shipping_options` - Shipping options offered for physical products (JSON)
* `event_settings` - Event address, check-in options and sale window, for event products (JSON)
* `event_batches` - Ticket batches with prices and ticket counters, for event products (JSON)
* `revenue_partners` - Co-producers sharing the revenue of the product (JSON)
Affiliates approved to promote the store's products, extracted as a full table. The API returns one record per affiliate and product, so the same person appears once for each product they promote.
**Key Fields:**
* `affiliate_id` - Unique identifier of the affiliate
* `product_id` / `product_name` - Product the affiliate promotes
* `name` - Full name of the affiliate
* `email` - Email address of the affiliate
* `company_name`, `director_cpf`, `company_cnpj` - Registration data of the affiliate
* `commission` - Commission the affiliate earns on the product, in cents
* `status` - Current status of the affiliation
* `created_at` - Affiliation creation timestamp
Ticket holders of the store's event products, incrementally synced by their last-change date. Products that are not events simply return no participants.
**Key Fields:**
* `id` - Unique identifier of the participant
* `product_id` - Event product the ticket belongs to
* `external_id` - Ticket code used for check-in at the event
* `batch_id` / `batch_name` - Ticket batch the ticket was sold in
* `name`, `email`, `cpf`, `phone` - Participant identification
* `order_id` - Sale that issued the ticket
* `checkin_at` - Check-in timestamp at the event
* `created_at` / `updated_at` - Ticket issue and last-change timestamps
Withdrawals requested from the account's balance, extracted as a full table.
**Key Fields:**
* `id` - Unique identifier of the payout
* `amount` - Amount withdrawn, in cents
* `status` - Current status of the payout (for example pending\_review or paid)
* `legal_entity_id` - Legal entity that received the payout
* `created_at` / `updated_at` - Request and last-change timestamps
Current balance of each legal entity of the account, extracted as a full table snapshot.
**Key Fields:**
* `legal_entity_id` - Legal entity the balance belongs to
* `available` - Balance already released for withdrawal, in cents
* `pending` - Balance still held by the platform's settlement period, in cents
Registration data of the connected Kiwify account, extracted as a full table.
**Key Fields:**
* `id` - Unique identifier of the account (store)
* `company_name` - Registered company name of the account
* `director_cpf` / `company_cnpj` - Tax identification of the account
* `legal_entities` - Legal entities that can receive payouts, with their company data and pix key (JSON)
Webhooks configured on the account, extracted as a full table.
**Key Fields:**
* `id` - Unique identifier of the webhook
* `name` - Name given to the webhook
* `url` - Endpoint Kiwify posts the events to
* `products` - Product the webhook listens to, or "all" when it covers every product
* `triggers` - Events that fire the webhook (compra\_aprovada, compra\_reembolsada, chargeback, and so on)
* `created_at` / `updated_at` - Creation and last-change timestamps
# Klaviyo as a data source
Source: https://docs.nekt.com/sources/klaviyo
Bring data from Klaviyo to Nekt.
## Overview
Klaviyo is an email and SMS marketing automation platform used for targeted campaigns, customer segmentation, and marketing analytics. This connector extracts campaign metadata, automation flows, event data, and metric definitions from the Klaviyo API.
## Configuration
To connect Klaviyo to Nekt, you need a **Private API Key** with read access.
| Field | Description |
| - | - |
| **Private API Key** | Your Klaviyo private API key. Found in **Settings > API Keys** in your Klaviyo account. The key must have read scopes for campaigns, events, flows, and metrics. |
| **Start Date** | *(Optional)* Start date for incremental data sync in ISO 8601 format (e.g., `2024-01-01T00:00:00Z`). Only applies to streams that support incremental replication. |
To create an API key, go to your Klaviyo account **Settings > API Keys > Create Private API Key**. Grant **Read** access to Campaigns, Events, Flows, and Metrics.
Choose which streams you want to sync. Available streams are listed in the **Streams and Fields** section below.
For each selected stream, configure:
* **Layer**: Choose the destination layer (e.g., Bronze).
* **Folder**: Organize data within the layer.
* **Table name**: The table name in your lakehouse.
* **Sync Type**: Incremental (recommended) or Full Sync.
Set up the source-level configuration:
* **Description**: Add a description for this data source.
* **Triggers**: Configure sync triggers (schedule or manual).
* **Delta Log Retention Days**: Number of days to retain delta log history.
* **Full Sync**: Optionally force a full sync on first run.
After saving, verify the source appears in your **Sources** list. Run a manual sync to validate that data flows correctly.
## Streams and Fields
Email campaign metadata including status, audiences, scheduling, and tracking configuration. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique campaign identifier |
| `name` | string | Campaign name |
| `status` | string | Campaign status (e.g., draft, scheduled, sent) |
| `archived` | boolean | Whether the campaign is archived |
| `audiences` | string | Target audience configuration (JSON) |
| `send_options` | string | Send options configuration (JSON) |
| `tracking_options` | string | Tracking settings (JSON) |
| `send_strategy` | string | Send strategy configuration (JSON) |
| `created_at` | datetime | When the campaign was created |
| `scheduled_at` | datetime | When the campaign is scheduled to send |
| `updated_at` | datetime | When the campaign was last updated |
| `send_time` | datetime | Actual send time |
Raw event data including email opens, clicks, bounces, purchases, and custom events. Supports incremental sync via `datetime`. This is the granular data behind Klaviyo's reporting, enabling custom aggregations downstream.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique event identifier |
| `timestamp` | integer | Event timestamp (Unix epoch) |
| `event_properties` | string | Event-specific properties and metadata (JSON) |
| `datetime` | datetime | Event date and time (ISO 8601) |
| `uuid` | string | Event UUID |
Automation flow definitions including status and trigger configuration. Supports incremental sync via `updated`.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique flow identifier |
| `name` | string | Flow name |
| `status` | string | Flow status (e.g., draft, manual, live) |
| `archived` | boolean | Whether the flow is archived |
| `created` | datetime | When the flow was created |
| `updated` | datetime | When the flow was last updated |
| `trigger_type` | string | What triggers the flow (e.g., segment, metric, date) |
Metric definitions that describe the types of events tracked in Klaviyo (e.g., "Opened Email", "Clicked Email", "Placed Order"). Reference data for joining with events.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique metric identifier |
| `name` | string | Metric name |
| `created` | datetime | When the metric was created |
| `updated` | datetime | When the metric was last updated |
| `integration` | string | Integration details for the metric (JSON) |
## Implementation Notes
* **Authentication**: Uses Klaviyo's private API key via the `Klaviyo-API-Key` authorization header.
* **Rate limits**: The Klaviyo API enforces rate limits per endpoint. The connector handles rate limiting automatically with backoff.
* **Campaigns filter**: The campaigns stream currently syncs email campaigns only. SMS and push campaigns use separate channel filters.
* **Events volume**: The events stream can produce a high volume of records. Use the Start Date configuration to limit the initial sync window.
* **JSON fields**: Fields like `audiences`, `send_strategy`, `tracking_options`, `event_properties`, and `integration` are serialized as JSON strings for maximum flexibility. Parse them downstream as needed.
# Kobana as a data source
Source: https://docs.nekt.com/sources/kobana
Bring data from Kobana to Nekt.
Kobana is a payment processing platform designed for Brazilian businesses. It provides tools for accepting online payments, managing recurring billing, and handling financial transactions securely, helping companies streamline their payment operations and improve revenue management.
## 1. Add your Kobana access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Kobana option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Token:** The API token to authenticate against the API service. For detailed instructions, [check their documentation](https://developers.kobana.com.br/reference/token-de-acesso).
* **Start Date:** The date of the first record you want to sync with. We will extract every record created or updated after this.
3. Click **Next**.
## 2. Select your Kobana streams
1. The next step is letting us know which streams you want to bring. A stream is an entity and you can select the whole entity (meaning all available attributes will come in each table) or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your Kobana data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Kobana data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
Kobana connector documentation as plain markdown, for use in AI agent contexts.
# Kommo as a data source
Source: https://docs.nekt.com/sources/kommo
Bring data from Kommo to Nekt.
Kommo (formerly amoCRM) is a messaging-powered CRM platform that helps businesses manage leads, contacts, and sales pipelines through a conversation-driven approach. It centralizes communication channels and provides tools for sales automation, pipeline management, and team collaboration.
![]()
## Configuring Kommo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Kommo option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need a long-lived API token from your Kommo account. To generate one, go to **Settings > Integrations** in your Kommo account and create an integration to obtain the token.
The following configurations are available:
* **API Key**: Long-lived token of a Kommo private integration. In Kommo, open Settings > Integrations, select (or create) your private integration and, under Keys and scopes, click Generate long-lived token. Paste the whole token, not the Secret key or the Integration ID, and without a 'Bearer' prefix. The conversation messages (`talk_messages`) stream also needs the 'External chat history' permission, not available on the Base and Advanced plans.
* **Subdomain**: Your Kommo account name only: for `https://mycompany.kommo.com`, type `mycompany`, without `https://` or `.kommo.com`. A full address is also accepted and will be automatically normalized.
* **Start Date**: (Optional) The earliest date from which records will be synced for incremental streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked *optional* are unselected by default.
| Stream | Slug | Description |
| - | - | - |
| Leads | `leads` | Sales leads (deals). Supports incremental sync via `updated_at`. |
| Contacts | `contacts` | CRM contacts (people). |
| Companies | `companies` | CRM companies / organizations. |
| Pipelines | `pipelines` | Sales pipelines. |
| Pipeline Statuses | `pipeline_statuses` | Stages (statuses) within each pipeline. |
| Tasks | `tasks` | Tasks associated with leads and contacts. |
| Events | `events` | Activity / change event log. |
| Users | `users` | Account users. |
| Talks | `talks` | Conversations from Kommo CRM. |
| Talk Messages | `talk_messages` | *Optional.* Messages exchanged inside a Kommo conversation. Requires the 'External chat history' permission. |
## Fields by stream
Below you'll find all available data streams from Kommo and their corresponding fields:
Sales leads (deals) in your Kommo CRM. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| - | - | - |
| id | integer | Lead identifier |
| name | string | Lead name |
| price | integer | Sale amount |
| responsible\_user\_id | integer | Assigned user ID |
| group\_id | integer | Group identifier |
| status\_id | integer | Current pipeline stage ID |
| pipeline\_id | integer | Pipeline identifier |
| loss\_reason\_id | integer | Reason for lead loss |
| created\_by | integer | Creator user ID |
| updated\_by | integer | Last editor user ID |
| created\_at | integer | Creation timestamp (Unix) |
| updated\_at | integer | Last update timestamp (Unix) |
| closed\_at | integer | Closure timestamp (Unix) |
| closest\_task\_at | integer | Next task completion date (Unix) |
| is\_deleted | boolean | Whether the lead is deleted |
| custom\_fields\_values | string | Custom field values (JSON) |
| score | integer | Lead score |
| account\_id | integer | Account identifier |
| labor\_cost | integer | Associated labor costs |
| is\_price\_computed | boolean | Whether price was computed |
| source\_id | integer | Lead source identifier |
| is\_price\_modified\_by\_robot | boolean | Whether price was set by automation |
| tags | string | Associated tags (JSON) |
Contacts (people) in your Kommo CRM. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| - | - | - |
| id | integer | Contact identifier |
| name | string | Contact full name |
| first\_name | string | First name |
| last\_name | string | Last name |
| responsible\_user\_id | integer | Assigned user ID |
| group\_id | integer | Group identifier |
| created\_by | integer | Creator user ID |
| updated\_by | integer | Last editor user ID |
| created\_at | integer | Creation timestamp (Unix) |
| updated\_at | integer | Last update timestamp (Unix) |
| closest\_task\_at | integer | Next task completion date (Unix) |
| is\_deleted | boolean | Whether the contact is deleted |
| is\_unsorted | boolean | Whether the contact is unsorted |
| custom\_fields\_values | string | Custom field values (JSON) |
| account\_id | integer | Account identifier |
| tags | string | Associated tags (JSON) |
Companies (organizations) in your Kommo CRM. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| - | - | - |
| id | integer | Company identifier |
| name | string | Company name |
| responsible\_user\_id | integer | Assigned user ID |
| group\_id | integer | Group identifier |
| created\_by | integer | Creator user ID |
| updated\_by | integer | Last editor user ID |
| created\_at | integer | Creation timestamp (Unix) |
| updated\_at | integer | Last update timestamp (Unix) |
| closest\_task\_at | integer | Next task completion date (Unix) |
| is\_deleted | boolean | Whether the company is deleted |
| custom\_fields\_values | string | Custom field values (JSON) |
| account\_id | integer | Account identifier |
| tags | string | Associated tags (JSON) |
Sales pipelines configuration. Full table sync.
| Field | Type | Description |
| - | - | - |
| id | integer | Pipeline identifier |
| name | string | Pipeline name |
| sort | integer | Sort order |
| is\_main | boolean | Whether this is the main pipeline |
| is\_unsorted\_on | boolean | Whether unsorted leads are enabled |
| is\_archive | boolean | Whether the pipeline is archived |
| account\_id | integer | Account identifier |
Pipeline stages/statuses. Full table sync. Each status belongs to a pipeline.
| Field | Type | Description |
| - | - | - |
| id | integer | Status identifier |
| name | string | Status name |
| sort | integer | Sort order within pipeline |
| is\_editable | boolean | Whether the status can be edited |
| pipeline\_id | integer | Parent pipeline identifier |
| color | string | Status color code |
| type | integer | Status type |
| account\_id | integer | Account identifier |
| conditions | string | Stage conditions (JSON) |
Tasks linked to leads, contacts, or companies. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| - | - | - |
| id | integer | Task identifier |
| created\_by | integer | Creator user ID |
| updated\_by | integer | Last editor user ID |
| created\_at | integer | Creation timestamp (Unix) |
| updated\_at | integer | Last update timestamp (Unix) |
| responsible\_user\_id | integer | Assigned user ID |
| group\_id | integer | Group identifier |
| entity\_id | integer | Linked entity ID |
| entity\_type | string | Linked entity type (lead, contact, company) |
| duration | integer | Task duration |
| is\_completed | boolean | Whether the task is completed |
| task\_type\_id | integer | Task type identifier |
| text | string | Task description |
| result | string | Task result |
| complete\_till | integer | Task deadline (Unix) |
| account\_id | integer | Account identifier |
Activity events and audit trail. Supports incremental sync via `created_at`.
| Field | Type | Description |
| - | - | - |
| id | string | Event identifier |
| type | string | Event type |
| entity\_id | integer | Related entity ID |
| entity\_type | string | Related entity type |
| created\_by | integer | User who triggered the event |
| created\_at | integer | Event timestamp (Unix) |
| value\_after | string | Value after the change (JSON) |
| value\_before | string | Value before the change (JSON) |
| account\_id | integer | Account identifier |
Team members and users. Full table sync.
| Field | Type | Description |
| - | - | - |
| id | integer | User identifier |
| name | string | User full name |
| email | string | User email address |
| lang | string | User language |
| rights | string | User permissions (JSON) |
Conversations from Kommo CRM.
| Field | Type | Description |
| - | - | - |
| talk\_id | integer | Unique identifier of the conversation |
| created\_at | integer | Moment the conversation started, as a Unix timestamp in seconds |
| updated\_at | integer | Moment the conversation last changed, as a Unix timestamp in seconds |
| rate | integer | Rating the client gave the conversation |
| contact\_id | integer | Identifier of the contact the conversation belongs to |
| chat\_id | string | Identifier of the chat behind the conversation |
| entity\_id | integer | Identifier of the entity the conversation is attached to (matches a lead) |
| entity\_type | string | Type of the attached entity ('lead') |
| status | string | Conversation status ('in\_work', 'closed', etc.) |
| is\_in\_work | boolean | Whether the conversation is still open |
| is\_read | boolean | Whether the conversation has been read by a user |
| origin | string | Channel the conversation travelled through |
Messages exchanged inside a Kommo conversation. Supports incremental sync via `created_at`.
| Field | Type | Description |
| - | - | - |
| id | string | Unique identifier of the message |
| talk\_id | integer | Identifier of the parent conversation |
| chat\_id | string | Identifier of the chat the message was sent in |
| type | string | Direction of the message ('incoming' or 'outgoing') |
| message\_type | string | Kind of content carried ('text', 'contact', 'file', etc.) |
| text | string | Text of the message |
| author | string | Who sent the message (JSON) |
| recipient | string | Who received the message (JSON) |
| attachment | string | File sent with the message (JSON) |
| delivery\_status | string | Delivery outcome reported by the channel |
| origin | string | Channel the message travelled through |
| created\_at | integer | Moment the message was sent, as a Unix timestamp in seconds |
| sec\_created\_at | integer | Moment the message was sent, as a Unix timestamp in milliseconds |
# Data Model
The following diagram illustrates the relationships between the Kommo streams:
```mermaid theme={null}
erDiagram
PIPELINES ||--o{ PIPELINE_STATUSES : "has stages"
PIPELINES ||--o{ LEADS : "contains"
PIPELINE_STATUSES ||--o{ LEADS : "current stage"
LEADS }o--|| USERS : "responsible"
LEADS }o--o{ CONTACTS : "linked to"
LEADS }o--o{ COMPANIES : "linked to"
CONTACTS }o--|| USERS : "responsible"
COMPANIES }o--|| USERS : "responsible"
TASKS }o--|| LEADS : "linked to"
TASKS }o--|| CONTACTS : "linked to"
TASKS }o--|| COMPANIES : "linked to"
TASKS }o--|| USERS : "responsible"
EVENTS }o--|| LEADS : "about"
EVENTS }o--|| CONTACTS : "about"
EVENTS }o--|| COMPANIES : "about"
EVENTS }o--|| USERS : "created by"
TALKS }o--|| LEADS : "attached to"
TALKS }o--|| CONTACTS : "belongs to"
TALK_MESSAGES }o--|| TALKS : "contains"
```
## Implementation Notes
### API Limits & Behavior
* **Timeout**: The connector enforces a strict 300-second timeout on all API calls to Kommo to ensure stalled requests are properly retried and avoid hanging runs.
* **Permissions Requirements**: The `talk_messages` stream requires the **External chat history** permission, which is not available on the Base and Advanced Kommo plans. This stream is not selected by default. If the provided API Key does not have this permission and the stream is enabled, the connector will skip the stream and log a warning, rather than failing the entire run. To sync this data, open Kommo > Settings > Integrations, select the private integration that issued the key, enable 'External chat history' under the Chats permissions, and generate a new long-lived token.
* **Connection Validation**: The connector automatically validates the connection during setup, normalizes the subdomain (extracting the account name if a full URL is provided), and checks for common errors like expired tokens or missing accounts.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| A certificate error naming an unexpected domain or missing account error | **Subdomain** was filled with an invalid format or an account that does not exist. | Verify that the Subdomain setting contains the correct account identifier (e.g., `mycompany` for `mycompany.kommo.com`). Full URLs will be automatically corrected, but typos will fail validation. |
| `401` with a message about an invalid user, password, or expired token | The token expired, was generated on another account, or the user has no administrator profile. | Generate a new long-lived token while logged into the correct account, with an administrator user. |
| The `talk_messages` stream returns no data or logs a skipped warning | The **External chat history** permission is not granted on the private integration that issued the key. | Grant the permission and generate a new token. See [API Limits & Behavior](#api-limits-behavior). |
## Skills for agents
Kommo connector documentation as plain markdown, for use in AI agent contexts.
# Konfidency as a data source
Source: https://docs.nekt.com/sources/konfidency
Bring data from Konfidency to Nekt.
Konfidency is a product reviews and reputation platform for e-commerce. It collects and moderates customer reviews, questions and answers, and NPS survey responses for online stores, helping brands showcase social proof and understand customer sentiment. This connector extracts data from the Konfidency Reviews Admin API.
![]()
## Configuring Konfidency as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Konfidency option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll authenticate with Konfidency using OAuth2 credentials (client credentials flow). You can request these from your Konfidency account manager.
The following configurations are available:
* **Client ID**: The OAuth2 client ID for the Konfidency Reviews Admin API.
* **Client Secret**: The OAuth2 client secret for the Konfidency Reviews Admin API.
* **Customer**: Your Konfidency customer identifier, used as the API path prefix (e.g. `brandili`).
* **Start Date**: The earliest record date to sync. This applies to the incremental streams (reviews and NPS responses).
Once you fill the configuration, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select all streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Konfidency and their corresponding fields:
Product reviews submitted by customers. Synced incrementally using the `created` timestamp.
**Key Fields:**
* `_id` - Unique identifier of the review
* `sku` - SKU of the reviewed product
* `name` - Display name of the review author
* `rating` - Star rating given by the reviewer, from 1 to 5
* `text` - Free-text body of the review
* `status` - Moderation status (published, removed, filtered, pending)
* `created` - Timestamp when the review was created (incremental replication key)
* `moderated_at` - Timestamp when the review was moderated
**Engagement:**
* `helpful` - Number of users who marked the review as helpful
* `unhelpful` - Number of users who marked the review as unhelpful
* `verified` - Whether the review comes from a verified purchase
* `recommended` - Whether the reviewer recommends the product
**Order & Product (JSON-encoded):**
* `order_id` - Identifier of the order associated with the review
* `order_name` - Customer name from the associated order
* `order_email` - Customer email from the associated order
* `order_phone` - Customer phone from the associated order
* `pictures` - List of picture URLs attached to the review
* `product` - Product details (name, sku, url, image, brand, mpn, gtin, categories, specifications)
* `seller_info` - Seller information (id, name) for marketplace orders
Product questions and answers submitted by customers. Synced as a full table.
**Key Fields:**
* `_id` - Unique identifier of the question
* `status` - Moderation status (published, removed, filtered, sent)
* `created` - Timestamp when the question was created
* `customer` - Konfidency customer identifier
* `sku` - SKU of the product the question refers to
* `name` - Display name of the person asking
* `text` - Free-text body of the question
* `email` - Email of the person asking
**Nested (JSON-encoded):**
* `reply` - Reply object (text, created, helpful, unhelpful)
* `product` - Product details (sku, brand, categories, image, name, url, variants, status, sum\_ratings, total\_reviews)
NPS survey responses collected from customers. Synced incrementally using the `created` timestamp.
**Key Fields:**
* `_id` - Unique identifier of the NPS response
* `customer` - Konfidency customer identifier
* `survey_id` - Identifier of the survey the response belongs to
* `request_id` - Identifier of the survey request sent to the customer
* `rating` - NPS score given by the respondent, from 0 to 10
* `comment` - Free-text comment left by the respondent
* `partial` - Whether the response was only partially completed
* `created` - Timestamp when the response was recorded (incremental replication key)
* `send_date` - Timestamp when the survey was sent to the customer
**Respondent & Order:**
* `name` - Name of the respondent
* `phone` - Phone number of the respondent
* `order_id` - Identifier of the order associated with the response
Product groups that bundle related SKUs together. Synced as a full table.
**Key Fields:**
* `_id` - Unique identifier of the product group
* `customer` - Konfidency customer identifier
* `name` - Display name of the product group
**Nested (JSON-encoded):**
* `skus` - List of SKUs that belong to the group
* `products` - List of product details (sku, brand, categories, image, name, url, variants)
## Skills for agents
Konfidency connector documentation as plain markdown, for use in AI agent contexts.
# Kualiz as a data source
Source: https://docs.nekt.com/sources/kualiz
Bring data from Kualiz to your Lakehouse.
Kualiz is a Brazilian omnichannel customer-service and CRM platform (WhatsApp, Instagram, Facebook, Telegram, voice and webchat in a single queue), with a built-in CRM of pipelines, opportunities, products and FAQs. Nekt extracts this data through the Kualiz integration API.
## Configuring Kualiz as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Kualiz option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are required:
* **Instance URL**: The base URL of your Kualiz instance, for example `https://your-company.kualiz.com.br` (no trailing path). The integration endpoints live under this URL.
* **API Key**: The global integration API key of your instance. Generate it inside Kualiz under **Configurações** and paste it here. The key is sent with every request, so keep it secret.
* **Start Date**: (Optional) The earliest date from which records will be synced for incremental streams.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Kualiz and their corresponding fields. All streams are extracted as full tables.
Service queues (filas) configured in the instance, with live counters.
**Key Fields:**
* `id` - Unique identifier of the queue
* `name` - Display name of the queue
* `type` - Type of the queue
* `enabled` - Whether the queue is enabled in the system
* `connected` - Whether the queue is connected to the messaging service
* `authenticatedNumber` - Phone number authenticated on the queue
* `openChats` - Number of chats currently open in the queue
* `todaysRespondedChats` - Chats already answered in this queue today
* `loggedAgentsCount` - Number of agents currently logged into the queue
CRM pipelines (funis) and their stages.
**Key Fields:**
* `id` - Unique identifier of the pipeline
* `name` - Display name of the pipeline
* `description` - Description of the pipeline
* `stageorders` - Ordered list of stage IDs in the pipeline
* `stages` - Ordered stages of the pipeline (JSON array of objects)
* `maxdaysonpipeline` - Maximum days an opportunity may stay in the pipeline
* `lossreasons` - Configured loss reasons (JSON array of objects)
* `permissions` - Pipeline permission rules (JSON array of objects)
CRM opportunities. Iterated per queue and pipeline.
The API returns only the currently **open** opportunities of a pipeline; already-closed (won or lost) opportunities are not returned, so this stream is a snapshot of the open pipeline.
**Key Fields:**
* `id` - Unique identifier of the opportunity
* `title` - Title of the opportunity
* `value` - Value as an integer multiplied by 100 (e.g. 10000 = 100.00)
* `status` - Opportunity status: 0 open, 1 won, 2 lost
* `probability` - Probability of closing, from 0 to 100
* `fkPipeline` - ID of the pipeline the opportunity belongs to
* `fkStage` - ID of the current stage
* `responsableid` - ID of the responsible user
* `mainphone` / `mainmail` - Primary phone and email
* `expectedclosedate` - Expected close date (ISO 8601)
* `createdAt` / `closedat` - Creation and close dates (ISO 8601)
* `formsdata` - Custom form data (JSON object)
* `queueId` - Source queue the opportunity was fetched from
Products registered in the instance. Iterated per queue.
**Key Fields:**
* `id` - Unique identifier of the product
* `name` - Name of the product
* `internalcode` - Internal reference of the product
* `gtin` - GTIN (barcode) of the product
* `value` - Price as an integer multiplied by 100
* `saleprice` - Promotional price as an integer multiplied by 100
* `currency` - ISO 4217 currency code (e.g. BRL, USD)
* `available` - Whether the product is available
* `groups` - IDs of product groups the product belongs to
* `files` - Files associated with the product (JSON array of objects)
Product groups configured in the instance.
**Key Fields:**
* `id` - Unique identifier of the product group
* `name` - Name of the product group
* `description` - Description of the product group
Contact tags (etiquetas de contatos). Iterated per queue.
**Key Fields:**
* `id` - Unique identifier of the tag
* `name` - Name of the tag
* `bgcolor` - Background color of the tag
* `fgcolor` - Font color of the tag
Chat tags (etiquetas de atendimento). Iterated per queue.
**Key Fields:**
* `id` - Unique identifier of the chat tag
* `name` - Name of the chat tag
* `description` - Description of the chat tag
* `marker` - Emoji marker of the chat tag
* `priority` - Priority of the chat tag
* `locktag` - Whether agents cannot remove this tag from a chat
* `alertonpanel` - Whether this tag raises an alert on the panel
Frequently asked questions configured in the instance.
**Key Fields:**
* `id` - Unique identifier of the FAQ
* `question` - The FAQ question text
* `description` - The FAQ answer / description
* `keywords` - Keywords used to match the FAQ
* `rating` - Average rating of the FAQ
* `votes` - Number of votes the FAQ received
* `faqgroups` - IDs of FAQ groups the FAQ belongs to
FAQ groups configured in the instance.
**Key Fields:**
* `id` - Unique identifier of the FAQ group
* `name` - Name of the FAQ group
* `items` - Number of FAQs in the group
# LaunchLMS as a data source
Source: https://docs.nekt.com/sources/launch-lms
Bring data from LaunchLMS to your Lakehouse.
LaunchLMS is a white-label learning platform used to run corporate universities and online course catalogues. The connector extracts your product catalogue and its contents, the participants enrolled in each product and how far along they are, their assessment and quiz results, and the commercial side of the operation: orders, transactions, coupons, affiliates and subscription plans. The result is a clear view of training engagement and revenue in your Lakehouse.
Because LaunchLMS is white-labelled, every organization answers on its own domain, so you inform your own platform address when configuring the source.
## Configuring LaunchLMS as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the LaunchLMS option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
LaunchLMS authenticates with a static **integration token**, sent in the `Authorization` header. In your LaunchLMS platform, go to **Settings > Integrations** and open the **API** tab to generate and copy the token.
The integration token has super administrator privileges over your organization. Keep it private and rotate it periodically: you can generate a new one in the same screen at any time, and then update it here.
The following configurations are available:
* **Platform address**: the address of your LaunchLMS platform, including `https://`. Use the same address you use to sign in, for example `https://cursos.yourcompany.com.br`. Required.
* **Organization ID**: the identifier of the organization you want to sync. Open your organization's settings in LaunchLMS and copy the code shown in the browser address bar. Required.
* **Integration token**: the token copied from **Settings > Integrations > API**. Required.
* **Initial sync date**: the earliest date from which records will be synced. It applies to the Transactions stream, the only endpoint LaunchLMS lets the connector filter by date; every other stream is always read in full.
* **Form IDs** (advanced): the identifiers of the forms whose submissions you want to sync. LaunchLMS has no endpoint that lists an organization's forms, so each one has to be informed here: open the form in the platform and copy the code shown in the browser address bar. Leave it empty to skip the Form responses stream.
* **Profiles to sync** (advanced): leave it empty to sync every profile of the organization, or choose `user` to sync only the learners and leave administrator accounts out of the Users stream.
* **Page size** (advanced): the number of records requested per page on the paginated endpoints. The default of 100 works for most platforms; lower it if your platform times out on large listings.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Most streams are read per product: the connector lists your products and then reads the contents, plans, coupons, affiliates, bonuses, reviews, orders and participants of each one. Extraction time therefore grows with the size of your catalogue.
The **Participant grades** stream is read one enrolment at a time (one request per participant per product), so it is by far the slowest stream of the connector. Select it only when you actually need the assessment and quiz results.
Only read operations are covered. LaunchLMS also exposes routes that create users, enrolments and subscriptions; those are write operations and are not part of this connector.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Transactions supports INCREMENTAL; every other stream is full table, since LaunchLMS exposes no "modified since" filter on the remaining endpoints. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How the data is modelled
A few conventions are worth knowing before you query these tables:
* Nested payloads (the plans of a product, the resources of a content, the participants of an order, the grades of an enrolment) are stored as **JSON text columns** rather than nested structures, so the table shape never changes when LaunchLMS adds a field. Read them with `JSON_VALUE(column, '$.field')`.
* The currency of every monetary value is unpacked into `currency_code`, `currency_symbol` and `currency_minor_unit`. Use `currency_minor_unit` to know how many decimal places the amounts carry.
* Products and subscription plans carry further configuration whose presence depends on your platform settings; those extra fields are kept together in an `additional_fields` JSON column.
* Form submissions return one key per question, named after the question text, so the answers are kept in an `answers` JSON object keyed by the original question titles. Submissions have no identifier of their own in LaunchLMS, so the connector derives a stable one from the content of the submission.
# Streams and Fields
Below you'll find all available data streams from LaunchLMS and their corresponding fields.
Spaces of the organization, the branded areas products are published in (`GET /api/organizations/{idOrganization}/spaces`). Every participant belongs to one, so it is the dimension used to split reporting by brand or business unit.
Primary key: `id_space`
| Field | Type | Description |
| :- | :- | :- |
| `id_space` | String | Identifier of the space. Used as the primary key. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `code` | String | Short code of the space, used in the platform address and to look the space up through the API. |
| `name` | String | Display name of the space. |
| `icon` | String | Address of the icon shown for the space. |
| `logo` | String | Address of the logo shown for the space in the light theme. |
| `logo_dark_theme` | String | Address of the logo shown for the space in the dark theme. |
| `banner` | String | Address of the banner image displayed on the space home. |
| `banner_link` | String | Address the space banner links to when clicked. |
| `zendesk_enabled` | Boolean | Whether the Zendesk help widget is enabled in the space. |
Every user registered in the organization, learners and staff alike (`GET /api/organizations/{idOrganization}/users`). This is the people dimension the enrolments join back to.
Primary key: `id_user`
| Field | Type | Description |
| :- | :- | :- |
| `id_user` | String | Identifier of the user. Used as the primary key. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `name` | String | Name of the user. |
| `email` | String | Email address of the user. |
| `username` | String | Name the user signs in with. |
| `phone` | String | Phone number of the user. |
| `country_dial_code` | String | International dialling code of the user's phone number. |
| `photo` | String | Address of the user's profile picture. |
| `admin` | Boolean | Whether the user administers the organization. |
| `superadmin` | Boolean | Whether the user has full super administrator access. |
| `blocked` | Boolean | Whether the user is blocked from signing in. |
| `id_space` | String | Identifier of the space the user belongs to. |
| `space_name` | String | Name of the space the user belongs to. |
| `id_group` | String | Identifier of the group the user belongs to. |
| `group_name` | String | Name of the group the user belongs to. |
| `language` | String | Language the platform is shown to the user in. |
| `theme` | String | Interface theme chosen by the user: 'light' or 'dark'. |
| `notifications` | String | Notification preferences of the user, as a JSON object. |
| `additional_data` | String | Extra profile data LaunchLMS returns for the user in its 'data' field, as a JSON value. Its shape depends on the fields the organization collects at sign-up. |
| `issue_date` | Datetime | Date the user was registered. |
| `last_access_date` | Datetime | Date the user last signed in. |
The catalogue: courses, trainings, events, communities, ebooks and learning paths (`GET /api/organizations/{idOrganization}/products`). Every product-scoped stream below expands from this one.
Primary key: `id_product`
| Field | Type | Description |
| :- | :- | :- |
| `id_product` | String | Identifier of the product. Used as the primary key. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `code` | String | Short code of the product, used in the platform address. |
| `name` | String | Name of the product. |
| `type` | String | Kind of product: 'course', 'training', 'community', 'ebook', 'event', 'webinar', 'mentoring', 'lecture', 'template', 'report', 'spreadsheet', 'enterprise' or 'learningPath'. |
| `status` | String | Whether the product is 'active' or 'archived'. |
| `visibility` | String | Whether the product is listed publicly ('public') or only reachable by direct enrolment ('private'). |
| `description` | String | Description of the product shown on its sales page. |
| `amount` | Number | Price of the product, in the currency below. Check 'currency\_minor\_unit' for how many decimal places the value carries. |
| `amount_description` | String | Price as displayed to the buyer, for example '12x R\$ 99,00'. |
| `is_free` | Boolean | Whether the product is offered free of charge. |
| `currency_code` | String | ISO code of the currency of the amounts, for example 'BRL'. |
| `currency_symbol` | String | Symbol used to display the currency, for example 'R\$'. |
| `currency_minor_unit` | Integer | Number of decimal places the currency uses, so the monetary amounts can be scaled correctly. |
| `language` | String | Language the product is delivered in. |
| `workload` | Number | Estimated workload of the product, in hours. |
| `custom_url` | String | Custom address configured for the product's sales page. |
| `category_name` | String | Name of the category the product is filed under. |
| `subcategory_name` | String | Name of the subcategory the product is filed under. |
| `content_count` | Integer | Number of public contents the product holds. |
| `plans` | String | Enrolment plans offered for the product, as a JSON array. The 'product\_plans' stream carries the same plans as rows. |
| `issue_date` | Datetime | Date the product was created. |
| `additional_fields` | String | Remaining product configuration returned by LaunchLMS but not modelled as its own column, as a JSON object. Which keys appear depends on the settings enabled for the organization. |
Recurring plans sold by the organization (`GET /api/organizations/{idOrganization}/subscriptionPlans`), as opposed to the one-off enrolment plans attached to a single product.
Primary key: `id_subscription_plan`
| Field | Type | Description |
| :- | :- | :- |
| `id_subscription_plan` | String | Identifier of the subscription plan. Used as the primary key. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `code` | String | Short code of the subscription plan. |
| `name` | String | Name of the subscription plan. |
| `slogan` | String | Tagline displayed with the plan on the checkout page. |
| `description` | String | Description of what the subscription plan grants access to. |
| `status` | String | Whether the plan is 'active' or 'inactive'. |
| `visibility` | String | Whether the plan is 'public' or 'private'. |
| `amount` | Number | Price charged on each billing cycle of the subscription. |
| `signup_fee` | Number | One-off fee charged when the subscription starts. |
| `interval` | String | Billing interval of the subscription: 'month' or 'year'. |
| `duration` | Integer | Length of the access the subscription grants. |
| `trial_days` | Integer | Number of free trial days offered before the first charge. |
| `number_of_invoices` | Integer | Number of invoices the subscription issues before it finishes. Empty when the subscription renews indefinitely. |
| `max_installments` | Integer | Maximum number of credit card instalments accepted. |
| `payment_methods` | String | Payment methods accepted by the plan, as a JSON array. |
| `id_currency` | String | Identifier of the currency the plan is charged in. |
| `language` | String | Language the subscription plan is presented in. |
| `custom_url` | String | Custom address configured for the plan's checkout page. |
| `issue_date` | Datetime | Date the subscription plan was created. |
| `additional_fields` | String | Remaining plan configuration returned by LaunchLMS but not modelled as its own column, as a JSON object. |
Every payment movement of the organization, for both orders and subscriptions (`GET /api/organizations/{idOrganization}/transactions`). This is the only stream replicated incrementally.
Primary key: `id_transaction`
Replication key: `issue_date` (incremental sync)
| Field | Type | Description |
| :- | :- | :- |
| `id_transaction` | String | Identifier of the transaction. Used as the primary key. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `status` | String | State of the payment: 'waiting', 'paid', 'cancelled', 'refunded' or 'error'. |
| `amount` | Number | Amount the transaction was issued for. |
| `paid_amount` | Number | Amount actually settled by the payer. |
| `fee_amount` | Number | Fee charged by the payment provider on the transaction. |
| `currency_code` | String | ISO code of the currency of the amounts, for example 'BRL'. |
| `currency_symbol` | String | Symbol used to display the currency, for example 'R\$'. |
| `currency_minor_unit` | Integer | Number of decimal places the currency uses, so the monetary amounts can be scaled correctly. |
| `payment_method` | String | Means of payment used, for example credit card, pix or boleto. |
| `installments` | Integer | Number of credit card instalments the payment was split into. |
| `track_number` | String | Identifier of the transaction inside the payment provider. |
| `id_order` | String | Identifier of the enrolment order being paid. Empty when the transaction belongs to a subscription instead. |
| `id_subscription` | String | Identifier of the subscription being charged. Empty when the transaction belongs to a one-off order instead. |
| `responsible_name` | String | Name of the person responsible for the order or subscription. |
| `responsible_email` | String | Email address of the person responsible for the order or subscription. |
| `issue_date` | Datetime | Date the transaction was issued, used as the incremental replication key. |
| `payment_date` | Datetime | Date the transaction was settled. |
| `refund_date` | Datetime | Date the transaction was refunded, when it was. |
The lessons, files, quizzes and forms that make up each product (`GET /api/organizations/{idOrganization}/products/{idProduct}/contents`).
Primary key: `id_product`, `id_content`
| Field | Type | Description |
| :- | :- | :- |
| `id_content` | String | Identifier of the content. Part of the primary key together with the product. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `title` | String | Title of the content. |
| `type` | String | Kind of content, for example a video, a file or a quiz. |
| `id_module` | String | Identifier of the module the content belongs to. |
| `module_name` | String | Name of the module the content belongs to. |
| `level` | Integer | Depth of the content in the product's hierarchy. |
| `position` | Integer | Order of the content inside its module. |
| `duration` | Integer | Estimated time to consume the content, in minutes. |
| `min_duration` | Integer | Minimum consumption time, in minutes, before the content counts as completed. |
| `visibility` | String | Who can see the content: 'public' for every participant, or 'instructors'. |
| `preview_enabled` | Boolean | Whether the content is open as a free preview. |
| `forum_enabled` | Boolean | Whether the discussion forum is enabled on the content. |
| `review_enabled` | Boolean | Whether participants can rate the content. |
| `available_in_days` | Integer | Number of days after enrolment before the content is released to the participant. |
| `id_pre_requisite_content` | String | Identifier of the content that must be completed before this one opens. |
| `start_date` | Datetime | Date the content is released. |
| `end_date` | Datetime | Date the content expires. |
| `resources` | String | Resources attached to the content (videos, files, forms and questionnaires), as a JSON array. |
| `tags` | String | Tags applied to the content, as a JSON value. |
Enrolment plans offered for each product: price, instalments and how long access lasts (`GET /api/organizations/{idOrganization}/products/{idProduct}/plans`).
Primary key: `id_plan`
| Field | Type | Description |
| :- | :- | :- |
| `id_plan` | String | Identifier of the plan. Used as the primary key. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `description` | String | Name of the plan as shown on the product's checkout page. |
| `notice` | String | Notice displayed alongside the plan at checkout. |
| `amount` | Number | Price charged for the plan. |
| `original_amount` | Number | Price of the plan before any discount. |
| `amount_description` | String | Price as displayed to the buyer, for example '12x R\$ 99,00'. |
| `installments` | Integer | Maximum number of credit card instalments accepted. |
| `duration` | Integer | Length of the access the plan grants, in 'duration\_type' units. |
| `duration_type` | String | Unit of the access length: 'days', 'months' or 'years'. |
| `quantity` | Integer | Maximum number of seats the plan offers. |
| `available` | Boolean | Whether the plan can currently be bought, considering its period, remaining seats and the product's participant limit. |
| `start_date` | Datetime | Date the plan becomes available for purchase. |
| `end_date` | Datetime | Date the plan stops being available for purchase. |
Discount coupons created for each product, with how many confirmed orders redeemed them (`GET /api/organizations/{idOrganization}/products/{idProduct}/coupons`).
Primary key: `id_coupon`
| Field | Type | Description |
| :- | :- | :- |
| `id_coupon` | String | Identifier of the coupon. Used as the primary key. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `code` | String | Code the buyer types at checkout to claim the discount. |
| `description` | String | Description of the coupon. |
| `category` | String | Category the coupon is filed under. |
| `type` | String | How the discount is calculated: 'percentage' or 'amount' (a fixed sum). |
| `amount` | Number | Fixed discount granted, when the coupon type is 'amount'. |
| `percentage` | Number | Percentage discount granted, when the coupon type is 'percentage'. |
| `status` | String | Whether the coupon is 'active' or 'inactive'. |
| `quantity` | Integer | Maximum number of times the coupon can be redeemed in total. |
| `number_of_uses` | Integer | How many times a single participant may redeem the coupon. |
| `used` | Integer | Number of confirmed orders that redeemed the coupon. |
| `issue_date` | Datetime | Date the coupon was created. |
| `end_date` | Datetime | Date the coupon stops being valid. |
Affiliates promoting each product, with their coupon, sales volume and commission (`GET /api/organizations/{idOrganization}/products/{idProduct}/affiliates`).
Primary key: `id_affiliate`
| Field | Type | Description |
| :- | :- | :- |
| `id_affiliate` | String | Identifier of the affiliation. Used as the primary key. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `id_user` | String | Identifier of the user who acts as the affiliate. |
| `name` | String | Name of the affiliate. |
| `email` | String | Email address of the affiliate. |
| `active` | Boolean | Whether the affiliation is currently active. |
| `product_name` | String | Name of the product the affiliation applies to. |
| `id_coupon` | String | Identifier of the coupon that credits sales to the affiliate. |
| `coupon_code` | String | Code of the affiliate's coupon. |
| `coupon_description` | String | Description of the affiliate's coupon. |
| `coupon_amount` | Number | Fixed discount the affiliate's coupon grants. |
| `coupon_percentage` | Number | Percentage discount the affiliate's coupon grants. |
| `commission_type` | String | How the commission is calculated: 'percentage' or 'amount' (a fixed sum). |
| `commission_amount` | Number | Fixed commission paid per sale. |
| `commission_percentage` | Number | Percentage commission paid per sale. |
| `orders` | Integer | Number of confirmed orders credited to the affiliate. |
| `order_total_amount` | Number | Total value of the confirmed orders credited to the affiliate. |
| `order_commission_amount` | Number | Total commission earned on those confirmed orders. |
Products handed over as a bonus when someone enrols in another product (`GET /api/organizations/{idOrganization}/products/{idProduct}/bonuses`).
Primary key: `id_bonus`
| Field | Type | Description |
| :- | :- | :- |
| `id_bonus` | String | Identifier of the bonus. Used as the primary key. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `id_bonus_product` | String | Identifier of the product handed over as the bonus. |
| `bonus_product_name` | String | Name of the product handed over as the bonus. |
| `product_status` | String | Status of the product handed over as the bonus. |
| `quantity` | Integer | Maximum number of bonuses available. |
| `available` | Boolean | Whether the bonus can currently be granted, considering its period, remaining quantity and the product's status. |
| `start_date` | Datetime | Date the bonus starts being offered. |
| `end_date` | Datetime | Date the bonus stops being offered. |
Ratings and written reviews participants left on each product (`GET /api/organizations/{idOrganization}/products/{idProduct}/reviews`).
Primary key: `id_review`
| Field | Type | Description |
| :- | :- | :- |
| `id_review` | String | Identifier of the review. Used as the primary key. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `id_user` | String | Identifier of the participant who wrote the review. |
| `user_name` | String | Name of the participant who wrote the review. |
| `user_email` | String | Email address of the participant who wrote the review. |
| `rating` | Number | Score the participant gave the product. |
| `headline` | String | Title the participant gave the review. |
| `review_text` | String | Body of the review. |
| `review_favorite` | Boolean | Whether the review is highlighted on the product page. |
| `review_date` | Datetime | Date the review was written. |
Enrolment orders placed for each product, whatever their payment state (`GET /api/organizations/{idOrganization}/products/{idProduct}/orders`). Join to Transactions on `id_order` for the money movements.
Primary key: `id_order`
| Field | Type | Description |
| :- | :- | :- |
| `id_order` | String | Identifier of the order. Used as the primary key. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `status` | String | State of the order: 'open', 'confirmed' or 'cancelled'. |
| `name` | String | Name of the first participant enrolled by the order. |
| `email` | String | Email address of the first participant enrolled by the order. |
| `total_amount` | Number | Total amount of the order, after discounts. |
| `currency_code` | String | ISO code of the currency of the amounts, for example 'BRL'. |
| `currency_symbol` | String | Symbol used to display the currency, for example 'R\$'. |
| `currency_minor_unit` | Integer | Number of decimal places the currency uses, so the monetary amounts can be scaled correctly. |
| `payment_method` | String | Means of payment chosen for the order. |
| `installments` | Integer | Number of credit card instalments the order was split into. |
| `affiliate_name` | String | Name of the affiliate credited with the order. |
| `coupon_description` | String | Description of the discount coupon applied to the order. |
| `fiscal_invoice_error` | String | Error reported while issuing the invoice for the order, when one occurred. |
| `participants` | String | Participants enrolled by the order, as a JSON array. |
| `transaction_logs` | String | Errors recorded by the payment provider for this order, as a JSON array. Useful to explain why an order never reached 'confirmed'. |
| `issue_date` | Datetime | Date the order was placed. |
Enrolments: who is taking each product and how far along they are (`GET /api/organizations/{idOrganization}/products/{idProduct}/users`). This is the core stream for learning reporting: progress, status and last access date.
Primary key: `id_product`, `id_user`
| Field | Type | Description |
| :- | :- | :- |
| `id_user` | String | Identifier of the participant. Part of the primary key together with the product. |
| `id_product` | String | Identifier of the product the record belongs to. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `name` | String | Name of the participant. |
| `email` | String | Email address of the participant. |
| `phone` | String | Phone number of the participant. |
| `country_dial_code` | String | International dialling code of the participant's phone number. |
| `photo` | String | Address of the participant's profile picture. |
| `status` | String | State of the enrolment: 'active' or 'closed'. |
| `progress` | Number | Share of the product the participant has completed, in percent. |
| `blocked` | Boolean | Whether the participant is blocked from this product. |
| `present` | Boolean | Whether attendance was recorded for the participant. |
| `id_space` | String | Identifier of the space the participant belongs to. |
| `space_name` | String | Name of the space the participant belongs to. |
| `issue_date` | Datetime | Date the participant was enrolled in the product. |
| `due_date` | Datetime | Date the participant's access to the product expires. |
| `last_access_date` | Datetime | Date the participant last opened a content of the product. |
Assessment and quiz results of one participant in one product (`GET /api/organizations/{idOrganization}/products/{idProduct}/users/{idUser}/grades`). LaunchLMS exposes grades one enrolment at a time, so this stream issues one request per participant per product.
Primary key: `id_product`, `id_user`
| Field | Type | Description |
| :- | :- | :- |
| `id_product` | String | Identifier of the product the grades belong to. Part of the primary key together with the participant. |
| `id_user` | String | Identifier of the participant the grades belong to. Part of the primary key together with the product. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `evaluations` | String | Assessments of the product and the participant's mark on each, as a JSON array of objects with 'idEvaluation', 'name', 'grade', 'comment' and 'available'. |
| `quizzes_results` | String | Results of the quizzes the participant finished, as a JSON array of objects with 'quiz', 'content', 'grade', 'percentage', 'approved' and 'responseDate'. |
Submissions of the forms listed in the Form IDs setting (`GET /api/organizations/{idOrganization}/forms/{idForm}/formResponses`).
Primary key: `id_form`, `id_form_response`
| Field | Type | Description |
| :- | :- | :- |
| `id_form` | String | Identifier of the form the submission belongs to, as configured on the source. Part of the primary key. |
| `id_form_response` | String | Fingerprint of the submission, derived from its content because LaunchLMS does not give a submission its own identifier. Stable across runs, so re-reading a form updates the rows instead of duplicating them. Part of the primary key. |
| `id_organization` | String | Identifier of the LaunchLMS organization the record belongs to, as configured on the source. |
| `date` | Datetime | Date the form was submitted. |
| `name` | String | Name of the participant who submitted the form. Only returned when the form is attached to a content. |
| `email` | String | Email address of the participant who submitted the form. Only returned when the form is attached to a content. |
| `content` | String | Title of the content the form is attached to, when it is attached to one. |
| `product` | String | Name of the product the form was submitted from, when the form is attached to a content. |
| `answers` | String | Answers given in the submission, as a JSON object keyed by the question titles exactly as they are written in the form. |
# Leadlovers as a data source
Source: https://docs.nekt.com/sources/leadlovers
Bring data from Leadlovers to your Lakehouse.
Leadlovers is a Brazilian marketing-automation platform built around funnels, called machines. Each machine holds landing pages, capture forms, email sequences and the leads moving through them, and the platform also covers tags, custom lead fields, a members area with courses and students, and an optional WhatsApp add-on. Its Web API exposes the lead book, the machine structure, the placement of every lead inside the email sequences, and the account's plan and quota data.
## Configuring Leadlovers as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Leadlovers option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the Web API token of the Leadlovers account that holds the data. Sign in to Leadlovers, open **General settings**, go to **Account data**, click **Preferences** and copy the **Web API token**. The token does not expire, is personal to the account, and reaches only that account's data, so each source needs the token of its own Leadlovers account.
The following configurations are available:
* **Web API Token**: The Leadlovers Web API token, used to authenticate every API call.
* **Initial sync date**: The earliest lead registration date from which records will be synced. It applies to the `leads` stream only, which is the sole endpoint with a server-side date filter.
Leadlovers answers a rejected token with a generic error and the message "Token inválido". If the extraction fails on the very first stream, regenerate the token in Preferences and update it here.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
The `email_sequences`, `levels`, `forms` and `pages` streams request one page per machine, and `students` and `courses` request one page per product. On accounts with many machines or products they take proportionally longer, so deselect them if you don't need the funnel or members-area structure.
The `zaplovers_numbers` and `zaplovers_templates` streams depend on the ZapLovers WhatsApp add-on. When the account does not have it, the connector logs a warning and skips those streams instead of failing the run.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Only `leads` supports incremental sync, by the lead's registration date. Every other stream is read in full on each run, because the Leadlovers API offers no date filter on those endpoints. Note that a lead edited after it was captured keeps its original registration date, so an incremental `leads` sync brings new leads but not later edits to old ones. Schedule an **Additional Full Sync** if you need those updates.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| User | `user` | Identity of the Leadlovers user the token belongs to. |
| Account Data | `account_data` | Plan, lead quota and expiry data of the account. |
| Machines | `machines` | Funnels configured in the account, with view and lead totals. |
| Leads | `leads` | The account's lead book, filtered by registration date. |
| Tags | `tags` | Tags defined in the account and how many leads carry each one. |
| Dynamic Fields | `dynamic_fields` | Custom fields defined on the account's lead records. |
| Dynamic Field Options | `dynamic_field_options` | Selectable options of each dynamic field. |
| Email Sequences | `email_sequences` | Email sequences configured inside each machine. |
| Levels | `levels` | Individual emails that make up each sequence. |
| Lead Sequences | `lead_sequences` | Current placement of every lead across the email sequences. |
| Lead Sequences History | `lead_sequences_history` | Historical placement of leads across the email sequences. |
| Forms | `forms` | Capture forms of each machine, with conversion counters. |
| Pages | `pages` | Pages of each machine, with view and conversion counters. |
| Products | `products` | Products (members areas) registered in the account. |
| Students | `students` | Students enrolled in each product, with their course progress. |
| Courses | `courses` | Full structure of each course, including modules and classes. |
| ZapLovers Numbers | `zaplovers_numbers` | WhatsApp sender numbers approved for the account. |
| ZapLovers Templates | `zaplovers_templates` | WhatsApp message templates approved for each sender number. |
## Fields by stream
Below you'll find all available data streams from Leadlovers and their corresponding fields. Lists of objects (a lead's tags, a product's courses, a student's progress, a course's modules) are delivered as JSON strings.
Identity of the Leadlovers user the Web API token belongs to. One record per source.
**Key Fields:**
* `user_code`: Unique identifier of the Leadlovers user
* `user_name`: Name of the user
* `user_email`: Contact email address registered for the user
* `user_access_email`: Email address the user signs in with
* `user_photo`: URL of the user's profile picture
Plan, quota and expiry data for every account tied to the token. Useful to watch how close the account is to its lead limit.
**Key Fields:**
* `user_code`: Identifier of the Leadlovers user that owns the account
* `order`: Sequence number of the plan or bonus within the account
**Plan and quota:**
* `plan_name`: Commercial name of the contracted plan
* `plan_leads`: Maximum number of leads allowed by the plan
* `active_leads`: Number of leads currently counted against the plan quota
* `leads`: Total number of leads stored in the account
* `purchase`: Timestamp the plan or bonus was purchased
* `expiration`: Timestamp the plan or bonus expires
* `pointer_plan_or_bonus`: Whether the entry refers to the main plan or to a bonus
* `register_type`: Internal code describing how the entry was registered
The funnels of the account. Machines are the parent of the sequences, forms and pages streams.
**Key Fields:**
* `machine_code`: Unique identifier of the machine
* `machine_name`: Name given to the machine
* `machine_image`: URL of the cover image displayed for the machine
**Metrics:**
* `views`: Total number of page views accumulated by the machine
* `leads`: Total number of leads currently stored in the machine
The account's lead book. Synced incrementally by registration date, the only server-side date filter the API offers.
**Key Fields:**
* `email`: Email address of the lead; the account's natural key
* `name`: Full name of the lead
* `phone`: Phone number registered for the lead
* `registration_date`: Timestamp the lead was registered, used as the incremental replication key
**Profile:**
* `birthday`: Birth date declared by the lead
* `city`: City declared by the lead
* `state`: State declared by the lead
* `company`: Company the lead is associated with
* `gender`: Gender declared by the lead
* `photo`: URL of the profile picture associated with the lead
* `score`: Engagement score accumulated by the lead
Tags defined in the account, with the number of leads currently carrying each one.
**Key Fields:**
* `id`: Unique identifier of the tag
* `title`: Display name of the tag
* `count`: Number of leads currently carrying this tag
Custom fields added to the account's lead records.
**Key Fields:**
* `id`: Unique identifier of the dynamic field
* `name`: Internal name of the field
* `label`: Label shown to the visitor when the field is filled in
* `tag`: Merge tag used to reference the field in emails and pages
**Settings:**
* `type`: Input type of the field (text, list, date, and so on)
* `status`: Whether the field is currently active in the account
* `register_date`: Timestamp the dynamic field was created
Selectable options of the dynamic fields that render as dropdowns or radio buttons.
**Key Fields:**
* `id`: Unique identifier of the option
* `dynamic_field_id`: Identifier of the dynamic field the option belongs to
* `value`: Value stored on the lead when the option is selected
**Settings:**
* `score`: Score added to the lead when the option is selected
* `status`: Whether the option is currently active
* `register_date`: Timestamp the option was created
Email sequences configured inside each machine.
**Key Fields:**
* `machine_code`: Identifier of the machine the sequence belongs to
* `sequence_code`: Unique identifier of the email sequence inside the machine
* `sequence_name`: Name given to the email sequence
The individual emails that make up a sequence, in the order they are sent.
**Key Fields:**
* `machine_code`: Identifier of the machine the level belongs to
* `sequence_code`: Identifier of the email sequence the level belongs to
* `model_code`: Unique identifier of the email model used by the level
**Content:**
* `sequence`: Position of the level within the sequence
* `subject`: Subject line of the email sent at this level
Where each lead currently sits inside the account's email sequences. The API returns the leads nested under each sequence; the connector expands that into one row per lead and sequence, so the table can be joined directly against `machines` and `email_sequences`.
**Key Fields:**
* `machine_code`: Identifier of the machine holding the sequence
* `machine_name`: Name of the machine holding the sequence
* `sequence_code`: Identifier of the email sequence the lead is placed in
* `sequence_name`: Name of the email sequence the lead is placed in
* `lead_code`: Identifier of the lead within the machine
* `lead_id`: Global identifier of the lead in the Leadlovers account
**Lead data:**
* `email`: Email address of the lead
* `name`: Full name of the lead
* `phone`: Phone number of the lead
* `birthday`: Birth date declared by the lead
* `photo`: URL of the profile picture associated with the lead
* `city`: City declared by the lead
* `state`: State declared by the lead
* `company`: Company the lead is associated with
* `gender`: Gender declared by the lead
* `score`: Engagement score accumulated by the lead
* `source`: Origin the lead was captured from
* `level`: Position of the lead inside the email sequence
* `status`: Current status of the lead in the sequence
* `message`: Free-text message stored with the lead
* `tags`: Tags applied to the lead, as a JSON array of objects
* `registration_date`: Timestamp the lead was registered in the account
The historical view of the same relationship: sequences a lead has been through, not only the current placement. Same fields as `lead_sequences`.
**Key Fields:**
* `machine_code`: Identifier of the machine holding the sequence
* `sequence_code`: Identifier of the email sequence the lead went through
* `lead_code`: Identifier of the lead within the machine
**Lead data:**
* `lead_id`, `email`, `name`, `phone`, `birthday`, `photo`, `city`, `state`, `company`, `gender`, `score`, `source`, `level`, `status`, `message`, `tags`, `registration_date`: same meaning as in the `lead_sequences` stream
Capture forms configured inside each machine, with their conversion counters.
**Key Fields:**
* `machine_code`: Identifier of the machine the form belongs to
* `form_code`: Unique identifier of the form
* `form_name`: Name given to the form
**Metrics and content:**
* `views`: Number of times the form was displayed
* `conversions`: Number of leads captured through the form
* `page_html`: Raw HTML snippet used to embed the form on a page
Pages published inside each machine, with their view and conversion counters.
**Key Fields:**
* `page_code`: Unique identifier of the page
* `machine_code`: Identifier of the machine the page belongs to
* `page_name`: Name given to the page
* `path`: Public URL path of the page
**Metrics and metadata:**
* `views`: Number of times the page was visited
* `conversions`: Number of leads captured on the page
* `page_image`: URL of the page's preview thumbnail
* `page_root`: Identifier of the template the page was created from
* `user_id`: Identifier of the Leadlovers user that owns the page
Products (members areas) registered in the account.
**Key Fields:**
* `product_id`: Unique identifier of the product
* `product_name`: Name given to the product
* `product_hash`: Opaque hash of the product, used by the students and courses endpoints
* `courses`: Courses published inside the product, as a JSON array of objects
Students enrolled in each product's members area, including their progress per course.
**Key Fields:**
* `product_id`: Identifier of the product the student is enrolled in
* `id`: Unique identifier of the student
* `name`: Full name of the student
* `email`: Email address used by the student to sign in
**Activity:**
* `date_register`: Date the student was enrolled
* `last_access`: Date of the student's most recent access to the product
* `photo`: URL of the student's profile picture
* `courses`: Per-course progress of the student, including completed classes and completion percentage, as a JSON array of objects
The full structure of each course published inside a product, read one course at a time.
**Key Fields:**
* `id`: Unique identifier of the course
* `product_id`: Identifier of the product the course belongs to
* `title`: Title of the course
* `modules`: Modules of the course and the classes inside each one, as a JSON array of objects
**Settings:**
* `image`: URL of the course cover image
* `enabled`: Whether the course is currently published to students
* `coach_id`: Identifier of the user assigned as the course coach
* `coach_name`: Name of the user assigned as the course coach
* `moderation_enabled`: Whether student comments on the course are moderated
* `register_date`: Timestamp the course was created
WhatsApp sender numbers approved for the account. Requires the ZapLovers add-on.
**Key Fields:**
* `sender_phone`: Approved WhatsApp sender number, in the country code + area code + number format
WhatsApp message templates approved for each sender number. Requires the ZapLovers add-on.
**Key Fields:**
* `sender_phone`: Sender number the template is approved for
* `id`: Unique identifier of the template
* `name`: Name of the template
**Approval:**
* `language`: Language code the template was approved in
* `status`: Approval status of the template with WhatsApp
* `compoments`: Body, header and button components of the template, as a JSON array of objects (spelled as the API returns it)
# Leads2b as a data source
Source: https://docs.nekt.com/sources/leads2b
Bring data from Leads2b to Nekt.
Leads2b is a B2B lead generation and prospecting platform that helps businesses find and connect with potential customers. It provides tools for company research, contact discovery, and lead qualification to support sales and marketing efforts.
## Configuring Leads2b as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Leads2b** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Leads2b data using an integration token.
The following configurations are available:
* **Auth Token**: The key to authenticate against the API service. This token can be obtained from the Leads2b platform, on Parameterizations > Integrations.
* **Initial sync date**: Starting point for data extraction. Only records created or updated after this date will be synced. Format: `YYYY-MM-DD`.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings (if available):
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### 5. Check your new source
Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
# Streams and Fields
Below you'll find the available data streams from Leads2b. They represent the core entities in your sales and prospecting workflows:
Stream containing data about your established customers within Leads2b.
Stream containing product or service items available in your Leads2b catalog.
Stream containing data about leads in your pipeline.
Stream containing data about ongoing sales opportunities.
Stream containing data about closed or processing orders.
Stream containing data about prospects being mapped and engaged before becoming qualified leads.
# Data Model
The following diagram conceptually illustrates the relationships between the core data streams in Leads2b.
```mermaid theme={null}
graph TD;
Prospects("Prospects") --> Leads("Leads");
Leads --> Opportunities("Opportunities");
Opportunities --> Orders("Orders");
Opportunities --> Customers("Customers");
Orders --> Items("Items");
```
## Skills for agents
Leads2b connector documentation as plain markdown, for use in AI agent contexts.
# Lemlist
Source: https://docs.nekt.com/sources/lemlist
Connect your Lemlist data to Nekt.
Lemlist is a cold-outreach and sales-engagement platform for running multichannel campaigns (email, LinkedIn, calls) and tracking how leads respond. This connector extracts your campaigns, the leads enrolled in them, the full activity funnel (sends, opens, clicks, replies, and more), team senders, unsubscribes, webhooks, and sending schedules.
## Configuring Lemlist as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Lemlist** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
To connect Lemlist, you need:
* **API Key**: Your Lemlist API key.
You can generate one in the Lemlist app under **Settings > Integrations > API & Webhooks**, then click **Generate key**. The key is shown only once, so copy it immediately.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis:
* **Campaigns**
* **Campaign Leads**
* **Activities**
* **Team**
* **Team Senders**
* **Unsubscribes**
* **Hooks**
* **Schedules**
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
The **Activities** stream supports incremental sync (based on each activity's `createdAt`). All other streams are full table.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated.
Optionally, you can define some additional settings:
* **Start Date** (optional): The earliest date to fetch activities from. Only activities on or after this date will be extracted on the first run.
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Lemlist and their corresponding fields:
Outreach campaigns in your account, including status, the sending sequence, schedules, and senders. Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique campaign identifier (primary key) |
| `name` | string | Campaign name |
| `status` | string | Campaign status (e.g. `running`, `paused`, `draft`, `ended`, `archived`, `errors`) |
| `labels` | array | Categorization labels |
| `sequenceId` | string | Identifier of the campaign's message sequence |
| `scheduleIds` | array | Sending schedule identifiers used by the campaign |
| `teamId` | string | Owning team identifier |
| `createdBy` | string | User identifier of the creator |
| `createdAt` | datetime | Campaign creation timestamp |
| `hasError` | boolean | Whether the campaign currently has errors |
| `errors` | string | Campaign errors (JSON) |
| `creator` | object | Creator details (`userId`, `userEmail`) |
| `senders` | array | Sending mailboxes (`id`, `email`, `sendUserMailboxId`) |
Full per-lead snapshot for each campaign, including contact details and the lead's current state in the funnel. Child of Campaigns. Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique lead identifier (primary key) |
| `email` | string | Lead email address |
| `emailStatus` | string | Email deliverability status (`deliverable`, `risky`, `undeliverable`) |
| `firstName` | string | Lead first name |
| `lastName` | string | Lead last name |
| `companyName` | string | Lead company name |
| `companyDomain` | string | Lead company domain |
| `jobTitle` | string | Lead job title |
| `phone` | string | Lead phone number |
| `linkedinUrl` | string | Lead LinkedIn profile URL |
| `picture` | string | Lead picture URL |
| `timezone` | string | Lead timezone |
| `icebreaker` | string | Personalized icebreaker text |
| `state` | string | Current lead state in the campaign |
| `stateSystem` | string | Internal system state |
| `lastState` | string | Previous lead state |
| `status` | string | Lead status |
| `campaign_id` | string | Identifier of the parent campaign |
The activity funnel: every event performed for a lead in a campaign (emails sent/opened/clicked/replied/bounced/unsubscribed, LinkedIn actions, calls, and more). Supports incremental sync based on `createdAt`.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique activity identifier (primary key) |
| `type` | string | Activity type (e.g. `emailsSent`, `emailsOpened`, `emailsClicked`, `emailsReplied`, `emailsBounced`, `emailsUnsubscribed`, `linkedinSent`, ...) |
| `leadId` | string | Lead the activity refers to |
| `campaignId` | string | Campaign the activity belongs to |
| `sequenceId` | string | Sequence identifier |
| `stepId` | string | Sequence step identifier |
| `sequenceStep` | integer | Zero-indexed position of the step in the sequence |
| `totalSequenceStep` | integer | Total number of steps in the sequence |
| `createdAt` | datetime | When the activity occurred (replication key) |
Your Lemlist team account (a single record). Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique team identifier (primary key) |
| `name` | string | Team name |
| `userIds` | array | Identifiers of users in the team |
| `createdBy` | string | User identifier of the team creator |
| `createdAt` | datetime | Team creation timestamp |
| `creatorDomain` | string | Email domain of the team creator |
Team members (senders) and the campaigns they participate in. Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `userId` | string | Sender user identifier (primary key) |
| `campaigns` | array | Campaigns the sender is involved in (`_id`, `name`, `status`, `sendingChannels`) |
Unsubscribed email addresses and domains. Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique unsubscribe identifier (primary key) |
| `email` | string | Unsubscribed email address |
| `campaignId` | string | Campaign the unsubscribe originated from |
| `campaignName` | string | Campaign name |
| `unsubscribedAt` | datetime | When the unsubscribe occurred |
| `scope` | string | Scope of the unsubscribe (`campaign`, `team`, or `global`) |
Webhooks configured for your team (configuration, not delivered events). Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique webhook identifier (primary key) |
| `targetUrl` | string | Destination URL the webhook posts to |
| `type` | string | Event type the webhook listens for |
| `campaignId` | string | Campaign the webhook is scoped to, if any |
| `createdAt` | datetime | Webhook creation timestamp |
| `zapId` | integer | Associated Zapier identifier, if any |
Sending schedules that control when campaigns send. Full table sync.
| Field | Type | Description |
| :- | :- | :- |
| `_id` | string | Unique schedule identifier (primary key) |
| `name` | string | Schedule name |
| `secondsToWait` | integer | Delay between sends, in seconds |
| `timezone` | string | Schedule timezone |
| `start` | string | Daily start time (e.g. `09:00`) |
| `end` | string | Daily end time (e.g. `18:00`) |
| `weekdays` | array | Active weekdays (1=Monday ... 7=Sunday) |
| `public` | boolean | Whether the schedule is shared across the team |
| `teamId` | string | Owning team identifier |
| `createdBy` | string | User identifier of the creator |
| `createdAt` | datetime | Schedule creation timestamp |
| `deletedAt` | datetime | Deletion timestamp, if deleted |
| `deletedBy` | string | User identifier who deleted the schedule |
# Linear as a data source
Source: https://docs.nekt.com/sources/linear
Bring data from Linear to Nekt.
Linear is a modern project management and issue tracking tool designed for software development teams. It provides a streamlined interface for managing issues, projects, cycles, and team workflows with powerful features for planning, tracking, and collaboration.
![]()
## Configuring Linear as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Linear option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You can authenticate with Linear using OAuth credentials for a secure, refreshable access to your Linear workspace. This is the preferred authentication method as it maintains long-term access without requiring manual token renewal.
* **Start Date**: The earliest record date to sync. This determines how far back in time the connector will fetch data during the initial sync.
Once you fill the configuration, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Linear and their corresponding fields:
Stream containing all issues/tickets in your Linear workspace.
**Key Fields:**
* `id` - Unique identifier for the issue
* `title` - Title of the issue
* `description` - Detailed description of the issue
* `priority` - Priority level (1-4, where 1 is highest)
* `estimate` - Estimated effort/story points
* `createdAt` - When the issue was created
* `updatedAt` - When the issue was last updated
* `startedAt` - When work on the issue started
* `completedAt` - When the issue was completed
* `dueDate` - Due date for the issue
* `identifier` - Human-readable identifier (e.g., ENG-123)
* `number` - Issue number
* `priorityLabel` - Priority as text (Urgent, High, Medium, Low)
* `url` - URL to the issue in Linear
**Nested Objects:**
* `state` - Issue state (id, name, type, color)
* `team` - Assigned team (id, name, key)
* `assignee` - Assigned user (id, name, email, displayName)
* `creator` - User who created the issue
* `project` - Associated project (id, name, description, state, priority)
* `cycle` - Associated cycle (id, name, number, startsAt, endsAt)
* `parent` - Parent issue if this is a sub-issue
* `labels` - Array of labels attached to the issue
* `subscribers` - Users subscribed to the issue
* `comments` - Comments on the issue
Stream containing all users in your Linear organization.
**Key Fields:**
* `id` - Unique identifier for the user
* `name` - Full name of the user
* `displayName` - Display name
* `email` - Email address
* `active` - Whether the user is active
* `admin` - Whether the user has admin privileges
* `guest` - Whether the user is a guest
* `createdAt` - When the user was created
* `updatedAt` - When the user was last updated
* `lastSeen` - When the user was last seen
* `avatarUrl` - URL to the user's avatar
* `timezone` - User's timezone
* `statusEmoji` - Current status emoji
* `statusLabel` - Current status label
* `createdIssueCount` - Number of issues created by this user
**Nested Objects:**
* `organization` - Organization the user belongs to
* `teams` - Array of teams the user is a member of
Stream containing all teams in your Linear organization.
**Key Fields:**
* `id` - Unique identifier for the team
* `name` - Name of the team
* `key` - Team key/abbreviation
* `description` - Team description
* `private` - Whether the team is private
* `color` - Team color
* `icon` - Team icon
* `createdAt` - When the team was created
* `updatedAt` - When the team was last updated
* `cyclesEnabled` - Whether cycles are enabled for this team
* `cycleDuration` - Duration of cycles in weeks
* `cycleStartDay` - Day of the week cycles start
* `defaultIssueEstimate` - Default estimate for new issues
* `issueEstimationType` - Type of estimation used
**Nested Objects:**
* `defaultIssueState` - Default state for new issues
* `organization` - Organization the team belongs to
Stream containing all projects in your Linear workspace.
**Key Fields:**
* `id` - Unique identifier for the project
* `name` - Name of the project
* `description` - Project description
* `state` - Current state of the project
* `priority` - Project priority
* `createdAt` - When the project was created
* `updatedAt` - When the project was last updated
* `startedAt` - When the project was started
* `completedAt` - When the project was completed
* `targetDate` - Target completion date
* `progress` - Progress percentage (0-1)
* `url` - URL to the project in Linear
* `color` - Project color
* `icon` - Project icon
**Nested Objects:**
* `creator` - User who created the project
* `lead` - Project lead
* `members` - Array of project members
* `teams` - Array of teams associated with the project
Stream containing all cycles (sprints) in your Linear workspace.
**Key Fields:**
* `id` - Unique identifier for the cycle
* `name` - Name of the cycle
* `description` - Cycle description
* `number` - Cycle number
* `startsAt` - When the cycle starts
* `endsAt` - When the cycle ends
* `createdAt` - When the cycle was created
* `updatedAt` - When the cycle was last updated
* `completedAt` - When the cycle was completed
* `progress` - Progress percentage (0-1)
* `completedIssueCountHistory` - Array of completed issue counts over time
* `issueCountHistory` - Array of total issue counts over time
* `scopeHistory` - Array of scope changes over time
**Nested Objects:**
* `team` - Team this cycle belongs to
Stream containing all issue labels in your Linear workspace.
**Key Fields:**
* `id` - Unique identifier for the label
* `name` - Name of the label
* `description` - Label description
* `color` - Label color
* `createdAt` - When the label was created
* `updatedAt` - When the label was last updated
**Nested Objects:**
* `creator` - User who created the label
* `team` - Team this label belongs to
* `parent` - Parent label if this is a child label
* `children` - Array of child labels
Stream containing all customers in your Linear workspace.
**Key Fields:**
* `id` - Unique identifier for the customer
* `name` - Customer name
* `domains` - Array of customer domains
* `externalIds` - Array of external identifiers
* `approximateNeedCount` - Approximate number of customer needs
* `createdAt` - When the customer was created
* `updatedAt` - When the customer was last updated
**Nested Objects:**
* `status` - Customer status
* `tier` - Customer tier
## Data Model
The following diagram illustrates the relationships between the core data streams in Linear. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
Issues("Issues")
Users("Users")
Teams("Teams")
Projects("Projects")
Cycles("Cycles")
IssueLabels("Issue Labels")
Customers("Customers")
Issues -- "assignee.id" --> Users
Issues -- "creator.id" --> Users
Issues -- "team.id" --> Teams
Issues -- "project.id" --> Projects
Issues -- "cycle.id" --> Cycles
Projects -- "creator.id" --> Users
Projects -- "lead.id" --> Users
Cycles -- "team.id" --> Teams
IssueLabels -- "creator.id" --> Users
IssueLabels -- "team.id" --> Teams
```
## Use Cases for Data Analysis
Here are some valuable business intelligence use cases when consolidating Linear data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Team Velocity and Sprint Performance
Analyze team productivity and sprint completion rates across cycles.
**Business Value:**
* Track team velocity over time
* Identify capacity planning opportunities
* Measure sprint completion rates
* Optimize cycle duration and scope
```sql theme={null}
WITH
cycle_metrics AS (
SELECT
c.id AS cycle_id,
c.name AS cycle_name,
c.number AS cycle_number,
c.team.name AS team_name,
c.startsAt AS cycle_start,
c.endsAt AS cycle_end,
c.progress AS cycle_progress,
DATE_DIFF('day', c.startsAt, c.endsAt) AS cycle_duration_days,
CASE
WHEN c.completedAt IS NOT NULL THEN 'Completed'
WHEN c.endsAt < CURRENT_TIMESTAMP THEN 'Overdue'
WHEN c.startsAt <= CURRENT_TIMESTAMP THEN 'Active'
ELSE 'Planned'
END AS cycle_status
FROM nekt_raw.linear_cycles c
WHERE c.startsAt >= CURRENT_DATE - INTERVAL '6' MONTH
),
issue_metrics AS (
SELECT
i.cycle.id AS cycle_id,
COUNT(*) AS total_issues,
COUNT(CASE WHEN i.completedAt IS NOT NULL THEN 1 END) AS completed_issues,
SUM(COALESCE(i.estimate, 0)) AS total_story_points,
SUM(CASE WHEN i.completedAt IS NOT NULL THEN COALESCE(i.estimate, 0) ELSE 0 END) AS completed_story_points,
AVG(DATE_DIFF('hour', i.createdAt, COALESCE(i.completedAt, CURRENT_TIMESTAMP))) AS avg_completion_time_hours
FROM nekt_raw.linear_issues i
WHERE i.cycle.id IS NOT NULL
GROUP BY i.cycle.id
)
SELECT
cm.team_name,
cm.cycle_name,
cm.cycle_number,
cm.cycle_start,
cm.cycle_end,
cm.cycle_duration_days,
cm.cycle_status,
ROUND(cm.cycle_progress * 100, 2) AS progress_percentage,
COALESCE(im.total_issues, 0) AS total_issues,
COALESCE(im.completed_issues, 0) AS completed_issues,
ROUND(
CAST(COALESCE(im.completed_issues, 0) AS DOUBLE) / NULLIF(im.total_issues, 0) * 100,
2
) AS completion_rate_percentage,
COALESCE(im.total_story_points, 0) AS total_story_points,
COALESCE(im.completed_story_points, 0) AS completed_story_points,
ROUND(im.avg_completion_time_hours / 24.0, 2) AS avg_completion_time_days
FROM cycle_metrics cm
LEFT JOIN issue_metrics im ON cm.cycle_id = im.cycle_id
ORDER BY cm.team_name, cm.cycle_start DESC
```
### 2. Issue Resolution Analysis
Track issue resolution patterns and identify bottlenecks in the development process.
**Business Value:**
* Identify issues that take longest to resolve
* Analyze resolution patterns by priority and team
* Track issue aging and identify blockers
* Optimize workflow efficiency
```sql theme={null}
WITH
issue_analysis AS (
SELECT
i.id,
i.identifier,
i.title,
i.priority,
i.priorityLabel,
i.team.name AS team_name,
i.assignee.displayName AS assignee_name,
i.state.name AS current_state,
i.state.type AS state_type,
i.createdAt,
i.startedAt,
i.completedAt,
i.estimate,
DATE_DIFF('day', i.createdAt, COALESCE(i.completedAt, CURRENT_TIMESTAMP)) AS age_days,
CASE
WHEN i.completedAt IS NOT NULL THEN
DATE_DIFF('day', i.createdAt, i.completedAt)
END AS resolution_time_days,
CASE
WHEN i.completedAt IS NOT NULL THEN 'Completed'
WHEN i.startedAt IS NOT NULL THEN 'In Progress'
ELSE 'Not Started'
END AS issue_status,
CASE
WHEN i.priority = 1 THEN 'Urgent'
WHEN i.priority = 2 THEN 'High'
WHEN i.priority = 3 THEN 'Medium'
WHEN i.priority = 4 THEN 'Low'
ELSE 'No Priority'
END AS priority_category
FROM nekt_raw.linear_issues i
WHERE i.createdAt >= CURRENT_DATE - INTERVAL '3' MONTH
),
team_summary AS (
SELECT
team_name,
priority_category,
COUNT(*) AS total_issues,
COUNT(CASE WHEN issue_status = 'Completed' THEN 1 END) AS completed_issues,
AVG(resolution_time_days) AS avg_resolution_days,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY resolution_time_days) AS median_resolution_days,
COUNT(CASE WHEN age_days > 30 AND issue_status != 'Completed' THEN 1 END) AS stale_issues
FROM issue_analysis
GROUP BY team_name, priority_category
)
SELECT
team_name,
priority_category,
total_issues,
completed_issues,
ROUND(
CAST(completed_issues AS DOUBLE) / NULLIF(total_issues, 0) * 100,
2
) AS completion_rate_percentage,
ROUND(avg_resolution_days, 2) AS avg_resolution_days,
ROUND(median_resolution_days, 2) AS median_resolution_days,
stale_issues,
ROUND(
CAST(stale_issues AS DOUBLE) / NULLIF(total_issues, 0) * 100,
2
) AS stale_issue_percentage
FROM team_summary
WHERE total_issues > 0
ORDER BY team_name,
CASE priority_category
WHEN 'Urgent' THEN 1
WHEN 'High' THEN 2
WHEN 'Medium' THEN 3
WHEN 'Low' THEN 4
ELSE 5
END
```
### 3. Project Progress and Resource Allocation
Monitor project health and team allocation across active projects.
**Business Value:**
* Track project progress and identify at-risk projects
* Analyze resource allocation across projects
* Monitor project timeline adherence
* Optimize team capacity planning
```sql theme={null}
WITH
project_metrics AS (
SELECT
p.id AS project_id,
p.name AS project_name,
p.state AS project_state,
p.priority AS project_priority,
p.progress AS project_progress,
p.createdAt AS project_created,
p.startedAt AS project_started,
p.completedAt AS project_completed,
p.targetDate AS target_date,
p.lead.displayName AS project_lead,
CASE
WHEN p.completedAt IS NOT NULL THEN 'Completed'
WHEN p.targetDate < CURRENT_DATE AND p.completedAt IS NULL THEN 'Overdue'
WHEN p.startedAt IS NOT NULL THEN 'In Progress'
ELSE 'Not Started'
END AS project_status,
CASE
WHEN p.targetDate IS NOT NULL THEN
DATE_DIFF('day', CURRENT_DATE, p.targetDate)
END AS days_to_target
FROM nekt_raw.linear_projects p
WHERE p.archivedAt IS NULL
),
project_issues AS (
SELECT
i.project.id AS project_id,
COUNT(*) AS total_issues,
COUNT(CASE WHEN i.completedAt IS NOT NULL THEN 1 END) AS completed_issues,
COUNT(CASE WHEN i.startedAt IS NOT NULL AND i.completedAt IS NULL THEN 1 END) AS in_progress_issues,
COUNT(CASE WHEN i.startedAt IS NULL THEN 1 END) AS not_started_issues,
SUM(COALESCE(i.estimate, 0)) AS total_story_points,
SUM(CASE WHEN i.completedAt IS NOT NULL THEN COALESCE(i.estimate, 0) ELSE 0 END) AS completed_story_points,
COUNT(DISTINCT i.assignee.id) AS unique_assignees,
COUNT(DISTINCT i.team.id) AS teams_involved
FROM nekt_raw.linear_issues i
WHERE i.project.id IS NOT NULL
GROUP BY i.project.id
)
SELECT
pm.project_name,
pm.project_state,
pm.project_status,
pm.project_lead,
pm.project_created,
pm.project_started,
pm.target_date,
pm.days_to_target,
ROUND(pm.project_progress * 100, 2) AS progress_percentage,
COALESCE(pi.total_issues, 0) AS total_issues,
COALESCE(pi.completed_issues, 0) AS completed_issues,
COALESCE(pi.in_progress_issues, 0) AS in_progress_issues,
COALESCE(pi.not_started_issues, 0) AS not_started_issues,
ROUND(
CAST(COALESCE(pi.completed_issues, 0) AS DOUBLE) / NULLIF(pi.total_issues, 0) * 100,
2
) AS issue_completion_rate,
COALESCE(pi.total_story_points, 0) AS total_story_points,
COALESCE(pi.completed_story_points, 0) AS completed_story_points,
ROUND(
CAST(COALESCE(pi.completed_story_points, 0) AS DOUBLE) / NULLIF(pi.total_story_points, 0) * 100,
2
) AS story_points_completion_rate,
COALESCE(pi.unique_assignees, 0) AS team_members_involved,
COALESCE(pi.teams_involved, 0) AS teams_involved,
CASE
WHEN pm.project_status = 'Overdue' THEN 'At Risk'
WHEN pm.days_to_target <= 7 AND pm.project_progress < 0.8 THEN 'At Risk'
WHEN pm.project_progress >= 0.8 THEN 'On Track'
ELSE 'Monitor'
END AS health_status
FROM project_metrics pm
LEFT JOIN project_issues pi ON pm.project_id = pi.project_id
ORDER BY
CASE pm.project_status
WHEN 'Overdue' THEN 1
WHEN 'In Progress' THEN 2
ELSE 3
END,
pm.days_to_target ASC NULLS LAST
```
## Implementation Notes
### Data Quality Considerations
* Monitor issue state transitions for workflow analysis
* Validate estimate accuracy against actual completion times
* Track label usage consistency across teams
* Ensure proper project and cycle assignment
### Authentication & Security
* OAuth authentication is recommended over legacy access tokens for better security
* Refresh tokens are automatically managed to maintain long-term access
* Use service accounts with appropriate permissions for production environments
### Automation Opportunities
* Schedule weekly team velocity reports
* Set up alerts for overdue issues and projects
* Automate sprint retrospective data collection
* Generate monthly team performance dashboards
### Best Practices
* Use incremental sync for large datasets to improve performance
* Set appropriate start\_date to avoid syncing unnecessary historical data
* Migrate from legacy access tokens to OAuth authentication when possible
These SQL transformations provide a foundation for comprehensive Linear analytics. Customize the date ranges, filters, and metrics based on your specific team workflows and reporting needs.
## Skills for agents
Linear connector documentation as plain markdown, for use in AI agent contexts.
# LinkedIn as a data source
Source: https://docs.nekt.com/sources/linkedin
Bring organic data from your LinkedIn Organization Page or Member Profile to Nekt.
LinkedIn is the world's largest professional networking platform, connecting professionals and businesses globally. The LinkedIn Community Management API provides access to organic page analytics, follower statistics, post performance, and audience demographics for your organization's LinkedIn Page, as well as personal analytics for individual Member Profiles.
![]()
This connector extracts **organic** data from your LinkedIn Organization Page or authenticated Member Profile. For paid advertising data, see the [LinkedIn Ads](/sources/linkedin-ads) connector.
## Configuring LinkedIn as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the LinkedIn option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your LinkedIn data. Click on the **LinkedIn Authorization** button and log in with your LinkedIn account.
If you intend to extract data from an Organization Page, make sure you're logged in with a LinkedIn account that has **Admin** or **Content Admin** access to that specific page.
Once you authorize your account, add the following information:
* **Extract data from organization account**: (Default: disabled) When enabled, extracts data from a LinkedIn organization page (posts, followers, page statistics). When disabled, extracts analytics data from the authenticated member's personal profile.
* **Organization ID**: (Conditional) The LinkedIn organization ID you want to extract data from. This is required only if you enabled the option to extract data from an organization account. This is the numeric ID that identifies your company page.
* **Start Date**: The earliest date from which records will be synced. Note that LinkedIn only returns up to **1 year of historical data** for daily reports and per-post statistics.
* **Lookback Window**: (Default: 30 days) Number of days to look back on each incremental run for the posts stream. This is applicable if extracting organization data. It ensures post statistics stay up to date for recently published posts. The lookback ensures that posts modified in the last N days have their latest metrics captured, while also preventing data loss during gaps between runs.
This is how you can find your Organization ID:
1. **From the Admin View URL**:
* Go to your LinkedIn Company Page
* Click on "Admin tools" or access the admin view
* Look at the URL: it will contain something like `/company/12345678/admin/`
* The number (e.g., `12345678`) is your Organization ID
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
The available streams depend on your configuration (whether you selected to extract data from an organization account or left it disabled for member profile data).
**Available Streams (Organization):**
*Organization Data:*
* Organization Details: Company profile information
*Daily Statistics (Time-series):*
* Daily Follower Statistics: Daily follower gains
* Daily Page Statistics: Daily page views and visitor analytics
* Daily Share Statistics: Daily aggregate engagement metrics
*Lifetime Statistics (Current Snapshot):*
* Lifetime Follower Statistics: Total follower count
* Follower Demographics: Lifetime follower demographic breakdown
* Lifetime Page Statistics: Lifetime page views and analytics
* Lifetime Share Statistics: Lifetime engagement metrics
*Content:*
* Posts: All posts from your page
* Post Statistics: Individual post performance
* Post Comments: Comments and replies on your organization's posts
* Video Analytics: Lifetime video metrics per organic video post
**Available Streams (Member Profile):**
*Member Profile Analytics:*
* Daily Member Follower Statistics: Daily follower count for the member
* Lifetime Member Follower Count: Total follower count for the member
* Daily Member Post Analytics: Daily post engagement metrics for the member
* Total Member Post Analytics: Lifetime post engagement metrics for the member
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: You can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: Every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: Every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from LinkedIn and their corresponding fields:
Company profile information for your LinkedIn organization.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | The LinkedIn numeric organization ID |
| `urn` | String | URN for the organization |
| `vanityName` | String | The LinkedIn public (vanity) name |
| `localizedName` | String | The localized name of the organization |
| `localizedDescription` | String | Localized description of the organization |
| `organizationType` | String | The organization type (e.g., PRIVATELY\_HELD) |
| `primaryOrganizationType` | String | Primary type (e.g., BRAND, NONE) |
| `staffCountRange` | String | Organization staff count range |
| `foundedOn` | Object | Founding information with year |
| `name` | Object | Structured name data with localized variants |
| `description` | Object | Structured description with localized variants |
| `website` | Object | Structured website URLs with localized variants |
| `localizedWebsite` | String | Localized website URL |
| `specialties` | Array | List of specialties for the organization |
| `localizedSpecialties` | Array | Localized specialties |
| `alternativeNames` | Array | Alternative names for the organization |
| `locations` | Array | Array of organization locations with address and type |
| `logoV2` | Object | Organization logo (cropped, original, cropInfo) |
| `coverPhotoV2` | Object | Cover photo (cropped, original, cropInfo) |
| `defaultLocale` | Object | Default locale (country, language) |
| `created` | Object | Creation metadata (actor, time) |
| `lastModified` | Object | Last modification metadata (actor, time) |
| `versionTag` | String | Version tag of the record |
| `autoCreated` | Boolean | Whether the organization was auto-created |
| `groups` | Array | List of group URNs associated with this organization |
Daily follower statistics for your organization, including follower gains.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the report (replication key) |
| `organizationalEntity` | String | The organization URN |
| `timeRange` | Object | Time range for the statistics (start/end timestamps) |
| `followerGains` | Object | Daily follower changes with organic and paid counts |
| `followerCountsByAssociationType` | Array | Followers by relationship (employee, etc.) |
| `followerCountsByGeoCountry` | Array | Followers by country |
| `followerCountsByFunction` | Array | Followers by job function |
| `followerCountsByIndustry` | Array | Followers by industry |
| `followerCountsByGeo` | Array | Followers by market area/region |
| `followerCountsBySeniority` | Array | Followers by seniority level |
| `followerCountsByStaffCountRange` | Array | Followers by company size |
The LinkedIn API omits demographic fields (`followerCountsBy*`) when fetching daily time-series data. To see demographic breakdowns of your followers, use the **Follower Demographics** stream instead.
Lifetime professional demographic breakdowns for your organization's followers. Provides current snapshot data for followers by seniority, function, industry, staff count range, and location.
| Field | Type | Description |
| :- | :- | :- |
| `organizationalEntity` | String | The organization URN |
| `followerCountsByAssociationType` | Array | Followers by relationship (employee, etc.) |
| `followerCountsByGeoCountry` | Array | Followers by country |
| `followerCountsByFunction` | Array | Followers by job function |
| `followerCountsByIndustry` | Array | Followers by industry |
| `followerCountsByGeo` | Array | Followers by market area/region |
| `followerCountsBySeniority` | Array | Followers by seniority level |
| `followerCountsByStaffCountRange` | Array | Followers by company size |
Total follower count for your organization (current snapshot).
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the report |
| `firstDegreeSize` | Integer | Total number of followers (network size) |
This stream provides the current total follower count, not historical data. Use Daily Follower Statistics for time-series follower growth data.
Daily page view and click statistics for your organization's LinkedIn page, with breakdowns by visitor demographics.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the report (replication key) |
| `organization` | String | The organization URN |
| `timeRange` | Object | Time range for the statistics (start/end timestamps) |
| `totalPageStatistics` | Object | Aggregate statistics with views and clicks |
| `allPageViews` | Object | Total page views across all devices |
| `allDesktopPageViews` | Object | All desktop page views |
| `allMobilePageViews` | Object | All mobile page views |
| `overviewPageViews` | Object | Overview/home tab views (all devices) |
| `desktopOverviewPageViews` | Object | Desktop overview views |
| `mobileOverviewPageViews` | Object | Mobile overview views |
| `aboutPageViews` | Object | About tab views (all devices) |
| `desktopAboutPageViews` | Object | Desktop about views |
| `mobileAboutPageViews` | Object | Mobile about views |
| `jobsPageViews` | Object | Jobs tab views (all devices) |
| `desktopJobsPageViews` | Object | Desktop jobs views |
| `mobileJobsPageViews` | Object | Mobile jobs views |
| `careersPageViews` | Object | Careers tab views (all devices) |
| `desktopCareersPageViews` | Object | Desktop careers views |
| `mobileCareersPageViews` | Object | Mobile careers views |
| `lifeAtPageViews` | Object | Life at \[Company] views (all devices) |
| `desktopLifeAtPageViews` | Object | Desktop life at views |
| `mobileLifeAtPageViews` | Object | Mobile life at views |
| `productsPageViews` | Object | Products tab views (all devices) |
| `desktopProductsPageViews` | Object | Desktop products views |
| `mobileProductsPageViews` | Object | Mobile products views |
| `peoplePageViews` | Object | People tab views (all devices) |
| `desktopPeoplePageViews` | Object | Desktop people views |
| `mobilePeoplePageViews` | Object | Mobile people views |
| `insightsPageViews` | Object | Insights tab views (all devices) |
| `desktopInsightsPageViews` | Object | Desktop insights views |
| `mobileInsightsPageViews` | Object | Mobile insights views |
| `pageStatisticsByGeoCountry` | Array | Views by country |
| `pageStatisticsByFunction` | Array | Views by job function |
| `pageStatisticsByIndustryV2` | Array | Views by industry (v2) |
| `pageStatisticsByGeo` | Array | Views by region/market area |
| `pageStatisticsBySeniority` | Array | Views by seniority |
| `pageStatisticsByStaffCountRange` | Array | Views by company size |
| `pageStatisticsByTargetedContent` | Array | Views by targeted content |
Lifetime page view statistics for your organization's LinkedIn page. Same structure as Daily Page Statistics but represents cumulative/lifetime data.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the report (replication key) |
| `organization` | String | The organization URN |
Contains the same fields as Daily Page Statistics (`totalPageStatistics`, all page view breakdowns, and demographic breakdowns), but represents lifetime cumulative data rather than daily increments.
Daily aggregate share and engagement statistics for your organization's content.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the report (replication key) |
| `organizationalEntity` | String | The organization URN |
| `share` | String | The share URN (blank for aggregate statistics) |
| `ugcPost` | String | The UGC post URN (blank for aggregate statistics) |
| `timeRange` | Object | Time range for the statistics (start/end timestamps) |
| `totalShareStatistics` | Object | Aggregate engagement metrics with clicks, comments, impressions, likes, shares, and engagement rate |
Lifetime aggregate share and engagement statistics for your organization's content.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the report (replication key) |
| `organizationalEntity` | String | The organization URN |
| `share` | String | The share URN (blank for aggregate statistics) |
| `ugcPost` | String | The UGC post URN (blank for aggregate statistics) |
| `timeRange` | Object | Time range for the statistics (start/end timestamps) |
| `totalShareStatistics` | Object | Cumulative engagement metrics with clicks, comments, impressions, likes, shares, and engagement rate |
All posts published by your organization on LinkedIn.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The post ID (URN) |
| `author` | String | The author URN (organization) |
| `createdAt` | Integer | Creation timestamp (epoch milliseconds) |
| `publishedAt` | Integer | Publication timestamp (epoch milliseconds) |
| `lastModifiedAt` | Integer | Last modification timestamp (replication key) |
| `lifecycleState` | String | Post state (PUBLISHED, DRAFT, etc.) |
| `commentary` | String | The post text/caption |
| `content` | Object | The attached content object (media, article, poll, etc.) |
| `contentCallToActionLabel` | String | CTA button label |
| `contentLandingPage` | String | CTA landing page URL |
| `visibility` | String | Visibility setting (PUBLIC, CONNECTIONS, etc.) |
| `distribution` | Object | Distribution settings including feed distribution and targeting |
| `adContext` | Object | Ad information if post is sponsored |
| `reshareContext` | Object | If this is a reshare, contains parent/root references |
| `isReshareDisabledByAuthor` | Boolean | Whether resharing is disabled |
| `lifecycleStateInfo` | Object | Additional state information |
Individual post performance metrics. This stream is linked to the Posts stream, providing statistics for each post.
| Field | Type | Description |
| :- | :- | :- |
| `postLastModifiedAt` | Integer | The last modified time of the post (replication key) |
| `organizationalEntity` | String | The organization URN |
| `postId` | String | The post ID (URN): links to the Posts stream |
| `share` | String | The share URN (for legacy shares) |
| `ugcPost` | String | The UGC post URN |
| `timeRange` | Object | Time range for the statistics (start/end timestamps) |
| `totalShareStatistics` | Object | Post performance metrics including clicks, comments, engagement rate, impressions, likes, shares, and unique impressions |
Comments and replies left on the organization's posts. This stream is linked to the Posts stream: it enumerates your organization's posts and fetches every comment thread, including replies to first-level comments.
| Field | Type | Description |
| :- | :- | :- |
| `commentUrn` | String | Unique composite URN of the comment, built from the thread URN and the comment ID (e.g., `urn:li:comment:(urn:li:activity:123,456)`) |
| `id` | String | The comment ID within its thread |
| `postId` | String | The organization's post URN (share or ugcPost) this comment belongs to: links to the Posts stream |
| `postLastModifiedAt` | Integer | Last modified time of the post the comment belongs to, in milliseconds since epoch (replication key) |
| `organizationalEntity` | String | The organization URN that authored the post |
| `object` | String | URN of the entity the comment was made on (typically the post's activity URN) |
| `parentComment` | String | URN of the parent comment when this comment is a reply; empty for first-level comments |
| `actor` | String | URN of the entity that authored the comment: a member (`urn:li:person`) or an organization/brand commenting as the page |
| `agent` | String | Member URN who wrote the comment on behalf of the organization, when the actor is an organization |
| `message` | Object | The comment text and its mentions (`text`, `attributes`) |
| `content` | Array | Media attached to the comment (e.g., images) |
| `created` | Object | Who created the comment and when (`actor`, `impersonator`, `time`) |
| `lastModified` | Object | Who last modified the comment and when (`actor`, `impersonator`, `time`) |
| `likesSummary` | Object | Like counts for the comment (`totalLikes`, `aggregatedTotalLikes`, `likedByCurrentUser`, `selectedLikes`) |
| `commentsSummary` | Object | Reply counts for the comment (`totalFirstLevelComments`, `aggregatedTotalComments`, `selectedComments`) |
Lifetime video metrics per organic video post. This stream enumerates your organization's video posts and provides aggregate viewing metrics for each one.
| Field | Type | Description |
| :- | :- | :- |
| `postLastModifiedAt` | Integer | The last modified time of the video post, in milliseconds since epoch (replication key) |
| `organizationalEntity` | String | The organization URN that authored the video post |
| `postId` | String | The video post ID (ugcPost URN) the analytics belong to |
| `video` | String | The video media asset URN attached to the post |
| `videoViews` | Integer | Number of video views (play-pause cycles of at least 3 seconds). Auto-looping counts as a single view. Available for up to six months after the post (VIDEO\_VIEW). |
| `viewers` | Integer | Number of unique viewers who made an engaged play on the video. Available for up to six months after the post (VIEWER). |
| `timeWatched` | Integer | Total time the video was watched, in milliseconds. Auto-looping keeps increasing this metric on each subsequent play (TIME\_WATCHED). |
| `watchTime` | Integer | Time watched, in milliseconds, for play-pause cycles of at least 3 seconds. Equals the 'Watch Time' figure shown in LinkedIn's UI (TIME\_WATCHED\_FOR\_VIDEO\_VIEWS). |
Daily follower count for the authenticated member. Returns one row per date with the number of new followers gained that day.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the follower data point |
| `member_followers_count` | Integer | Number of new followers gained on this date |
Lifetime total follower count for the authenticated member.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date when the data was fetched |
| `member_followers_count` | Integer | Total number of followers |
Daily aggregated post analytics for the authenticated member.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date of the analytics data point |
| `impressions` | Integer | Number of times posts were displayed |
| `reactions` | Integer | Number of reactions received |
| `comments` | Integer | Number of comments received |
| `reshares` | Integer | Number of reshares/reposts |
| `post_saves` | Integer | Number of times posts were saved |
| `post_sends` | Integer | Number of times posts were sent via messaging |
| `premium_cta_clicks` | Integer | Number of premium CTA button clicks |
Lifetime aggregated post analytics for the authenticated member. Returns a single row with the total count for each metric across all posts.
| Field | Type | Description |
| :- | :- | :- |
| `date` | Date | The date when the data was fetched |
| `impressions` | Integer | Total number of impressions |
| `members_reached` | Integer | Total unique members reached |
| `reactions` | Integer | Total number of reactions |
| `comments` | Integer | Total number of comments |
| `reshares` | Integer | Total number of reshares/reposts |
| `post_saves` | Integer | Total number of post saves |
| `post_sends` | Integer | Total number of post sends |
| `link_clicks` | Integer | Total number of link clicks |
| `premium_cta_clicks` | Integer | Total premium CTA clicks |
# Data Model
The following diagram illustrates the relationships between the core data streams in LinkedIn. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Organization Profile"
OrgDetails("Organization Details");
end
subgraph "Daily Analytics"
DailyFollower("Daily Follower Statistics");
DailyPage("Daily Page Statistics");
DailyShare("Daily Share Statistics");
end
subgraph "Lifetime Analytics"
LifetimeFollower("Lifetime Follower Statistics");
FollowerDemog("Follower Demographics");
LifetimePage("Lifetime Page Statistics");
LifetimeShare("Lifetime Share Statistics");
end
subgraph "Content"
Posts("Posts");
PostStats("Post Statistics");
PostComments("Post Comments");
VideoAnalytics("Video Analytics");
end
subgraph "Member Profile Analytics"
DailyMemberFollowers("Daily Member Follower Statistics");
LifetimeMemberFollowers("Lifetime Member Follower Count");
DailyMemberPosts("Daily Member Post Analytics");
TotalMemberPosts("Total Member Post Analytics");
end
OrgDetails -- "id/urn" --> Organization["Organization ID"];
DailyFollower -- "organizationalEntity" --> Organization;
DailyPage -- "organization" --> Organization;
DailyShare -- "organizationalEntity" --> Organization;
LifetimeFollower -- "organization_id" --> Organization;
FollowerDemog -- "organizationalEntity" --> Organization;
LifetimePage -- "organization" --> Organization;
LifetimeShare -- "organizationalEntity" --> Organization;
Posts -- "author" --> Organization;
PostStats -- "postId" --> Posts;
PostComments -- "postId" --> Posts;
PostComments -- "organizationalEntity" --> Organization;
VideoAnalytics -- "postId" --> Posts;
VideoAnalytics -- "organizationalEntity" --> Organization;
DailyMemberFollowers -- "date" --> MemberProfile["Member Profile (Authenticated)"];
LifetimeMemberFollowers -- "date" --> MemberProfile;
DailyMemberPosts -- "date" --> MemberProfile;
TotalMemberPosts -- "date" --> MemberProfile;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating LinkedIn organic data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
## Content Performance Analysis
### 1. Post Engagement Overview
Track the performance of your LinkedIn posts to identify top-performing content.
**Business Value:**
* Identify which content types drive the most engagement
* Understand optimal posting patterns
* Inform content strategy with data-driven insights
```sql theme={null}
WITH post_metrics AS (
SELECT
p.id AS post_id,
p.commentary AS post_text,
FROM_UNIXTIME(p."publishedAt" / 1000) AS published_date,
p."lifecycleState" AS status,
CASE
WHEN p.content.media IS NOT NULL THEN 'MEDIA'
WHEN p.content.article IS NOT NULL THEN 'ARTICLE'
WHEN p.content.poll IS NOT NULL THEN 'POLL'
WHEN p.content."multiImage" IS NOT NULL THEN 'MULTI_IMAGE'
WHEN p.content.carousel IS NOT NULL THEN 'CAROUSEL'
ELSE 'TEXT_ONLY'
END AS content_type,
ps."totalShareStatistics"."impressionCount" AS impressions,
ps."totalShareStatistics"."uniqueImpressionsCount" AS unique_impressions,
ps."totalShareStatistics"."clickCount" AS clicks,
ps."totalShareStatistics"."likeCount" AS likes,
ps."totalShareStatistics"."commentCount" AS comments,
ps."totalShareStatistics"."shareCount" AS shares,
ps."totalShareStatistics".engagement AS engagement_rate
FROM
nekt_raw.linkedin_posts p
LEFT JOIN nekt_raw.linkedin_post_statistics ps
ON p.id = ps."postId"
WHERE
p."lifecycleState" = 'PUBLISHED'
)
SELECT
post_id,
SUBSTRING(post_text, 1, 100) AS post_preview,
published_date,
content_type,
impressions,
unique_impressions,
clicks,
likes,
comments,
shares,
ROUND(engagement_rate * 100, 2) AS engagement_rate_pct,
(likes + comments + shares) AS total_engagements
FROM
post_metrics
ORDER BY
total_engagements DESC
LIMIT 20
```
```sql theme={null}
WITH post_metrics AS (
SELECT
p.id AS post_id,
p.commentary AS post_text,
TIMESTAMP_MILLIS(CAST(p.publishedAt AS INT64)) AS published_date,
p.lifecycleState AS status,
CASE
WHEN p.content.media IS NOT NULL THEN 'MEDIA'
WHEN p.content.article IS NOT NULL THEN 'ARTICLE'
WHEN p.content.poll IS NOT NULL THEN 'POLL'
WHEN p.content.multiImage IS NOT NULL THEN 'MULTI_IMAGE'
WHEN p.content.carousel IS NOT NULL THEN 'CAROUSEL'
ELSE 'TEXT_ONLY'
END AS content_type,
ps.totalShareStatistics.impressionCount AS impressions,
ps.totalShareStatistics.uniqueImpressionsCount AS unique_impressions,
ps.totalShareStatistics.clickCount AS clicks,
ps.totalShareStatistics.likeCount AS likes,
ps.totalShareStatistics.commentCount AS comments,
ps.totalShareStatistics.shareCount AS shares,
ps.totalShareStatistics.engagement AS engagement_rate
FROM
`nekt_raw.linkedin_posts` p
LEFT JOIN `nekt_raw.linkedin_post_statistics` ps
ON p.id = ps.postId
WHERE
p.lifecycleState = 'PUBLISHED'
)
SELECT
post_id,
SUBSTR(post_text, 1, 100) AS post_preview,
published_date,
content_type,
impressions,
unique_impressions,
clicks,
likes,
comments,
shares,
ROUND(engagement_rate * 100, 2) AS engagement_rate_pct,
(likes + comments + shares) AS total_engagements
FROM
post_metrics
ORDER BY
total_engagements DESC
LIMIT 20
```
| post\_id | post\_preview | published\_date | content\_type | impressions | unique\_impressions | clicks | likes | comments | shares | engagement\_rate\_pct | total\_engagements |
| - | - | - | - | - | - | - | - | - | - | - | - |
| urn:li:share:123456 | We're excited to announce our new product... | 2024-11-15 14:30:00 | MEDIA | 45,230 | 38,450 | 1,234 | 892 | 156 | 78 | 4.82 | 1,126 |
| urn:li:ugcPost:789012 | Join us for our upcoming webinar on... | 2024-11-12 09:00:00 | ARTICLE | 32,150 | 28,320 | 956 | 543 | 89 | 45 | 3.21 | 677 |
| urn:li:share:345678 | Proud to share that our team has been... | 2024-11-10 16:45:00 | MULTI\_IMAGE | 28,900 | 24,670 | 445 | 678 | 234 | 123 | 5.12 | 1,035 |
### 2. Follower Growth Analysis
Track your organization's follower growth over time and understand the source of new followers.
**Business Value:**
* Monitor organic vs paid follower acquisition
* Track growth trends over time
* Measure the impact of content and campaigns on follower growth
```sql theme={null}
WITH daily_followers AS (
SELECT
DATE(date) AS report_date,
"followerGains"."organicFollowerGain" AS organic_gained,
"followerGains"."paidFollowerGain" AS paid_gained,
("followerGains"."organicFollowerGain" + "followerGains"."paidFollowerGain") AS total_gained
FROM
nekt_raw.linkedin_daily_follower_statistics
WHERE
date >= CURRENT_DATE - INTERVAL '30' DAY
),
running_totals AS (
SELECT
report_date,
organic_gained,
paid_gained,
total_gained,
SUM(total_gained) OVER (ORDER BY report_date) AS cumulative_growth,
AVG(total_gained) OVER (
ORDER BY report_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
) AS seven_day_avg
FROM
daily_followers
)
SELECT
report_date,
organic_gained,
paid_gained,
total_gained,
cumulative_growth,
ROUND(seven_day_avg, 1) AS seven_day_moving_avg,
ROUND(organic_gained * 100.0 / NULLIF(total_gained, 0), 1) AS organic_pct
FROM
running_totals
ORDER BY
report_date DESC
```
```sql theme={null}
WITH daily_followers AS (
SELECT
DATE(date) AS report_date,
followerGains.organicFollowerGain AS organic_gained,
followerGains.paidFollowerGain AS paid_gained,
(followerGains.organicFollowerGain + followerGains.paidFollowerGain) AS total_gained
FROM
`nekt_raw.linkedin_daily_follower_statistics`
WHERE
date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
),
running_totals AS (
SELECT
report_date,
organic_gained,
paid_gained,
total_gained,
SUM(total_gained) OVER (ORDER BY report_date) AS cumulative_growth,
AVG(total_gained) OVER (
ORDER BY report_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
) AS seven_day_avg
FROM
daily_followers
)
SELECT
report_date,
organic_gained,
paid_gained,
total_gained,
cumulative_growth,
ROUND(seven_day_avg, 1) AS seven_day_moving_avg,
ROUND(SAFE_DIVIDE(organic_gained * 100.0, total_gained), 1) AS organic_pct
FROM
running_totals
ORDER BY
report_date DESC
```
| report\_date | organic\_gained | paid\_gained | total\_gained | cumulative\_growth | seven\_day\_moving\_avg | organic\_pct |
| - | - | - | - | - | - | - |
| 2024-11-27 | 45 | 12 | 57 | 1,234 | 48.3 | 78.9 |
| 2024-11-26 | 38 | 8 | 46 | 1,177 | 46.7 | 82.6 |
| 2024-11-25 | 52 | 15 | 67 | 1,131 | 45.1 | 77.6 |
| 2024-11-24 | 28 | 5 | 33 | 1,064 | 43.9 | 84.8 |
| 2024-11-23 | 31 | 7 | 38 | 1,031 | 44.2 | 81.6 |
### 3. Page Visitor Demographics
Analyze who is visiting your LinkedIn page to understand your audience.
**Business Value:**
* Understand your audience composition
* Identify high-value visitor segments
* Inform content targeting decisions
```sql theme={null}
WITH seniority_stats AS (
SELECT
s.seniority,
SUM(s."pageStatistics".views."allPageViews"."pageViews") AS total_views,
SUM(s."pageStatistics".views."allPageViews"."uniquePageViews") AS unique_views
FROM
nekt_raw.linkedin_daily_page_statistics ps
CROSS JOIN UNNEST(ps."pageStatisticsBySeniority") AS t(s)
WHERE
DATE(ps.date) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
s.seniority
),
total AS (
SELECT SUM(total_views) AS grand_total FROM seniority_stats
)
SELECT
seniority,
total_views,
unique_views,
ROUND(total_views * 100.0 / grand_total, 1) AS pct_of_total
FROM
seniority_stats, total
ORDER BY
total_views DESC
```
```sql theme={null}
WITH seniority_stats AS (
SELECT
s.seniority,
SUM(s.pageStatistics.views.allPageViews.pageViews) AS total_views,
SUM(s.pageStatistics.views.allPageViews.uniquePageViews) AS unique_views
FROM
`nekt_raw.linkedin_daily_page_statistics` ps,
UNNEST(ps.pageStatisticsBySeniority) AS s
WHERE
DATE(ps.date) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
s.seniority
),
total AS (
SELECT SUM(total_views) AS grand_total FROM seniority_stats
)
SELECT
seniority,
total_views,
unique_views,
ROUND(SAFE_DIVIDE(total_views * 100.0, grand_total), 1) AS pct_of_total
FROM
seniority_stats, total
ORDER BY
total_views DESC
```
| seniority | total\_views | unique\_views | pct\_of\_total |
| - | - | - | - |
| urn:li:seniority:5 | 12,450 | 9,230 | 28.4 |
| urn:li:seniority:6 | 9,870 | 7,456 | 22.5 |
| urn:li:seniority:4 | 8,234 | 6,123 | 18.8 |
| urn:li:seniority:3 | 6,543 | 4,890 | 14.9 |
| urn:li:seniority:7 | 4,321 | 3,210 | 9.9 |
## Implementation Notes
LinkedIn analytics data typically has a delay of 24-48 hours. The most recent data available may be from 1-2 days ago.
* **Daily reports**: Updated once per day
* **Post statistics**: Updated as engagement occurs, with some delay
* **Lifetime demographics**: Represent current state, not historical
LinkedIn limits historical data access:
* **Daily statistics** (Follower, Page, Share): Up to **1 year** of historical data
* **Posts**: All published posts are available
* **Post statistics**: Available for a rolling **12-month window**. Older data is not available from the LinkedIn API.
* **Video analytics**: View and viewer metrics (`videoViews`, `viewers`) are available for up to **six months** after the post is published.
This connector requires the following LinkedIn permissions depending on the streams extracted:
* `r_organization_social` - Read organization posts and social data
* `r_organization_social_feed` - Read comments and replies on the organization's posts
* `r_member_profileAnalytics` - Read analytics for the authenticated member's profile
* `r_member_postAnalytics` - Read analytics for the authenticated member's posts
* Access to **Community Management API**
If extracting data from an organization, the LinkedIn account used for authorization must have **Admin** or **Content Admin** access to the organization page.
LinkedIn enforces daily quotas on API requests. The connector detects both daily limits:
* **Combined Application & Member limit** (`APPLICATION_AND_MEMBER DAY`)
* **Resource-level limit** (`APPLICATION DAY`)
How an exhausted quota is handled depends on the sync type:
* **Incremental Syncs**: The connector stops gracefully, saves the records extracted so far, and advances the bookmark only over what was emitted. The extraction resumes from this point on the next run after LinkedIn resets quotas (midnight UTC). This is what lets a first sync larger than the daily quota converge over multiple days.
* **Full Table Syncs**: The connector fails the run instead. A full-table stream replaces its whole table on every sync, so continuing would overwrite your existing data with an incomplete snapshot.
LinkedIn uses URNs (Uniform Resource Names) to identify entities:
* **Organizations**: `urn:li:organization:{id}`
* **Posts**: `urn:li:share:{id}` or `urn:li:ugcPost:{id}`
* **Seniority levels**: `urn:li:seniority:{id}`
* **Industries**: `urn:li:industry:{id}`
* **Functions**: `urn:li:function:{id}`
* **Countries**: `urn:li:geo:{id}` or `urn:li:country:{code}`
These URNs are used as identifiers and can be decoded to get the underlying IDs.
LinkedIn's `engagement` field is calculated as:
```
engagement = (clicks + likes + comments + shares) / impressions
```
This is returned as a decimal (e.g., 0.0482 = 4.82%). Multiply by 100 to get the percentage.
The Posts, Post Statistics, Video Analytics, and Post Comments streams support incremental sync with a lookback window:
* **Posts**: Uses `lastModifiedAt` as the replication key for incremental sync.
* **Post Statistics**: Uses `postLastModifiedAt` as the replication key, derived from the parent post's modification time. It drives its enumeration from its own bookmark, meaning a full sync will properly backfill from the `Start Date` (up to the 12-month API limit).
* **Video Analytics**: Uses `postLastModifiedAt` as the replication key, derived from the parent video post's modification time. Like Post Statistics, it enumerates posts from its own bookmark, so a full sync backfills from the `Start Date`.
* **Post Comments**: Uses `postLastModifiedAt` as the replication key, derived from the parent post's modification time. It also enumerates posts from its own bookmark, so a full sync backfills comments from the `Start Date`.
* **Lookback Window**: Configurable number of days (default: 30) to look back on each run to ensure recently modified posts have up-to-date statistics and newly added comments are captured.
* **API Rate Limiting & Resume**: The connector handles LinkedIn's daily rate limits gracefully. For per-post statistics, video analytics and post comments, data is fetched oldest-first so that if a quota limit interrupts a long backfill, the bookmark lands at the oldest unfetched post and the next run resumes there without leaving data gaps.
To spare LinkedIn's quota-throttled share statistics endpoint, the Post Statistics stream requests statistics in batches of 10 posts per API call instead of one call per post. Each batch is fetched as a single page (the connector never follows LinkedIn's `next` paging link), which prevents the pagination loop that previously aborted syncs. If LinkedIn ever reports an additional page for a batch, the connector processes the first page and logs a warning rather than getting stuck.
The connector automatically percent-encodes target URNs to comply with LinkedIn's API requirements and prevent syntax exceptions. Additionally, if LinkedIn rejects a specific target (such as a deleted post or an unsupported URN type returning a 400 or 404 error), the connector will gracefully skip it and continue the extraction process for the remaining records. This ensures that a single invalid item does not fail the entire extraction.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| The source stops extracting after a change to the LinkedIn application | When the application's scopes change on the provider side, the token has to be refreshed on every source that uses that account. | Refresh the token on each affected source and re-run. |
| The **Post Comments** stream comes back empty | This stream only covers company pages, and the authorizing user must be an administrator of the page. | Grant the user the page administrator role and re-authorize. |
| Old posts do not appear in **Post Statistics** | The API limits history per post, and the stream is incremental on the last-modified date. | Run an initial full sync. Posts outside the provider's window are not recoverable. |
## Skills for agents
LinkedIn connector documentation as plain markdown, for use in AI agent contexts.
# LinkedIn Ads as a data source
Source: https://docs.nekt.com/sources/linkedin-ads
Bring data from LinkedIn Ads to Nekt.
LinkedIn Ads is a powerful tool for B2B marketers, offering unique targeting capabilities to reach a professional audience. With various ad formats and robust analytics, it helps businesses achieve their marketing objectives, from lead generation to brand awareness.
![]()
## Configuring LinkedIn Ads as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the LinkedIn Ads option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your LinkedIn Ads OAuth credentials for this connection. You can generate these in your LinkedIn Developer portal. Once you have them, add the account access, then click **Next**.
* **Authorization**: Log in with a Linkedin account that has proper access to the LinkedIn Ads account you want to extract data from.
* **Account ID**: The ID for the LinkedIn Ads account you want to extract data from. This can be found on `https://www.linkedin.com/campaignmanager` right below the organization name on the left-side panel.
![]()
* **Start Date**: The earliest record date to sync. Records created or updated after the start date will be extracted.
* **Lookback window (days)**: (Default: 28) Number of days to subtract from the bookmark in incremental mode to re-fetch records. The effective start date will never be earlier than the configured start date.
* **API Version**: (Default: `202609`) The version of the LinkedIn Marketing API to use. LinkedIn supports each API version for one year from its release before it is sunset.
Linkedin only returns up to 1-year of historical data for daily performance reports.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from LinkedIn Ads and their corresponding fields:
Stream for fetching ad accounts information.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | The account ID. |
| `name` | String | The account name. |
| `type` | String | The account type. |
| `status` | String | Current status of the account. |
| `currency` | String | The account's currency. |
| `reference` | String | The LinkedIn organization reference URN. |
| `changeAuditStamps` | Object | Audit timestamps for account creation and modification, including creator, modifier, and timestamps. |
| `notifiedOnCampaignOptimization` | Boolean | Whether notifications are enabled for campaign optimization. |
| `notifiedOnCreativeApproval` | Boolean | Whether notifications are enabled for creative approvals. |
| `notifiedOnCreativeRejection` | Boolean | Whether notifications are enabled for creative rejections. |
| `notifiedOnEndOfCampaign` | Boolean | Whether notifications are enabled for campaign end. |
| `servingStatuses` | Array | List of serving statuses for the account. |
| `test` | Boolean | Whether this is a test account. |
| `version` | Object | Version information for the account. |
Stream for fetching campaign groups.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the record. |
| `name` | String | Name of the record. |
| `status` | String | Current status of the record. |
| `account` | String | Identifier of the associated account. |
| `runSchedule` | Object | Schedule for when the campaign group runs. |
| `changeAuditStamps` | Object | Audit timestamps for campaign group creation and modification. |
| `test` | Boolean | Whether this is a test record. |
| `servingStatuses` | Array | List of serving statuses for the campaign group. |
| `allowedCampaignTypes` | Array | List of campaign types allowed in the group. |
| `backfilled` | Boolean | Whether the campaign group was backfilled. |
Stream for fetching campaigns.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the campaign. |
| `name` | String | Name of the campaign. |
| `status` | String | Current status of the campaign. |
| `type` | String | Type of campaign (e.g. SPONSORED\_UPDATES). |
| `account` | String | URN of the associated advertising account. |
| `campaignGroup` | String | URN of the campaign group. |
| `servingStatuses` | Array | Current serving status of the campaign. |
| `locale` | Object | Locale settings for the campaign, including country and language. |
| `version` | Object | Version information for the campaign. |
| `associatedEntity` | String | URN of the associated LinkedIn organization. |
| `test` | Boolean | Whether this is a test campaign. |
| `runSchedule` | Object | Campaign schedule information, including start and end timestamps. |
| `targeting` | Object | Deprecated by LinkedIn. Only returned for legacy campaigns that were never migrated to targetingCriteria; campaigns created or edited in the current Campaign Manager expose their audience in targetingCriteria instead. |
| `targetingCriteria` | String | Audience the campaign targets, as a raw JSON string. LinkedIn models it as an AND/OR boolean expression with 'include' and 'exclude' blocks whose keys are targeting facet URNs. Replaces the deprecated targeting field. |
| `optimizationPreference` | Object | Campaign optimization preferences. |
| `optimizationTargetType` | String | Target type for campaign optimization. |
| `changeAuditStamps` | Object | Audit timestamps for campaign creation and last modification. |
| `dailyBudget` | Object | Daily budget settings, including amount and currency code. |
| `unitCost` | Object | Cost per unit settings, including amount and currency code. |
| `creativeSelection` | String | Creative selection method (e.g. OPTIMIZED). |
| `costType` | String | Cost type for the campaign (e.g. CPC). |
| `offsiteDeliveryEnabled` | Boolean | Whether offsite delivery is enabled. |
| `audienceExpansionEnabled` | Boolean | Whether audience expansion is enabled. |
Stream for fetching creatives and their associated contents.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the creative. |
| `status` | String | Current status of the creative. |
| `campaign` | String | URN of the associated campaign. |
| `account` | String | URN of the associated account. |
| `createdAt` | Integer | Timestamp when the creative was created. |
| `servingHoldReasons` | Array | List of reasons why the creative is not serving. |
| `lastModifiedAt` | Integer | Timestamp when the creative was last modified. |
| `isTest` | Boolean | Whether this is a test creative. |
| `createdBy` | String | URN of the user who created the creative. |
| `isServing` | Boolean | Whether the creative is currently serving. |
| `lastModifiedBy` | String | URN of the user who last modified the creative. |
| `intendedStatus` | String | Intended status of the creative. |
| `content` | String | URN reference to the creative content. |
Stream for fetching aggregated analytics per campaign.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `_id` | String | Unique identifier for the analytics data. |
| `account_id` | String | Account ID. |
| `campaign_id` | String | Campaign ID. |
| `date` | String | Date of the analytics data. |
| `impressions` | Integer | Number of impressions served. |
| `clicks` | Integer | Number of clicks received. |
| `totalEngagements` | Integer | Total engagements (clicks, likes, shares, etc.). |
| `costInLocalCurrency` | Number | Total cost in local currency. |
| `oneClickLeads` | Integer | Number of one-click leads. |
| `externalWebsitePostClickConversions` | Integer | Post-click conversions on external websites. |
| `externalWebsitePostViewConversions` | Integer | Post-view conversions on external websites. |
| `videoViews` | Integer | Number of video views. |
| `videoCompletions` | Integer | Number of completed video views. |
| `likes` | Integer | Number of likes. |
| `shares` | Integer | Number of shares. |
| `follows` | Integer | Number of follows. |
| `comments` | Integer | Number of comments. |
| `reactions` | Integer | Number of reactions. |
| `landingPageClicks` | Integer | Clicks to landing page. |
| `companyPageClicks` | Integer | Clicks to company page. |
| `sends` | Integer | Number of messages sent. |
| `opens` | Integer | Number of messages opened. |
Stream for fetching creative analytics.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `_id` | String | Unique identifier for the analytics data. |
| `account_id` | String | Account ID. |
| `creative_id` | String | Creative ID. |
| `date` | String | Date of the analytics data. |
| `impressions` | Integer | Number of impressions served. |
| `clicks` | Integer | Number of clicks received. |
| `totalEngagements` | Integer | Total engagements (clicks, likes, shares, etc.). |
| `costInLocalCurrency` | Number | Total cost in local currency. |
| `oneClickLeads` | Integer | Number of one-click leads. |
| `externalWebsitePostClickConversions` | Integer | Post-click conversions on external websites. |
| `externalWebsitePostViewConversions` | Integer | Post-view conversions on external websites. |
| `videoViews` | Integer | Number of video views. |
| `videoCompletions` | Integer | Number of completed video views. |
| `likes` | Integer | Number of likes. |
| `shares` | Integer | Number of shares. |
| `follows` | Integer | Number of follows. |
| `comments` | Integer | Number of comments. |
| `reactions` | Integer | Number of reactions. |
| `landingPageClicks` | Integer | Clicks to landing page. |
| `companyPageClicks` | Integer | Clicks to company page. |
| `sends` | Integer | Number of messages sent. |
| `opens` | Integer | Number of messages opened. |
# Data Model
The following diagram illustrates the relationships between the core data streams in LinkedIn Ads. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
AdAccounts("Ad Accounts");
CampaignGroups("Campaign Groups");
Campaigns("Campaigns");
Creatives("Creatives");
end
subgraph "Performance Data"
CampaignAnalytics("Campaign Analytics");
CreativeAnalytics("Creative Analytics");
end
CampaignGroups -- "account" --> AdAccounts;
Campaigns -- "campaignGroup" --> CampaignGroups;
Campaigns -- "account" --> AdAccounts;
Creatives -- "campaign" --> Campaigns;
Creatives -- "account" --> AdAccounts;
CampaignAnalytics -- "account_id" --> AdAccounts;
CampaignAnalytics -- "campaign_id" --> Campaigns;
CreativeAnalytics -- "account_id" --> AdAccounts;
CreativeAnalytics -- "creative_id" --> Creatives;
```
## Implementation Notes
### API Resilience and Retries
The connector features built-in retry mechanisms and connection pooling for fetching creative content (such as InMail contents and sponsored conversation flows). These requests are routed through a robust SDK session utilizing backoff decorators. This ensures that transient network failures (e.g., "Network is unreachable" or general `ConnectionError` statuses) are automatically retried instead of crashing your sync process.
### API Limitations
* LinkedIn restricts daily performance reports to a maximum of **1-year of historical data**. If your start date requests records older than one year for analytics streams, only the past 365 days will be returned. Ensure your initial backfill expectations align with this API limitation.
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| The source stops extracting after a change to the LinkedIn application | When the application's scopes change on the provider side, the token has to be refreshed on every source that uses that account. | Refresh the token on each affected source (**Refresh Token** in the source settings) and re-run. |
| Reports return no history beyond one year | LinkedIn caps daily performance reports at 365 days. | Align the initial sync date with the window. See [API Limitations](#api-limitations). |
## Skills for agents
LinkedIn Ads connector documentation as plain markdown, for use in AI agent contexts.
# Linx Microvix as a data source
Source: https://docs.nekt.com/sources/linx-microvix
Bring data from Linx Microvix to your Lakehouse.
Linx Microvix is a retail ERP from Linx, widely used by Brazilian retail chains and franchise networks to run stores, stock, sales and finance. Its "Web Service de Saída Padrão" exposes the registrations and movements of every store in a portal, which Nekt uses to bring your sales, product, customer, stock and financial data into the Lakehouse.
## Before you start
The Web Service de Saída Padrão is a **paid, separately contracted service** from Linx. Before configuring the source, confirm with Linx that it is activated for your portal. Linx provides the access key required below during activation.
Linx documents that integration keys generating excessive traffic may be deactivated without prior notice. This connector follows the vendor's recommended pattern: it reads incrementally by row version and paces its requests. Avoid forcing repeated full re-extractions.
## Configuring Linx Microvix as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Linx Microvix option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **Access Key**: the `chave` provided by Linx when the Web Service de Saída Padrão was activated for your portal. It is a unique identifier such as `7d73503f-db26-4112-ac66-94033ac4986p`.
* **Group Name**: (Optional) the name of the chain (*rede*) your stores belong to, as registered in Microvix. It only applies to chains and franchise networks: a portal registered outside any chain matches no group, so leave it empty to extract every store your access key covers.
* **Portal ID**: only required when your access key serves more than one portal in Microvix.
* **Store CNPJs**: restrict the extraction to specific stores. Leave it empty to extract every store your access key covers.
* **Start Date**: (Default: `2000-01-01`) lower bound for the date filters that some Linx methods require. The default covers the full history on the first sync.
* **Inventory Date**: (Default: the day the sync runs) the date used to read the stock position in the `Product Inventory` stream.
* **Start Row Version**: (Default: `0`) the Microvix row version to start reading from on the first sync. Leave it at `0` to load the full history once.
* **Portal-wide Extraction**: (Default: `false`) reads the whole portal in a single pass for the methods that support it (movements, invoices, sales and purchase orders, cash movements) instead of querying store by store. It is much faster for large chains, but it only covers the portal your access key belongs to. Leave it disabled if your stores span more than one portal.
* **Interval Between Requests**: (Default: `0.2` seconds) the pause applied between requests. Lower it only with care, given the traffic warning above.
* **Use Acceptance Portal**: (Default: `false`) connects to the Linx acceptance (homologação) environment instead of production.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Your access key is scoped to the Microvix modules your company contracted. If a selected stream belongs to a module you don't have, Nekt logs that the table was skipped and continues with the remaining streams.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How incremental sync works
Microvix does not expose an "updated at" date that can be trusted across all routines. Instead, every record carries a `timestamp` column, which despite the name is **not a date**: it is a SQL Server row version, a counter that increases on every database operation.
Nekt uses it exactly as Linx recommends: each extraction asks for the records above the highest row version already retrieved, and stores the new highest value for the next run. This keeps extractions small and avoids re-reading history, which is what protects your integration key from being throttled.
A few streams (`Store Groups`, `Product Inventory`, `Customer Contacts`, `Payment Plan Card Brands`) carry no row version and are therefore always extracted as full tables.
# Streams and Fields
Below you'll find all available data streams from Linx Microvix and their corresponding fields:
### Stores and portal structure
Every store (CNPJ) your access key can read, narrowed to one chain when a Group Name is set.
Source method: `LinxGrupoLojas`. Full table sync (this method returns no row version).
**Primary key:** `portal`, `cnpj`
**Fields:**
* `cnpj` - CNPJ (Brazilian tax ID) of the store
* `nome_empresa` - Store name
* `id_empresas_rede` - Chain code in Microvix
* `rede` - Chain name in Microvix
* `portal` - Portal code in Microvix
* `nome_portal` - Portal name in Microvix
* `empresa` - Company code within the portal
* `lojas_proprias` - Whether the portal belongs to the chain's own stores
* `classificacao_portal` - Portal classification
Registration data of each store of the portal.
Source method: `LinxLojas`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `empresa`
**Fields:**
* `portal` - Portal code in Microvix
* `empresa` - Company code in Microvix
* `nome_emp` - Trade name of the company
* `razao_emp` - Legal company name
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `inscricao_emp` - State tax registration number
* `endereco_emp` - Street address
* `num_emp` - Street number of the address
* `complement_emp` - Address complement
* `bairro_emp` - Neighbourhood
* `cep_emp` - Postal code (CEP)
* `cidade_emp` - City
* `estado_emp` - State abbreviation (UF)
* `fone_emp` - Phone number of the customer
* `email_emp` - Email address
* `cod_ibge_municipio` - IBGE code of the municipality
* `data_criacao_emp` - Date the company was created in Microvix
* `data_criacao_portal` - Date the portal was created in Microvix
* `sistema_tributacao` - Taxation regime of the company
* `regime_tributario` - Tax regime of the company
* `area_empresa` - Floor area of the store in square metres
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `sigla_empresa` - Short code of the company
* `id_classe_fiscal` - Identifier of the fiscal class attached to the company
* `centro_distribuicao` - Whether the store is a distribution centre
* `inscricao_municipal_emp` - Municipal tax registration number of the company
* `cnae_emp` - CNAE economic activity code of the company
* `cod_cliente_linx` - Linx ERP customer/supplier code of the store
* `tabela_preco_preferencial` - Code of the company's preferred price table in Microvix
* `subrede` - Name of the subchain
* `produto_disponivel_empresa` - Whether the portal differentiates products between companies
* `id_grupo` - Identifier of the group
### Sales movements
Sales and purchase movements, one row per item of each fiscal invoice or sales ticket.
Source method: `LinxMovimento`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `transacao`, `ordem`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `transacao` - Internal Microvix code
* `usuario` - User that performed the movement
* `documento` - Fiscal document number
* `chave_nf` - Access key of the fiscal invoice (NF-e)
* `ecf` - Fiscal printer (ECF) number
* `numero_serie_ecf` - Serial number of the fiscal printer (ECF)
* `modelo_nf` - Fiscal invoice model code
* `data_documento` - Date the record was generated in Microvix
* `data_lancamento` - Date the fiscal document was posted
* `codigo_cliente` - Customer code in Microvix
* `serie` - Series of the document
* `desc_cfop` - Description of the fiscal CFOP
* `id_cfop` - Code of the fiscal CFOP
* `cod_vendedor` - Salesperson code in Microvix
* `quantidade` - Quantity of the product
* `preco_custo` - Cost price of the product
* `valor_liquido` - Net amount of the product
* `desconto` - Discount applied to the sale
* `cst_icms` - ICMS tax situation code (CST)
* `cst_pis` - PIS tax situation code (CST)
* `cst_cofins` - COFINS tax situation code (CST)
* `cst_ipi` - IPI tax situation code (CST)
* `valor_icms` - ICMS tax amount
* `aliquota_icms` - ICMS tax rate
* `base_icms` - ICMS tax base amount
* `valor_pis` - PIS tax amount
* `aliquota_pis` - PIS tax rate
* `base_pis` - PIS tax base amount
* `valor_cofins` - COFINS tax amount
* `aliquota_cofins` - COFINS tax rate
* `base_cofins` - COFINS tax base amount
* `valor_icms_st` - ICMS ST tax amount
* `aliquota_icms_st` - ICMS ST tax rate
* `base_icms_st` - ICMS ST tax base amount
* `valor_ipi` - IPI tax amount
* `aliquota_ipi` - IPI tax rate
* `base_ipi` - IPI tax base amount
* `valor_total` - Total amount of the item
* `forma_dinheiro` - Whether the sale was paid in cash
* `total_dinheiro` - Total amount paid in cash
* `forma_cheque` - Whether the sale was paid by cheque
* `total_cheque` - Total amount paid by cheque
* `forma_cartao` - Whether the sale was paid by card
* `total_cartao` - Total amount paid by card
* `forma_crediario` - Whether the sale was paid with store credit (crediário)
* `total_crediario` - Total amount paid with store credit (crediário)
* `forma_convenio` - Whether the sale was paid through a corporate agreement (convênio)
* `total_convenio` - Total amount paid through a corporate agreement (convênio)
* `frete` - Freight amount
* `tipo_transacao` - Operation of the movement: E inbound, S outbound, DE inbound return, DS outbound return, N neutral, C receipt-replacing invoice
* `cod_produto` - Service order role of the movement: A for service order billing, O for service order consumption write-off, empty to consider only the operation
* `cod_barra` - Barcode of the product
* `cancelado` - Whether the document is cancelled (S) or valid (N)
* `excluido` - Whether the record was deleted (S) or is valid (N)
* `soma_relatorio` - Whether the movement uses an operation nature that is excluded from report totals
* `deposito` - Name of the warehouse
* `obs` - Note about the document
* `preco_unitario` - Unit price of the item at the time of sale
* `hora_lancamento` - Time the movement was posted
* `natureza_operacao` - Description of the operation nature
* `tabela_preco` - Code of the price table used
* `nome_tabela_preco` - Name of the price table
* `cod_sefaz_situacao` - Status code of the document at SEFAZ
* `desc_sefaz_situacao` - Description of the document status at SEFAZ
* `protocolo_aut_nfe` - Authorisation protocol of the fiscal invoice for outbound operations
* `dt_update` - Timestamp of the last update of the record in the database, including cancellations
* `forma_cheque_prazo` - Whether the sale was paid by post-dated cheque
* `total_cheque_prazo` - Total amount paid by post-dated cheque
* `cod_natureza_operacao` - Code of the operation nature
* `preco_tabela_epoca` - Price of the product at the time of sale according to the default price table
* `desconto_total_item` - Discount amount applied to the item
* `conferido` - Whether the transfer note was checked: S for yes, N for no
* `transacao_pedido_venda` - Transaction number of the related sales order
* `codigo_modelo_nf` - Code of the fiscal invoice model
* `acrescimo` - Surcharge applied to the sale
* `mob_checkout` - Whether the sale was made through the mobile checkout
* `aliquota_iss` - ISS tax rate
* `base_iss` - ISS tax base amount
* `ordem` - Sequence of the item within the transaction in the ERP
* `codigo_rotina_origem` - Code of the ERP routine that originated the record
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `troco` - Change given for the cash payment
* `transportador` - Code of the carrier registered in Microvix
* `icms_aliquota_desonerado` - ICMS exemption tax rate
* `icms_valor_desonerado_item` - ICMS exemption amount for the item
* `empresa` - Company code within the portal
* `desconto_item` - Discount applied to the item
* `aliq_iss` - ISS tax rate
* `iss_base_item` - Base amount of the item
* `despesas` - Expense amount
* `seguro_total_item` - Total insurance charged to the item
* `acrescimo_total_item` - Total surcharge applied to the item
* `despesas_total_item` - Total expenses charged to the item
* `forma_pix` - Whether the sale was paid by Pix
* `total_pix` - Total amount paid by Pix
* `forma_deposito_bancario` - Whether the sale was paid by bank deposit
* `total_deposito_bancario` - Total amount paid by bank deposit
* `id_venda_produto_b2c` - Identifier of the product sale in the B2C channel
* `item_promocional` - Whether the item is part of a promotion
* `acrescimo_item` - Surcharge applied to the item
* `icms_st_antecipado_aliquota` - Anticipated ICMS ST tax rate
* `icms_st_antecipado_margem` - Anticipated ICMS ST margin
* `icms_st_antecipado_percentual_reducao` - Reduction percentage of the anticipated ICMS ST
* `icms_st_antecipado_valor_item` - Anticipated ICMS ST amount for the item
* `icms_base_desonerado_item` - ICMS exemption base amount for the item
* `codigo_status_nfe` - Status code of the electronic invoice (NF-e)
* `preco_produto_epoca` - Price of the product before any promotional discount
* `ficha` - Card (ficha) number
* `numero_sequencial_pos` - Sequential number of the POS terminal
* `dt_insert` - Date the record was inserted into the ERP database
* `custo_medio_epoca` - Average cost of the product at the time the entry was posted
* `conhecimento` - Freight bill of lading
Payment plans used to settle each movement.
Source method: `LinxMovimentoPlanos`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `identificador`, `plano`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `identificador` - Identifier that groups the records belonging to the same fiscal document
* `plano` - Internal payment plan code in Microvix
* `desc_plano` - Description of the payment plan
* `total` - Total amount settled through the payment plan
* `qtde_parcelas` - Number of instalments
* `indice_plano` - Index of the payment plan applied to the movement
* `cod_forma_pgto` - Code of the payment method
* `forma_pgto` - Name of the payment method
* `tipo_transacao` - Card payment type: D for debit, C for credit
* `taxa_financeira` - Financial fee percentage charged by the card administrators
* `ordem_cartao` - Sequence of the card (1, 2, 3, ...); only filled for card payment plans
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `empresa` - Company code within the portal
Card transactions captured on each sales movement.
Source method: `LinxMovimentoCartoes`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `identificador`, `ordem_cartao`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `codlojasitef` - Sitef store code
* `data_lancamento` - Date the document was posted
* `identificador` - Identifier of the movement
* `cupomfiscal` - Fiscal receipt number
* `credito_debito` - Card transaction type: C for credit, D for debit
* `id_cartao_bandeira` - Identifier of the card brand registry
* `descricao_bandeira` - Description of the card brand used
* `valor` - Amount charged to the card
* `ordem_cartao` - Sequence number of the card when several cards are used
* `nsu_host` - Host NSU (unique sequential number) of the transaction
* `nsu_sitef` - Sitef NSU (unique sequential number) of the transaction
* `cod_autorizacao` - Authorisation code of the transaction
* `id_antecipacoes_financeiras` - Identifier of the financial advance, when there is one
* `texto_comprovante` - Receipt text printed for the card transaction
* `id_maquineta_pos` - Identifier of the card terminal registry (non-TEF)
* `descricao_maquineta` - Description given to the card terminal
* `serie_maquineta` - Serial number of the card terminal
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `cartao_prepago` - Whether a prepaid card was used
* `transacao_cancelada_smartpos` - Whether the SmartPOS transaction was cancelled
* `usuario_cancelamento_transacao_smartpos` - User that cancelled the SmartPOS transaction
* `data_hora_cancelamento_transacao_smartpos` - Date the SmartPOS transaction was cancelled
Exchange vouchers applied to sales movements.
Source method: `LinxMovimentoTrocas`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `identificador`, `num_vale`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `identificador` - Identifier that groups the records belonging to the same fiscal document
* `num_vale` - Number of the exchange voucher
* `valor_vale` - Amount of the exchange voucher
* `motivo` - Description of the reason for the exchange
* `doc_origem` - Number of the source document related to the return
* `serie_origem` - Series of the source document related to the return
* `doc_venda` - Number of the document related to the sale
* `serie_venda` - Series related to the sale
* `excluido` - Whether the exchange was deleted
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `desfazimento` - Set to 1 when the exchange has a null identifier, a negative balance and doc\_venda equal to 0
* `empresa` - Company code within the portal
* `vale_cod_cliente` - Code of the customer of the sale where the standalone credit voucher was used as payment
* `vale_codigoproduto` - Product code recorded on the exchange voucher
* `id_vale_ordem_servico_externa` - Identifier of the external service order voucher
* `doc_venda_origem` - Source sale document, shown for nationwide exchange operations
* `serie_venda_origem` - Source sale series, shown for nationwide exchange operations
* `cod_cliente` - Code of the customer chosen when the standalone credit voucher was created
* `data_vencimento` - Expiry date of the exchange voucher
* `valor_original` - Original amount of the exchange voucher; negative when it offsets a balance
* `id_antecipacoes_financeiras` - Identifier of the financial advance in the antecipacoes\_financeiras table
Promotional campaigns applied to sales movements.
Source method: `LinxMovimentoAcoesPromocionais`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `identificador`, `id_acoes_promocionais`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `identificador` - Identifier that groups the records belonging to the same fiscal document
* `transacao` - Internal Microvix code
* `id_acoes_promocionais` - Code of the promotional campaign
* `desconto_item` - Discount amount granted by the promotional campaign
* `quantidade` - Number of promotional campaigns applied to the sale
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
### Products and stock
Product catalogue of the portal.
Source method: `LinxProdutos`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cod_produto`
**Fields:**
* `portal` - Portal code in Microvix
* `cod_produto` - Product code in Microvix
* `cod_barra` - Barcode of the product
* `nome` - Name of the product
* `ncm` - NCM tax classification code of the product
* `cest` - CEST tax classification code of the product
* `referencia` - Reference code of the product
* `cod_auxiliar` - Auxiliary code of the product
* `unidade` - Measurement unit the product is registered with
* `desc_cor` - Description of the product's second grid dimension (colour) in Microvix
* `desc_tamanho` - Description of the product's first grid dimension (size) in Microvix
* `desc_setor` - Description of the product sector in Microvix
* `desc_linha` - Description of the product line in Microvix
* `desc_marca` - Description of the product brand in Microvix
* `desc_colecao` - Description of the product collection in Microvix
* `dt_update` - Date the product was last changed
* `cod_fornecedor` - Internal Microvix code of the supplier linked to the product
* `desativado` - Whether the product is deactivated
* `desc_espessura` - Description of the product thickness in Microvix
* `id_espessura` - Product thickness code in Microvix
* `desc_classificacao` - Description of the product classification in Microvix
* `id_classificacao` - Product classification code in Microvix
* `origem_mercadoria` - Code of the goods origin in Microvix
* `peso_liquido` - Net weight of the product in Microvix
* `peso_bruto` - Gross weight of the product in Microvix
* `id_cor` - Code of the product's second grid dimension (colour) in Microvix
* `id_tamanho` - Code of the product's first grid dimension (size) in Microvix
* `id_setor` - Product sector code in Microvix
* `id_linha` - Product line code in Microvix
* `id_marca` - Product brand code in Microvix
* `id_colecao` - Product collection code in Microvix
* `dt_inclusao` - Date the product was added
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `fator_conversao` - Conversion factor of the unit
* `codigo_integracao_ws` - For customers integrated through the inbound web service: the code sent on integration
* `codigo_integracao_reshop` - For customers integrated through the inbound web service: the code field sent on integration
* `id_produtos_opticos_tipo` - Code of the optical product type
* `id_sped_tipo_item` - SPED item type identifier
* `componente` - Whether the product is a component: S for yes, N for no
* `altura_para_frete` - Height of the product considered for freight calculation
* `largura_para_frete` - Width of the product considered for freight calculation
* `comprimento_para_frete` - Product length considered for freight calculation
* `codigoproduto_io` - Product code in Linx IO
* `cod_comprador` - Buyer code
* `codigo_integracao_oms` - Product code used by the OMS
* `obrigatorio_identificacao_cliente` - Whether identifying the customer is mandatory on sales
* `descricao_basica` - Basic description of the product
* `curva` - ABC curve classification of the product
* `tributa_fcp` - Whether the product is subject to the poverty relief fund tax (FCP)
* `observacao` - Note about the product, limited to 1000 characters; only returned when the exibe\_observacao parameter is set to 1
* `codigo_adicional_reshop` - Additional product code in the ERP, used by Reshop
* `brinde` - Whether the product is a giveaway
* `exige_controle` - Whether the product requires stock control: S for yes, N for no
* `multiplo_venda` - Sales multiple of the product: the quantity increment it can be sold in
Per-store product data: current stock balance, prices, costs and tax setup.
Source method: `LinxProdutosDetalhes`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `cod_produto`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `cod_produto` - Product code in Microvix
* `cod_barra` - Barcode of the product
* `quantidade` - Current stock balance of the product
* `preco_custo` - Cost price of the product including ICMS
* `preco_venda` - Sale price of the item in the default price table
* `custo_medio` - Average cost of the product
* `id_config_tributaria` - Code of the tax configuration attached to the product
* `desc_config_tributaria` - Description of the tax configuration attached to the product
* `despesas1` - Amount recorded in the expenses field
* `qtde_minima` - Minimum stock quantity
* `qtde_maxima` - Maximum stock quantity
* `ipi` - IPI tax rate
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `custototal` - Total cost of the product
* `empresa` - Company code within the portal
* `id_produtos_status_compra` - Identifier of the status in the produtos\_status\_compra table
* `localizacao` - Storage location of the product
Barcodes registered for each product.
Source method: `LinxProdutosCodBar`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cod_produto`, `cod_barra`
**Fields:**
* `portal` - Portal code in Microvix
* `cod_produto` - Product code in Microvix
* `cod_barra` - Barcode of the product
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
Stock position of each product on the requested date.
Source method: `LinxProdutosInventario`. Full table sync (this method returns no row version).
**Primary key:** `portal`, `cnpj_emp`, `cod_produto`, `cod_deposito`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `cod_produto` - Product code in Microvix
* `cod_barra` - Barcode of the product
* `quantidade` - Stock balance on the requested date
* `cod_deposito` - Warehouse code, only returned when at least one warehouse is requested
* `empresa` - Company code within the portal
Warehouses (depósitos) registered for the stores.
Source method: `LinxProdutosDepositos`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cod_deposito`
**Fields:**
* `portal` - Portal code in Microvix
* `cod_deposito` - Warehouse code
* `nome_deposito` - Name of the warehouse
* `disponivel` - Whether the warehouse is available for sales
* `disponivel_transferencia` - Whether the warehouse is available for transfers between branches
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
### Product classification
Product brands registered in the portal.
Source method: `LinxMarcas`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_marca`
**Fields:**
* `id_marca` - Identifier of the brand
* `desc_marca` - Description of the brand
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `portal` - Portal code in Microvix
Product lines registered in the portal.
Source method: `LinxLinhas`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_linha`
**Fields:**
* `id_linha` - Identifier of the product line
* `desc_linha` - Description of the product line
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `portal` - Portal code in Microvix
* `coeficiente_comissao` - Commission coefficient
Product collections registered in the portal.
Source method: `LinxColecoes`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_colecao`
**Fields:**
* `id_colecao` - Identifier of the collection
* `desc_colecao` - Description of the collection
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `portal` - Portal code in Microvix
Product sectors registered in the portal.
Source method: `LinxSetores`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_setor`
**Fields:**
* `id_setor` - Identifier of the sector
* `desc_setor` - Description of the sector
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `ativo` - Whether the record is active (1) or inactive (0)
* `portal` - Portal code in Microvix
Product sizes registered in the portal.
Source method: `LinxTamanhos`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_tamanho`
**Fields:**
* `id_tamanho` - Identifier of the size
* `desc_tamanho` - Description of the size
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `ativo` - Whether the record is active (1) or inactive (0)
* `portal` - Portal code in Microvix
Product colours registered in the portal.
Source method: `LinxCores`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_cor`
**Fields:**
* `id_cor` - Identifier of the colour
* `desc_cor` - Description of the colour
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `ativo` - Whether the record is active (1) or inactive (0)
* `portal` - Portal code in Microvix
Product classifications registered in the portal.
Source method: `LinxClassificacoes`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `id_classificacao`
**Fields:**
* `id_classificacao` - Identifier of the classification
* `desc_classificacao` - Description of the classification
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_integracao_ws` - Code assigned by the web service integration
* `ativo` - Whether the record is active (1) or inactive (0)
* `portal` - Portal code in Microvix
Measurement units registered in the portal.
Source method: `LinxUnidades`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `unidade`
**Fields:**
* `unidade` - Measurement unit
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `portal` - Portal code in Microvix
### Customers and suppliers
Customers and suppliers registered in the stores.
Source method: `LinxClientesFornec`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cod_cliente`
**Fields:**
* `portal` - Portal code in Microvix
* `cod_cliente` - Customer/supplier code in Microvix
* `razao_cliente` - Legal name of the customer or supplier
* `nome_cliente` - Name of the customer or supplier
* `doc_cliente` - CPF or CNPJ (Brazilian tax ID) of the customer
* `tipo_cliente` - Whether the customer is an individual or a company
* `endereco_cliente` - Street address
* `numero_rua_cliente` - Street number of the address
* `complement_end_cli` - Address complement
* `bairro_cliente` - Neighbourhood
* `cep_cliente` - Postal code (CEP)
* `cidade_cliente` - City
* `uf_cliente` - State abbreviation (UF)
* `pais` - Country
* `fone_cliente` - Phone number of the customer
* `email_cliente` - Email address
* `sexo` - Gender
* `data_cadastro` - Date the customer was registered
* `data_nascimento` - Date of birth of the customer
* `cel_cliente` - Mobile phone number of the customer
* `ativo` - Whether the customer is active (S) or not (N)
* `dt_update` - Date the record was last changed
* `inscricao_estadual` - State tax registration number
* `incricao_municipal` - Municipal tax registration number
* `identidade_cliente` - RG (Brazilian ID document) of the customer
* `cartao_fidelidade` - Loyalty card number in Microvix, when the loyalty module is used
* `cod_ibge_municipio` - IBGE code of the municipality
* `classe_cliente` - Code and name of the class linked to the customer; only one class is returned
* `matricula_conveniado` - Code of the affiliated party (conveniado)
* `tipo_cadastro` - Registry type: C for customers, F for suppliers, A for both
* `empresa_cadastro` - Whether the record is a carrier (T)
* `id_estado_civil` - Marital status: 1 not informed, 2 married, 3 single, 4 divorced, 5 widowed
* `fax_cliente` - Fax number of the customer
* `site_cliente` - Website of the customer
* `timestamp` - Row version of the underlying table, used as the incremental replication key
* `cliente_anonimo` - Whether the customer is anonymous
* `limite_compras` - Purchase limit registered as store credit (crediário) in the customer and supplier registry
* `codigo_ws` - Code sent through the inbound web service integration
* `limite_credito_compra` - Purchase limit registered for cash and card payments in the customer and supplier registry
* `id_classe_fiscal` - Identifier of the fiscal class assigned to the customer
* `obs` - Note about the customer or supplier
* `mae` - Mother's name of the customer
* `cliente_contribuinte` - Whether the customer is a registered taxpayer
* `id_tipo_registro_profissional` - Identifier in the tipo\_registro\_profissional table
* `registro_profissional` - Description of the professional registration
* `recebe_newsletter` - Whether the customer subscribes to the newsletter
* `diferencial_icms_fornec` - Whether an ICMS rate differential applies
* `perc_diferencial_icms_fornec` - ICMS rate differential percentage
* `id_classe_preferencial` - Code of the preferred class linked to the customer
* `id_subclasse_preferencial` - Code of the preferred subclass linked to the customer
Contacts attached to each customer or supplier.
Source method: `LinxClientesFornecContatos`. Full table sync (this method returns no row version).
**Primary key:** `portal`, `cod_cliente`, `nome_contato`
**Fields:**
* `portal` - Portal code in Microvix
* `cod_cliente` - Customer/supplier code in Microvix
* `nome_contato` - Contact name
* `sexo_contato` - Gender of the contact: F for female, M for male
* `contatos_clientes_parentesco` - Relationship code of the contact
* `fone1_contato` - Primary phone number of the contact
* `fone2_contato` - Secondary phone number of the contact
* `celular_contato` - Mobile phone number of the contact
* `email_contato` - Email address of the contact
* `data_nasc_contato` - Date of birth of the contact
* `tipo_contato` - Type of contact
### Finance
Accounts receivable and payable instalments.
Source method: `LinxFaturas`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `codigo_fatura`, `ordem_parcela`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `codigo_fatura` - Internal invoice code in Microvix
* `data_emissao` - Issue date of the invoice
* `cod_cliente` - Code of the customer of the invoice
* `nome_cliente` - Name of the customer or supplier of the invoice
* `data_vencimento` - Due date of the invoice
* `data_baixa` - Date the invoice was settled
* `valor_fatura` - Original amount of the invoice
* `valor_pago` - Amount paid when the invoice was settled
* `valor_desconto` - Discount amount granted on the invoice
* `valor_juros` - Interest amount charged on the invoice
* `documento` - Number of the document that generated the invoice
* `serie` - Series of the document generated for the invoice
* `ecf` - Number of the fiscal printer (ECF) that generated the invoice
* `observacao` - Note about the invoice
* `qtde_parcelas` - Total number of instalments of the invoice
* `ordem_parcela` - Instalment sequence number
* `receber_pagar` - Whether the invoice is receivable or payable
* `vendedor` - Code of the salesperson linked to the invoice
* `excluido` - Whether the invoice was deleted
* `cancelado` - Whether the invoice is cancelled
* `identificador` - Identifier that references the records belonging to the same fiscal document
* `nsu` - NSU of the invoice settled through TEF
* `cod_autorizacao` - Authorisation code of the invoice settled through TEF
* `documento_sem_tef` - Document number entered manually for sales without TEF
* `autorizacao_sem_tef` - Authorisation code entered manually for sales without TEF
* `plano` - Internal payment plan code in Microvix
* `conta_credito` - Credit account code of the accounting entry
* `conta_debito` - Debit account code of the accounting entry
* `conta_fluxo` - Cash-flow account code of the accounting entry
* `cod_historico` - Code of the history entry in Microvix
* `forma_pgto` - Name of the payment method
* `ordem_cartao` - Sequence of the card (1, 2, 3, ...); only filled for card payment plans
* `banco_codigo` - Bank code, used for cheque invoices
* `banco_agencia` - Bank branch code, used for cheque invoices
* `banco_conta` - Bank account code, used for cheque invoices
* `banco_autorizacao_garantidora` - Guarantor code, used for cheque invoices
* `numero_bilhete_seguro` - Only used with the third-party services sales module
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `empresa` - Company code within the portal
* `id_categorias_financeiras` - Identifier of the financial category
* `taxa_financeira` - Financial fee applied to the invoice
* `valor_abatimento` - Rebate amount applied to the invoice
* `valor_multa` - Penalty amount charged on the invoice
* `centrocusto` - Identifier of the cost centre
* `perc_taxa_adquirente` - Fee percentage charged by the card acquirer
* `fatura_origem_importacao_erp` - Original number of the invoice imported into the ERP
* `cod_portador` - Code of the invoice holder (portador)
* `nsu_host` - Host NSU (unique sequential number) of the transaction
* `nsu_sitef` - Sitef NSU (unique sequential number) of the transaction
* `previsao` - Whether the invoice is a forecast: S for yes, N for no
* `data_lanc` - Date the invoice was posted
* `dt_insert` - Date the record was inserted
* `plano_pagamento_baixa` - Identifier of the payment plan used to settle the invoice
Payment plans registered in the portal.
Source method: `LinxPlanos`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `plano`
**Fields:**
* `portal` - Portal code in Microvix
* `plano` - Internal payment plan code in Microvix
* `desc_plano` - Description of the payment plan
* `qtde_parcelas` - Number of instalments
* `prazo_entre_parcelas` - Number of days between instalments
* `tipo_plano` - Transaction type: C for purchase, V for sale
* `indice_plano` - Index of the payment plan
* `cod_forma_pgto` - Code of the payment method
* `forma_pgto` - Name of the payment method
* `conta_central` - Central account code linked to the customer/supplier registry
* `tipo_transacao` - Card payment type: D for debit, C for credit
* `taxa_financeira` - Financial fee percentage charged by the card administrators
* `dt_upd` - Date the record was last changed
* `desativado` - Whether the payment plan is deactivated: S for yes, N for no
* `usa_tef` - Whether the plan uses TEF: S (yes) or N (no)
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `codigo_ws` - Code received from the web service integration
* `valor_minimo_parcela` - Minimum instalment amount
Card brands linked to each payment plan.
Source method: `LinxPlanosBandeiras`. Full table sync (this method returns no row version).
**Primary key:** `portal`, `plano`, `bandeira`
**Fields:**
* `portal` - Portal code in Microvix
* `plano` - Internal payment plan code in Microvix
* `bandeira` - Card brand description in Microvix
* `tipo_bandeira` - Description of the linked card brand type
* `adquirente` - Code of the card acquirer or operator, linked to the customer/supplier registry
* `nome_adquirente` - Name of the card acquirer or operator, linked to the customer/supplier registry
* `codigo_bandeira_sitef` - Sitef card brand code
Cash withdrawals (sangrias) and supplies registered at the cash register.
Source method: `LinxSangriaSuprimentos`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `id_sangria_suprimentos`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `usuario` - User that performed the movement
* `data` - Date of the record
* `valor` - Amount of the cash withdrawal or cash supply
* `obs` - Description of the cash withdrawal or supply
* `cancelado` - Whether the record is cancelled (S) or valid (N)
* `conferido` - Whether the document has been checked (1) or not (0)
* `cod_historico` - Code of the history entry in Microvix
* `desc_historico` - Description of the history entry
* `id_sangria_suprimentos` - Identifier of the cash withdrawal or supply entry
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
### Salespeople
Salespeople registered in the stores.
Source method: `LinxVendedores`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cod_vendedor`
**Fields:**
* `portal` - Portal code in Microvix
* `cod_vendedor` - Salesperson code in Microvix
* `nome_vendedor` - Name of the customer or supplier
* `tipo_vendedor` - Role: V for salesperson, C for buyer, A for both
* `end_vend_rua` - Street address
* `end_vend_numero` - Street number of the address
* `end_vend_complemento` - Address complement
* `end_vend_bairro` - Neighbourhood
* `end_vend_cep` - Postal code (CEP)
* `end_vend_cidade` - City
* `end_vend_uf` - State abbreviation (UF)
* `fone_vendedor` - Phone number of the customer
* `mail_vendedor` - Email address
* `dt_upd` - Date the record was last changed
* `cpf_vendedor` - CPF (Brazilian individual tax ID) of the salesperson
* `ativo` - Whether the salesperson is active: S for yes, N for no
* `data_admissao` - Hiring date of the salesperson
* `data_saida` - Termination date of the salesperson
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `matricula` - Employee registration number of the salesperson
* `id_tipo_venda` - Identifier of the sale type
* `descricao_tipo_venda` - Description of the sale type
* `cargo` - Job title of the salesperson
* `nome_exibicao` - Display name of the salesperson in Microvix
* `funcao` - Job function of the salesperson
* `empresa` - Company the salesperson is assigned to
* `codigo_integracao_ws` - Code sent through the web service integration
* `meta_peso` - Weight of the sales target
Sales targets assigned to salespeople.
Source method: `LinxMetasVendedores`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `id_meta`, `cod_vendedor`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `id_meta` - Sales target code in Microvix
* `descricao_meta` - Description of the sales target
* `data_inicial_meta` - Start date of the sales target
* `data_final_meta` - End date of the sales target
* `valor_meta_loja` - Sales target amount for the store
* `valor_meta_vendedor` - Sales target amount for the salesperson
* `cod_vendedor` - Salesperson code in Microvix
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
### Promotions
Promotional campaigns registered in the stores.
Source method: `LinxAcoesPromocionais`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `id_acoes_promocionais`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `id_acoes_promocionais` - Code of the promotional campaign
* `vigencia_inicio` - Start date of the validity period
* `vigencia_fim` - End date of the validity period
* `observacao` - Note about the promotional campaign
* `ativa` - Whether the promotional campaign is active
* `excluida` - Whether the promotional campaign was deleted
* `integrada` - Whether the promotional campaign was integrated
* `qtde_integrada` - Maximum quantity integrated by the franchisor
* `valor_pago_franqueadora` - Amount paid by the franchisor
* `id_acoes_promocionais_origem` - Identifier of the originating promotional campaign
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
### Orders
Sales orders, one row per item.
Source method: `LinxPedidosVenda`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `cod_pedido`, `posicao_item`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `cod_pedido` - Order code
* `data_lancamento` - Date of the order
* `hora_lancamento` - Time the order was posted
* `transacao` - Internal Microvix code
* `usuario` - User that placed the order
* `codigo_cliente` - Customer code
* `cod_produto` - Product code
* `quantidade` - Quantity of the product
* `valor_unitario` - Unit price of the product
* `cod_vendedor` - Salesperson code in Microvix
* `valor_frete` - Freight amount
* `valor_total` - Total amount of the order
* `desconto_item` - Discount applied to the item
* `cod_plano_pagamento` - Code of the payment plan
* `plano_pagamento` - Name of the payment plan
* `obs` - Note about the order
* `aprovado` - Whether the record is already an order (S) or still a quotation (N)
* `cancelado` - Flag with values S (yes) or N (no)
* `data_aprovacao` - Date the order was approved
* `data_alteracao` - Date the record was last changed
* `tipo_frete` - Freight type: 1 for CIF, 2 for FOB
* `natureza_operacao` - Description of the operation nature
* `tabela_preco` - Code of the price table used
* `nome_tabela_preco` - Name of the price table
* `previsao_entrega` - Expected delivery date
* `realizado_por` - Code of the customer the order was placed for
* `pontuacao_ser` - Customer code in the Pontuação Ser loyalty programme
* `venda_externa` - Flag with values S (yes) or N (no)
* `nf_gerada` - Comma-separated numbers of the fiscal invoices generated when the order was billed
* `status` - Billing status: F fully billed, P partially billed, N not billed
* `numero_projeto_officina` - Project code
* `cod_natureza_operacao` - Code of the operation nature
* `margem_contribuicao` - Contribution margin of the store
* `doc_origem` - Source document number
* `posicao_item` - Position of the item within the sales order
* `orcamento_origem` - Number the quotation originated from; 0 when it has no origin
* `transacao_origem` - Original transaction kept when a sales order is partially approved and a new transaction is generated
* `timestamp` - Original transaction the record refers to, used as the incremental replication key
* `desconto` - Total discount of the sales order
* `transacao_ws` - Transaction sent through the web service integration
* `empresa` - Company code within the portal
* `transportador` - Identifier of the carrier registered in the portal
* `deposito` - Identifier of the warehouse in the portal
* `texto_agrupamento` - Grouping text of the order
* `posicao` - Code of the quotation position
* `encerrado` - Whether the quotation or order was closed
Purchase orders, one row per item.
Source method: `LinxPedidosCompra`. Incremental sync on the `timestamp` row version.
**Primary key:** `portal`, `cnpj_emp`, `cod_pedido`, `cod_produto`
**Fields:**
* `portal` - Portal code in Microvix
* `cnpj_emp` - CNPJ (Brazilian tax ID) of the store
* `cod_pedido` - Order code
* `data_pedido` - Date of the order
* `transacao` - Internal Microvix code
* `usuario` - User that placed the order
* `codigo_fornecedor` - Supplier code
* `cod_produto` - Product code
* `quantidade` - Quantity of the product
* `valor_unitario` - Unit price of the product
* `cod_comprador` - Buyer code in Microvix
* `valor_frete` - Freight amount
* `valor_total` - Total amount of the order
* `cod_plano_pagamento` - Code of the payment plan
* `plano_pagamento` - Name of the payment plan
* `obs` - Note about the order
* `aprovado` - Flag with values S (yes) or N (no)
* `cancelado` - Flag with values S (yes) or N (no)
* `encerrado` - Flag with values S (yes) or N (no)
* `data_aprovacao` - Date the order was approved
* `numero_ped_fornec` - Supplier's own order number, when informed
* `tipo_frete` - Freight type code
* `natureza_operacao` - Description of the operation nature
* `previsao_entrega` - Expected delivery date
* `numero_projeto_officina` - Project code
* `status_pedido` - Order status: E for delivered, P for pending
* `qtde_entregue` - Quantity of items already delivered
* `descricao_frete` - Description of the freight
* `integrado_linx` - Whether the order was integrated by Linx (1) or not (0)
* `nf_gerada` - Number of the fiscal invoice generated when the order was billed; only the first invoice is shown
* `timestamp` - Row version of the underlying table, used as the incremental replication key (not a date: it's a SQL Server row-version counter)
* `empresa` - Company code within the portal
* `nf_origem_ws` - Number of the fiscal invoice created through the LinxCadastraNotas method
* `id_compras_pedido_status` - Identifier of the purchase order status
* `qtde_desejada` - Quantity requested
* `motivo_ajuste` - Reason for the adjustment
* `saldo_disponivel_api` - Balance still available through the API
# Lixlog as a data source
Source: https://docs.nekt.com/sources/lixlog
Bring data from Lixlog to Nekt.
Lixlog is a Brazilian freight and logistics platform. It quotes shipping across multiple carriers, books the collection at your warehouse and tracks the delivery through to the addressee. This connector brings that whole operation into your catalog: the shipments and their delivery status, the carrier tracking events, the collection requests, and the full history of freight quotes with every carrier option that was offered.
![]()
## Configuring Lixlog as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Lixlog option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the API token of your Lixlog account. Generate it in the Lixlog panel; it starts with `flx_`.
The following configurations are available:
* **API token**: The API token of your Lixlog account. It is stored encrypted and is sent to Lixlog as a bearer token.
* **Initial sync date**: (Advanced) The starting point for orders, pickups and freight quote history. Products, warehouses and carriers are always read in full. When left empty, orders and pickups load their entire history and freight quotes load the last 90 days.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Lixlog only exposes tracking events one shipment at a time, so the **Order Trackings** stream makes one request per order. On accounts with a large order history the extraction takes considerably longer with this stream selected. Leave it unselected if you do not need the carrier's event-by-event history.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Orders and pickups change over their lifetime (an order created today is delivered days later), so INCREMENTAL keeps them up to date by their last update timestamp. Freight quotes are never edited after the fact, so their history only ever grows. Products, warehouses and carriers are small reference tables and are read in full on every run.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Orders | `orders` | Incremental | Shipments registered in Lixlog, with carrier, warehouse, addressee, invoice and delivery status |
| Order Trackings | `order_trackings` | Full table | Carrier tracking events for each order |
| Pickups | `pickups` | Incremental | Collection requests placed with the carriers |
| Quotation Logs | `quotation_logs` | Incremental | Freight quotes answered, with every carrier option offered |
| Products | `products` | Full table | Products with the weight and dimensions freight is priced from |
| Warehouses | `warehouses` | Full table | Shipping origins registered in the account |
| Carriers | `carriers` | Full table | Carriers available to the account |
## Fields by stream
Below you'll find all available data streams from Lixlog and their corresponding fields:
The central table of the connector: one row per shipment, carrying the carrier that moves it, the warehouse it leaves from, the addressee it goes to, the invoice attached to it and the timestamps of each step of the delivery.
It also carries the freight rate chosen for the shipment, flattened into the `selected_rate_*` columns. That block is what makes delivery SLA measurable: it holds the delivery time the carrier committed to and the date it promised, for essentially every order, where the only other estimate in this connector is the one some carriers volunteer in the tracking feed.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the order in Lixlog |
| `reference` | String | Order identifier in your own system |
| `tracking_code` | String | Tracking code Lixlog assigned to the order, used to join the Order Trackings stream |
| `carrier_tracking_code` | String | Tracking code the carrier assigned to the shipment |
| `status` | String | Current delivery status, such as `initial`, `available_for_pickup`, `in_transit`, `out_for_delivery`, `delivered`, `failure` or `cancelled` |
| `pickuped_at` | Timestamp | When the carrier collected the order at the warehouse |
| `out_for_delivery_at` | Timestamp | When the order went out for its final delivery attempt |
| `delivered_at` | Timestamp | When the order was delivered to the addressee |
| `archived_at` | Timestamp | When the order was archived in Lixlog |
| `created_at` | Timestamp | When the order was created in Lixlog |
| `updated_at` | Timestamp | When the order was last updated (incremental replication key) |
| `carrier_id` | Integer | Identifier of the carrier moving this order, joins the Carriers stream |
| `carrier_name` | String | Legal name of the carrier |
| `carrier_federal_tax` | String | CNPJ of the carrier |
| `warehouse_id` | Integer | Identifier of the warehouse the order ships from, joins the Warehouses stream |
| `warehouse_name` | String | Name of the warehouse the order ships from |
| `selected_rate_id` | Integer | Identifier of the freight rate chosen for this order |
| `selected_rate_rate_id` | String | Identifier of the tariff the chosen rate was priced from, as Lixlog's own quote identifier. Joins the carrier option of the same id inside the 'response' of the quotation\_logs stream |
| `selected_rate_order_id` | Integer | Identifier of the order this rate was chosen for. Always this order's own id |
| `selected_rate_carrier_id` | Integer | Identifier of the carrier that quoted the chosen rate. Joins the carriers stream, and is normally the carrier that ends up moving the order |
| `selected_rate_service` | String | Service the carrier quoted for this order, such as Normal or Expresso. Carriers price and commit to a delivery time per service |
| `selected_rate_delivery_days` | Integer | Number of business days committed to for this delivery, counted from the quotation date in 'selected\_rate\_created\_at', not from the order's creation date, which on a re-quoted order can be months earlier. This is the final figure, after any freight rule of the account extended the carrier's own quote; 'selected\_rate\_log\_freight\_rules\_applied' shows whether one did |
| `selected_rate_delivery_date` | Date | Delivery date the carrier committed to when the rate was quoted. This is the promised date to measure delivery SLA against, next to 'delivered\_at' |
| `selected_rate_value` | Float | Final freight price charged for this order, in BRL |
| `selected_rate_cost` | Float | What the carrier charges Lixlog for this shipment, in BRL, before the freight rules that produce the final price |
| `selected_rate_cubed_weight` | Float | Cubed weight the rate was priced on, in kilograms. Carriers bill the greater of the real and the cubed weight, so this is what the price was based on |
| `selected_rate_log_details` | JSON String | Composition of the freight tariff, as a JSON array of objects with one entry per charge line ('fee': FRETE, ADV, TDA, GRIS, PEDAGIO and so on) carrying its cost, minimum, percentage and ICMS rate |
| `selected_rate_log_deadline_details` | JSON String | Breakdown of the carrier's own lead time, as a JSON array of objects with the days assigned to each stage of the delivery ('type': despatch, separation, pickup and transit). The stages sum to the lead time the carrier quoted, which is what the account's freight rules are then applied on top of, so the sum matches 'selected\_rate\_delivery\_days' only when no rule changed the delivery days |
| `selected_rate_log_freight_rules_applied` | JSON String | Freight rules the account applied on top of the carrier's quote, as a JSON array of objects carrying each rule's id and name and the value or the delivery days it changed, before and after |
| `selected_rate_created_at` | Timestamp | Timestamp when the chosen rate was quoted. This is the base date the carrier counts 'selected\_rate\_delivery\_days' from |
| `selected_rate_updated_at` | Timestamp | Timestamp when the chosen rate was last updated in Lixlog |
| `addressee_name` | String | Name of the person or company receiving the shipment |
| `addressee_email` | String | Email address of the addressee |
| `addressee_phone` | String | Phone number of the addressee |
| `addressee_federal_tax` | String | CPF or CNPJ of the addressee |
| `addressee_street` | String | Street name of the delivery address |
| `addressee_number` | String | Street number of the delivery address |
| `addressee_complement` | String | Additional address information, such as unit or floor |
| `addressee_neighborhood` | String | Neighborhood of the delivery address |
| `addressee_city` | String | City of the delivery address |
| `addressee_state` | String | State of the delivery address, as a two-letter code |
| `addressee_zip` | String | Postal code (CEP) of the delivery address |
| `addressee_residential` | Boolean | Whether the address is residential, which some carriers price differently |
| `addressee_id` | Integer | Identifier of the address record in Lixlog |
| `addressee_created_at` | Timestamp | When the address was created in Lixlog |
| `addressee_updated_at` | Timestamp | When the address was last updated in Lixlog |
| `invoice_id` | Integer | Identifier of the invoice (nota fiscal) attached to the order |
| `invoice_number` | String | Number of the invoice |
| `invoice_serie` | String | Series of the invoice |
| `invoice_key` | String | Access key (chave de acesso) of the electronic invoice attached to the order |
| `link_tracking` | String | Public URL where the addressee can follow the delivery |
| `items` | JSON String | Products shipped in this order: reference (SKU), price, weight, volume, quantity and package dimensions |
| `warehouse_address` | JSON String | Full address of the warehouse the order ships from |
Tracking events reported by the carrier for each order. One row per event.
| Field | Type | Description |
| :- | :- | :- |
| `order_id` | Integer | Identifier of the order this event belongs to, joins the Orders stream |
| `tracking_code` | String | Lixlog tracking code the event was reported against |
| `id` | Integer | Unique identifier of the tracking event |
| `status` | String | Delivery status this event puts the shipment in: `in_transit`, `out_for_delivery`, `delivered` or `failure` |
| `title` | String | Short title of the event |
| `message` | String | Full description of what happened to the shipment |
| `code` | String | Event code reported by the carrier, when the carrier sends one |
| `city` | String | City where the event was recorded |
| `state` | String | State where the event was recorded |
| `occurred_at` | Timestamp | When the event happened |
| `delivery_date` | Timestamp | Delivery date the carrier reported with this event |
| `created_at` | Timestamp | When Lixlog recorded the event |
Collection requests placed with the carriers, one row per request.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the pickup in Lixlog |
| `status` | String | Current status of the request, such as `initial`, `accepted`, `cancelled` or `finished` |
| `pickup_date` | Timestamp | Date and time scheduled with the carrier for the collection |
| `carrier_pickup_code` | String | Code the carrier assigned to this request |
| `note` | String | Free-text note sent to the carrier with the request |
| `accepted_at` | Timestamp | When the carrier accepted the request |
| `performed_at` | Timestamp | When the carrier actually performed the collection |
| `finished_at` | Timestamp | When the collection was closed in Lixlog |
| `canceled_at` | Timestamp | When the request was cancelled |
| `created_at` | Timestamp | When the request was created in Lixlog |
| `updated_at` | Timestamp | When the request was last updated (incremental replication key) |
| `carrier_id` | Integer | Identifier of the carrier asked to collect, joins the Carriers stream |
| `carrier_name` | String | Legal name of the carrier |
| `carrier_federal_tax` | String | CNPJ of the carrier |
| `warehouse_id` | Integer | Identifier of the warehouse the collection happens at, joins the Warehouses stream |
| `warehouse_name` | String | Name of the warehouse the collection happens at |
| `integration_status` | String | Outcome of the automated request Lixlog sent to the carrier's own system |
| `company_id` | Integer | Identifier of the Lixlog account the pickup belongs to |
| `created_by_id` | Integer | Identifier of the Lixlog user who requested the collection |
| `link_public_url` | String | Public URL the carrier uses to see and confirm this collection |
| `integration_request` | JSON String | Payload Lixlog sent to the carrier's system to book the collection |
| `integration_response` | JSON String | Response the carrier's system returned for the booking |
| `warehouse_address` | JSON String | Full address of the warehouse the collection happens at |
Every freight quote Lixlog answered, with all the carrier options it returned. This is the freight-pricing history: what a shipment could have cost, next to what it did cost in the Orders stream.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the quote request, and its incremental replication key |
| `channel` | String | Sales channel the quote was requested from, when the caller identified one |
| `zip_from` | String | Origin postal code (CEP) the freight was quoted from |
| `zip_to` | String | Destination postal code (CEP) the freight was quoted to |
| `quoted_options_count` | Integer | Number of carrier options returned. Zero means no carrier could serve the route, which is the signal to look for coverage gaps |
| `min_freight_value` | Float | Cheapest final freight price among the options returned |
| `request` | JSON String | What was asked to be quoted. Its `parcels` array carries one entry per package with the declared price, weight, dimensions, quantity and product reference |
| `response` | JSON String | Every carrier option returned: the final freight price, Lixlog's own cost, the delivery estimate in days, the cubed weight, the carrier (id, code and service) and the freight rules applied |
Products registered in Lixlog, with the measurements freight is priced from. This is the table to check when a shipment is quoted heavier or bulkier than expected.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the product in Lixlog |
| `reference` | String | Product code (SKU) in your own system |
| `description` | String | Description of the product |
| `price` | Float | Declared value of one unit, used to insure the shipment |
| `weight` | Float | Weight of one unit, in kilograms |
| `volume` | Float | Total volume of the product's packages, in cubic meters |
| `deadline` | Integer | Extra days this product adds to the delivery deadline of an order |
| `created_at` | Timestamp | When the product was created in Lixlog |
| `updated_at` | Timestamp | When the product was last updated |
| `volumes` | JSON String | Packages the product ships in, with the height, length and width in meters and the quantity of identical packages |
Shipping origins registered in the account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the warehouse in Lixlog |
| `name` | String | Name of the warehouse |
| `description` | String | Description of the warehouse |
| `zip` | String | Postal code (CEP) freight is quoted from for this warehouse |
| `default` | Boolean | Whether this is the warehouse used when an order does not name one |
| `address_street` | String | Street name of the warehouse address |
| `address_number` | String | Street number of the warehouse address |
| `address_complement` | String | Additional address information |
| `address_neighborhood` | String | Neighborhood of the warehouse address |
| `address_city` | String | City of the warehouse address |
| `address_state` | String | State of the warehouse address, as a two-letter code |
| `address_zip` | String | Postal code (CEP) of the warehouse address |
| `address_name` | String | Contact name at the warehouse |
| `address_email` | String | Contact email at the warehouse |
| `address_phone` | String | Contact phone at the warehouse |
| `address_federal_tax` | String | CPF or CNPJ registered for the warehouse contact |
| `address_residential` | Boolean | Whether the address is residential |
| `address_id` | Integer | Identifier of the address record in Lixlog |
| `address_created_at` | Timestamp | When the address was created in Lixlog |
| `address_updated_at` | Timestamp | When the address was last updated in Lixlog |
| `created_at` | Timestamp | When the warehouse was created in Lixlog |
| `updated_at` | Timestamp | When the warehouse was last updated |
Carriers available to the account.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the carrier in Lixlog |
| `name` | String | Legal name of the carrier |
| `display_name` | String | Short name of the carrier as shown in the Lixlog panel |
| `code` | String | Lixlog code of the carrier. It is the value that appears as the carrier code in the freight quotes of the Quotation Logs stream |
| `tms` | String | Transport management system Lixlog integrates with to reach this carrier |
| `federal_tax` | String | CNPJ of the carrier |
| `state_tax` | String | State tax registration of the carrier |
| `phone` | String | Contact phone number of the carrier |
| `created_at` | Timestamp | When the carrier was added in Lixlog |
| `updated_at` | Timestamp | When the carrier was last updated |
# Data Model
The following diagram illustrates the relationships between the core data streams in Lixlog. The arrows indicate the join keys that link the different entities, providing a clear overview of the logistics structure.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Orders("Orders");
Warehouses("Warehouses");
Carriers("Carriers");
end
subgraph "History and Logistics"
OrderTrackings("Order Trackings");
Pickups("Pickups");
QuotationLogs("Quotation Logs");
end
OrderTrackings -- "order_id" --> Orders;
Orders -- "warehouse_id" --> Warehouses;
Orders -- "carrier_id" --> Carriers;
Pickups -- "warehouse_id" --> Warehouses;
Pickups -- "carrier_id" --> Carriers;
```
## Implementation Notes
### Data Quality Considerations
* **Nested Data**: Some columns hold structured data that is stored as a JSON string, for example the items of an order, the packages of a product, or the carrier options of a quote. Use your warehouse's JSON functions (`JSON_EXTRACT`, etc.) to expand them in a Query or Notebook.
### API Limits & Performance
* **Tracking Events**: Lixlog only exposes tracking events one shipment at a time, so the `Order Trackings` stream makes one request per order. On accounts with a large order history, this extraction takes considerably longer.
* **Pagination and Collection Limits**: The Lixlog API (which runs on Rails/Pagy) handles pagination dynamically. When a request asks for a page beyond the end of a collection (e.g., page 2 of a 1-page result), it returns an HTTP `400 Bad Request` instead of an empty page. The Nekt connector safely handles these page-overflow errors, gracefully interpreting them as the end of the collection rather than aborting the extraction run.
* **Dynamic Page Optimization**: To speed up your extractions and prevent sending doomed out-of-bounds requests, the connector memorizes the maximum observed page size dynamically, stopping pagination early when a page is returned shorter than the known limit.
## Skills for agents
Lixlog connector documentation as plain markdown, for use in AI agent contexts.
# Lomadee as a data source
Source: https://docs.nekt.com/sources/lomadee
Bring data from Lomadee to Nekt.
Lomadee is one of Latin America's largest affiliate networks: brands pay commission to affiliates for the orders their links and coupons generate. This connector reads either side of the network. As an **affiliate**, you bring your orders and commissions, your short links and their clicks, and the catalog of brands, campaigns and products you promote. As a **brand**, you bring the orders your affiliates generated and the affiliates of your program.
## Before you start
Lomadee has one API for each side of the network, and each side has its own key. Pick the side your Lomadee account is on and create the matching key.
### If you are an affiliate
In the Lomadee panel, open the **Chaves de API** page.
Click **Criar chave de API**, give it a name (for example *Nekt*) and select the permissions for the tables you want: **Pedidos**, **Marcas**, **Campanhas**, **Canais**, **Produtos** and **Encurtador (leitura)**.
The key is shown **only once**. Copy it before closing the window. A new key may show the status *Em análise* until Lomadee approves it.
A table whose permission the key does not have is skipped and reported in the run, and every other table still syncs. To add it later, create a new key with that permission and update the source.
### If you are a brand
In the Lomadee panel, go to **Configurações » Integrações**.
Under **Dados da integração por SDK**, copy the **Chave de API**.
If your brand panel has no **Dados da integração por SDK** section, your brand does not have API access enabled yet. Ask your Lomadee contact to enable the SDK integration for your brand, then copy the key.
## Configuring Lomadee as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button in the top-right corner, then select Lomadee from the list of connectors.
Click **Next** and add your access details.
### 1. Add account access
The following configuration fields are available:
* **Account type**: **Affiliate** if you promote brands and earn commission, or **Brand** if you run an affiliate program on Lomadee. It decides which tables the source offers.
* **API key**: The key you copied in [Before you start](#before-you-start). An affiliate key does not work on a brand source, and the other way around.
* **Initial sync date** (advanced, affiliates only): The earliest order date read by the **Orders** table on the first run. Later runs continue from where the previous one stopped. Leave it empty to read your full order history.
* **Lookback window (days)** (advanced, affiliates only): How many days before the last synced order every run reads again, so status and commission changes are picked up while the brand validates an order. Defaults to 90.
* **Include expired campaigns** (advanced, affiliates only): Also read campaigns whose period has ended. By default only running and scheduled campaigns are read.
* **Product brand IDs** (advanced, affiliates only): Comma-separated ids of the brands whose products to read, taken from the `id` column of the **Brands** table. Leave it empty to read the whole catalog.
* **Requests per minute** (advanced): Maximum number of API calls per minute. Lomadee allows 60 per key; the default of 50 leaves room for other integrations that use the same key.
After configuring access, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the ones relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how data should appear in your Catalog, including layer, table naming, and sync type.
* **Layer**: Select the destination layer where extracted tables will be created.
* **Folder**: A folder can be created inside the selected layer to group all tables from this source.
* **Table name**: Use the suggested name or customize it. You can also apply a prefix to all tables at once.
* **Sync Type**: Choose between `INCREMENTAL` and `FULL_TABLE`. See [Sync types](#sync-types): only the affiliate **Orders** table supports `INCREMENTAL`.
When you're done, click **Next**.
### 4. Configure data source
Add a clear description for this source so your team can identify it easily, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), choose how often data should be extracted (for example, daily, weekly, or at fixed times).
Optionally, configure additional settings:
* Delta Log Retention, which controls how long old table states are kept. Learn more [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync), to periodically complement incremental runs with a complete refresh.
When you are ready, click **Next** to finalize setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, trigger a run manually from the source actions.
To see data in the [Catalog](https://app.nekt.ai/catalog), at least one source run must complete successfully.
## Sync types
The affiliate **Orders** table is the only one that syncs incrementally. It is the only Lomadee listing that filters by date, so each run reads the orders created since the last one, plus the lookback window, because an order's status and commission keep changing for weeks while the brand validates it.
Every other table is read in full on every run. Lomadee offers no "modified since" filter on them, so a full read is what keeps the Catalog equal to Lomadee. That includes the brand **Orders** table, whose endpoint has no date filter at all.
## Good to know
* **Two sides, two sets of tables.** An affiliate source offers Orders, Brands, Campaigns, Channels, Products and Short links. A brand source offers Orders and Affiliates. Both have an Orders table, with different columns, because each side sees the order differently.
* **Order statuses change after the sale.** An affiliate order starts as `pending` and becomes `approved`, `rejected` or `repproved` once the brand validates it, and `commission_status` follows. The lookback window is what brings those updates in.
* **The full order is always kept.** Both Orders tables carry a `raw_payload` column with the complete record exactly as Lomadee returned it, including any field that has no column of its own.
* **Brand order values follow your integration.** Brand order values are stored exactly as your store's integration sent them to Lomadee. The Lomadee pixel sends them in cents (`157490` is R\$ 1.574,90), so divide by 100 in a [Query](https://docs.nekt.com/queries/overview) if that is how your integration works.
* **Nested data is JSON text.** Order items, value breakdowns, product variants and similar lists are stored as JSON arrays. Parse them in a Query or Notebook when you need one row per item.
* **Products can be a large table.** It holds the catalog of every brand you can promote. Use **Product brand IDs** to limit it to the brands you work with.
* **A product id is unique per brand.** Two brands can use the same product id, so the key of the Products table is `organization_id` plus `id`.
* **Channels are capped at 20.** Lomadee returns at most 20 channels per account; if you have more, the run says so.
* **Short links include inactive ones.** The table holds both active and deactivated links, with their click counts; filter on `active` when you need only the live ones.
* **Rate limit.** Lomadee allows 60 requests per minute per key. The connector paces itself below it and waits automatically if Lomadee asks it to slow down, so a large first sync takes a while but loses no data.
# Streams and Fields
## Affiliate account
| Stream | Slug | Description |
| - | - | - |
| Orders | `orders` | Orders your links and coupons generated, with their value, status and commission. |
| Brands | `brands` | Brands you can promote, with the commission they pay and their ratings. |
| Campaigns | `campaigns` | Offers and coupons the brands publish, with their period and links. |
| Channels | `channels` | Your own promotion channels, such as a blog or a push list. |
| Products | `products` | The product catalog of the brands you can promote, with variants, prices and stock. |
| Short links | `short_links` | Short links you generated, with their click counts. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the order in the Lomadee network. |
| `order_id` | String | Identifier of the order in the brand's own store system. |
| `organization_id` | String | Identifier of the brand the order was placed with. |
| `status` | String | Status of the order: pending, approved, rejected or repproved. |
| `value` | Number | Total value of the order, exactly as reported by Lomadee. |
| `commission` | Number | Commission the affiliate earns on the order, exactly as reported by Lomadee. |
| `commission_status` | String | Status of the commission: pending while the brand validates the order, then approved, rejected or repproved. |
| `mdasc` | String | Custom tracking identifier the affiliate appended to the link (the mdasc parameter), used to attribute the order to a campaign or placement. |
| `items` | String | Items of the order, each with its id, name, quantity and price (JSON array of objects). |
| `created_at` | Date | Date and time the order was created, used as the incremental replication key. |
| `raw_payload` | String | The complete order record exactly as Lomadee returned it (JSON object), including any field that has no column of its own. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the brand (its organization id). |
| `name` | String | Name of the brand. |
| `slug` | String | URL-friendly slug of the brand. |
| `logo` | String | URL of the brand's logo. |
| `site` | String | URL of the brand's website. |
| `segment` | String | Market segment the brand belongs to. |
| `network_active` | Boolean | Whether the brand is active on the Lomadee network. |
| `network_is_public` | Boolean | Whether the brand is public on the network or restricted to approved affiliates. |
| `is_exclusive` | Boolean | Whether the brand is exclusive to Lomadee. |
| `is_highlight` | Boolean | Whether Lomadee highlights the brand. |
| `is_favorite` | Boolean | Whether the affiliate marked the brand as a favourite. |
| `rating_approval` | Number | Approval rate of the brand, from 0 to 5. |
| `rating_commission` | Number | Commission rating of the brand, in percent. |
| `rating_conversion` | Number | Conversion rate of the brand, in percent. |
| `rating_popularity` | Number | Popularity index of the brand, from 0 to 100. |
| `rating_validation` | Number | Order validation rate of the brand, from 0 to 5. |
| `commission_value` | Number | Commission the brand pays the affiliate, in percent of the order value. |
| `commission_transfer` | String | Commission payment model, such as cpa (cost per action). |
| `channels` | String | The affiliate's channels for this brand, each with its short links or the restriction that blocks them (JSON array of objects). |
| `created_at` | Date | Date and time the brand joined the network. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the campaign (its operation id, used to generate links). |
| `name` | String | Name of the campaign. |
| `type` | String | Kind of campaign: PersonalCoupon, GenericCoupon or Offer. |
| `offer_type` | String | For offers, how the offer is delivered: Url or Spreadsheet. |
| `description` | String | Description of the campaign. |
| `code` | String | Coupon code of the campaign, when it is a coupon. |
| `url` | String | Landing page URL of the campaign. |
| `organization_id` | String | Identifier of the brand the campaign belongs to. |
| `is_highlight` | Boolean | Whether Lomadee highlights the campaign. |
| `status` | String | Status derived from the campaign period: onTime (running), scheduled (not started) or expired (ended). |
| `period_start_at` | Date | Start of the campaign period; empty for a permanent campaign. |
| `period_end_at` | Date | End of the campaign period; empty for a permanent campaign. |
| `categories` | String | Categories of the campaign (JSON array). |
| `channels` | String | The affiliate's channels for this campaign, each with its short links (JSON array of objects). |
| `media_kit` | String | Marketing material of the campaign, such as banner URLs (JSON object). |
| `created_at` | Date | Date and time the campaign was created. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the channel. |
| `name` | String | Type of the channel, such as Blog or PushNotification. |
| `active` | Boolean | Whether the channel is active. |
| `created_at` | Date | Date and time the channel was created. |
| Field | Type | Description |
| :- | :- | :- |
| `organization_id` | String | Identifier of the brand that sells the product. |
| `id` | String | Identifier of the product in the brand's catalog (unique per brand). |
| `object_id` | String | Internal Lomadee identifier of the catalog entry. |
| `name` | String | Name of the product. |
| `description` | String | Description of the product (HTML). |
| `url` | String | URL of the product page in the brand's store. |
| `available` | Boolean | Whether the product is available for sale. |
| `integration` | String | E-commerce platform the product was imported from, such as vtex. |
| `images` | String | Images of the product (JSON array of objects). |
| `options` | String | Variants of the product, each with its EAN, seller, prices in cents and stock (JSON array of objects). |
| `metadata` | String | Additional product data (JSON array). |
| `created_at` | Date | Date and time the product entered the catalog. |
| `updated_at` | Date | Date and time the product was last updated. |
| Field | Type | Description |
| :- | :- | :- |
| `url_code` | String | Unique code of the short link, the last part of its URL. |
| `short_url` | String | The short link itself. |
| `original_url` | String | The URL the short link redirects to. |
| `organization_id` | String | Identifier of the brand the link points to. |
| `link_type` | String | What the link points to: Coupon, Offer, BrandPage or Custom. |
| `active` | Boolean | Whether the short link is active. |
| `click_count` | Integer | Number of clicks the short link received. |
| `created_at` | Date | Date and time the short link was created. |
| `updated_at` | Date | Date and time the short link was last updated. |
## Brand account
| Stream | Slug | Description |
| - | - | - |
| Orders | `orders` | Orders attributed to your affiliates, with their value breakdown and customer. |
| Affiliates | `affiliates` | Affiliates of your program, with their invitation status and profile. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the order in Lomadee. |
| `order_id` | String | Identifier of the order in the brand's own store system. |
| `status` | String | Status of the order: Pending, Approved, Reproved or Canceled. |
| `value` | Number | Total value of the order as the brand's integration sent it; the Lomadee pixel sends values in cents (157490 is R\$ 1,574.90). |
| `items_value` | Number | Total value of the order's products, in the same unit as value (the Items sub- item). |
| `shipping_value` | Number | Shipping charged on the order, in the same unit as value (the Shipping sub-item). |
| `discounts_value` | Number | Discounts applied to the order, as a negative number in the same unit as value (the Discounts sub-item). |
| `coupon` | String | Coupon code used on the order, if any. |
| `affiliate_id` | String | Identifier of the affiliate the order is attributed to. |
| `channel_id` | String | Identifier of the affiliate channel the order came through. |
| `group` | String | Grouping label the brand attached to the order. |
| `customer_id` | String | Identifier of the customer in the brand's store. |
| `customer_email` | String | E-mail address of the customer. |
| `customer_first_name` | String | First name of the customer. |
| `customer_last_name` | String | Last name of the customer. |
| `customer_document_type` | String | Type of the customer's document, such as CPF or CNPJ. |
| `customer_document` | String | Document number of the customer. |
| `customer_phone` | String | Phone number of the customer. |
| `items` | String | Products of the order, each with its id, name, image, price, list price and quantity (JSON array of objects). |
| `sub_items` | String | Breakdown of the order value into Items, Shipping, Discounts and other entries (JSON array of objects). |
| `created_at` | Date | Date and time the order was registered in Lomadee. |
| `updated_at` | Date | Date and time the order was last updated. |
| `raw_payload` | String | The complete order record exactly as Lomadee returned it (JSON object), including any field that has no column of its own. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the affiliate in the brand's program. |
| `user_organization_id` | String | Identifier of the link between the affiliate's user and the brand. |
| `invite_status` | String | Status of the affiliate's invitation: pendent, accept, need\_approval or rejected. |
| `active` | Boolean | Whether the affiliate is active in the program. |
| `user_id` | String | Identifier of the affiliate's Lomadee user. |
| `user_email` | String | E-mail address of the affiliate. |
| `user_username` | String | Username of the affiliate on Lomadee. |
| `user_trade_name` | String | Trade name of the affiliate, for a company. |
| `user_legal_name` | String | Legal name of the affiliate, for a company. |
| `user_document_type` | String | Type of the affiliate's document: cpf, cnpj, rg, passport or international. |
| `user_document` | String | Document number of the affiliate. |
| `user_phone` | String | Phone number of the affiliate. |
| `user_birth_date` | String | Birth date of the affiliate. |
| `user_position` | String | Job position the affiliate declared. |
| `user_avatar` | String | URL of the affiliate's profile picture. |
| `user_onboarding_status` | String | Whether the affiliate finished signing up: Completed or Pending. |
| `user_access_count` | Integer | Number of times the affiliate accessed the Lomadee panel. |
| `user_last_login_at` | Date | Date and time the affiliate last logged in to Lomadee. |
| `user_address` | String | Address of the affiliate (JSON object). |
| `user_categories` | String | Categories the affiliate promotes (JSON). |
| `clusters` | String | Affiliate groups of the brand's program the affiliate belongs to, each with its name and whether it is the default group (JSON array of objects). |
| `user_created_at` | Date | Date and time the affiliate's Lomadee user was created. |
| `user_updated_at` | Date | Date and time the affiliate's Lomadee user was last updated. |
# Magalu as a data source
Source: https://docs.nekt.com/sources/magalu
Bring data from Magalu Marketplace to Nekt.
Magalu (Magazine Luiza) is one of Brazil's largest retail and e-commerce platforms. The Magalu Seller API provides access to your marketplace seller data including orders (with their embedded deliveries), delivery invoices (NF-e), product catalog (SKUs), pricing, and inventory. This connector enables you to extract and analyze your Magalu Marketplace operations data.
![]()
## Configuring Magalu as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Magalu option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Magalu Seller data. Click on the `Magalu Authorization` button and log in with your Magalu account. Grant the necessary permissions for the seller account you want to extract data from.
Make sure you're logged in with a Magalu account that has **seller access** and appropriate permissions to view orders, deliveries, and product catalog data.
The following configurations are available:
* **Start Date**: (Optional) Lower bound for the incremental **Orders** sync (`updated_at__gte` filter on the API). **Deliveries invoices** are fetched for the deliveries of the synced orders, so they follow the same bound indirectly (their own bookmark is `issued_at`). If omitted, orders are synced from the full history available in the API. SKUs, Prices and Stocks are full-table extractions and ignore this setting.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Magalu and their corresponding fields:
Product catalog stream containing all SKUs (Stock Keeping Units) from your Magalu seller portfolio.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `sku` | String | SKU identifier, unique code that identifies the product |
| `title` | String | Product title or name |
| `description` | String | Detailed product description |
| `brand` | String | Brand name of the product |
| `status` | String | Current status (e.g., 'active', 'inactive', 'pending', 'rejected') |
| `active` | Boolean | Whether the SKU is currently active and available for sale |
| `condition` | String | Product condition (e.g., 'new', 'used', 'refurbished') |
| `type` | String | Product type classification |
| `created_at` | Timestamp | When the SKU was created |
| `updated_at` | Timestamp | When the SKU was last updated (the **skus** stream syncs as full table; this field is useful for change tracking) |
| `creator` | String | User who created the SKU |
| `updater` | String | User who last updated the SKU |
| `attributes` | Array | Array of product attributes (name/value pairs for characteristics like Color, Size) |
| `datasheet` | Array | Array of technical specifications (name/value pairs like Voltage, Power, Warranty) |
| `extra_data` | Array | Array of additional custom data fields |
| `identifiers` | Array | Array of product identifier objects (type, value) |
| `has_ean` | Boolean | Whether the product has an EAN barcode |
| `group` | Object | Product group information linking related SKUs/variants (id) |
| `categories` | Array | Array of category classifications (categories, sub\_categories, product\_type, channel) |
| `channels` | Array | Array of sales channels where product is available (id) |
| `url_marketplace` | Array | Array of marketplace listing URLs (channel, url) |
| `dimensions` | Array | Array of dimension sets (name, height, length, width, weight; each measure as value/unit) |
| `images` | Array | Array of product images (reference, type) |
| `videos` | Array | Array of product videos (reference, type) |
| `podcasts` | Array | Array of related podcasts (reference, type) |
| `fulfillment` | Boolean | Whether the product uses fulfillment services |
| `perishable` | Boolean | Whether the product is perishable |
Orders stream containing all marketplace orders with customer, payment, and delivery information.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the order |
| `code` | String | Order code in the platform |
| `status` | String | Order status (e.g., 'new', 'approved', 'cancelled', 'finished') |
| `created_at` | Timestamp | When the order was created |
| `updated_at` | Timestamp | When the order was last updated (replication key) |
| `approved_at` | Timestamp | When the order was approved |
| `purchased_at` | Timestamp | When the order was purchased |
| `channel` | Object | Sales channel information (id, extras, marketplace) |
| `customer` | Object | Customer details (name, email, document\_number, customer\_type, birth\_date, phones) |
| `amounts` | Object | Order totals (currency, normalizer, total, commission, discount, freight, tax) |
| `foreign_amounts` | Object | Foreign currency amounts (currency, normalizer, total, exchange\_rate) |
| `deliveries` | Array | Array of order deliveries (id, code, status, purchased\_at, seller, amounts, items). Each delivery carries a `shipping` object with `pickup_details`, `drop_details`, `recipient`, `handling_time`, `deadline`, `provider` (id, name, description, extras), `tracking_url`, `tracking`, `shipped_at`, `delivered_at` and `cancelled_at` |
| `payments` | Array | Array of payment methods (amount, currency, method, method\_brand, description, installments, normalizer, integration\_type, authorization\_code, gateway, extras) |
**Performance Impact:** This stream calls the invoices endpoint once per delivery embedded in the synced **Orders** (`deliveries[]`). Large delivery volumes increase API traffic and extraction time. Enable it only when you need NF-e (Nota Fiscal eletrônica) keys or XML payloads.
Brazilian electronic invoice (NF-e) metadata and XML for each delivery. Child stream of **Orders**: one request is made per delivery listed in each order, except deliveries with status `cancelled`, which never carry an invoice and are skipped. Deliveries for which the API answers 403 or 404 (no invoice available) are skipped with a warning instead of failing the run.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `key` | String | NF-e access key (chave de acesso); primary key for this stream |
| `delivery_id` | String | Delivery identifier (from the parent order's `deliveries[].id`) |
| `status` | String | Invoice status (e.g., approved) |
| `issued_at` | Timestamp | When the invoice was issued (ISO 8601); replication key for incremental sync |
| `xml` | String | Full NF-e XML content |
**Performance Impact:** This stream requires an additional API request for each SKU in your catalog. If you have a large number of SKUs, selecting this stream will significantly increase extraction time. Only enable this stream if you need real-time pricing data per channel.
SKUs that have no pricing document in Magalu (the API answers `404 Document not found`) are skipped with a warning instead of failing the sync.
Pricing information for each SKU per sales channel. This is a child stream of SKUs.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `sku` | String | SKU identifier, unique code that identifies the product |
| `channel_id` | String | Channel identifier where this price is valid |
| `created_at` | Timestamp | When the price record was created |
| `updated_at` | Timestamp | When the price was last updated |
| `price` | Number | Current selling price of the product |
| `list_price` | Number | List price or original price before discounts |
| `currency` | String | Currency code (e.g., 'BRL') |
| `normalizer` | Number | Price normalizer value used for price calculations |
| `channel` | Object | Channel information (id) |
**Performance Impact:** This stream requires an additional API request for each SKU in your catalog. If you have a large number of SKUs, selecting this stream will significantly increase extraction time. Only enable this stream if you need real-time stock levels per channel.
SKUs that have no inventory document in Magalu (the API answers `404 Document not found`) are skipped with a warning instead of failing the sync.
Stock/inventory information for each SKU per sales channel. This is a child stream of SKUs.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `sku` | String | SKU identifier, unique code that identifies the product |
| `channel_id` | String | Channel identifier where this stock is available |
| `created_at` | Timestamp | When the stock record was created |
| `updated_at` | Timestamp | When the stock was last updated |
| `quantity` | Number | Available quantity of the product in stock |
| `type` | String | Type of stock (e.g., 'AVAILABLE') |
# Data Model
The following diagram illustrates the relationships between the core data streams in Magalu. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Product Catalog"
SKUs("SKUs");
Prices("Prices");
Stocks("Stocks");
end
subgraph "Sales Data"
Orders("Orders");
DeliveriesInvoices("Deliveries invoices");
end
SKUs -- "sku" --> Prices;
SKUs -- "sku" --> Stocks;
Orders -- "deliveries[].items[].info.sku" --> SKUs;
Orders -- "deliveries[].id = delivery_id" --> DeliveriesInvoices;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Magalu Marketplace data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Order Status Overview
Track the distribution of order statuses to understand your sales pipeline and identify potential bottlenecks.
**Business Value:**
* Monitor order fulfillment rates
* Identify issues with specific order statuses
* Track cancellation rates and reasons
```sql theme={null}
WITH order_stats AS (
SELECT
status,
COUNT(*) AS order_count,
SUM(amounts.total / COALESCE(NULLIF(amounts.normalizer, 0), 1)) AS total_revenue,
AVG(amounts.total / COALESCE(NULLIF(amounts.normalizer, 0), 1)) AS avg_order_value
FROM
nekt_raw.magalu_orders
WHERE
DATE(created_at) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
status
),
total AS (
SELECT SUM(order_count) AS total_orders FROM order_stats
)
SELECT
os.status,
os.order_count,
ROUND(os.order_count * 100.0 / t.total_orders, 2) AS percentage,
ROUND(os.total_revenue, 2) AS total_revenue,
ROUND(os.avg_order_value, 2) AS avg_order_value
FROM
order_stats os, total t
ORDER BY
os.order_count DESC
```
```sql theme={null}
WITH order_stats AS (
SELECT
status,
COUNT(*) AS order_count,
SUM(amounts.total / COALESCE(NULLIF(amounts.normalizer, 0), 1)) AS total_revenue,
AVG(amounts.total / COALESCE(NULLIF(amounts.normalizer, 0), 1)) AS avg_order_value
FROM
`nekt_raw.magalu_orders`
WHERE
DATE(created_at) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
status
),
total AS (
SELECT SUM(order_count) AS total_orders FROM order_stats
)
SELECT
os.status,
os.order_count,
ROUND(SAFE_DIVIDE(os.order_count * 100.0, t.total_orders), 2) AS percentage,
ROUND(os.total_revenue, 2) AS total_revenue,
ROUND(os.avg_order_value, 2) AS avg_order_value
FROM
order_stats os, total t
ORDER BY
os.order_count DESC
```
| status | order\_count | percentage | total\_revenue | avg\_order\_value |
| - | - | - | - | - |
| finished | 1,245 | 62.25 | 892,340.50 | 716.74 |
| approved | 456 | 22.80 | 312,450.00 | 685.20 |
| new | 189 | 9.45 | 145,230.00 | 768.41 |
| cancelled | 110 | 5.50 | 78,560.00 | 714.18 |
### 2. Top Selling Products
Identify your best-performing products based on sales volume and revenue.
**Business Value:**
* Understand which products drive the most revenue
* Optimize inventory for high-demand items
* Inform marketing and promotional strategies
```sql theme={null}
WITH item_sales AS (
SELECT
i.info.sku AS sku,
i.info.name AS product_name,
SUM(i.quantity) AS total_quantity,
SUM(i.amounts.total / COALESCE(NULLIF(i.amounts.normalizer, 0), 1)) AS total_revenue,
COUNT(DISTINCT o.id) AS order_count
FROM
nekt_raw.magalu_orders o
CROSS JOIN UNNEST(o.deliveries) AS t(d)
CROSS JOIN UNNEST(d.items) AS t2(i)
WHERE
o.status NOT IN ('cancelled')
AND DATE(o.created_at) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
i.info.sku,
i.info.name
)
SELECT
sku,
product_name,
total_quantity,
order_count,
ROUND(total_revenue, 2) AS total_revenue,
ROUND(total_revenue / NULLIF(total_quantity, 0), 2) AS avg_unit_price
FROM
item_sales
ORDER BY
total_revenue DESC
LIMIT 20
```
```sql theme={null}
WITH item_sales AS (
SELECT
i.info.sku AS sku,
i.info.name AS product_name,
SUM(i.quantity) AS total_quantity,
SUM(i.amounts.total / COALESCE(NULLIF(i.amounts.normalizer, 0), 1)) AS total_revenue,
COUNT(DISTINCT o.id) AS order_count
FROM
`nekt_raw.magalu_orders` o,
UNNEST(o.deliveries) AS d,
UNNEST(d.items) AS i
WHERE
o.status NOT IN ('cancelled')
AND DATE(o.created_at) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
i.info.sku,
i.info.name
)
SELECT
sku,
product_name,
total_quantity,
order_count,
ROUND(total_revenue, 2) AS total_revenue,
ROUND(SAFE_DIVIDE(total_revenue, total_quantity), 2) AS avg_unit_price
FROM
item_sales
ORDER BY
total_revenue DESC
LIMIT 20
```
| sku | product\_name | total\_quantity | order\_count | total\_revenue | avg\_unit\_price |
| - | - | - | - | - | - |
| SKU-12345 | Smart TV 55" 4K Ultra HD | 234 | 230 | 467,532.00 | 1,998.00 |
| SKU-67890 | Smartphone Galaxy S24 | 189 | 185 | 341,811.00 | 1,809.00 |
| SKU-24680 | Notebook Gamer 16GB RAM | 78 | 75 | 312,000.00 | 4,000.00 |
| SKU-13579 | Air Fryer Digital 5.5L | 456 | 420 | 228,456.00 | 501.00 |
| SKU-11223 | Wireless Earbuds Pro | 892 | 845 | 178,400.00 | 200.00 |
### 3. Delivery Performance Analysis
Monitor delivery times and shipping provider performance to optimize logistics, using the deliveries embedded in each order.
**Business Value:**
* Track delivery success rates
* Identify slow shipping providers
* Optimize fulfillment processes
```sql theme={null}
WITH delivery_metrics AS (
SELECT
d.shipping.provider.name AS provider_name,
d.status AS delivery_status,
COUNT(*) AS delivery_count,
AVG(d.shipping.deadline.value) AS avg_deadline_days,
SUM(CASE WHEN d.status = 'delivered' THEN 1 ELSE 0 END) AS delivered_count,
SUM(CASE WHEN d.status = 'cancelled' THEN 1 ELSE 0 END) AS cancelled_count
FROM
nekt_raw.magalu_orders o
CROSS JOIN UNNEST(o.deliveries) AS t(d)
WHERE
DATE(d.purchased_at) >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
d.shipping.provider.name,
d.status
)
SELECT
provider_name,
SUM(delivery_count) AS total_deliveries,
SUM(delivered_count) AS total_delivered,
SUM(cancelled_count) AS total_cancelled,
ROUND(SUM(delivered_count) * 100.0 / NULLIF(SUM(delivery_count), 0), 2) AS delivery_success_rate,
ROUND(AVG(avg_deadline_days), 1) AS avg_deadline_days
FROM
delivery_metrics
GROUP BY
provider_name
ORDER BY
total_deliveries DESC
```
```sql theme={null}
WITH delivery_metrics AS (
SELECT
d.shipping.provider.name AS provider_name,
d.status AS delivery_status,
COUNT(*) AS delivery_count,
AVG(d.shipping.deadline.value) AS avg_deadline_days,
SUM(CASE WHEN d.status = 'delivered' THEN 1 ELSE 0 END) AS delivered_count,
SUM(CASE WHEN d.status = 'cancelled' THEN 1 ELSE 0 END) AS cancelled_count
FROM
`nekt_raw.magalu_orders` o,
UNNEST(o.deliveries) AS d
WHERE
DATE(d.purchased_at) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
d.shipping.provider.name,
d.status
)
SELECT
provider_name,
SUM(delivery_count) AS total_deliveries,
SUM(delivered_count) AS total_delivered,
SUM(cancelled_count) AS total_cancelled,
ROUND(SAFE_DIVIDE(SUM(delivered_count) * 100.0, SUM(delivery_count)), 2) AS delivery_success_rate,
ROUND(AVG(avg_deadline_days), 1) AS avg_deadline_days
FROM
delivery_metrics
GROUP BY
provider_name
ORDER BY
total_deliveries DESC
```
| provider\_name | total\_deliveries | total\_delivered | total\_cancelled | delivery\_success\_rate | avg\_deadline\_days |
| - | - | - | - | - | - |
| Magalu Entregas | 1,456 | 1,389 | 42 | 95.40 | 3.2 |
| Correios | 892 | 834 | 35 | 93.50 | 5.8 |
| Jadlog | 456 | 421 | 18 | 92.32 | 4.5 |
| Total Express | 234 | 218 | 8 | 93.16 | 4.1 |
## Implementation Notes
### Data normalization
Magalu uses a normalizer pattern for monetary values. To get the actual value, divide by the normalizer:
```sql theme={null}
-- Example: Get actual order total
SELECT
amounts.total / COALESCE(NULLIF(amounts.normalizer, 0), 1) AS actual_total
FROM orders
```
Common normalizers:
* `100` for values stored as cents
* `1` for values already in the base currency unit
### Order status flow
Orders in Magalu typically follow this status flow:
1. `new` - Order just placed
2. `approved` - Payment confirmed
3. `invoiced` - Invoice generated
4. `shipped` - Order dispatched
5. `delivered` - Order delivered to customer
6. `finished` - Order completed
Orders can also be:
* `cancelled` - Order was cancelled
* `returned` - Customer returned the order
### Brazilian context
This connector is designed for the Brazilian marketplace:
* **Currency**: Values are in BRL (Brazilian Real)
* **Documents**: Customer documents are CPF (individuals) or CNPJ (companies)
* **Addresses**: Brazilian address format with CEP (postal code), state abbreviations
* **Shipping**: Includes Brazilian carriers like Correios, Jadlog, and Magalu's own logistics
### Incremental sync and replication keys
* **Orders**: Incremental when configured, using `updated_at` as the replication key. API requests are sorted by `updated_at` and filtered with `updated_at__gte` from the bookmark or **Start Date**.
* **Deliveries invoices**: Incremental bookmark on `issued_at`. The invoices endpoint itself is not date-filtered; the set of deliveries to query comes from the orders returned by the incremental **Orders** sync.
* **SKUs**, **Prices**, **Stocks**: Full table sync in the tap (no replication key on those streams). **Start Date** does not apply to them.
### Child streams and API volume
* **Prices** and **Stocks** are children of **SKUs**: one extra API request per SKU per stream when enabled.
* **Deliveries invoices** is a child of **Orders**: one extra API request per delivery listed in each synced order (deliveries with status `cancelled` are skipped). The API's standalone `/deliveries` endpoint is not used because of its pagination limits, so delivery data itself lives in the `deliveries` array of **Orders**.
Extraction time scales with SKU count (prices/stocks) and delivery count (in invoices). Prefer off-peak schedules for large catalogs, and enable child streams only when you need that granularity.
### Missing documents in child streams
* **Prices** and **Stocks**: if a SKU listed in your catalog has no pricing or stock document on the seller side, Magalu answers `404 Document not found`. The connector skips that SKU (a warning is logged once per stream, naming the first affected SKU) and continues the extraction normally.
* **Deliveries invoices**: a 403 or 404 for a delivery means no invoice is available for it; the delivery is skipped with a warning and the run continues.
### Catalog size limits and pagination
The Magalu API enforces a strict offset limit of 20,000 records on the SKUs listing. To ensure complete extraction of large catalogs, the connector automatically escalates through multiple pagination strategies (cursor-based, keyset re-anchoring, and bidirectional per-status sweeps) to bypass this limit.
If your catalog has an extremely high volume of SKUs sharing the exact same status (over \~40,000 items), the API's hard limits might still truncate the extraction for that specific status. In this rare scenario, the connector logs a warning indicating how many SKUs were synced and skips the unreachable ones. Note that any SKUs truncated this way will also be missing from the child **Prices** and **Stocks** streams.
### Nested data structures
Magalu payloads are deeply nested. When querying:
* Use `UNNEST` (or `CROSS JOIN UNNEST` in Athena) to flatten arrays
* Access nested objects using dot notation (e.g., `customer.name`)
* Handle NULL values in nested fields with `COALESCE`
Example for accessing delivery items from nested orders:
```sql theme={null}
-- AWS Athena
SELECT d.items
FROM orders o
CROSS JOIN UNNEST(o.deliveries) AS t(d)
-- GCP BigQuery
SELECT i.*
FROM orders o, UNNEST(o.deliveries) AS d, UNNEST(d.items) AS i
```
## Skills for agents
Magalu connector documentation as plain markdown, for use in AI agent contexts.
# Magazord as a data source
Source: https://docs.nekt.com/sources/magazord
Bring data from Magazord to your Lakehouse.
Magazord is a Brazilian e-commerce platform with a built-in ERP, covering online stores, marketplaces, orders, inventory, billing and logistics in a single back office. Nekt extracts this data through the Magazord Integration API.
## Configuring Magazord as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Magazord option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are required:
* **Panel URL**: The base URL of your Magazord administrative panel, for example `https://yourstore.painel.magazord.com.br` (no trailing path). The API is served under this domain.
* **API Token**: The API token of your Magazord account. If you don't have one yet, request your integration credentials from Magazord (for example via your account manager or [integracao@magazord.com.br](mailto:integracao@magazord.com.br)).
* **API Password**: The API password paired with the token. Both are sent using HTTP Basic authentication.
Optionally, under advanced settings:
* **Start Date**: The earliest date from which records will be synced for incremental streams (defaults to 2023-01-01).
* **Store Timezone**: IANA timezone used to render date filters in store-local time (defaults to America/Sao\_Paulo).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Magazord and their corresponding fields. Orders, customers, products, carts, invoices, prices and stock movements sync incrementally; the remaining streams are extracted as full tables.
Orders across all stores and marketplaces, incrementally synced by their last-change timestamp.
**Key Fields:**
* `id` - Internal ID of the order
* `codigo` - Order code
* `data_hora` - Date/time when the order was created
* `valor_produto` - Total product value of the order
* `valor_frete` - Shipping value
* `valor_desconto` - Discount value granted at purchase time
* `valor_total` - Total order value paid by the customer
* `pessoa_id` / `pessoa_nome` / `pessoa_cpf_cnpj` - Customer identification
* `forma_pagamento_id` / `forma_pagamento_nome` - Payment method
* `condicao_pagamento_nome` - Payment condition (cash, installments)
* `pedido_situacao` / `pedido_situacao_descricao` - Order status code and description
* `loja_id` - Store the order belongs to
* `loja_do_marketplace_id` / `loja_do_marketplace_nome` - Marketplace store, when applicable
* `synced_at` - Sync timestamp used as the incremental replication key
People and companies registered in the account, with their addresses and contacts, incrementally synced by their last-update date.
**Key Fields:**
* `id` - Internal ID of the person
* `tipo` - Person type (individual or company)
* `nome` - Name of the person
* `email` - Email address
* `cpf_cnpj` - CPF/CNPJ of the person
* `data_cadastro` - Registration date
* `data_hora_atualizacao` - Date of the last update
* `nome_fantasia` / `inscricao_estadual` - Company details, when applicable
* `pessoa_endereco` - Addresses (JSON array)
* `pessoa_contato` - Contacts such as phones and emails (JSON array)
Products in the catalog with their variations, incrementally synced by their last-update timestamp.
**Key Fields:**
* `id` - ID of the product
* `nome` - Name of the product
* `codigo` - Code of the product
* `tipo` - Type (product, service or kit)
* `marca` - ID of the product's brand
* `ativo` - Whether the product is active
* `ncm` / `cest` / `origem_fiscal` - Fiscal codes
* `peso` / `altura` / `largura` / `comprimento` - Dimensions
* `data_lancamento` - Launch date
* `categorias` - IDs of the categories (the first is the main one)
* `derivacoes` - Variations/SKUs of the product (JSON array)
Shopping carts (open, abandoned or converted), incrementally synced by their update date.
**Key Fields:**
* `id` - ID of the cart
* `status` - Status (open, abandoned or purchased)
* `data_inicio` - When the cart was created
* `data_atualizacao` - Last update to the cart and its items (replication key)
* `hash` - Unique hash of the cart
* `pedido` - Order linked to the cart, when converted (JSON)
* `draft` - Partially filled cart data (JSON)
Fiscal invoices (NF-e) with their line items, incrementally synced by emission date.
**Key Fields:**
* `id` - ID of the fiscal invoice
* `tipo` - Inbound or outbound invoice
* `situacao` - Status (typed, closed, authorized, cancelled, etc.)
* `numero` / `serie` / `chave` - Invoice number, series and access key
* `data_emissao` - Emission date/time (replication key)
* `itens` - Line items with product, quantity, values and taxes (JSON array)
Financial titles to pay, extracted as a full table.
**Key Fields:**
* `id` - ID of the financial title
* `numero` - Number of the title
* `data_abertura` / `data_geracao` / `data_vencimento` - Opening, generation and due dates
* `valor_original` / `valor_liquidado` / `saldo` - Original, settled and outstanding values
* `parcela` - Installment indicator
* `situacao` - Status of the title (JSON)
* `pessoa` - Supplier linked to the title (JSON)
* `ultimo_movimento` - Latest settlement movement (JSON)
Financial titles to receive, extracted as a full table.
**Key Fields:**
* `id` - ID of the financial title
* `numero` - Number of the title
* `data_abertura` / `data_geracao` / `data_vencimento` - Opening, generation and due dates
* `valor_original` / `valor_liquidado` / `saldo` - Original, settled and outstanding values
* `parcela` - Installment indicator
* `situacao` - Status of the title (JSON)
* `pessoa` - Customer linked to the title (JSON)
* `origem` - Origin of the title, including the linked order (JSON)
Current stock position per product variation (SKU) and warehouse, extracted as a full table.
**Key Fields:**
* `produto` - Code of the product variation (SKU)
* `deposito` - ID of the warehouse
* `quantidade_disponivel_venda` - Quantity available for sale
* `quantidade_fisica` - Physical quantity in stock
* `quantidade_reservado_saida` - Quantity reserved for outbound orders
* `quantidade_prevista_entrada` - Quantity expected to arrive
* `custo_medio` / `custo_virtual` - Average and virtual costs
Stock movement history, incrementally synced by movement date/time.
**Key Fields:**
* `movimentacao` - ID of the stock movement
* `produto` - Code of the product variation (SKU)
* `deposito` - ID of the warehouse
* `quantidade` - Quantity moved
* `valor_movimentacao` - Total value of the movement
* `data_hora_movimentacao` - Date/time of the movement (replication key)
* `tipo_operacao` - Adjustment, inbound or outbound
* `origem` - Origin of the movement
* `pedido_id` / `nota_fiscal_id` - Related order and invoice
Prices per product variation and price table, incrementally synced by update time.
**Key Fields:**
* `produto` - Code of the product variation (SKU)
* `tabela_preco` / `tabela_nome` - Price table
* `preco_venda` - Current sale price
* `preco_antigo` - Previous (crossed-out) price
* `percentual_desconto` - Discount percentage
* `data_hora_atualizacao` - Last price update (replication key)
Product categories, extracted as a full table.
**Key Fields:**
* `id` - Internal ID of the category
* `nome` - Name of the category
* `pai` - ID of the parent category
* `descricao` - Description of the category
Product brands, extracted as a full table.
**Key Fields:**
* `id` - Internal ID of the brand
* `nome` - Name of the brand
* `producao_propria` - Whether the brand is produced in-house
Stores configured in the account, extracted as a full table.
**Key Fields:**
* `id` - Internal ID of the store
* `codigo` - Code of the store
* `nome` - Name of the store
* `ativo` - Whether the store is active
* `url` - URL of the store
* `grupo` - Store group (JSON)
Shipping carriers, extracted as a full table.
**Key Fields:**
* `id` - Internal ID of the carrier
* `nome` - Name of the carrier
* `pessoa` - Carrier person registered in Magazord (JSON)
Price tables, extracted as a full table.
**Key Fields:**
* `id` - ID of the price table
* `nome` - Name of the price table
* `ativa` - Whether the price table is active
* `data_inicio` / `data_fim` - Validity period
* `opcoes` - Pricing options and rules (JSON)
Discount coupons, extracted as a full table.
**Key Fields:**
* `id` - Identifier of the coupon
* `codigo` - Code entered by the customer at checkout
* `ativo` - Whether the coupon is active
* `tipo_desconto` - Discount type (value, percentage, fixed price, etc.)
* `valor_desconto` - Discount value or percentage
* `valido_de` / `valido_ate` - Validity period
* `loja` - Store the coupon belongs to
If you have any questions about the Magazord connector, feel free to reach out through our support channels.
# Mailchimp as a data source
Source: https://docs.nekt.com/sources/mailchimp
Bring data from Mailchimp to Nekt.
## Overview
Mailchimp is an email marketing and automation platform used for audience management, email campaigns, and marketing analytics. This connector extracts audiences and their members, segments, email campaigns, campaign reports, and automation workflows from the Mailchimp Marketing API.
## Configuration
You can connect Mailchimp using either an **API key** or **OAuth**.
| Field | Description |
| - | - |
| **API Key** | Your Mailchimp API key. The data center is read automatically from its suffix (e.g. the `us9` in `abc123-us9`). Found under **Account → Extras → API keys**. |
| **Start Date** | *(Optional)* Start date for incremental data sync in ISO 8601 format (e.g., `2024-01-01T00:00:00Z`). Only applies to streams that support incremental replication. |
| **Data Center** | *(Optional)* Data center override (e.g. `us9`). Inferred from the API key suffix when omitted. |
To create an API key, go to **Account → Extras → API keys → Create A Key** in your Mailchimp account. Copy the key immediately: Mailchimp shows it only once.
Instead of an API key, you can click **Authorization** to connect via OAuth and grant access without copying a key. Use one method or the other.
Choose which streams you want to sync. Available streams are listed in the **Streams and Fields** section below.
For each selected stream, configure:
* **Layer**: Choose the destination layer (e.g., Bronze).
* **Folder**: Organize data within the layer.
* **Table name**: The table name in your lakehouse.
* **Sync Type**: Incremental (recommended) or Full Sync.
Set up the source-level configuration:
* **Description**: Add a description for this data source.
* **Triggers**: Configure sync triggers (schedule or manual).
* **Delta Log Retention Days**: Number of days to retain delta log history.
* **Full Sync**: Optionally force a full sync on first run.
After saving, verify the source appears in your **Sources** list. Run a manual sync to validate that data flows correctly.
## Streams and Fields
Audiences (lists) in the account, including contact details, signup settings, and aggregate stats. Synced as a full table.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique audience identifier |
| `web_id` | integer | Numeric ID used in the Mailchimp web app |
| `name` | string | Audience name |
| `contact` | object | Contact information shown in campaign footers |
| `permission_reminder` | string | Permission reminder shown to subscribers |
| `use_archive_bar` | boolean | Whether the archive bar is enabled |
| `campaign_defaults` | object | Default from name, email, subject, and language |
| `notify_on_subscribe` | string | Email notified on new subscribes |
| `notify_on_unsubscribe` | string | Email notified on unsubscribes |
| `date_created` | datetime | When the audience was created |
| `list_rating` | integer | Star rating of the audience |
| `email_type_option` | boolean | Whether subscribers can choose an email type |
| `subscribe_url_short` | string | Shortened signup form URL |
| `subscribe_url_long` | string | Full signup form URL |
| `beamer_address` | string | Email-to-campaign (Mailchimp Beamer) address |
| `visibility` | string | Audience visibility (`pub` or `prv`) |
| `double_optin` | boolean | Whether double opt-in is enabled |
| `has_welcome` | boolean | Whether a welcome email is configured |
| `marketing_permissions` | boolean | Whether GDPR marketing permissions are enabled |
| `modules` | array | Enabled audience modules |
| `stats` | object | Aggregate stats (member counts, open/click rates, last activity) |
Members (contacts) of each audience. Child of `lists`. Supports incremental sync via `last_changed`.
| Field | Type | Description |
| - | - | - |
| `id` | string | MD5 hash of the lowercased email, used as the member identifier |
| `email_address` | string | Member email address |
| `unique_email_id` | string | Unique email ID across the account |
| `contact_id` | string | Contact identifier |
| `full_name` | string | Member full name |
| `web_id` | integer | Numeric ID used in the Mailchimp web app |
| `email_type` | string | Preferred email type (`html` or `text`) |
| `status` | string | Subscription status (e.g. `subscribed`, `unsubscribed`, `cleaned`) |
| `unsubscribe_reason` | string | Reason for unsubscribing, if provided |
| `consents_to_one_to_one_messaging` | boolean | Whether the member consents to 1:1 messaging |
| `merge_fields` | string | Audience-specific merge fields (JSON; keys vary per audience) |
| `interests` | string | Interest group memberships (JSON; keys vary per audience) |
| `stats` | object | Member open/click rates and ecommerce data |
| `ip_signup` | string | IP address at signup |
| `timestamp_signup` | datetime | Signup timestamp |
| `ip_opt` | string | IP address at opt-in |
| `timestamp_opt` | datetime | Opt-in timestamp |
| `member_rating` | integer | Member engagement rating |
| `last_changed` | datetime | When the member record was last changed |
| `language` | string | Member language |
| `vip` | boolean | Whether the member is marked VIP |
| `email_client` | string | Last known email client |
| `location` | object | Geolocation (lat/long, country, timezone) |
| `marketing_permissions` | array | GDPR marketing permission states |
| `last_note` | object | Most recent note on the member |
| `source` | string | Signup source |
| `tags_count` | integer | Number of tags applied |
| `tags` | array | Tags applied to the member |
| `list_id` | string | Parent audience identifier |
Segments within each audience. Child of `lists`. Supports incremental sync via `updated_at`.
| Field | Type | Description |
| - | - | - |
| `id` | integer | Unique segment identifier (numeric) |
| `name` | string | Segment name |
| `member_count` | integer | Number of members in the segment |
| `type` | string | Segment type (e.g. `saved`, `static`, `fuzzy`) |
| `created_at` | datetime | When the segment was created |
| `updated_at` | datetime | When the segment was last updated |
| `options` | string | Match logic and conditions (JSON; structure varies by segment) |
| `list_id` | string | Parent audience identifier |
Email campaigns including recipients, settings, tracking, and a report summary. Supports incremental sync via `create_time`.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique campaign identifier |
| `web_id` | integer | Numeric ID used in the Mailchimp web app |
| `parent_campaign_id` | string | Parent campaign ID (for variations) |
| `type` | string | Campaign type (e.g. `regular`, `automation`, `rss`) |
| `create_time` | datetime | When the campaign was created |
| `archive_url` | string | Archive URL |
| `long_archive_url` | string | Full archive URL |
| `status` | string | Campaign status (e.g. `save`, `schedule`, `sent`) |
| `emails_sent` | integer | Number of emails sent |
| `send_time` | datetime | When the campaign was sent |
| `content_type` | string | Content type |
| `needs_block_refresh` | boolean | Whether content blocks need refreshing |
| `resendable` | boolean | Whether the campaign can be resent |
| `recipients` | object | Target audience and segment details |
| `settings` | object | Subject, from name, template, and footer settings |
| `tracking` | object | Open/click/ecommerce tracking settings |
| `report_summary` | object | Aggregate performance summary (opens, clicks, ecommerce) |
Per-campaign aggregate reports. The `id` matches the campaign id. Supports incremental sync via `send_time`.
| Field | Type | Description |
| - | - | - |
| `id` | string | Report (campaign) identifier |
| `campaign_title` | string | Campaign title |
| `type` | string | Campaign type |
| `list_id` | string | Audience identifier |
| `list_is_active` | boolean | Whether the audience is active |
| `list_name` | string | Audience name |
| `subject_line` | string | Email subject line |
| `preview_text` | string | Email preview text |
| `emails_sent` | integer | Number of emails sent |
| `abuse_reports` | integer | Number of abuse reports |
| `unsubscribed` | integer | Number of unsubscribes |
| `send_time` | datetime | When the campaign was sent |
| `rss_last_send` | datetime | Last RSS send time |
| `bounces` | object | Hard/soft bounces and syntax errors |
| `forwards` | object | Forward counts and opens |
| `opens` | object | Open totals, unique opens, open rate, last open |
| `clicks` | object | Click totals, unique clicks, click rate, last click |
| `facebook_likes` | object | Facebook like counts |
| `list_stats` | object | Audience sub/unsub/open/click rates |
Classic automation workflows including recipients, settings, and a report summary. Supports incremental sync via `create_time`.
| Field | Type | Description |
| - | - | - |
| `id` | string | Unique automation (workflow) identifier |
| `create_time` | datetime | When the automation was created |
| `start_time` | datetime | When the automation started |
| `status` | string | Automation status (e.g. `save`, `sending`, `paused`) |
| `emails_sent` | integer | Number of emails sent |
| `recipients` | object | Target audience details |
| `settings` | object | Title, from name, and footer settings |
| `tracking` | object | Open/click/ecommerce tracking settings |
| `trigger_settings` | string | Workflow trigger configuration (JSON) |
| `report_summary` | object | Aggregate performance summary (opens, clicks) |
## Implementation Notes
* **Authentication**: HTTP Basic Auth with the API key (any username, API key as password), or OAuth via the **Authorization** button. The data center (e.g. `us9`) is resolved from the API key suffix or the OAuth metadata endpoint.
* **Pagination**: The connector paginates with `count` and `offset`, fetching all pages up to each endpoint's total.
* **Incremental sync**: `list_members`, `segments`, `campaigns`, `reports`, and `automations` sync incrementally using Mailchimp's `since_*` date filters. `lists` is full-table only, since Mailchimp exposes only a creation date for audiences.
* **JSON fields**: Fields with audience-specific or variable structure (`merge_fields`, `interests` for members, `options` for segments, and `trigger_settings` for automations) are serialized as JSON strings. Parse them downstream as needed.
* **Child streams**: `list_members` and `segments` are synced per audience, using the parent `lists` stream to provide each `list_id`.
# Mais Controle as a data source
Source: https://docs.nekt.com/sources/mais-controle
Bring data from Mais Controle ERP to Nekt.
Mais Controle is a Brazilian ERP for the construction industry, covering budgets, schedules, projects (obras), billing, payments, and financial management. This connector extracts your operational data through the Mais Controle **API para Relatórios** (Reports API), landing projects, financial entries, payments, billing, receipts, schedules, budget items, and proposals in your Catalog.
## Configuring Mais Controle as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Mais Controle option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Mais Controle uses a dedicated API credential (HTTP Basic authentication). You first need to enable the Reports API inside Mais Controle, then copy the generated credential into Nekt.
**Enable the API in Mais Controle** (a key user / *usuário-chave* must do this):
1. Log in to Mais Controle and go to **Configurações → Integração**.
2. On **API para Relatórios**, click **Configurar**, then **Ativar API para Relatórios** (the free tier updates data once a week).
3. The screen shows an **API login (e-mail)** and an **API password ("Senha da API")**. Use the copy button to grab the full password: it is shown only once.
The **API Password** is the "Senha da API" generated on that activation screen. It is **not** your Mais Controle account/login password. Copy it with the copy button (the displayed value is truncated) right after activation, since it is shown only once.
Back in Nekt, provide:
* **API Login (e-mail)**: the API login shown on the activation screen.
* **API Password**: the "Senha da API" copied from that same screen.
Optionally, under advanced settings you can define:
* **CSV Delimiter**: override the delimiter used to parse the API's CSV responses. Leave empty to auto-detect (`;` or `,`).
* **Request Timeout (seconds)**: HTTP timeout for each report download (default `300`).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
Some routines depend on the modules your account has contracted. If a module is not enabled (for example, Medições), that stream's report returns an error and the connector skips it, so it produces no rows.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: every stream is extracted as **FULL\_TABLE**; each run fetches the current full report for the routine.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
On the **free** Reports API tier, Mais Controle refreshes the exported data **once a week** and limits how many requests you can make per period (exceeding it returns HTTP 429). Schedule this source no more frequently than your contracted update cadence (weekly, daily, or hourly) to avoid quota errors and redundant runs.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Mais Controle and their corresponding fields. Each report is delivered as a CSV export and synced as **FULL\_TABLE** (no primary key or incremental filter).
All fields are extracted as **strings**. Mais Controle returns Brazilian-formatted values: decimals use a comma (`1.234,56`) and dates use `dd/mm/yyyy`, neither of which are valid numeric/date types. Cast them downstream (for example in a Query) when you need typed values.
Construction projects registry (*Cadastro de Obras*). **Sync:** Full table.
* `id` - Project (obra) ID
* `nome` - Project name
* `status_obra` - Project status
* `codigo_obra` - Project code
* `tipo_obra` - Project type
* `responsavel_obra` - Project owner
* `responsavel_tecnico` - Technical lead
* `area_total` - Total area
* `unidade_area_total` - Total area unit
* `cliente` - Client
Financial entries (*Lançamentos*). **Sync:** Full table.
* `id_lancamento` - Entry ID
* `id_obra` - Related project (obra) ID
* `centro_de_custo` - Cost center
* `fornecedor` - Supplier
* `descricao` - Description
* `numero_documento` - Document number
* `data_competencia` - Accrual date
* `valor_total_lancamento` - Total entry amount
* `conta_bancaria` - Bank account
* `categoria` - Category
* `condicao_pagamento` - Payment terms
* `numero_parcela` - Installment number
* `valor_parcela` - Installment amount
* `data_vencimento` - Due date
* `indice_item_orcamento` - Linked budget item index
* `possui_vinculo_estoque` - Whether it is linked to inventory
* `ordem_de_compra` - Purchase order
* `data_lancamento` - Entry date
Payments made against payables (*Pagamentos*). **Sync:** Full table.
* `id_lancamento` - Related entry ID
* `data_competencia` - Accrual date
* `data_vencimento` - Due date
* `data_pagamento` - Payment date
* `numero_parcela` - Installment number
* `valor_parcela` - Installment amount
* `valor_pago` - Amount paid
* `valor_desconto` - Discount amount
* `valor_juros_e_multa` - Interest and penalty amount
* `fornecedor` - Supplier
* `descricao` - Description
* `numero_documento` - Document number
* `categoria` - Category
* `grupo` - Group
* `plano_de_conta` - Chart-of-accounts entry
* `condicao_pagamento` - Payment terms
* `forma_pagamento` - Payment method
* `quem_paga` - Who pays
* `conta_bancaria` - Bank account
* `id_obra` - Project (obra) ID
* `centro_de_custo` - Cost center
* `indice_item_orcamento` - Linked budget item index
* `ordem_de_compra` - Purchase order
Billing (*Faturamentos*). **Sync:** Full table.
* `id_faturamento` - Billing ID
* `id_obra` - Project (obra) ID
* `centro_de_custo` - Cost center
* `cliente` - Client
* `descricao` - Description
* `data_competencia` - Accrual date
* `valor_bruto` - Gross amount
* `impostos` - Taxes
* `valor_liquido` - Net amount
* `numero_parcela` - Installment number
* `valor_parcela` - Installment amount
* `data_vencimento` - Due date
* `conta_bancaria` - Bank account
* `natureza` - Nature
* `condicao_recebimento` - Receipt terms
* `numero_documento` - Document number
* `vendedor` - Salesperson
* `calculo_reajuste_acumulado` - Accumulated adjustment calculation
* `tipo` - Type
* `valor` - Amount
* `tabela_reajuste` - Adjustment table
* `reajuste` - Adjustment
* `reajuste_acumulado` - Accumulated adjustment
* `data_faturamento` - Billing date
Receipts against billing (*Recebimentos*). **Sync:** Full table.
* `id_faturamento` - Related billing ID
* `data_competencia` - Accrual date
* `data_vencimento` - Due date
* `data_recebimento` - Receipt date
* `numero_parcela` - Installment number
* `valor_parcela` - Installment amount
* `valor_recebido` - Amount received
* `valor_desconto` - Discount amount
* `valor_juros_e_multa` - Interest and penalty amount
* `id_obra` - Project (obra) ID
* `centro_de_custo` - Cost center
* `cliente` - Client
* `descricao` - Description
* `numero_documento` - Document number
* `condicao_recebimento` - Receipt terms
* `conta_bancaria` - Bank account
* `forma_recebimento` - Receipt method
* `natureza` - Nature
Project schedules (*Cronogramas*). **Sync:** Full table.
* `id_obra` - Project (obra) ID
* `obra` - Project name
* `indice` - Item index
* `etapa_item` - Stage / item
* `descricao` - Description
* `dias_uteis_desconsiderar` - Business days to disregard
* `dias_uteis_planejado` - Planned business days
* `dias_corridos_planejado` - Planned calendar days
* `data_inicio_planejado` - Planned start date
* `data_fim_planejado` - Planned end date
* `dias_uteis_realizado` - Actual business days
* `dias_corridos_realizado` - Actual calendar days
* `data_inicio_realizado` - Actual start date
* `data_fim_realizado` - Actual end date
Budget items (*Orçamentos*). **Sync:** Full table.
* `id_obra` - Project (obra) ID
* `obra` - Project name
* `indice` - Item index
* `etapa_item` - Stage / item
* `codigo_item` - Item code
* `base` - Reference base (e.g. SINAPI)
* `tipo` - Type
* `descricao` - Description
* `unid` - Unit
* `qtde` - Quantity
* `custo_unitario_mao_de_obra` - Unit labor cost
* `custo_unitario_material` - Unit material cost
* `custo_unitario_equipamento` - Unit equipment cost
* `custo_unitario_outros` - Unit other cost
* `custo_unitario_item` - Unit item cost
* `custo_total_mao_de_obra` - Total labor cost
* `custo_total_material` - Total material cost
* `custo_total_equipamento` - Total equipment cost
* `custo_total_outros` - Total other cost
* `custo_total_item` - Total item cost
* `bdi_item` - Item BDI (markup)
* `preco_unitario_mao_de_obra` - Unit labor price
* `preco_unitario_material` - Unit material price
* `preco_unitario_equipamento` - Unit equipment price
* `preco_unitario_outros` - Unit other price
* `preco_unitario_item` - Unit item price
* `preco_total_mao_de_obra` - Total labor price
* `preco_total_material` - Total material price
* `preco_total_equipamento` - Total equipment price
* `preco_total_outros` - Total other price
* `preco_total_item` - Total item price
Proposals (*Propostas*). **Sync:** Full table.
* `id_obra` - Project (obra) ID
* `obra` - Project name
* `cod_orcamento` - Budget code
* `cliente` - Client
* `status_proposta` - Proposal status
* `data_criacao` - Creation date
* `data_entrega` - Delivery date
* `data_venda` - Sale date
* `responsavel` - Owner
* `cond_pgto` - Payment terms
* `desconto` - Discount
* `preco_total_com_desconto` - Total price with discount
Measurements (*Medições*). **Sync:** Full table.
Requires the **Medições** module. If your account has not contracted it, the report returns an error and the stream is skipped. When available, the fields are read directly from the report's CSV header.
# Marisa as a data source
Source: https://docs.nekt.com/sources/marisa
Bring data from the Marisa marketplace Seller Center to Nekt.
Marisa is one of Brazil's largest fashion retail chains, and its marketplace lets third-party
sellers list and sell products on marisa.com.br. The Seller Center runs on the Omnik
marketplace platform and exposes the seller's orders, catalogue, reference data and
commission settlements through a REST API.
## Configuring Marisa as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Marisa option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need an API credential created inside the Marisa Seller Center. Sign in to the
Seller Center with your seller account and go to **Credential Maintenance**. Creating a
credential asks for your Seller Center password and
returns two values, both required below.
The following configurations are available:
* **API Token**: the token issued in Credential Maintenance. It is sent on every request and is stored encrypted.
* **Application ID**: the identifier shown next to the token. The API rejects a token sent without it.
* **Seller Tenant ID**: only needed when the credential belongs to a marketplace operator acting on a seller's behalf. A seller's own credential already resolves to its tenant, so leave this empty in the usual case.
* **Start Date**: the earliest record update date to sync. It applies to the streams that support incremental replication.
* **Marketplace IDs**: the marketplaces whose category tree should be synced. Used only by the `marketplace_categories` stream; leave it empty and that stream is skipped while every other stream runs normally.
* **Environment**: (Default: `production`) the Seller Center environment to read from. Change it only if your account was explicitly given access to a test environment.
* **Lookback Window**: (Default: 0 days) the number of days to re-read before the last synced update date, to pick up records the marketplace updated late.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Good to know
* **Rate limit.** The Seller Center API accepts 100 requests per minute per credential. Nekt stays inside that budget and backs off automatically when the marketplace answers with a rate-limit error, so a large first extraction simply takes longer rather than failing.
* **Page sizes are set by the marketplace.** Orders are returned 50 at a time and every other endpoint 10 at a time. A first full extraction of a large catalogue is therefore slower than the order history of the same account.
* **`Order Financial Details` costs one request per order.** It is the only stream that queries the marketplace once per order, so selecting it makes the run noticeably longer on accounts with a high order volume. Select it when you need commission and settlement data; leave it out otherwise.
* **Nested data is stored as JSON text.** Fields such as `customer_data`, `payment_data` or `images` hold structured content and are stored as JSON strings. Read them in SQL with `JSON_VALUE(customer_data, '$.name')` rather than `customer_data.name`.
* **Streams your credential cannot reach are skipped.** If the credential lacks permission for an endpoint, that stream is skipped with a message in the run log and the rest of the extraction completes normally.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Orders | `orders` | Orders placed with the seller, with customer, invoice, delivery, payment and commission data. |
| Order Items | `order_items` | One row per item line of each order, with price, quantity and line-level freight. |
| Order Financial Details | `order_financial_details` | Commission, promotional subsidies and settlement entries for each order. One request per order. |
| Products | `products` | Products the seller publishes, with dimensions, tax data and category assignment. |
| Product SKUs | `product_skus` | Variants of each product, with price, stock, images and marketplace listing status. |
| Brands | `brands` | Brands registered by the seller. |
| Marketplace Brands | `marketplace_brands` | Brands the marketplace recognises for the seller's listings. |
| Variants | `variants` | Variant axes such as colour or size, with their allowed values. |
| Marketplace Categories | `marketplace_categories` | Marketplace category tree and the attributes each category requires. Requires **Marketplace IDs**. |
| Financial Entries | `financial_entries` | Manual financial entries recorded against the seller's settlement cycle. |
## Fields by stream
Below you'll find all available data streams from Marisa and their corresponding fields:
Orders placed with the seller on the marketplace. Synced incrementally by `last_update`.
**Key Fields:**
* `market_place_id` - Identifier the marketplace assigned to the order (primary key)
* `id` - Identifier of the order in the seller's own numbering
* `status` - Current stage of the order (approved, invoiced, sent, delivered, cancelled)
* `last_update` - Timestamp of the last change, used as the incremental replication key
* `create_date` - Timestamp the order was first recorded in the Seller Center
* `last_event` - Name of the most recent event applied to the order
**Dates and Channel:**
* `order_date` - Timestamp the customer placed the order
* `approval_date` - Timestamp the payment was approved
* `not_approval_date` - Timestamp the payment was refused, when it was
* `cancel_date` - Timestamp the order was cancelled, when it was
* `market_place_id_order` - Order identifier as shown to the customer
* `shipping_id` - Identifier of the shipment the order was dispatched under
* `sales_channel` - Sales channel the order came through
* `place_of_sale` - Point of sale credited with the order
* `exclude_from_seller_index` - Whether the order is left out of the seller's performance index
**Order Values:**
* `order_values_items` - Number of item lines in the order
* `order_values_value` - Total value of the items before discounts and interest
* `order_values_discount` - Total discount applied
* `order_values_interest` - Total interest charged
* `order_values_net_value` - Value after discounts and interest
* `order_values_gross_value` - Value including freight and every charge
**Commission:**
* `commission_percentage` - Percentage charged on the order's products
* `commission_value` - Amount charged on the order's products
* `freight_commission_percentage` - Percentage charged on the order's freight
* `freight_commission_value` - Amount charged on the order's freight
* `is_external_calculation` - Whether the commission was calculated outside the Seller Center
* `paid_with_bonus` - Whether the commission was settled with bonus credit
**Account:**
* `tenant` - Identifier of the seller account that owns the order
* `operator` - Identifier of the marketplace operator (the hub)
**JSON Fields:**
* `marketplace_data` - Marketplace name, marketplace order ID, site ID and active flag
* `customer_data` - Buyer name, document, e-mail, address and phone numbers
* `invoice_data` - Invoice access key, number, series, CFOP, SEFAZ status and XML/DANFE links
* `delivery_data` - Carrier, tracking code and URL, shipping method, dates, address and exceptions
* `shipping_data` - Dispatch and expiry dates and the shipping label links
* `freight_data` - Freight type, charged value, transit time, modal and take rates
* `platform_data` - Platform that originated the order
* `payment_data` - Payer, billing address and each payment method with its transaction identifiers
* `cancellations_data` - History of cancellations applied to the order
* `devolutions_data` - History of returns, with reason, returned items and reverse logistics
* `receiving_period` - Delivery window agreed with the buyer
* `campaigns`, `integrators`, `postage_data`, `credit_information_data`, `price_monitoring_data`, `marketing` - Additional data the marketplace attaches to the order
One row per item line of an order. The lines come from the order payload, so this stream
adds no extra requests to the marketplace.
**Key Fields:**
* `market_place_id` - Marketplace identifier of the order (part of the primary key)
* `item_index` - Position of the line inside the order, starting at 0 (part of the primary key)
* `order_id` - Identifier of the order in the seller's own numbering
* `order_last_update` - Timestamp of the last change to the parent order
* `tenant` - Identifier of the seller account
**SKU:**
* `id` - Identifier of the SKU sold on this line
* `sku` - Seller's SKU code
* `sku_name` - Name of the SKU as published
* `sku_original` - Original SKU code, when the marketplace remapped it
* `gtin` - Global trade item number (EAN/GTIN)
* `model` - Manufacturer model
* `product_type` - Product type assigned to the item
* `product_condition` - Condition of the item sold
* `active` - Whether the item was active when the order was placed
* `crossdocking_days` - Days the seller needs to hand the item to the carrier
* `supplier_code`, `erp_code`, `establishment_code`, `platform_id`, `ref_id` - External codes for the item
* `moderation_status` - Marketplace moderation status
* `moderation_date` - Timestamp of the last moderation decision
* `description`, `description_html` - Descriptions of the item
* `measurement_unit` - Unit the item is sold in
* `unit_multiplier` - How many base units one sold unit represents
**Price and Quantity:**
* `unit_price` - Price of a single unit before discounts
* `discount_unit` - Discount applied to a single unit
* `net_price` - Line value after discounts
* `gross_price` - Line value including every charge
* `quantity` - Units sold on this line
* `returned_quantity` - Units of this line that were returned
**Stock:**
* `stock_stock` - Stock balance recorded when the order was placed
* `stock_min_stock` - Minimum stock configured for the SKU
* `stock_move` - Whether the order moved the SKU's stock balance
**JSON Fields:**
* `freight_data` - Freight attributed to this line, with type, charged value and take rates
* `product_data` - Product the SKU belongs to, when the marketplace sends it
* `vendors`, `campaigns`, `operator_campaigns`, `monitored_price_data` - Additional data attached to the line
Commission, promotional subsidies and settlement entries for each order. This stream
queries the marketplace **once per order**, so it makes the run longer on accounts with a
high order volume.
**Key Fields:**
* `market_place_id` - Marketplace identifier of the order (primary key)
* `order_id` - Identifier of the order in the seller's own numbering
**Commission:**
* `itens_commission_percent` - Commission percentage applied to the products
* `itens_commission_value` - Commission amount charged on the products
* `freight_commission_percent` - Commission percentage applied to the freight
* `freight_commission_value` - Commission amount charged on the freight
* `total_commission_value` - Total commission charged on the order
**Settlement:**
* `total_transfer_seller` - Amount to be transferred to the seller for this order
* `financial_cicle_type` - Settlement cycle the order was assigned to
* `financial_transfer_days_after_end_cicle` - Days after the cycle closes before the transfer is released
**JSON Fields:**
* `commission_origins` - Per-SKU commission breakdown with product and freight percentages and values
* `discounts` - Promotions applied and how their subsidy was split between marketplace and seller
* `financial_statements` - Ledger entries generated by the order, with type, status, value, release date and instalment position
Products the seller publishes to the marketplace. Synced incrementally by `last_update`.
Stock is always requested fresh, so the SKU rows match the Seller Center.
**Key Fields:**
* `id` - Unique identifier of the product (primary key)
* `product_name` - Commercial name of the product
* `active` - Whether the product is currently active
* `last_update` - Timestamp of the last change, used as the incremental replication key
* `create_date` - Timestamp the product was created
**Description:**
* `description` - Plain-text description
* `description_html` - HTML description as published
* `brand` - Brand the product belongs to
* `tags` - Tags attached to the product
* `warranty` - Warranty period offered, in months
* `variant` - Whether the product is sold in variants (more than one SKU)
* `unit` - Unit the product is sold in
* `unit_initials` - Abbreviation of that unit
**Dimensions:**
* `product_width`, `product_height`, `product_depth`, `product_gross_weight` - Dimensions of the product itself
* `package_width`, `package_height`, `package_depth`, `package_gross_weight` - Dimensions of the shipping package
**Category and Account:**
* `category_id`, `category_code`, `category_name` - The seller's own category for the product
* `tenant` - Identifier of the seller account
* `operator` - Identifier of the marketplace operator (the hub)
* `account` - Identifier of the seller's account the product belongs to
**JSON Fields:**
* `tax_data` - ICMS origin and the NCM code the product is declared under
* `codes` - Additional codes registered for the product
* `categories` - Marketplace category paths, up to eight levels per channel
* `attributes` - Product specifications, with whether the marketplace requires each one
* `marketplaces` - Marketplaces the product is published to, with commission and active flag
Variants of each product, with price and stock. The SKUs come from the product payload,
so this stream adds no extra requests.
**Key Fields:**
* `product_id` - Identifier of the parent product (part of the primary key)
* `id` - Unique identifier of the SKU (part of the primary key)
* `sku` - Seller's own SKU code
* `sku_name` - Name of the SKU as published
* `active` - Whether the SKU is currently active
* `product_last_update` - Timestamp of the last change to the parent product
* `tenant` - Identifier of the seller account
**Price and Stock:**
* `from_price` - Reference (struck-through) price shown for the SKU
* `price` - Selling price
* `cost_price` - Cost price registered by the seller
* `stock` - Current stock balance
* `min_stock` - Minimum stock configured
**Identification:**
* `gtin` - Global trade item number (EAN/GTIN)
* `model` - Manufacturer model
* `product_type` - Product type assigned to the SKU
* `product_condition` - Condition the SKU is sold in
* `crossdocking_days` - Days the seller needs to hand the SKU to the carrier
* `supplier_code`, `erp_code`, `establishment_code` - External codes for the SKU
* `moderation_status` - Marketplace moderation status of the listing
* `moderation_date` - Timestamp of the last moderation decision
* `description`, `description_html` - Descriptions of the SKU
**Package Dimensions:**
* `package_width`, `package_height`, `package_depth`, `package_gross_weight` - Dimensions of the shipping package
**JSON Fields:**
* `codes` - Additional codes registered for the SKU
* `errors` - Errors the marketplace returned for this SKU's listing
* `images` - Images in each rendered size, with the main-image flag and display order
* `attributes` - Specifications specific to the SKU
* `marketplaces` - Marketplaces the SKU is published to, with commission and active flag
Brands registered by the seller. Synced incrementally by `last_update`.
**Key Fields:**
* `id` - Unique identifier of the brand (primary key)
* `code` - Code registered for the brand
* `name` - Display name of the brand
* `tenant` - Identifier of the seller account
* `operator` - Identifier of the marketplace operator (the hub)
* `create_date` - Timestamp the brand was created
* `last_update` - Timestamp of the last change, used as the incremental replication key
Brands the marketplace recognises for the seller's listings. Same fields as `brands`.
**Key Fields:**
* `id` - Unique identifier of the brand (primary key)
* `code` - Code registered for the brand
* `name` - Display name of the brand
* `tenant` - Identifier of the seller account
* `operator` - Identifier of the marketplace operator (the hub)
* `create_date` - Timestamp the brand was created
* `last_update` - Timestamp of the last change, used as the incremental replication key
Variant axes such as colour or size. Synced incrementally by `last_update`.
**Key Fields:**
* `id` - Unique identifier of the variant axis (primary key)
* `code` - Code registered for the axis
* `name` - Display name of the axis, such as Colour or Size
* `tenant` - Identifier of the seller account
* `operator` - Identifier of the marketplace operator (the hub)
* `create_date` - Timestamp the axis was created
* `last_update` - Timestamp of the last change, used as the incremental replication key
**JSON Fields:**
* `values` - Allowed values of the axis, each with its code, name and nickname
The marketplace's category tree and the attributes each category requires. This stream
only runs when **Marketplace IDs** is filled in; otherwise it is skipped with a note in
the run log.
**Key Fields:**
* `marketplace` - Marketplace whose tree the row belongs to (part of the primary key)
* `id` - Unique identifier of the category (part of the primary key)
* `channel` - Sales channel the category belongs to
* `ean_required` - Whether the marketplace requires an EAN for products in this category
* `last_update` - Timestamp of the last change, used as the incremental replication key
* `create_date` - Timestamp the category was created
* `tenant` - Identifier of the seller account the category was read for
* `operator` - Identifier of the marketplace operator (the hub)
**Category Path:**
* `category_channel` - Channel recorded inside the category path
* `category_tag` - Tag the marketplace uses for the path
* `category_id1` … `category_id8` - Identifier of each level of the path
* `category_name1` … `category_name8` - Name of each level of the path
**JSON Fields:**
* `attributes` - Attributes the marketplace defines for the category, with whether each is required and the accepted values
Manual financial entries recorded against the seller's settlement cycle. The endpoint
accepts no filters, so this stream is always a full table.
**Key Fields:**
* `id` - Unique identifier of the entry (primary key)
* `type` - Type of the entry, such as a sale or an adjustment
* `status` - Current status of the entry
* `value` - Amount of the entry
* `description` - Description recorded with the entry
**Cycle and Payment:**
* `type_of_cycle` - Type of settlement cycle the entry belongs to
* `financial_cycle_status` - Status of that cycle
* `beginning_of_cycle` - Event that opens the cycle
* `end_of_cycle` - Event that closes the cycle
* `payment_status` - Payment status of the entry
* `payment_error` - Reason the payment failed, when it did
* `transfer_date` - Date the amount is transferred to the seller
* `release_date` - Date the amount is released for transfer
* `statements` - Statement the entry was consolidated into
* `statement_id` - Identifier of that statement
**References:**
* `order_id` - Order the entry relates to, in the seller's own numbering
* `marketplace_id` - Marketplace identifier of that order
* `tenant` - Identifier of the seller account
* `seller_tenant` - Identifier of the seller the entry settles
* `operator` - Identifier of the marketplace operator (the hub)
* `operator_name` - Name of that operator
* `application_id` - Identifier of the API credential that created the entry
* `create_date` - Timestamp the entry was created
* `last_update` - Timestamp of the last change to the entry
**JSON Fields:**
* `typed_label` - Classification of the entry, with type identifier, label and description
# Martz as a data source
Source: https://docs.nekt.com/sources/martz
Bring data from Martz to Nekt.
Martz is a Brazilian retention CRM for e-commerce. It ingests customers, orders, products and abandoned carts from your store's own platform (Nuvemshop, Shopify, Tray, VTEX, Bling, Omie and others), segments the customer base into groups, and runs WhatsApp, e-mail and SMS campaigns against it. This connector brings what Martz has already ingested and computed into your Nekt Catalog, including the message-by-message log of every campaign and the coupons those campaigns handed out.
## Before you start
You need a **public API key** for your Martz store. You generate it yourself, in the Martz panel:
In [painel.martz.com.br](https://painel.martz.com.br/settings/api-keys), go to **Configurações » Chaves de Acesso**.
Look at the **Chave da API pública** box. If a key is already there, copy it.
If the box says *Nenhuma chave criada*, click **gerar nova chave**.
Generating a new key immediately stops the previous one working. If other integrations already use your Martz API key, only regenerate it if you are ready to update them all at the same time.
The key does not expire and is tied to one store, so one Nekt source reads one Martz store.
## Configuring Martz as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button in the top-right corner, then select Martz from the list of connectors.
Click **Next** and add your access details.
### 1. Add account access
The following configuration fields are available:
* **API Key**: The public API key you copied from the Martz panel. It is sent as the `X-API-KEY` header on every request.
* **Initial sync date** (advanced): The earliest scheduled send date read by the **Campaign activities** table on the first run. Later runs continue from where the previous one stopped. Leave it empty to read your full history. It does not affect the other tables; see [Sync types](#sync-types) below.
* **Lookback window (days)** (advanced): How many days before the last sync the Campaign activities table re-reads on every run. Defaults to 1.
* **Page size** (advanced): Records requested per page. Martz caps this at 100, which is also the default and the fastest option.
* **Requests per minute** (advanced): Maximum number of API calls per minute this source makes. Defaults to 120.
* **API URL** (advanced): Overrides the Martz API address. Only change it if Martz told you to use a different host.
After configuring access, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the ones relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how data should appear in your Catalog, including layer, table naming, and sync type.
* **Layer**: Select the destination layer where extracted tables will be created.
* **Folder**: A folder can be created inside the selected layer to group all tables from this source.
* **Table name**: Use the suggested name or customize it. You can also apply a prefix to all tables at once.
* **Sync Type**: Choose between `INCREMENTAL` and `FULL_TABLE`. See [Sync types](#sync-types); only **Campaign activities** supports `INCREMENTAL`.
When you're done, click **Next**.
### 4. Configure data source
Add a clear description for this source so your team can identify it easily, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), choose how often data should be extracted (for example, daily, weekly, or at fixed times).
Optionally, configure additional settings:
* Delta Log Retention, which controls how long old table states are kept. Learn more [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync), to periodically complement incremental runs with a complete refresh.
When you are ready, click **Next** to finalize setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, trigger a run manually from the source actions.
To see data in the [Catalog](https://app.nekt.ai/catalog), at least one source run must complete successfully.
## Sync types
**Campaign activities** is the only table that can sync incrementally, and it is also the largest: it holds one row per message sent to one customer. Its endpoint is the only one in the Martz API that accepts a date filter, so every run reads only what was scheduled since the last one, plus the lookback window.
Every other table is read in full on every run. The Martz API publishes no "modified since" filter on any other endpoint, so there is no reliable way to ask it for only what changed. Reading them in full is what guarantees the Catalog matches Martz; a full read also self-corrects, so a table never drifts.
In practice this matters most for **Customers** and **Orders**, the two tables that grow with the store. If they are large, prefer a daily trigger over an hourly one.
## Good to know
* **Money is in cents.** `subtotal`, `freight`, `discount`, `total`, `price` and the coupon values are whole numbers in the smallest unit of your currency (`19000` is R\$ 190,00). Divide by 100 in a [Query](https://docs.nekt.com/queries/overview) when you present them.
* **Two views of an order's status.** `status` is Martz's normalised status; `raw_status` is the exact wording your own e-commerce platform used. Use `raw_status` when a report has to agree with your back office.
* **Every commerce record keeps its origin.** `integration_name` and `integration_id` carry the platform the record came from and its identifier there, so you can join Martz data back to your store's own exports.
* **Order lines and cart lines are JSON text.** `orders.products` and `abandoned_carts.products` hold the line items as a JSON array. Parse them in a Query or Notebook when you need one row per item.
* **Leads and buyers share one table.** Martz calls a contact with no paid order a *lead*. Both are in **Customers**; `order_amount` equal to zero is what marks a lead.
* **Deleted coupons are still returned.** Rows in **Coupons** carrying a `deleted_at` value are ones Martz removed. Filter them out if your report should ignore them.
* **Campaign activities can be scheduled into the future.** `scheduled_to` is when a message is due to go out, so rows with dates ahead of today are normal.
* **A table your plan does not include is skipped, not failed.** If Martz refuses one of the endpoints for your store, that table is left untouched and the run reports it; every other table still syncs.
* **One key, one store.** The key is issued per store, so syncing two Martz stores means two sources in Nekt.
# Streams and Fields
## Available streams
| Stream | Slug | Description |
| - | - | - |
| Store | `store` | The Martz account itself: name, site, category and connected platforms. One row. |
| Customers | `customers` | Every customer and lead Martz holds, with lifetime order totals and average ticket. |
| Orders | `orders` | Orders imported from your e-commerce platform, with their line items and customer. |
| Products | `products` | Products imported from your e-commerce platform. |
| Product categories | `product_categories` | Category tree imported from your e-commerce platform. |
| Abandoned carts | `abandoned_carts` | Carts filled in the store and never checked out, with their line items. |
| Campaigns | `campaigns` | Retention campaigns configured in Martz, with status and period. |
| Campaign activities | `campaign_activities` | One row per message a campaign scheduled for one customer, with its send status. |
| Coupons (bonus) | `bonus` | Discount coupons Martz generated, and the orders that earned and spent them. |
| Customer groups | `customer_clusters` | Segments the store targets campaigns at, drawn by hand or maintained by Martz. |
| Customer tags | `customer_tags` | Labels the store can attach to its customers. |
| Custom attributes | `resource_custom_attributes` | Extra fields the store defined on its customers or orders. |
| Popups | `popups` | Popups and newsletter forms shown on the store's site. |
| Popup sign-ups | `popup_signins` | One row per visitor who submitted a popup form. |
| Collaborators | `collaborators` | People with access to the store's Martz account, and their roles. |
| Dashboards | `dashboards` | Dashboards saved in the Martz panel. |
| WhatsApp templates | `whatsapp_templates` | Message templates submitted to Meta, with their approval status. |
| WhatsApp Business numbers | `whatsapp_business_numbers` | WhatsApp Business numbers connected through the app. |
| WhatsApp official senders | `whatsapp_official_numbers` | Numbers registered with Meta's official WhatsApp Business API. |
## Fields by stream
Below you'll find all available data streams from Martz and their corresponding fields:
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the store in Martz. |
| `name` | String | Name of the store. |
| `description` | String | Description of the store as written in the Martz panel. |
| `site_url` | String | Address of the store's own site. |
| `logo` | String | Address of the store's logo image. |
| `enabled` | Boolean | Whether the store's Martz account is active. |
| `store_category_name` | String | Retail category the store was classified under. |
| `created_at` | Date | When the store's Martz account was created. |
| `updated_at` | Date | When the store's Martz account was last changed. |
| `store_integrations` | String | Platforms connected to this Martz account, as a JSON array of objects carrying each integration's name and whether it is enabled. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the customer in Martz. |
| `email` | String | E-mail address of the customer. |
| `first_name` | String | First name of the customer. |
| `last_name` | String | Last name of the customer. |
| `gender` | String | Gender recorded for the customer, as a single letter (M or F) when the store's platform provides it. |
| `phone` | String | Phone number of the customer, without the country code. |
| `phone_country_code` | String | Country calling code of the phone number, without the plus sign (55 for Brazil). |
| `enabled` | Boolean | Whether the customer is active in Martz and may receive campaign messages. |
| `average_ticket` | Integer | Average value of the customer's paid orders, in cents. |
| `order_amount` | Integer | Number of paid orders the customer has placed. Zero means Martz treats this contact as a lead rather than a buyer. |
| `order_total` | Integer | Sum of the customer's paid orders, in cents. |
| `last_order_date` | Date | Date of the customer's most recent order. |
| `integration_created_at` | Date | When the customer was created in the store's own platform, as reported by the integration that imported them. |
| `is_valid_whatsapp_number` | Boolean | Whether Martz verified the customer's phone number as a reachable WhatsApp account. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the order in Martz. |
| `order_number` | String | Order number as shown in the store's own platform. |
| `number` | String | Street number of the delivery address. |
| `status` | String | Order status normalised by Martz, for example PAID, CANCELED or WAITING\_PAYMENT. |
| `raw_status` | String | Order status exactly as the store's own platform reported it, before Martz normalised it. |
| `channel` | String | Sales channel the order came through. |
| `payment_method` | String | Payment method used, as named by the store's platform (for example 'Boleto - Yapay'). |
| `subtotal` | Integer | Value of the ordered items before freight and discounts, in cents. |
| `freight` | Integer | Shipping cost charged on the order, in cents. |
| `discount` | Integer | Discount applied to the order, in cents. |
| `financial_discount` | Integer | Additional financial discount applied to the order, in cents. |
| `total` | Integer | Total value of the order including freight and discounts, in cents. |
| `coupon_name` | String | Code of the discount coupon used on the order, when one was. |
| `ordered_at` | Date | When the order was placed in the store's platform. |
| `created_at` | Date | When the order record was created in Martz. |
| `updated_at` | Date | When the order record was last changed in Martz. |
| `integration_name` | String | Name of the integration the order was imported from, for example tray, nuvemshop or vtex. |
| `integration_id` | String | Identifier of the order in the store's own platform. |
| `integration_updated_at` | Date | When the order was last changed in the store's own platform. |
| `integration_paid_at` | Date | When the order was paid, as reported by the store's own platform. Empty while the order is unpaid. |
| `integration_cancelled_at` | Date | When the order was cancelled in the store's own platform. Empty unless it was cancelled. |
| `integration_seller` | String | Seller credited with the order in the store's own platform. |
| `address` | String | Street of the delivery address. |
| `complement` | String | Additional detail of the delivery address, such as an apartment or block. |
| `locality` | String | Neighbourhood of the delivery address. |
| `city` | String | City of the delivery address. |
| `state` | String | State or region of the delivery address. |
| `country` | String | Country of the delivery address, as a two-letter code. |
| `zipcode` | String | Postal code of the delivery address. |
| `freight_name` | String | Shipping method chosen for the order. |
| `freight_tracking_code` | String | Carrier tracking code for the shipment. |
| `freight_tracking_url` | String | Address where the shipment can be tracked. |
| `customer_id` | String | Identifier in Martz of the customer this record belongs to. |
| `customer_email` | String | E-mail address of the customer this record belongs to. |
| `customer_first_name` | String | First name of the customer this record belongs to. |
| `customer_last_name` | String | Last name of the customer this record belongs to. |
| `products` | String | Items in the order, as a JSON array of objects. Each entry carries the line's id, product\_id, product\_name, product\_category\_id, product\_category\_name, integration\_id, unit price in cents, quantity, created\_at and updated\_at. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the product in Martz. |
| `name` | String | Display name of the product. |
| `sku` | String | Stock keeping unit of the product in the store's own platform. |
| `price` | Integer | Price of the product, in cents. |
| `integration_id` | String | Identifier of the product in the store's own platform. |
| `integration_name` | String | Name of the integration the product was imported from, for example vtex, tray or nuvemshop. |
| `created_at` | Date | When the product record was created in Martz. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the product category in Martz. |
| `name` | String | Display name of the category. |
| `main_category_id` | String | Identifier in Martz of the parent category, or empty when this is a top-level category. |
| `integration_id` | String | Identifier of the category in the store's own platform. |
| `integration_name` | String | Name of the integration the category was imported from. |
| `created_at` | Date | When the category record was created in Martz. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the abandoned cart in Martz. |
| `store_id` | String | Identifier in Martz of the store the cart belongs to. |
| `url` | String | Checkout address that reopens this cart in the store. |
| `subtotal` | Integer | Value of the items in the cart before freight and discounts, in cents. |
| `freight` | Integer | Shipping cost estimated for the cart, in cents. |
| `discount` | Integer | Discount applied to the cart, in cents. |
| `total` | Integer | Total value of the cart including freight and discounts, in cents. |
| `payment_method` | String | Payment method selected before the cart was abandoned, or not\_informed when the customer never reached that step. |
| `coupon_name` | String | Code of the discount coupon applied to the cart, when one was. |
| `channel` | String | Sales channel the cart was created in. |
| `city` | String | City of the address filled in before the cart was abandoned. |
| `state` | String | State or region of the address filled in before the cart was abandoned. |
| `country` | String | Country of the address, as a two-letter code. |
| `zipcode` | String | Postal code of the address filled in before the cart was abandoned. |
| `freight_name` | String | Shipping method selected before the cart was abandoned. |
| `integration_id` | String | Identifier of the cart or checkout session in the store's own platform. |
| `integration_name` | String | Name of the integration the cart was imported from. |
| `integration_created_at` | Date | When the cart was created in the store's own platform. |
| `created_at` | Date | When the cart record was created in Martz. |
| `updated_at` | Date | When the cart record was last changed in Martz. |
| `customer_id` | String | Identifier in Martz of the customer this record belongs to. |
| `customer_email` | String | E-mail address of the customer this record belongs to. |
| `customer_first_name` | String | First name of the customer this record belongs to. |
| `customer_last_name` | String | Last name of the customer this record belongs to. |
| `products` | String | Items in the cart, as a JSON array of objects. Each entry carries the line's id, product\_id, product\_name, unit price in cents, quantity, created\_at and updated\_at. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the campaign in Martz. |
| `name` | String | Name given to the campaign in the Martz panel. |
| `status` | String | Current state of the campaign: ACTIVATED, PAUSED, FINISHED, CANCELED or LOADING. |
| `started_at` | Date | When the campaign starts sending. |
| `ended_at` | Date | When the campaign stops sending. Campaigns meant to run indefinitely carry a date far in the future. |
| `customer_total` | Integer | Number of customers currently associated with the campaign. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the activity in Martz. |
| `message` | String | Text of the message as it was sent to the customer, with the campaign's variables already replaced. |
| `message_type` | String | Form of the message, for example TEXT or a media type. |
| `channel` | String | Channel the message went out on, for example OFFICIAL\_WHATSAPP, WHATSAPP, EMAIL or SMS. |
| `status` | String | Martz's own view of the send: DELAYED, NOT\_SENDED, SENDED, UNACHIEVABLE, WAITING\_FOR\_SEND, DELIVERED, SENT, FAILED or READED. |
| `remote_status` | String | Status the messaging channel itself reported back, for example READ or DELIVERED. |
| `unachievable` | Boolean | Whether the customer could not be reached on this channel, for example because the number is not a valid WhatsApp account. |
| `last_send_error` | String | Error reported the last time the send was attempted. |
| `scheduled_to` | Date | When the message is or was scheduled to go out. Used as the incremental replication key, and the field the API's date filter compares against. |
| `executed_at` | Date | When the send actually ran. Empty while the activity is still waiting. |
| `created_at` | Date | When the activity was created in Martz. |
| `execution_mode` | String | Whether the send was triggered by the campaign itself (AUTOMATIC) or by a person in the panel. |
| `executor_name` | String | Name of the collaborator who triggered the send, when it was not automatic. |
| `generated_from_ai` | Boolean | Whether the message text was written by Martz's AI assistant. |
| `offset` | Integer | Position of this message in the campaign's sequence of steps, counted from the trigger. |
| `sender_id` | String | Identifier of the WhatsApp number or e-mail identity the message was sent from. |
| `mail_subject` | String | Subject line, for activities sent by e-mail. |
| `mail_from` | String | Sending address, for activities sent by e-mail. |
| `mail_sender_name` | String | Display name of the sender, for activities sent by e-mail. |
| `asset_resource_file` | String | Address of the image or file attached to the message. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the coupon in Martz. |
| `code` | String | Coupon code the customer types at checkout. |
| `type` | String | How the discount is calculated, for example PERCENTAGE or a fixed value. |
| `value` | Integer | Size of the discount. For a fixed coupon this is in cents; for a percentage coupon it is the percentage times one hundred. |
| `max_value` | Integer | Cap on the discount a percentage coupon may give, in cents. |
| `minimum_for_use` | Integer | Smallest order value the coupon may be used on, in cents. |
| `purchase_value` | Integer | Value of the order that earned the customer this coupon, in cents. |
| `used_purchase_value` | Integer | Value of the order the coupon was spent on, in cents. |
| `channel` | String | Where the coupon can be redeemed, for example ECOMMERCE. |
| `campaign_id` | String | Identifier in Martz of the campaign that generated the coupon. |
| `order_id` | String | Identifier in Martz of the order that earned the customer this coupon. |
| `used_order_id` | String | Identifier in Martz of the order the coupon was spent on. |
| `used_at` | Date | When the coupon was redeemed. Empty while it is unused. |
| `expires_at` | Date | When the coupon stops being valid. |
| `canceled_at` | Date | When the coupon was cancelled. Empty unless it was. |
| `deleted_at` | Date | When the coupon was removed in Martz. Rows carrying a value here are still returned by the API. |
| `created_at` | Date | When the coupon was created in Martz. |
| `updated_at` | Date | When the coupon was last changed in Martz. |
| `customer_id` | String | Identifier in Martz of the customer the coupon belongs to. |
| `customer_email` | String | E-mail address of the customer the coupon belongs to. |
| `customer_first_name` | String | First name of the customer the coupon belongs to. |
| `customer_last_name` | String | Last name of the customer the coupon belongs to. |
| `customer_document` | String | Tax document (CPF or CNPJ) of the customer the coupon belongs to. |
| `customer_phone` | String | Phone number of the customer the coupon belongs to. |
| `limitations` | String | Days and time ranges the coupon may be used in, as a JSON object keyed by weekday. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the customer group in Martz. |
| `name` | String | Name given to the group in the Martz panel. |
| `visibility` | String | Whether the group is visible to the whole store or only to the collaborator who created it. |
| `mode` | String | Whether membership is recalculated as customers change (DYNAMIC) or fixed at the moment the group was built. |
| `update_mode` | String | How a recalculation treats customers that no longer match, for example WITH\_REMOVAL. |
| `creation_type` | String | Whether the group was created by a person in the panel (BY\_USER) or maintained by Martz itself (BY\_SYSTEM). |
| `status` | String | State of the group's last recalculation, for example UPDATED. |
| `customers_amount` | Integer | Number of customers currently in the group. |
| `created_at` | Date | When the group was created in Martz. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the tag in Martz. |
| `name` | String | Label shown on the tag. |
| `color` | String | Colour the tag is drawn in, as a hexadecimal code. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the custom attribute in Martz. |
| `name` | String | Name the attribute is shown under in the Martz panel. |
| `path` | String | Location the value is read from inside the source record, written as a dotted path such as products.id. |
| `resource` | String | Entity the attribute belongs to: CUSTOMER or ORDER. |
| `enabled` | Boolean | Whether the attribute is currently being collected. |
| `created_at` | Date | When the attribute was created in Martz. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the popup in Martz. |
| `name` | String | Name given to the popup in the Martz panel. |
| `title` | String | Headline shown to the visitor. |
| `enabled` | Boolean | Whether the popup is currently being shown on the store's site. |
| `integration_names` | Array | Names of the store platforms this popup is displayed on. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the sign-up in Martz. |
| `popup_id` | String | Identifier in Martz of the popup that collected this sign-up. |
| `customer_id` | String | Identifier in Martz of the customer created or matched by the sign-up. |
| `page` | String | Page of the store's site the visitor was on when they signed up. |
| `newsletter_version` | Integer | Version of the popup's content that was on screen at the time. |
| `created_at` | Date | When the visitor submitted the form. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of this person's membership of the store in Martz. |
| `role` | String | What the person may do in the account: OWNER, MANAGER, VISUALIZER or SELLER. |
| `enabled` | Boolean | Whether the membership is active. |
| `created_at` | Date | When the person was added to the store. |
| `updated_at` | Date | When the membership was last changed. |
| `collaborator_id` | String | Identifier of the person in Martz, stable across every store they belong to. |
| `collaborator_name` | String | Name of the person. |
| `collaborator_email` | String | E-mail address the person signs in with. |
| `collaborator_enabled` | Boolean | Whether the person's own Martz account is active. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the dashboard in Martz. |
| `name` | String | Name given to the dashboard in the Martz panel. |
| `featured` | Boolean | Whether the dashboard is pinned as the account's default view. |
| `created_at` | Date | When the dashboard was created. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the template in Martz. |
| `name` | String | Name the template was registered under with Meta. |
| `body` | String | Text of the template, with its variables written as numbered placeholders. |
| `language` | String | Language the template was submitted in, for example pt\_BR. |
| `category` | String | Category the template was submitted under, for example MARKETING or UTILITY. |
| `content_type` | String | Form of the template, for example CALL\_TO\_ACTION. |
| `approval_status` | String | Where the template stands with Meta: NOT\_SUBMITTED, PENDING, APPROVED or REJECTED. |
| `reason` | String | Reason Meta gave for the current status, or NONE. |
| `quality_rating` | String | Quality rating Meta assigned to the template based on how recipients reacted to it. |
| `media_url` | String | Address of the image or file the template sends. |
| `media_type` | String | Kind of media the template sends. |
| `external_id` | String | Identifier of the template inside Meta's systems. |
| `meta_business_manager_id` | String | Identifier of the Meta Business Manager the template belongs to. |
| `whatsapp_business_account_id` | String | Identifier of the WhatsApp Business Account the template belongs to. |
| `sample_data` | String | Example values submitted for the template's placeholders, as a JSON object keyed by placeholder number. |
| `metadata` | String | Additional template detail Meta returned, as a JSON object. |
| `created_at` | Date | When the template was created in Martz. |
| `updated_at` | Date | When the template was last changed in Martz. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the connected number in Martz. |
| `number` | String | The WhatsApp number itself. |
| `name` | String | Name given to the connection in the Martz panel. |
| `is_active` | Boolean | Whether the number is enabled for sending. |
| `is_online` | Boolean | Whether the connected WhatsApp session is currently reachable. |
| `store_collaborator_id` | String | Identifier of the collaborator membership this number belongs to, joining to the collaborators table. |
| `store_collaborator_name` | String | Name of the collaborator who answers on this number. |
| `created_at` | Date | When the number was connected. |
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the sender in Martz. |
| `number` | String | The sending number, in international format. |
| `business_display_name` | String | Business name recipients see next to the number. |
| `status` | String | Whether the sender is currently connected, for example ONLINE. |
| `is_active` | Boolean | Whether the sender is enabled for sending. |
| `quality_rating` | String | Quality rating Meta assigned to the number based on how recipients reacted to its messages. |
| `messaging_limits` | String | Daily sending tier Meta granted the number, which caps how many customers a campaign may reach. |
| `external_id` | String | Identifier of the phone number inside Meta's systems. |
| `whatsapp_business_account_id` | String | Identifier of the WhatsApp Business Account the sender belongs to. |
| `created_at` | Date | When the sender was registered in Martz. |
| `updated_at` | Date | When the sender was last changed in Martz. |
# Mautic as a data source
Source: https://docs.nekt.com/sources/mautic
Bring data from Mautic to Nekt.
Mautic is an open-source marketing automation and CRM platform. It manages contacts and companies, sends email and SMS campaigns, hosts landing pages and forms, and scores contacts through points and lifecycle stages. Mautic is self-hosted, so each organization runs its own instance and this connector reads from that instance directly.
## Before you start
Mautic ships with its API switched **off**. An administrator of your instance has to turn it on before Nekt can read anything.
In Mautic, go to **Settings → Configuration → API Settings** and set **API enabled** to *Yes*. Save and close.
Choose one of the two methods below. OAuth 2 is the one Mautic recommends.
**OAuth 2 (recommended).** Sign in as a user with an **administrator** role, go to **Settings → API Credentials**, and add a new credential. Mautic asks for a name and a Redirect URI; the Redirect URI is never used by this connection, so any valid URL will do. Save, then copy the **Client ID** and **Client Secret**.
**Basic authentication.** Back in **Settings → Configuration → API Settings**, also set **Enable HTTP basic auth** to *Yes*. Then use the username and password of an existing Mautic user.
The API credential must be created while signed in as an **administrator**. Mautic only enables the machine-to-machine sign-in this connector uses when an administrator creates the credential, and it only does so at the moment of creation; editing the credential afterwards does not add it. A credential created by any other role is refused at sign-in with no explanation of why. If that happens, delete it and create a new one as an administrator.
What the connection can read is governed by the role of the user behind it. With OAuth 2 that is the role of the administrator who created the credential; with basic authentication it is the role of the user whose password you supplied. A role without permission on part of Mautic makes the matching streams come back empty, and the run log says which ones.
## Configuring Mautic as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Mautic option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are available:
* **Mautic URL**: The address of your Mautic installation, with no trailing slash, for example `https://mautic.example.com`. This is your own server, not a shared Mautic address.
* **Authentication Method**: pick *OAuth 2* to sign in with an API credential, or *Username and password* to sign in as a Mautic user. The credential fields below change to match what you pick, so only the ones that apply are shown.
* **Client ID** and **Client Secret**: the two keys from **Settings → API Credentials**. Shown when you pick OAuth 2.
* **Username** and **Password**: the credentials of a Mautic user. Shown when you pick username and password.
* **Start Date**: The earliest change to read on the first sync. Later runs continue from where the previous one stopped. Leave it empty to read your full history.
* **Page Size**: (Default: 100) Records requested per API call. Mautic's own default is 30. A larger page means fewer calls but a heavier response for your server to build; lower it if your instance times out or runs out of memory on large tables.
* **Requests Per Minute**: (Default: 120) The pace this connection holds itself to, so a full sync does not overwhelm your Mautic server. Set it to 0 to remove the limit.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Mautic exposes two different families of data, and this connector reads both.
**Records** are the things people create and edit in the Mautic interface: contacts, companies, campaigns, emails, forms, segments. **Activity tables** are the event log behind those records: every send, open, page hit, form submission, campaign step, point change and segment membership. Most analysis needs both: the records give you the *what*, the activity tables give you the *when* and *how often*.
## How syncing works
Records sync **incrementally** on a column named `last_changed_at`. Mautic leaves its own `date_modified` empty until a record is edited for the first time, so this connector derives `last_changed_at` from the modification date and falls back to the creation date. The original `date_modified` is still available as its own column.
Activity tables sync **incrementally on `id`** where they have one, which reads each new row exactly once. Tables whose rows are updated after they are written (an email's open counters, a push notification's clicks) say so on their `id` field: choose a full sync for those streams if you need the counters to keep up. Association tables such as `segment_memberships` have no row id at all and always sync in full.
All timestamps are stored by Mautic in UTC. Activity tables return them without a timezone and this connector reads them as UTC; record endpoints return them with the offset your instance is configured for, and that offset is preserved.
## Custom fields
Mautic lets you add your own fields to contacts and companies. The `contacts` and `companies` streams have real columns for every field Mautic installs by default (`email`, `firstname`, `city`, `companyname`, and so on), and a `fields` column holding the complete set as JSON, including anything your instance added. Use the `contact_fields` stream to see what each key in that JSON means and what type it holds.
## Record streams
| Stream | Slug | Sync | Description |
| - | - | - | - |
| Contacts | `contacts` | Incremental | People known to Mautic, with their built-in and custom fields. |
| Companies | `companies` | Incremental | Organizations known to Mautic, with their built-in and custom fields. |
| Campaigns | `campaigns` | Incremental | Automation campaigns and the event tree that drives each one. |
| Emails | `emails` | Incremental | Email templates and campaign emails, with their aggregate send counters. |
| Forms | `forms` | Incremental | Forms, with their field definitions and submit actions. |
| Segments | `segments` | Incremental | Contact segments and the filters that define their membership. |
| Assets | `assets` | Incremental | Downloadable files hosted or linked by Mautic. |
| Notes | `notes` | Incremental | Notes people write on a contact's timeline. |
| Stages | `stages` | Incremental | Lifecycle stages a contact can be moved through. |
| Tags | `tags` | Full table | Tags available to label contacts. |
| Categories | `categories` | Incremental | Categories used to file emails, forms, campaigns, and other records. |
| Contact Fields | `contact_fields` | Incremental | Definitions of the contact fields this instance has, including custom ones. |
| Point Groups | `point_groups` | Incremental | Point groups, which let a contact hold separate scores side by side. |
| Push Notifications | `notifications` | Incremental | Web push notifications defined on the instance. |
| Text Messages | `text_messages` | Incremental | SMS messages defined on the instance. |
| Users | `users` | Incremental | Mautic's own staff accounts, and the role each one holds. |
| Webhooks | `webhooks` | Incremental | Outgoing webhooks configured on the instance. |
## Activity streams
These read Mautic's own database tables through its stats endpoint. The **Mautic table** column is the name Mautic uses for it, in case you need to cross-reference its documentation.
| Stream | Mautic table | Sync | Description |
| - | - | - | - |
| `email_stats` | `email_stats` | Incremental | One row per email sent to a contact, with its delivery and open state. |
| `email_replies` | `email_stat_replies` | Incremental | Replies Mautic matched back to an email it sent. |
| `email_open_devices` | `email_stats_devices` | Incremental | Which device each email open came from. |
| `page_hits` | `page_hits` | Incremental | Every tracked page view, on a Mautic landing page or a site carrying the tracking script. |
| `form_submissions` | `form_submissions` | Incremental | One row per form submission. The submitted values themselves stay in Mautic's form results tables and are not exposed by the API. |
| `asset_downloads` | `asset_downloads` | Incremental | Every download of a file hosted as a Mautic asset. |
| `campaign_event_log` | `campaign_lead_event_log` | Incremental | Every campaign step taken for a contact, including steps still scheduled. |
| `campaign_memberships` | `campaign_leads` | Full table | Which contacts are in which campaign. Keyed by the campaign and contact pair rather than by a row id, and edited in place when someone leaves, so it always syncs in full. |
| `segment_memberships` | `lead_lists_leads` | Full table | Which contacts are in which segment. Keyed by the segment and contact pair rather than by a row id, and edited in place on removal, so it always syncs in full. |
| `company_contacts` | `companies_leads` | Full table | Which contacts belong to which company. Keyed by the company and contact pair, so it always syncs in full. |
| `contact_categories` | `lead_categories` | Incremental | Category subscriptions a contact holds, as used by preference centres. |
| `contact_event_log` | `lead_event_log` | Incremental | Audit trail of what changed on each contact and who changed it. |
| `contact_point_changes` | `lead_points_change_log` | Incremental | Every point award or deduction, with what caused it. |
| `contact_stage_changes` | `lead_stages_change_log` | Incremental | Every move of a contact from one lifecycle stage to another. |
| `contact_company_changes` | `lead_companies_change_log` | Incremental | History of contacts being linked to and unlinked from companies. |
| `do_not_contact` | `lead_donotcontact` | Incremental | Contacts that must not be contacted on a given channel, and why. |
| `contact_devices` | `lead_devices` | Incremental | Devices seen for each contact, as detected from their browser. |
| `contact_utm_tags` | `lead_utmtags` | Incremental | UTM parameters recorded the times a contact arrived from a tagged link. |
| `contact_frequency_rules` | `lead_frequencyrules` | Incremental | Per-contact limits on how often each channel may be used. |
| `point_action_log` | `point_lead_action_log` | Full table | Which point actions have already fired for which contact, so they do not fire twice. Keyed by the point and contact pair, so it syncs in full. |
| `point_event_log` | `point_lead_event_log` | Full table | Which point-group events have fired for which contact. Keyed by the event and contact pair, so it syncs in full. |
| `stage_action_log` | `stage_lead_action_log` | Full table | Which stage actions have fired for which contact. Keyed by the stage and contact pair, so it syncs in full. |
| `tracked_links` | `channel_url_trackables` | Full table | Click totals per tracked link and channel. Holds running counters rather than individual clicks, and has no row id, so it syncs in full. |
| `link_redirects` | `page_redirects` | Incremental | The tracked links themselves, with their click counters. |
| `text_message_stats` | `sms_message_stats` | Incremental | One row per text message sent to a contact. |
| `push_notification_stats` | `push_notification_stats` | Incremental | One row per web push notification sent to a contact, with its click state. |
| `dynamic_content_stats` | `dynamic_content_stats` | Incremental | How often each dynamic content variant was shown to a contact. |
| `dynamic_content_slots` | `dynamic_content_lead_data` | Incremental | Which dynamic content slot a contact last saw. |
| `video_hits` | `video_hits` | Incremental | Views of videos embedded with Mautic's video tracking. |
| `ip_addresses` | `ip_addresses` | Incremental | The IP addresses every other table refers to by ip\_id, with whatever geolocation Mautic resolved for them. |
| `audit_log` | `audit_log` | Incremental | Mautic's own audit trail of who changed what across the instance. |
| `focus_stats` | `focus_stats` | Incremental | Views and interactions with Focus items: the pop-ups, notices and banners Mautic injects into a page. Present on Mautic 5 and removed in later versions, where this stream simply reports itself unavailable. |
| `tweet_stats` | `tweet_stats` | Incremental | Tweets Mautic sent on a contact's behalf. Present on Mautic 5 and removed in later versions, where this stream simply reports itself unavailable. |
| `webhook_logs` | `webhook_logs` | Incremental | Delivery attempts of the instance's outgoing webhooks. |
A stream can come back empty because the table does not exist on your version of Mautic, or because the role behind this connection may not read it. Either way the run log names the stream and says which of the two it was, so nothing fails silently. `focus_stats` and `tweet_stats` are the common case: they exist on Mautic 5 and were removed in later versions.
## Fields by stream
People known to Mautic, with their built-in and custom fields.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the contact
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `points` - Current point score of the contact, as awarded by point actions and triggers
* `last_active` - Date and time the contact was last seen doing anything Mautic tracks
* `date_identified` - Date and time an anonymous visitor became an identified contact. Empty while the contact is still anonymous
* `color` - Hex colour a point trigger assigned to the contact, used to highlight it in the Mautic interface
* `owner_id` - Identifier of the Mautic user who owns the contact
* `owner_username` - Login name of the Mautic user who owns the contact
* `owner_first_name` - Given name of the Mautic user who owns the contact
* `owner_last_name` - Family name of the Mautic user who owns the contact
* `title` - Honorific or salutation, such as Mr or Dr
* `firstname` - Given name of the contact
* `lastname` - Family name of the contact
* `company` - Free-text company name on the contact itself, which is separate from the company records the contact is linked to
* `position` - Job title of the contact
* `email` - Primary email address of the contact
* `mobile` - Mobile phone number of the contact
* `phone` - Landline phone number of the contact
* `fax` - Fax number of the contact
* `address1` - First line of the contact's address
* `address2` - Second line of the contact's address
* `city` - City of the contact's address
* `state` - State, province or region of the contact's address
* `zipcode` - Postal code of the contact's address
* `country` - Country of the contact's address
* `preferred_locale` - Locale the contact prefers to be contacted in
* `timezone` - Timezone recorded for the contact
* `attribution` - Monetary value attributed to the contact by the campaigns and points configuration
* `attribution_date` - Date the attribution value was assigned
* `website` - Website of the contact
* `facebook` - Facebook handle of the contact
* `foursquare` - Foursquare handle of the contact
* `instagram` - Instagram handle of the contact
* `linkedin` - LinkedIn handle of the contact
* `skype` - Skype handle of the contact
* `twitter` - Twitter handle of the contact
* `fields` - Every contact field and its value, as a JSON object keyed by field alias. This is the complete set for this instance, so it also carries any custom field added locally that has no column of its own
* `tags` - Tags applied to the contact (JSON array)
* `utmtags` - UTM parameters recorded for the contact (JSON array of objects)
* `do_not_contact` - Channels the contact must not be contacted on, with the reason for each (JSON array of objects)
* `ip_addresses` - IP addresses seen for the contact, with any geolocation details Mautic resolved (JSON object keyed by address)
Organizations known to Mautic, with their built-in and custom fields.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the company
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `score` - Score assigned to the company by Mautic's company scoring
* `owner_id` - Identifier of the Mautic user who owns the company
* `owner_username` - Login name of the Mautic user who owns the company
* `owner_first_name` - Given name of the Mautic user who owns the company
* `owner_last_name` - Family name of the Mautic user who owns the company
* `companyname` - Name of the company
* `companyemail` - Email address of the company
* `companyphone` - Phone number of the company
* `companywebsite` - Website of the company
* `companyaddress1` - First line of the company's address
* `companyaddress2` - Second line of the company's address
* `companycity` - City of the company's address
* `companystate` - State, province or region of the company's address
* `companyzipcode` - Postal code of the company's address
* `companycountry` - Country of the company's address
* `companyindustry` - Business sector the company operates in
* `companydescription` - Free-text description of the company
* `companynumber_of_employees` - Headcount recorded for the company
* `companyannual_revenue` - Annual revenue recorded for the company
* `companyfax` - Fax number of the company
* `fields` - Every company field and its value, as a JSON object keyed by field alias, including any custom field added on this instance
Automation campaigns and the event tree that drives each one.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the campaign
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Name of the campaign
* `description` - Free-text description of the campaign
* `alias` - URL-safe alias generated from the campaign name
* `events` - The campaign's decisions and actions with their triggers and ordering (JSON array of objects)
Email templates and campaign emails, with their aggregate send counters.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the email
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Internal name of the email
* `subject` - Subject line sent to recipients
* `language` - Language code of the email
* `from_address` - Address the email is sent from
* `from_name` - Display name the email is sent from
* `reply_to_address` - Address replies are sent to
* `bcc_address` - Address blind-copied on every send
* `use_owner_as_mailer` - Whether the email is sent from the contact's owner instead of the configured sender
* `preheader_text` - Preview text shown after the subject line in most inboxes
* `custom_html` - Rendered HTML body of the email
* `plain_text` - Plain-text body of the email
* `template` - Theme used to style the email
* `email_type` - Whether the email is sent to a segment (list) or used by a campaign action (template)
* `public_preview` - Whether the email can be viewed at a public URL
* `read_count` - Number of times the email has been opened, counted by Mautic
* `sent_count` - Number of times the email has been sent, counted by Mautic
* `revision` - Revision number of the email
* `variant_start_date` - Date the A/B test on this email started
* `variant_sent_count` - Number of sends attributed to variants of this email
* `variant_read_count` - Number of opens attributed to variants of this email
* `variant_parent` - Identifier of the email this one is an A/B variant of
* `translation_parent` - Identifier of the email this one is a translation of
* `send_to_dnc` - Whether the email is sent to contacts marked do-not-contact
* `utm_tags` - UTM parameters appended to links (JSON object)
* `asset_attachments` - Assets attached to the email (JSON array)
* `variant_children` - Identifiers of the A/B variants of this email (JSON array)
* `translation_children` - Identifiers of the translations of this email (JSON array)
* `dynamic_content` - Dynamic content blocks and their filters (JSON array of objects)
* `lists` - Segments this email is sent to (JSON array of objects)
* `headers` - Custom email headers (JSON object)
* `settings` - Per-email settings (JSON object)
* `grapesjsbuilder` - Configuration saved by the drag-and-drop email builder (JSON object)
* `unsubscribe_form` - Form used as the unsubscribe page for this email (JSON object)
Forms, with their field definitions and submit actions.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the form
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Name of the form
* `alias` - URL-safe alias of the form
* `description` - Description of the form
* `form_type` - Whether the form stands alone or belongs to a campaign
* `template` - Theme used to style the form
* `language` - Language code of the form
* `in_kiosk_mode` - Whether submissions are recorded without cookie tracking or IP association, as on a shared device at an event
* `render_style` - Whether Mautic includes its own CSS when rendering the form
* `post_action` - What happens after a submission: nothing, a redirect, or a message
* `post_action_property` - The redirect URL or the success message that goes with the post action
* `no_index` - Whether the form page asks search engines not to index it
* `form_attributes` - Custom HTML attributes added to the form tag
* `submission_limit` - Maximum number of submissions the form accepts. Empty means unlimited
* `submission_limit_message` - Message shown once the submission limit is reached
* `submission_count` - Number of submissions recorded for the form
* `cached_html` - The HTML and CSS Mautic generated for the form
* `fields` - Field definitions of the form, including type, validation and the contact field each one maps to (JSON object keyed by field alias)
* `actions` - Actions run when the form is submitted (JSON object keyed by action alias)
Contact segments and the filters that define their membership.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the segment
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Internal name of the segment
* `public_name` - Name shown to contacts where the segment is public
* `alias` - URL-safe alias of the segment
* `description` - Description of the segment
* `is_global` - Whether every Mautic user can use the segment, rather than only the user who created it
* `is_preference_center` - Whether contacts can subscribe to the segment themselves from a preference centre
* `filters` - Rules that decide which contacts belong to the segment (JSON array of objects). Membership itself is in the segment\_memberships stream
Downloadable files hosted or linked by Mautic.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the asset
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `title` - Title of the asset
* `description` - Description of the asset
* `alias` - URL-safe alias of the asset
* `language` - Language code of the asset
* `download_count` - Total number of downloads, counting repeat downloads by the same contact
* `unique_download_count` - Number of distinct contacts who downloaded the asset
* `revision` - Revision number of the asset
* `extension` - File extension of the asset
* `mime` - MIME type of the asset
* `size` - File size of the asset in bytes
* `download_url` - Public URL the asset is served at
Notes people write on a contact's timeline.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the note
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `contact_id` - Identifier of the contact the note is attached to
* `text` - Body of the note, which may contain HTML
* `type` - Kind of note: general, email, call or meeting
* `date_time` - Date and time the note refers to, which the author sets and which can differ from when the note was written
Lifecycle stages a contact can be moved through.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the stage
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Name of the stage
* `description` - Description of the stage
* `weight` - Position of the stage in the lifecycle. Mautic requires it to be unique across stages, so it orders them
Tags available to label contacts.
**Key:** `id`. **Sync:** full table
**Fields:**
* `id` - Unique identifier of the tag
* `tag` - Text of the tag
* `description` - Description of the tag
Categories used to file emails, forms, campaigns, and other records.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the category
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `title` - Name of the category
* `alias` - URL-safe alias of the category
* `description` - Description of the category
* `color` - Hex colour assigned to the category
* `bundle` - Part of Mautic the category applies to, such as email, form or campaign
Definitions of the contact fields this instance has, including custom ones.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the field
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `label` - Name of the field as people see it
* `alias` - Key the field's value appears under in the contacts stream's fields column
* `description` - Description of the field
* `type` - Kind of field, such as text, number, date, select or country
* `group` - Group the field is shown under
* `order` - Position of the field within its group
* `object` - Whether the field belongs to contacts or to companies
* `default_value` - Value used when none is given
* `is_required` - Whether a value must be supplied
* `is_publicly_updatable` - Whether a public request, such as the tracking pixel, may set this field
* `is_unique_identifier` - Whether two contacts sharing this value are merged into one
* `properties` - Type-specific configuration, such as the options of a select field (JSON object)
Point groups, which let a contact hold separate scores side by side.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the point group
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `name` - Name of the point group
* `description` - Description of the point group
Web push notifications defined on the instance.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the notification
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Internal name of the notification
* `heading` - Heading shown to the recipient
* `message` - Body shown to the recipient
* `url` - Address opened when the notification is clicked
* `language` - Language code of the notification
* `read_count` - Number of times the notification was read
* `sent_count` - Number of times the notification was sent
SMS messages defined on the instance.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the text message
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `publish_up` - Date and time from which the record becomes active
* `publish_down` - Date and time from which the record stops being active
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Internal name of the text message
* `message` - Body of the text message
* `language` - Language code of the text message
* `sent_count` - Number of times the message was sent
Mautic's own staff accounts, and the role each one holds.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the user
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `username` - Login name of the user
* `first_name` - Given name of the user
* `last_name` - Family name of the user
* `email` - Email address of the user
* `position` - Job title of the user
* `timezone` - Timezone the user works in
* `locale` - Language the interface is shown in
* `last_login` - Date and time the user last signed in
* `last_active` - Date and time the user was last active
* `signature` - Email signature of the user
* `role_id` - Identifier of the role assigned to the user
* `role_name` - Name of the role assigned to the user
* `role_is_admin` - Whether the role grants full access to the instance. Only an administrator can create the API credential this connector uses
Outgoing webhooks configured on the instance.
**Key:** `id`. **Sync:** incremental on `last_changed_at`
**Fields:**
* `id` - Unique identifier of the webhook
* `is_published` - Whether the record is currently published and in use
* `date_added` - Date and time the record was created
* `created_by` - Identifier of the Mautic user who created the record
* `created_by_user` - Name of the Mautic user who created the record
* `date_modified` - Date and time the record was last edited. Empty on a record that has never been edited since it was created
* `modified_by` - Identifier of the Mautic user who last edited the record
* `modified_by_user` - Name of the Mautic user who last edited the record
* `last_changed_at` - Date and time the record last changed, falling back to the creation date for a record that was never edited. Derived by this connector and used as the incremental replication key
* `category_id` - Identifier of the category the record is filed under
* `category_title` - Name of the category the record is filed under
* `category_alias` - URL-safe alias of the category the record is filed under
* `name` - Name of the webhook
* `description` - Description of the webhook
* `webhook_url` - URL Mautic posts the payload to
* `events_orderby_dir` - Order the batched events are sent in
* `triggers` - Event types that make the webhook fire (JSON array)
One row per email sent to a contact, with its delivery and open state. Reads Mautic's `email_stats` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the send. Rows in this table keep changing after they are created, so an incremental sync keyed on this column records each row as it was when first read. Choose a full sync for this stream if those later changes have to stay current.
* `email_id` - Identifier of the email involved
* `lead_id` - Identifier of the contact the row belongs to
* `list_id` - Identifier of the segment the send was addressed to, when it came from a segment rather than a campaign
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `email_address` - Address the email was actually sent to
* `date_sent` - Date and time the email was sent
* `is_read` - Whether the recipient has opened the email at least once
* `date_read` - Date and time of the first open
* `open_count` - Number of times the email has been opened
* `last_opened` - Date and time of the most recent open
* `open_details` - Details of each open, such as when and from where (JSON)
* `is_failed` - Whether delivery failed
* `retry_count` - Number of delivery attempts made
* `viewed_in_browser` - Whether the recipient opened the web version instead of the email itself
* `tracking_hash` - Unique hash identifying this send in tracking links
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
* `tokens` - Personalisation tokens resolved for this send (JSON object)
* `copy_id` - Identifier of the stored copy of the message body, shared by every send of the same content
Replies Mautic matched back to an email it sent. Reads Mautic's `email_stat_replies` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `stat_id` - Identifier of the send record this row refers to
* `date_replied` - Date and time the reply arrived
* `message_id` - Message-ID header of the reply
Which device each email open came from. Reads Mautic's `email_stats_devices` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `stat_id` - Identifier of the send record this row refers to
* `device_id` - Identifier of the device record involved
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `date_opened` - Date and time the email was opened
Every tracked page view, on a Mautic landing page or a site carrying the tracking script. Reads Mautic's `page_hits` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the page view. Rows in this table keep changing after they are created, so an incremental sync keyed on this column records each row as it was when first read. Choose a full sync for this stream if those later changes have to stay current.
* `page_id` - Identifier of the landing page involved
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `email_id` - Identifier of the email involved
* `redirect_id` - Identifier of the tracked link involved
* `device_id` - Identifier of the device record involved
* `date_hit` - Date and time the page was opened
* `date_left` - Date and time the visitor left the page. Written after the row is created, and empty when the visitor never triggered the tracking script again
* `url` - Address involved
* `url_title` - Title of the page that was viewed
* `referer` - Address the visitor arrived from
* `query` - Query string of the address (JSON object)
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
* `tracking_id` - Anonymous tracking cookie value of the visitor
* `user_agent` - Browser user agent string reported by the visitor
* `browser_languages` - Languages the visitor's browser asked for (JSON array)
* `page_language` - Language of the page that was viewed
* `remote_host` - Hostname resolved for the visitor's IP address
* `city` - City resolved from the visitor's IP address
* `region` - Region resolved from the visitor's IP address
* `country` - Country resolved from the visitor's IP address
* `isp` - Internet provider resolved from the visitor's IP address
* `organization` - Organisation resolved from the visitor's IP address
* `code` - HTTP status code returned to the visitor
One row per form submission. The submitted values themselves stay in Mautic's form results tables and are not exposed by the API. Reads Mautic's `form_submissions` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `form_id` - Identifier of the form involved
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `page_id` - Identifier of the landing page the form was submitted from
* `date_submitted` - Date and time the form was submitted
* `referer` - Address the visitor arrived from
* `tracking_id` - Anonymous tracking cookie value of the visitor
Every download of a file hosted as a Mautic asset. Reads Mautic's `asset_downloads` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `asset_id` - Identifier of the asset involved
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `email_id` - Identifier of the email involved
* `date_download` - Date and time the file was downloaded
* `code` - HTTP status code returned to the visitor
* `referer` - Address the visitor arrived from
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
* `tracking_id` - Anonymous tracking cookie value of the visitor
* `utm_source` - Value of the utm\_source column in asset\_downloads
* `utm_medium` - Value of the utm\_medium column in asset\_downloads
* `utm_campaign` - Value of the utm\_campaign column in asset\_downloads
* `utm_content` - Value of the utm\_content column in asset\_downloads
* `utm_term` - Value of the utm\_term column in asset\_downloads
Every campaign step taken for a contact, including steps still scheduled. Reads Mautic's `campaign_lead_event_log` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `campaign_id` - Identifier of the campaign involved
* `event_id` - Identifier of the event involved
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `channel` - Channel involved, such as email, sms or notification
* `channel_id` - Identifier of the record within that channel
* `date_triggered` - Date and time the step actually ran. Empty while it is still scheduled
* `trigger_date` - Date and time the step is due to run
* `is_scheduled` - Whether the step is still waiting to run
* `system_triggered` - Whether Mautic ran the step by itself rather than a person forcing it
* `non_action_path_taken` - Whether the contact followed the negative branch of a decision
* `rotation` - Which pass through the campaign this row belongs to, for campaigns a contact can re-enter
* `metadata` - Details of the step's outcome (JSON object)
Which contacts are in which campaign. Keyed by the campaign and contact pair rather than by a row id, and edited in place when someone leaves, so it always syncs in full. Reads Mautic's `campaign_leads` table.
**Key:** `campaign_id`, `lead_id`. **Sync:** full table
**Fields:**
* `campaign_id` - Identifier of the campaign involved
* `lead_id` - Identifier of the contact the row belongs to
* `date_added` - Date and time the contact entered the campaign
* `date_last_exited` - Date and time the contact last left the campaign
* `manually_added` - Whether someone added this link by hand rather than it being applied by a filter
* `manually_removed` - Whether someone removed this link by hand even though a filter still matches
* `rotation` - Number of times the contact has gone through the campaign
Which contacts are in which segment. Keyed by the segment and contact pair rather than by a row id, and edited in place on removal, so it always syncs in full. Reads Mautic's `lead_lists_leads` table.
**Key:** `leadlist_id`, `lead_id`. **Sync:** full table
**Fields:**
* `leadlist_id` - Identifier of the segment
* `lead_id` - Identifier of the contact the row belongs to
* `date_added` - Date and time the contact joined the segment
* `manually_added` - Whether someone added this link by hand rather than it being applied by a filter
* `manually_removed` - Whether someone removed this link by hand even though a filter still matches
Which contacts belong to which company. Keyed by the company and contact pair, so it always syncs in full. Reads Mautic's `companies_leads` table.
**Key:** `company_id`, `lead_id`. **Sync:** full table
**Fields:**
* `company_id` - Identifier of the company involved
* `lead_id` - Identifier of the contact the row belongs to
* `date_added` - Date and time the contact was linked to the company
* `is_primary` - Whether this is the contact's main company
* `manually_added` - Whether someone added this link by hand rather than it being applied by a filter
* `manually_removed` - Whether someone removed this link by hand even though a filter still matches
Category subscriptions a contact holds, as used by preference centres. Reads Mautic's `lead_categories` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `category_id` - Identifier of the category involved
* `date_added` - Date and time the row was recorded
* `manually_added` - Whether someone added this link by hand rather than it being applied by a filter
* `manually_removed` - Whether someone removed this link by hand even though a filter still matches
Audit trail of what changed on each contact and who changed it. Reads Mautic's `lead_event_log` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `action` - Action performed, such as create, update or delete
* `bundle` - Part of Mautic the action happened in
* `object` - Kind of record the action applied to
* `object_id` - Identifier of the record the action applied to
* `properties` - What the change was, before and after (JSON object)
* `user_id` - Identifier of the Mautic user who performed the action
* `user_name` - Name of the Mautic user who performed the action
* `date_added` - Date and time the row was recorded
Every point award or deduction, with what caused it. Reads Mautic's `lead_points_change_log` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `type` - Internal type of the change
* `event_name` - Human-readable name of the event that caused the change
* `action_name` - Human-readable name of the action that caused the change
* `delta` - Points added, or removed when negative
* `group_id` - Identifier of the point group the change belongs to, when the instance keeps separate scores side by side
* `date_added` - Date and time the points changed
Every move of a contact from one lifecycle stage to another. Reads Mautic's `lead_stages_change_log` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `stage_id` - Identifier of the stage involved
* `event_name` - Human-readable name of the event that caused the change
* `action_name` - Human-readable name of the action that caused the change
* `date_added` - Date and time the stage changed
History of contacts being linked to and unlinked from companies. Reads Mautic's `lead_companies_change_log` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `company_id` - Identifier of the company involved
* `type` - Internal type of the change
* `event_name` - Human-readable name of the event that caused the change
* `action_name` - Human-readable name of the action that caused the change
* `date_added` - Date and time the row was recorded
Contacts that must not be contacted on a given channel, and why. Reads Mautic's `lead_donotcontact` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `channel` - Channel involved, such as email, sms or notification
* `channel_id` - Identifier of the record within that channel
* `reason` - Why contact is blocked, as Mautic's own code: 1 unsubscribed, 2 bounced, 3 marked as spam
* `comments` - Free-text note recorded with the block
* `date_added` - Date and time the block was recorded
Devices seen for each contact, as detected from their browser. Reads Mautic's `lead_devices` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `client_info` - Everything detected about the browser (JSON object)
* `device` - Kind of device, such as desktop, smartphone or tablet
* `device_brand` - Manufacturer of the device
* `device_model` - Model of the device
* `device_os_name` - Operating system of the device
* `device_os_platform` - Platform of the operating system
* `device_os_shortname` - Short code of the operating system
* `device_os_version` - Version of the operating system
* `device_fingerprint` - Fingerprint used to recognise the device again
* `tracking_id` - Anonymous tracking cookie value of the visitor
* `date_added` - Date and time the row was recorded
UTM parameters recorded the times a contact arrived from a tagged link. Reads Mautic's `lead_utmtags` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `utm_source` - Value of the utm\_source parameter
* `utm_medium` - Value of the utm\_medium parameter
* `utm_campaign` - Value of the utm\_campaign parameter
* `utm_content` - Value of the utm\_content parameter
* `utm_term` - Value of the utm\_term parameter
* `url` - Address involved
* `referer` - Address the visitor arrived from
* `query` - Query string of the address (JSON object)
* `remote_host` - Hostname resolved for the visitor's IP address
* `user_agent` - Browser user agent string reported by the visitor
* `date_added` - Date and time the row was recorded
Per-contact limits on how often each channel may be used. Reads Mautic's `lead_frequencyrules` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `channel` - Channel involved, such as email, sms or notification
* `frequency_number` - How many messages are allowed in the period
* `frequency_time` - Length of the period, such as DAY, WEEK or MONTH
* `preferred_channel` - Whether this is the contact's preferred channel
* `pause_from_date` - Start of a period the contact asked not to be contacted in
* `pause_to_date` - End of that period
* `date_added` - Date and time the row was recorded
Which point actions have already fired for which contact, so they do not fire twice. Keyed by the point and contact pair, so it syncs in full. Reads Mautic's `point_lead_action_log` table.
**Key:** `point_id`, `lead_id`. **Sync:** full table
**Fields:**
* `point_id` - Identifier of the point action
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `date_fired` - Date and time the rule fired for this contact
Which point-group events have fired for which contact. Keyed by the event and contact pair, so it syncs in full. Reads Mautic's `point_lead_event_log` table.
**Key:** `event_id`, `lead_id`. **Sync:** full table
**Fields:**
* `event_id` - Identifier of the event involved
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `date_fired` - Date and time the rule fired for this contact
Which stage actions have fired for which contact. Keyed by the stage and contact pair, so it syncs in full. Reads Mautic's `stage_lead_action_log` table.
**Key:** `stage_id`, `lead_id`. **Sync:** full table
**Fields:**
* `stage_id` - Identifier of the stage involved
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `date_fired` - Date and time the rule fired for this contact
Click totals per tracked link and channel. Holds running counters rather than individual clicks, and has no row id, so it syncs in full. Reads Mautic's `channel_url_trackables` table.
**Key:** `channel`, `channel_id`, `redirect_id`. **Sync:** full table
**Fields:**
* `channel` - Channel involved, such as email, sms or notification
* `channel_id` - Identifier of the record within that channel
* `redirect_id` - Identifier of the tracked link involved
* `hits` - Total clicks on the link, counting repeats
* `unique_hits` - Number of distinct contacts who clicked the link
The tracked links themselves, with their click counters. Reads Mautic's `page_redirects` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the link. Rows in this table keep changing after they are created, so an incremental sync keyed on this column records each row as it was when first read. Choose a full sync for this stream if those later changes have to stay current.
* `redirect_id` - Public token the redirect URL is built from
* `url` - Address the link sends the visitor to
* `hits` - Total clicks on the link, counting repeats
* `unique_hits` - Number of distinct contacts who clicked the link
* `is_published` - Whether the link is active
* `date_added` - Date and time the row was recorded
* `date_modified` - Date and time the link was last edited
* `created_by` - Identifier of the Mautic user who created the link
* `created_by_user` - Name of the Mautic user who created the link
* `modified_by` - Identifier of the Mautic user who last edited the link
* `modified_by_user` - Name of the Mautic user who last edited the link
* `checked_out` - Date and time the link was locked for editing
* `checked_out_by` - Identifier of the Mautic user editing the link
* `checked_out_by_user` - Name of the Mautic user editing the link
One row per text message sent to a contact. Reads Mautic's `sms_message_stats` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `sms_id` - Identifier of the text message involved
* `lead_id` - Identifier of the contact the row belongs to
* `list_id` - Identifier of the segment the send was addressed to, when it came from a segment rather than a campaign
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `date_sent` - Date and time the message was sent
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
* `tokens` - Personalisation tokens resolved for this send (JSON object)
* `tracking_hash` - Unique hash identifying this send in tracking links
* `is_failed` - Value of the is\_failed column in sms\_message\_stats
* `details` - Value of the details column in sms\_message\_stats
One row per web push notification sent to a contact, with its click state. Reads Mautic's `push_notification_stats` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the send. Rows in this table keep changing after they are created, so an incremental sync keyed on this column records each row as it was when first read. Choose a full sync for this stream if those later changes have to stay current.
* `notification_id` - Identifier of the push notification involved
* `lead_id` - Identifier of the contact the row belongs to
* `list_id` - Identifier of the segment the send was addressed to, when it came from a segment rather than a campaign
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `date_sent` - Date and time the notification was sent
* `date_read` - Date and time the notification was displayed
* `is_clicked` - Whether the recipient clicked the notification
* `date_clicked` - Date and time of the first click
* `click_count` - Number of times the notification was clicked
* `last_clicked` - Date and time of the most recent click
* `click_details` - Details of each click (JSON)
* `retry_count` - Number of delivery attempts made
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
* `tokens` - Personalisation tokens resolved for this send (JSON object)
* `tracking_hash` - Unique hash identifying this send in tracking links
How often each dynamic content variant was shown to a contact. Reads Mautic's `dynamic_content_stats` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row. Rows in this table keep changing after they are created, so an incremental sync keyed on this column records each row as it was when first read. Choose a full sync for this stream if those later changes have to stay current.
* `dynamic_content_id` - Identifier of the dynamic content block involved
* `lead_id` - Identifier of the contact the row belongs to
* `date_sent` - Date and time the variant was first shown
* `last_sent` - Date and time the variant was most recently shown
* `sent_count` - Number of times the variant was shown
* `sent_details` - Details of each display (JSON)
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
* `tokens` - Personalisation tokens resolved for this send (JSON object)
Which dynamic content slot a contact last saw. Reads Mautic's `dynamic_content_lead_data` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `dynamic_content_id` - Identifier of the dynamic content block involved
* `lead_id` - Identifier of the contact the row belongs to
* `slot` - Name of the slot on the page or email
* `date_added` - Date and time the row was recorded
Views of videos embedded with Mautic's video tracking. Reads Mautic's `video_hits` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `lead_id` - Identifier of the contact the row belongs to
* `ip_id` - Identifier of the IP address record involved, which resolves against the ip\_addresses stream
* `channel` - Channel involved, such as email, sms or notification
* `channel_id` - Identifier of the record within that channel
* `guid` - Identifier of the viewing session
* `url` - Address involved
* `date_hit` - Date and time the video started playing
* `date_left` - Date and time the viewer left
* `duration` - Total length of the video in seconds
* `time_watched` - Seconds of the video actually watched
* `referer` - Address the visitor arrived from
* `query` - Query string of the address (JSON object)
* `user_agent` - Browser user agent string reported by the visitor
* `browser_languages` - Languages the visitor's browser asked for (JSON array)
* `page_language` - Language of the page that was viewed
* `remote_host` - Hostname resolved for the visitor's IP address
* `city` - City resolved from the visitor's IP address
* `region` - Region resolved from the visitor's IP address
* `country` - Country resolved from the visitor's IP address
* `isp` - Internet provider resolved from the visitor's IP address
* `organization` - Organisation resolved from the visitor's IP address
* `code` - HTTP status code returned to the visitor
The IP addresses every other table refers to by ip\_id, with whatever geolocation Mautic resolved for them. Reads Mautic's `ip_addresses` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `ip_address` - The IP address itself
* `ip_details` - Geolocation resolved for the address: city, region, country, provider and coordinates (JSON object)
Mautic's own audit trail of who changed what across the instance. Reads Mautic's `audit_log` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `bundle` - Part of Mautic the action happened in
* `object` - Kind of record the action applied to
* `object_id` - Identifier of the record the action applied to
* `action` - Action performed, such as create, update or delete
* `details` - What changed, before and after (JSON object)
* `user_id` - Identifier of the Mautic user who performed the action
* `user_name` - Name of the Mautic user who performed the action
* `ip_address` - IP address the change was made from
* `date_added` - Date and time the row was recorded
Views and interactions with Focus items: the pop-ups, notices and banners Mautic injects into a page. Present on Mautic 5 and removed in later versions, where this stream simply reports itself unavailable. Reads Mautic's `focus_stats` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `focus_id` - Identifier of the Focus item
* `lead_id` - Identifier of the contact the row belongs to
* `type` - What the contact did, such as view or click
* `type_id` - Identifier of the record that interaction refers to
* `date_added` - Date and time the row was recorded
Tweets Mautic sent on a contact's behalf. Present on Mautic 5 and removed in later versions, where this stream simply reports itself unavailable. Reads Mautic's `tweet_stats` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `tweet_id` - Identifier of the tweet template in Mautic
* `lead_id` - Identifier of the contact the row belongs to
* `twitter_tweet_id` - Identifier the network gave the posted tweet
* `handle` - Account the tweet was addressed to
* `date_sent` - Date and time the tweet was sent
* `is_failed` - Whether sending failed
* `retry_count` - Number of delivery attempts made
* `favorite_count` - Number of favourites the tweet received
* `retweet_count` - Number of retweets the tweet received
* `response_details` - What the network answered (JSON)
* `source` - What caused this row, such as email, campaign or form: Mautic's internal source name
* `source_id` - Identifier of the record named by source
Delivery attempts of the instance's outgoing webhooks. Reads Mautic's `webhook_logs` table.
**Key:** `id`. **Sync:** incremental on `id`
**Fields:**
* `id` - Unique identifier of the row
* `webhook_id` - Identifier of the webhook involved
* `status_code` - HTTP status the receiving endpoint returned
* `runtime` - How long the delivery took, in seconds
* `note` - Error text recorded when the delivery failed
* `date_added` - Date and time the row was recorded
# Meetime as a data source
Source: https://docs.nekt.com/sources/meetime
Bring data from Meetime to Nekt.
Meetime is a sales engagement and meeting scheduling platform that helps sales teams connect with prospects and customers. It provides tools for meeting scheduling, follow-up automation, and sales pipeline management to improve conversion rates and customer relationships.
![]()
## Configuring Meetime as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Meetime option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Meetime data.
The following configurations are available:
* **API Token**: Your Meetime authorization token. Check [Meetime's documentation](https://meetime.readme.io/reference/autentica%C3%A7%C3%A3o) to discover where you can find this parameter.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Cadences | `cadences` | Prospecting cadences (sequences of outreach steps) configured in Meetime. |
| Calls | `calls` | Sales calls logged in Meetime, including outcome and metadata. |
| Demos | `demos` | Product demos and meetings scheduled with leads. |
| Leads | `leads` | Leads (prospects) tracked in Meetime. |
| Prospections | `prospections` | Prospection records tying leads to cadences during outreach. |
| Prospections Activities | `prospections_activities` | Individual activities executed within each prospection. |
| Prospections Lost Reasons | `prospections_lost_reasons` | Reasons registered when a prospection is marked as lost. |
| Users | `users` | Meetime users (sales reps and team members) in the account. |
## Fields by stream
Below you'll find all available data streams from Meetime:
Stream for retrieving cadence data from Meetime.
Stream for retrieving call data from Meetime.
Stream for retrieving demo data from Meetime.
Stream for retrieving lead data from Meetime, including custom fields.
Stream for retrieving prospection data from Meetime.
Stream for retrieving prospection activities data from Meetime.
Stream for retrieving prospection lost reasons from Meetime.
Stream for retrieving user data from Meetime.
## Skills for agents
Meetime connector documentation as plain markdown, for use in AI agent contexts.
# MeetRox as a data source
Source: https://docs.nekt.com/sources/meetrox
Bring data from MeetRox to Nekt.
MeetRox is a conversation intelligence platform that records, transcribes, and analyzes sales calls. It provides AI-powered insights, scorecards, and interaction metrics to help sales teams improve performance and close more deals.
![]()
## Configuring MeetRox as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the MeetRox option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your MeetRox data.
The following configurations are available:
* **API Key**: Your MeetRox API key for authentication. Contact MeetRox support or check your account settings to obtain this key.
* **Initial sync date** (optional, advanced): Starting point for data extraction. Only records created or updated after this date will be synced. If not set, all available data will be extracted.
* **Fetch Call Details** (optional, advanced): When enabled, the connector makes one additional API request per call to fetch the full call details (analysis, scorecard, evaluators and screen-share percentage). This increases sync time and API usage, so keep it off unless you need those fields.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from MeetRox:
Stream for retrieving call records from MeetRox, including interaction metrics, business information, CRM associations and attendee details. Records are synced incrementally by `updated_at`. Nested objects are delivered as structs; their inner fields are listed below with dot notation.
**Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique call identifier |
| `workspace_id` | Integer | Workspace identifier |
| `title` | String | Call title |
| `duration` | Number | Call duration in seconds |
| `timestamp` | DateTime | When the call occurred |
| `inserted_at` | DateTime | Record creation timestamp |
| `updated_at` | DateTime | Record last update timestamp (replication key) |
| `format` | String | Call format (audio, video, file, or chat) |
| `public` | Boolean | Whether the call is publicly accessible |
| `source` | Object | Meeting system the call came from |
| `source.meeting_system` | String | Meeting system name |
| `source.meeting_system_url` | String | Meeting system URL |
| `source.meeting_system_id` | String | Identifier of the call in the meeting system |
| `url` | String | Call URL |
| `interaction_metrics` | Object | Conversation metrics computed for the call |
| `interaction_metrics.rep_talk_ratio` | Number | Representative talk ratio |
| `interaction_metrics.call_exchanges` | Number | Number of call exchanges |
| `interaction_metrics.longest_rep_monologue` | Number | Longest representative monologue duration |
| `interaction_metrics.longest_lead_story` | Number | Longest lead story duration |
| `interaction_metrics.silence_percentage` | Number | Percentage of silence during the call |
| `interaction_metrics.ai_score` | Number | AI-generated score |
| `interaction_metrics.score` | Number | Overall score |
| `business_info` | Object | Deal information attached to the call |
| `business_info.lead_name` | String | Lead name |
| `business_info.lead_company` | String | Lead company |
| `business_info.won` | Boolean | Whether the deal was won |
| `associations` | Array | CRM associations flattened into a single array of objects. Each object contains an `entity_type` (e.g., crm\_deals, crm\_contacts, crm\_companies, crm\_activities, or crm\_custom\_objects), `system`, `system_id`, `label`, and `url`. |
| `agent` | Object | Sales representative who owns the call |
| `agent.id` | Integer | Agent identifier |
| `agent.name` | String | Agent name |
| `agent.email` | String | Agent email |
| `attendees` | Array | List of attendee emails |
| `type` | Object | Call type |
| `type.id` | Integer | Call type identifier |
| `type.name` | String | Call type name |
| `type.description` | String | Call type description |
| `type.association_mode` | String | Call type association mode |
| `screen_share_percentage` | Number | Percentage of the call with screen sharing. Only populated when **Fetch Call Details** is enabled. |
| `analysis` | Object | AI analysis of the call. Only populated when **Fetch Call Details** is enabled. |
| `analysis.summary` | String | AI-generated call summary as a JSON string. The API returns an object with dynamic section-title keys, so it is serialized rather than typed as a fixed struct. |
| `analysis.transcription` | String | Call transcription |
| `scorecard` | Object | Scorecard template applied to the call. Only populated when **Fetch Call Details** is enabled. |
| `scorecard.id` | Integer | Scorecard identifier |
| `scorecard.name` | String | Scorecard name |
| `scorecard.active` | Boolean | Whether the scorecard is active |
| `scorecard.inserted_at` | String | When the scorecard was created, as ISO-8601 text |
| `scorecard.updated_at` | String | When the scorecard was last edited, as ISO-8601 text |
| `scorecard.categories` | Array | List of scorecard categories |
| `scorecard.categories[].id` | Integer | Category identifier |
| `scorecard.categories[].name` | String | Category name |
| `scorecard.categories[].sort_number` | Integer | Category sort number |
| `scorecard.categories[].inserted_at` | String | When the category was created, as ISO-8601 text |
| `scorecard.categories[].updated_at` | String | When the category was last edited, as ISO-8601 text |
| `scorecard.categories[].questions` | Array | List of questions in the category |
| `scorecard.categories[].questions[].id` | Integer | Question identifier |
| `scorecard.categories[].questions[].sort_number` | Integer | Question sort number |
| `scorecard.categories[].questions[].weight` | Number | Question weight in the score |
| `scorecard.categories[].questions[].description` | String | Question description |
| `scorecard.categories[].questions[].ai_description` | String | AI description of the question |
| `scorecard.categories[].questions[].default_answer` | String | Default answer for the question |
| `scorecard.categories[].questions[].text` | String | Question text |
| `scorecard.categories[].questions[].inserted_at` | String | When the question was created, as ISO-8601 text |
| `scorecard.categories[].questions[].updated_at` | String | When the question was last edited, as ISO-8601 text |
| `evaluators` | Array | Evaluations of the call against the scorecard. Only populated when **Fetch Call Details** is enabled. |
| `evaluators[].id` | Integer | Evaluation identifier |
| `evaluators[].type` | String | Evaluator type |
| `evaluators[].score` | Number | Score given by this evaluator |
| `evaluators[].user_id` | Integer | Identifier of the evaluating user |
| `evaluators[].name` | String | Evaluator name |
| `evaluators[].email` | String | Evaluator email |
| `evaluators[].answers` | Array | Answers given to each scorecard question |
| `evaluators[].answers[].id` | Integer | Answer identifier |
| `evaluators[].answers[].answer` | String | Answer value |
| `evaluators[].answers[].category_id` | Integer | Scorecard category the answer belongs to |
| `evaluators[].answers[].question_id` | Integer | Scorecard question the answer refers to |
## Implementation Notes
### Incremental Sync
Calls are requested ordered by `updated_at` ascending and filtered by `updated_at >=` the saved bookmark (or the **Initial sync date** on the first run), so each run only fetches calls created or edited since the previous one. Pagination follows the API's cursor (`meta.end_cursor`).
### Data Type Coercion
The MeetRox API is loose about scalar types: decimals such as `duration` or `interaction_metrics.ai_score` may arrive as strings (e.g. `"1696.14"`), and text fields such as `default_answer` may arrive as numbers or booleans. Because the loader builds each column from the declared type, a single mismatched value would fail the whole batch. The connector therefore reconciles scalars with the declared schema type in both directions during extraction:
* Text that spells a number is parsed into the declared Integer or Number type.
* Numbers and booleans found where the schema declares a String are cast to text.
* A string that cannot be parsed as a number (e.g. `"n/a"`) is kept as text and a warning is logged, rather than dropping the record.
### CRM Associations
The API returns `associations` as an object grouped by entity type (`crm_deals`, `crm_contacts`, ...). The connector flattens these groups into a single array of link objects, keeping the group name in `entity_type`, so the field always matches the declared array schema.
### Scorecard Timestamps
Timestamps inside the `scorecard` struct (`inserted_at` / `updated_at` of the scorecard, its categories and questions) are intentionally delivered as ISO-8601 text rather than DateTime. Nested date-times are not converted by the loader and would break the batch, and these values are only metadata of the scorecard template. The API also misspells the scorecard's `updateted_at`, which the connector maps to `updated_at`.
### Call Details and Rate Limiting
The MeetRox API allows 100 requests per minute. When **Fetch Call Details** is enabled, detail requests are spaced roughly 0.7 seconds apart, and `429` responses honour the `X-RateLimit-Reset` header before retrying. A detail request that still fails after 5 attempts is skipped (the call record is kept without its detail fields) and a warning is logged; after 6 skipped calls, further warnings are suppressed to keep the log readable.
## Skills for agents
MeetRox connector documentation as plain markdown, for use in AI agent contexts.
# MemberKit as a data source
Source: https://docs.nekt.com/sources/member-kit
Bring data from MemberKit to Nekt.
MemberKit is a membership management platform designed for subscription-based businesses. It provides tools for managing member subscriptions, content access, and billing to help businesses create and maintain successful membership programs.
## 1. Add your MemberKit access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the MemberKit option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access. Check [MemberKit documentation](https://ajuda.memberkit.com.br/referencia-api/introducao#autenticacao) to discover where you can find the API Key for the connection.
3. Click **Next**.
## 2. Select your MemberKit streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your MemberKit data streams
1. Customize how you want your data to appear in your catalog. Select the layer, a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: companies in the Growth plan can choose in which [layer](https://docs.nekt.com/get-started/core-concepts/catalog-layers) the tables with the extracted data will be placed.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your MemberKit data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Skills for agents
MemberKit connector documentation as plain markdown, for use in AI agent contexts.
# Mercado Livre as a data source
Source: https://docs.nekt.com/sources/mercado-livre
Bring data from Mercado Livre Marketplace and Advertising to Nekt.
Mercado Livre is Latin America's largest e-commerce and fintech platform, operating across Brazil, Argentina, Mexico, and other countries in the region. This connector provides access to your seller account data including orders (with shipment details, fiscal invoices, and buyer billing profiles where available), catalog listings via a scan-based item list plus full item payloads, advertising campaigns (Product Ads and Brand Ads), and item-level performance metrics such as visits, quality scores, and buyer experience ratings.
![]()
## Configuring Mercado Livre as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Mercado Livre option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to create a Mercado Livre application and generate OAuth credentials to connect your seller account to Nekt.
The following configurations are required:
* **Client ID**: The application ID from your Mercado Livre OAuth app.
* **Client Secret**: The secret key from your Mercado Livre OAuth app.
* **Refresh Token**: A refresh token obtained through the Mercado Livre OAuth flow, from a seller account with enough permissions to access the data.
* **Metrics Granularity**: The time granularity for item-level metrics (visits, ad performance, etc.). Available options:
* `day` - Daily metrics (most granular, but slower extraction)
* `week` - Weekly metrics
* `month` - Monthly metrics (default, fastest extraction)
* **Start Date**: The earliest date from which records will be synced.
The more granular the metrics setting, the longer the extraction will take, as more API calls are required.
Follow these steps to create an OAuth application in Mercado Livre and obtain the credentials needed for Nekt:
1. Go to [Mercado Livre Developer Center](https://developers.mercadolivre.com.br/devcenter) (use the appropriate country link for your account)
2. Log in with your Mercado Livre seller account
3. Click **Create new application**
4. Fill in the application details:
* **Application name**: e.g., "Nekt Data Integration"
* **Short description**: e.g., "Data extraction for analytics"
* **Redirect URI**: `https://developers.google.com/oauthplayground`
5. Under **Scopes**, select:
* `read`: Read access to your account data
* `offline_access`: Required to generate a refresh token
6. Click **Create application**
7. Copy your **App ID** (this is your Client ID) and **Secret Key** (this is your Client Secret)
Keep your Client Secret secure. Never expose it in frontend code or public repositories.
Click the link below to open Google OAuth Playground pre-configured for Mercado Livre:
[**Open Mercado Livre OAuth Playground →**](https://developers.google.com/oauthplayground/#step1\&url=https%3A%2F%2F\&content_type=application%2Fjson\&http_method=GET\&useDefaultOauthCred=unchecked\&oauthEndpointSelect=Custom\&oauthAuthEndpointValue=https%3A%2F%2Fauth.mercadolivre.com.br%2Fauthorization\&oauthTokenEndpointValue=https%3A%2F%2Fapi.mercadolibre.com%2Foauth%2Ftoken\&oauthClientId=ML_CLIENT_ID\&oauthClientSecret=ML_CLIENT_SECRET\&includeCredentials=checked\&accessTokenType=bearer\&autoRefreshToken=unchecked\&accessType=offline\&prompt=consent\&response_type=code\&wrapLines=on)
In the OAuth Playground, click the **gear icon (⚙️)** on the right side to open "OAuth 2.0 configuration" and update these values:
1. Replace `ML_CLIENT_ID` with your **App ID** (Client ID)
2. Replace `ML_CLIENT_SECRET` with your **Secret Key** (Client Secret)
3. Verify the endpoints:
* **OAuth authorization endpoint**: `https://auth.mercadolivre.com.br/authorization`
* **OAuth token endpoint**: `https://api.mercadolibre.com/oauth/token`
If your account is from **Argentina**, use `https://auth.mercadolibre.com.ar/authorization`. For **Mexico**, use `https://auth.mercadolibre.com.mx/authorization`. The token endpoint is always `https://api.mercadolibre.com/oauth/token` regardless of region.
4. Click **Close**
1. In Step 1 of the playground, click **Authorize APIs**
2. You'll be redirected to Mercado Livre. Log in and grant permissions
3. After returning to the playground, click **Exchange authorization code for tokens** (Step 2)
4. Your **refresh token** will appear in the response on the right side
```json theme={null}
{
"access_token": "APP_USR-...",
"token_type": "Bearer",
"expires_in": 21600,
"scope": "offline_access read",
"user_id": 123456789,
"refresh_token": "TG-..."
}
```
You now have everything needed to configure Mercado Livre in Nekt:
* **Client ID**: Your App ID from the Developer Center
* **Client Secret**: Your Secret Key from the Developer Center
* **Refresh Token**: The `refresh_token` from the OAuth response (starts with `TG-`)
The refresh token will be automatically renewed by Nekt on each extraction. You don't need to manually refresh it.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can pick individual streams or combine marketplace, fiscal, and advertising tables as needed.
**Important:** Advertising streams (Product Ads Campaigns, Product Ads Campaign Metrics, Brand Ads Campaigns, Brand Ads Campaign Metrics) are only available if your account has an Advertiser ID registered with Mercado Livre Advertising.
**Order-related child streams** (**Shipments**, **Invoices**, **Billing Info**) run one API request per qualifying order and can materially increase runtime for high-volume sellers.
**Items List** is not selected by default; it holds scan batches of item IDs used to drive the **Items** stream. Enable it only if you want the intermediate ID batches as their own table in the catalog.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Mercado Livre and their corresponding fields:
Stream for all orders from your seller account, including payment details, items, and buyer/seller information.
**Key Fields:**
* `id` - Unique identifier for the order
* `status` - Order status
* `status_detail` - Detailed status information
* `date_created` - When the order was created
* `date_closed` - When the order was closed
* `date_last_updated` - Last update timestamp (replication key)
* `last_updated` - Last modification timestamp
* `expiration_date` - Order expiration date
**Financial:**
* `total_amount` - Total order amount
* `paid_amount` - Amount paid
* `shipping_cost` - Shipping cost
* `currency_id` - Currency code (e.g., BRL, ARS, MXN)
* `coupon` - Coupon information
* `amount` - Coupon discount amount
* `id` - Coupon ID
* `taxes` - Tax information
* `amount` - Tax amount
* `currency_id` - Tax currency
* `id` - Tax ID
**Payments:**
* `payments` - Array of payment details
* `id` - Payment ID
* `status` - Payment status
* `status_code` - Payment status code
* `status_detail` - Payment status detail
* `total_paid_amount` - Total amount paid
* `transaction_amount` - Transaction amount
* `transaction_amount_refunded` - Refunded amount
* `payment_method_id` - Payment method identifier
* `payment_type` - Payment type
* `installments` - Number of installments
* `installment_amount` - Amount per installment
* `date_created` - Payment creation date
* `date_approved` - Payment approval date
* `date_last_modified` - Last modification date
* `authorization_code` - Authorization code
* `operation_type` - Operation type
* `taxes_amount` - Taxes on payment
* `shipping_cost` - Shipping cost in payment
* `coupon_id` - Applied coupon ID
* `coupon_amount` - Coupon discount amount
* `overpaid_amount` - Overpaid amount
* `currency_id` - Currency ID
* `site_id` - Site ID
* `payer_id` - Payer ID
* `order_id` - Associated order ID
* `card_id` - Card ID (if applicable)
* `issuer_id` - Card issuer ID
* `collector` - Collector information (id)
* `available_actions` - Available actions array
* `activation_uri` - Activation URI
* `deferred_period` - Deferred period
* `atm_transfer_reference` - ATM transfer reference (transaction\_id, company\_id)
* `transaction_order_id` - Transaction order ID
* `reason` - Payment reason
**Order Items:**
* `order_items` - Array of purchased items
* `item` - Item details
* `id` - Item ID (MLB...)
* `title` - Item title
* `category_id` - Category ID
* `variation_id` - Variation ID
* `seller_custom_field` - Seller custom field
* `seller_sku` - Seller SKU
* `global_price` - Global price
* `net_weight` - Net weight
* `variation_attributes` - Variation attributes array
* `warranty` - Warranty info
* `condition` - Item condition
* `quantity` - Quantity purchased
* `unit_price` - Unit price
* `full_unit_price` - Full unit price (before discounts)
* `currency_id` - Currency ID
* `sale_fee` - Mercado Livre sale fee
* `listing_type_id` - Listing type
* `manufacturing_days` - Manufacturing days
* `picked_quantity` - Picked quantity
* `requested_quantity` - Requested quantity (measure, value)
* `base_exchange_rate` - Base exchange rate
* `base_currency_id` - Base currency ID
* `bundle` - Bundle info
* `element_id` - Element ID
**Buyer & Seller:**
* `buyer` - Buyer information (id, nickname)
* `seller` - Seller information (id, nickname)
**Shipping & Fulfillment:**
* `shipping` - Shipping information (id)
* `fulfilled` - Whether the order is fulfilled
* `pack_id` - Pack ID for grouped shipments
* `pickup_id` - Pickup ID
**Additional:**
* `tags` - Array of order tags
* `comment` - Order comment
* `manufacturing_ending_date` - Manufacturing ending date
* `feedback` - Feedback info (buyer, seller)
* `order_request` - Order request info (change, return)
* `context` - Context information
* `application` - Application
* `product_id` - Product ID
* `channel` - Channel
* `site` - Site
* `flows` - Flows array
Shipment details for each order that has a shipping ID. This stream is a child of **Orders** and calls `GET /shipments/{shipping_id}` so you get fields such as `logistic_type` (for example fulfillment vs. self-service) that are not on the order payload alone.
**Key Fields:**
* `id` - Shipment ID (primary key)
* `order_id` - Associated order ID (injected from the parent order)
* `order_last_updated` - Parent order `date_last_updated` (replication key)
* `status` - Shipment status
* `substatus` - Shipment substatus
* `date_created` - When the shipment was created
* `last_updated` - When the shipment was last updated
* `logistic_type` - Shipping / fulfillment type (for example `fulfillment`, `self_service`, `drop_off`, `xd_drop_off`, `cross_docking`)
* `mode` - Shipment mode (for example `me2`)
* `sender_id` - Sender (seller) ID
* `receiver_id` - Receiver (buyer) ID
* `tracking_number` - Carrier tracking number
* `tracking_method` - Tracking method used
**Shipping option:**
* `shipping_option` - Selected option including:
* `id`, `name`, `currency_id`, `cost`
* `speed` - `handling`, `shipping`
* `estimated_delivery_time` - `date`, `pay_before`, `type`
**Contents:**
* `shipping_items` - Items in the shipment (`id`, `quantity`, `description`, `dimensions`)
Brazilian electronic invoice (NF-e style) data for orders, when Mercado Livre exposes an invoice for that order. This stream is a child of **Orders**; orders without an invoice may be skipped with a warning.
**Key Fields:**
* `id` - Invoice record ID
* `order_id` - Marketplace order ID
* `order_last_updated` - Parent order last update (replication key)
* `status` - Invoice workflow status
* `transaction_status` - Fiscal transaction processing status
* `issued_date` - Official issuance timestamp
* `invoice_series` - Fiscal series
* `invoice_number` - Sequential number within the series
* `site_id` - Mercado Livre site code
* `pack_id` - Associated fulfillment pack ID
* `amount` - Total monetary amount
* `items_amount` - Sum of line amounts before global adjustments
* `items_quantity` - Number of line items
* `errors` - Validation or emission errors
**Issuer & recipient:**
* `issuer` - Legal issuer including `user_id`, `brand_name`, `name`, `phone` (`area_code`, `number`), `address` (street, city, zip, state, country), `identifications` (`cnpj`, `crt`, `ie`, `ie_type`)
* `recipient` - Buyer/recipient including `external_recipient_id`, `name`, `phone`, `address`, `identifications` (`cpf`, `cnpj`, `crt`, `ie`, `ie_type`)
**Shipment block on the invoice:**
* `shipment` - `id`, `site_id`, `mode`, `logistic_type`, `buyer_cost`, `paid_by`, `carrier` (name, phone, address, identifications), `volumes` (`net_weight`, `gross_weight`), `fiscal_model_id`, `shipping_locations`, `destination`
**Line items (`items` array):**
* Identifiers: `id`, `invoice_id`, `seller_id`, `pack_id`, `external_order_id`, `external_product_id`
* `attributes` - `ean`, `sku`, `type`, `bundle_quantity`
* `product_name`, `quantity`, `total_amount`, `shipping_buyer_cost`, `discount_amount` (`unconditional`, `conditional`)
* `fiscal_data` - Brazilian tax payload per line (NCM, CEST, CFOP, CSOSN, tax rules with ICMS/PIS/COFINS bases and amounts, and nested `messages`)
* `payments` - Payments linked to the line (`payment_method_id`, `installments`, `transaction_amount`, `charges_details`, etc.)
* `additional_info`, `other_amount`
**Header attributes (`attributes`):**
* Fiscal metadata such as `invoice_key`, `environment_type`, `xml_version`, Sefaz `status_code` / `status_description`, `receipt`, `receipt_date`, `protocol`, `authorization_date`, cancellation fields, `danfe`, `xml_location`, `document_type`, `tags`, and related NF-e fields
**Invoice-level fiscal totals (`fiscal_data`):**
* `customer_type`, `transaction_type`, `transaction_type_description`, `messages`
* `fiscal_amounts` - Named totalizers with consolidated tax amounts (`vbc`, `vicms`, `vst`, `vpis`, `vcofins`, etc.)
* `state_calculation_type` - State-specific ICMS calculation method
**Other:**
* `payments` - Serialized payment summary on the invoice header (string)
* `additional_info`, `custom_issuer_address`, `other_amount`
Buyer billing profile for each order (name, document, address, and tax metadata). This stream is a child of **Orders** and calls `/orders/{order_id}/billing_info` with API version `2`. Some orders return **403** if the authenticated account cannot read billing data; those orders are skipped with a warning.
**Key Fields:**
* `cust_id` - Buyer customer ID (primary key in the stream schema)
* `date_last_updated` - Parent order last update timestamp carried into the row
* `name` - Buyer first name for invoicing
* `last_name` - Buyer last name for invoicing
**Identification:**
* `identification` - `type`, `number` (tax or personal ID)
**Taxes:**
* `taxes` - Wrapper for buyer tax metadata including `taxpayer_type` (`id`, `description`)
**Address:**
* `address` - `street_name`, `street_number`, `city_name`, `comment`, `neighborhood`, `state` (`code`, `name`), `zip_code`, `country_id`
**Attributes:**
* `attributes` - `vat_discriminated_billing`, `new_billing_info`, `normalized`, `cust_type`
Scan-based discovery of all item IDs for the seller (`/users/{seller_id}/items/search?search_type=scan`). Each row is one batch of IDs returned from the search API. This stream is the **parent** of the **Items** stream: full item records are loaded per batch. By default this stream is **not** selected; enable it if you want the batched ID lists as a separate table in addition to item details.
**Key Fields:**
* `id_list` - Comma-separated list of item IDs in one batch (primary key)
Stream for all product listings from your seller account. Rows are loaded in batches discovered by the **Items List** parent stream (each batch of IDs becomes one or more full item payloads).
**Key Fields:**
* `id` - Unique item identifier (e.g., MLB12345678)
* `title` - Item title
* `permalink` - Direct link to the listing
* `condition` - Item condition (new, used)
* `status` - Item status (active, paused, closed, etc.)
* `site_id` - Site identifier
* `seller_id` - Seller ID
* `category_id` - Category ID
* `domain_id` - Domain ID
* `catalog_product_id` - Catalog product ID (if linked to catalog)
* `catalog_listing` - Whether item is a catalog listing
* `listing_type_id` - Listing type (gold\_special, gold\_pro, etc.)
* `buying_mode` - Buying mode (buy\_it\_now, auction)
* `start_time` - Listing start time
* `stop_time` - Listing end time
* `end_time` - Listing end time
* `expiration_time` - Listing expiration time
* `date_created` - Item creation date
* `last_updated` - Last update timestamp
* `health` - Item health score
* `family_name` - Family name
* `family_id` - Family ID
* `user_product_id` - User product ID
* `parent_item_id` - Parent item ID (for variations)
* `seller_custom_field` - Custom seller field
**Pricing:**
* `price` - Current price
* `base_price` - Base price
* `original_price` - Original price (before discount)
* `currency_id` - Currency code
**Inventory:**
* `available_quantity` - Available stock quantity
* `initial_quantity` - Initial quantity
* `sold_quantity` - Total units sold
* `inventory_id` - Inventory ID
**Media:**
* `thumbnail` - Thumbnail URL
* `thumbnail_id` - Thumbnail ID
* `video_id` - Video ID
* `pictures` - Array of item pictures
* `id` - Picture ID
* `url` - Picture URL
* `secure_url` - Secure picture URL
* `size` - Picture size
* `max_size` - Max picture size
* `quality` - Picture quality
**Seller & Store:**
* `official_store_id` - Official store ID
**Shipping:**
* `shipping` - Shipping configuration
* `store_pick_up` - Store pickup available
* `local_pick_up` - Local pickup available
* `free_shipping` - Free shipping enabled
* `logistic_type` - Logistic type (fulfillment, xd\_drop\_off, etc.)
* `mode` - Shipping mode
* `tags` - Shipping tags
* `methods` - Available shipping methods
* `dimensions` - Package dimensions
* `international_delivery_mode` - International delivery mode
**Location:**
* `seller_address` - Seller's address
* `address_line` - Address line
* `zip_code` - ZIP code
* `city` - City (id, name)
* `state` - State (id, name)
* `country` - Country (id, name)
* `latitude` - Latitude
* `longitude` - Longitude
* `search_location` - Search location (neighborhood, city, state)
* `geolocation` - Geolocation coordinates (latitude, longitude)
**Attributes:**
* `attributes` - Array of item attributes
* `id` - Attribute ID
* `name` - Attribute name
* `value_id` - Value ID
* `value_name` - Value name
* `attribute_group_id` - Attribute group ID
* `attribute_group_name` - Attribute group name
* `value_struct` - Structured value (number, unit)
* `values` - Values array
* `source` - Attribute source
* `value_type` - Value type
**Sale Terms:**
* `sale_terms` - Array of sale terms
* `id` - Term ID
* `name` - Term name
* `value_id` - Value ID
* `value_name` - Value name
* `value_type` - Value type
* `values` - Values array (id, name)
**Variations:**
* `variations` - Array of product variations
* `id` - Variation ID
* `price` - Variation price
* `available_quantity` - Available quantity for this variation
* `sold_quantity` - Sold quantity for this variation
* `inventory_id` - Inventory ID
* `user_product_id` - User product ID
* `catalog_product_id` - Catalog product ID
* `seller_custom_field` - Seller custom field
* `picture_ids` - Picture IDs array
* `attribute_combinations` - Attribute combinations array (id, name, value\_id, value\_name, value\_type, values)
* `sale_terms` - Sale terms for this variation
* `item_relations` - Related items (id, variation\_id, stock\_relation)
**Additional:**
* `accepts_mercadopago` - Accepts MercadoPago
* `sub_status` - Sub-status array
* `tags` - Item tags array
* `warranty` - Warranty information
* `deal_ids` - Active deal IDs
* `automatic_relist` - Whether item is auto-relisted
* `item_relations` - Related items array (id, variation\_id, stock\_relation)
* `channels` - Sales channels array
* `listing_source` - Listing source
* `descriptions` - Item descriptions
* `non_mercado_pago_payment_methods` - Non-MercadoPago payment methods
* `coverage_areas` - Coverage areas
* `warnings` - Item warnings
* `differential_pricing` - Differential pricing info
Customer experience and reputation metrics for each item. This stream is a child of the Items stream.
**Key Fields:**
* `item_id` - The item identifier
**Status:**
* `status` - Current status (id)
* `freeze` - Freeze information (text)
* `title` - Title information (text)
**Reputation:**
* `reputation` - Overall reputation
* `color` - Reputation color indicator
* `text` - Reputation text
* `value` - Reputation score
**Subtitles:**
* `subtitles` - Array of subtitle information
* `order` - Display order
* `text` - Subtitle text
* `placeholders` - Placeholder values
**Actions:**
* `actions` - Available actions array
* `order` - Action order
* `text` - Action text
**Metrics Details:**
* `metrics_details` - Detailed metrics breakdown
* `problems` - Array of problems
* `order` - Problem order
* `key` - Problem key
* `color` - Problem severity color
* `quantity` - Quantity string
* `cancellations` - Number of cancellations
* `claims` - Number of claims
* `tag` - Problem tag
* `level_two` - Level two details (key, title)
* `level_three` - Level three details (key, title, remedy)
* `distribution` - Rating distribution
* `from` - Period start
* `to` - Period end
* `level_one` - Level one breakdown array
* `key` - Key
* `title` - Title (order, text)
* `color` - Color
* `percentage` - Percentage
* `quantities_level_two` - Level two quantities
Visit statistics for each item over time. This stream is a child of the Items stream.
**Key Fields:**
* `item_id` - The item identifier
* `date_from` - Period start date (replication key)
* `date_to` - Period end date
**Metrics:**
* `total_visits` - Total number of visits in the period
**Visit Details:**
* `visits_detail` - Breakdown by source
* `company` - Source/company name
* `quantity` - Number of visits from this source
Quality score and performance metrics for each item. This stream is a child of the Items stream.
Quality score is only available for non-catalog items. Items linked to the Mercado Livre catalog will be skipped.
**Key Fields:**
* `entity_type` - Entity type
* `entity_id` - Entity identifier (item ID)
* `score` - Overall quality score
* `level` - Quality level
* `level_wording` - Quality level description
* `calculated_at` - Calculation timestamp (replication key)
**Buckets:**
* `buckets` - Quality score buckets array
* `key` - Bucket key
* `type` - Bucket type
* `status` - Bucket status
* `score` - Bucket score
* `title` - Bucket title
* `calculated_at` - Calculation timestamp
* `variables` - Variables array
* `key` - Variable key
* `status` - Variable status
* `score` - Variable score
* `calculated_at` - Calculation timestamp
* `title` - Variable title
* `rules` - Rules array
* `key` - Rule key
* `status` - Rule status
* `progress` - Progress value
* `mode` - Rule mode
* `calculated_at` - Calculation timestamp
* `wordings` - Rule wordings (title, label, link)
Product Ads campaign configurations and settings. Only available for accounts with an Advertiser ID.
**Key Fields:**
* `id` - Campaign ID
* `name` - Campaign name
* `status` - Campaign status (active, paused, etc.)
* `date_created` - Creation timestamp
* `last_updated` - Last update timestamp
**Budget & Bidding:**
* `budget` - Daily budget amount
* `currency_id` - Budget currency
* `acos_target` - Target ACoS (Advertising Cost of Sale)
* `acos_top_search_target` - Target ACoS for top search positions
* `strategy` - Bidding strategy
**Channel:**
* `channel` - Campaign channel
Daily performance metrics for Product Ads campaigns. This stream is a child of Product Ads Campaigns.
**Key Fields:**
* `campaign_id` - Campaign identifier
* `date` - Metrics date (replication key)
**Traffic Metrics:**
* `prints` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `cpc` - Cost per click
* `cost` - Total advertising cost
**Conversion Metrics:**
* `direct_units_quantity` - Units sold from direct clicks
* `indirect_units_quantity` - Units sold from indirect attribution
* `units_quantity` - Total units sold
* `direct_amount` - Revenue from direct conversions
* `indirect_amount` - Revenue from indirect conversions
* `total_amount` - Total attributed revenue
* `direct_items_quantity` - Items sold from direct clicks
* `indirect_items_quantity` - Items sold from indirect attribution
* `advertising_items_quantity` - Total items from advertising
**Organic Metrics:**
* `organic_units_quantity` - Organic units sold
* `organic_units_amount` - Organic revenue
* `organic_items_quantity` - Organic items sold
**Efficiency Metrics:**
* `acos` - Advertising Cost of Sale (cost/revenue)
* `cvr` - Conversion rate
* `roas` - Return on Ad Spend
**Share of Voice:**
* `sov` - Share of Voice
* `impression_share` - Impression share
* `top_impression_share` - Top position impression share
* `lost_impression_share_by_budget` - Lost impressions due to budget
* `lost_impression_share_by_ad_rank` - Lost impressions due to ad rank
**Benchmark:**
* `acos_benchmark` - ACoS benchmark for comparison
Brand Ads campaign configurations and settings. Only available for accounts with an Advertiser ID.
**Key Fields:**
* `campaign_id` - Campaign ID
* `name` - Campaign name
* `status` - Campaign status
* `moderation_status` - Content moderation status
* `campaign_type` - Campaign type
* `start_date` - Campaign start date
* `end_date` - Campaign end date
**Account:**
* `advertiser_id` - Advertiser ID
* `site_id` - Site ID
* `eshop_id` - E-shop ID
* `official_store_id` - Official store ID
* `destination_id` - Destination ID
**Creative:**
* `headline` - Campaign headline
**Budget & Bidding:**
* `budget` - Budget information
* `amount` - Budget amount
* `currency` - Currency code
* `cpc` - Cost per click bid
**Items:**
* `items` - Array of campaign items
* `campaign_id` - Campaign ID
* `status` - Item status
* `item_id` - Item ID
**Keywords:**
* `keywords` - Array of campaign keywords
* `campaign_id` - Campaign ID
* `type` - Keyword type
* `term` - Keyword term
* `match_type` - Match type (exact, phrase, broad)
* `is_negative` - Is negative keyword
* `cpc` - Keyword CPC bid
Daily performance metrics for Brand Ads campaigns. This stream is a child of Brand Ads Campaigns.
**Key Fields:**
* `campaign_id` - Campaign identifier
* `date` - Metrics date (replication key)
**Traffic Metrics:**
* `prints` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `cost_per_clicks` - Cost per click (CPC)
* `consumed_budget` - Budget consumed
**Efficiency Metrics:**
* `cvr` - Conversion rate
* `acos` - Advertising Cost of Sale
* `roas` - Return on Ad Spend
**Attribution Metrics:**
* `attribution_order_conversions` - Number of orders attributed to the campaign
* `attribution_order_amount` - Revenue attributed to the campaign
* `leads` - Number of leads generated
# Data Model
The following diagram illustrates the relationships between the core data streams in Mercado Livre. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Seller Data"
Orders("Orders");
Shipments("Shipments");
Invoices("Invoices");
BillingInfo("Billing Info");
ItemsList("Items List");
Items("Items");
end
subgraph "Item Metrics"
VisitsPerItem("Visits Per Item");
QualityScore("Quality Score Per Item");
BuyerExperience("Buyer Experience Per Item");
end
subgraph "Product Ads"
ProductAdsCampaigns("Product Ads Campaigns");
ProductAdsMetrics("Product Ads Campaign Metrics");
end
subgraph "Brand Ads"
BrandAdsCampaigns("Brand Ads Campaigns");
BrandAdsMetrics("Brand Ads Campaign Metrics");
end
Orders -- "shipping_id" --> Shipments;
Orders -- "order_id" --> Invoices;
Orders -- "per order" --> BillingInfo;
ItemsList -- "id_list batches" --> Items;
Items -- "item_id" --> VisitsPerItem;
Items -- "item_id" --> QualityScore;
Items -- "item_id" --> BuyerExperience;
Items -- "item.id" --> Orders;
ProductAdsCampaigns -- "campaign_id" --> ProductAdsMetrics;
BrandAdsCampaigns -- "campaign_id" --> BrandAdsMetrics;
BrandAdsCampaigns -- "item_id" --> Items;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Mercado Livre data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Product Ads Campaign Performance Overview
Analyze the performance of your Product Ads campaigns to understand ROI and optimize budget allocation.
**Business Value:**
* Identify top-performing campaigns by ROAS and ACoS
* Understand which campaigns drive the most conversions
* Optimize budget distribution across campaigns
```sql theme={null}
WITH campaign_summary AS (
SELECT
c.id AS campaign_id,
c.name AS campaign_name,
c.status AS campaign_status,
c.strategy,
c.acos_target,
SUM(m.prints) AS total_impressions,
SUM(m.clicks) AS total_clicks,
SUM(m.cost) AS total_cost,
SUM(m.total_amount) AS total_revenue,
SUM(m.units_quantity) AS total_units_sold,
AVG(m.ctr) AS avg_ctr,
AVG(m.cvr) AS avg_cvr
FROM
nekt_raw.mercadolivre_product_ads_campaigns c
LEFT JOIN nekt_raw.mercadolivre_product_ads_campaign_metrics m
ON c.id = m.campaign_id
WHERE
m.date >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
c.id, c.name, c.status, c.strategy, c.acos_target
)
SELECT
campaign_name,
campaign_status,
strategy,
total_impressions,
total_clicks,
ROUND(total_cost, 2) AS total_cost,
ROUND(total_revenue, 2) AS total_revenue,
total_units_sold,
ROUND(avg_ctr * 100, 2) AS ctr_percent,
ROUND(avg_cvr * 100, 2) AS cvr_percent,
ROUND(CASE WHEN total_revenue > 0 THEN total_cost / total_revenue * 100 ELSE 0 END, 2) AS acos_percent,
ROUND(acos_target * 100, 2) AS target_acos_percent,
ROUND(CASE WHEN total_cost > 0 THEN total_revenue / total_cost ELSE 0 END, 2) AS roas
FROM
campaign_summary
ORDER BY
total_revenue DESC
```
```sql theme={null}
WITH campaign_summary AS (
SELECT
c.id AS campaign_id,
c.name AS campaign_name,
c.status AS campaign_status,
c.strategy,
c.acos_target,
SUM(m.prints) AS total_impressions,
SUM(m.clicks) AS total_clicks,
SUM(m.cost) AS total_cost,
SUM(m.total_amount) AS total_revenue,
SUM(m.units_quantity) AS total_units_sold,
AVG(m.ctr) AS avg_ctr,
AVG(m.cvr) AS avg_cvr
FROM
`nekt_raw.mercadolivre_product_ads_campaigns` c
LEFT JOIN `nekt_raw.mercadolivre_product_ads_campaign_metrics` m
ON c.id = m.campaign_id
WHERE
DATE(m.date) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
c.id, c.name, c.status, c.strategy, c.acos_target
)
SELECT
campaign_name,
campaign_status,
strategy,
total_impressions,
total_clicks,
ROUND(total_cost, 2) AS total_cost,
ROUND(total_revenue, 2) AS total_revenue,
total_units_sold,
ROUND(avg_ctr * 100, 2) AS ctr_percent,
ROUND(avg_cvr * 100, 2) AS cvr_percent,
ROUND(SAFE_DIVIDE(total_cost, total_revenue) * 100, 2) AS acos_percent,
ROUND(acos_target * 100, 2) AS target_acos_percent,
ROUND(SAFE_DIVIDE(total_revenue, total_cost), 2) AS roas
FROM
campaign_summary
ORDER BY
total_revenue DESC
```
| campaign\_name | campaign\_status | strategy | total\_impressions | total\_clicks | total\_cost | total\_revenue | total\_units\_sold | ctr\_percent | cvr\_percent | acos\_percent | target\_acos\_percent | roas |
| - | - | - | - | - | - | - | - | - | - | - | - | - |
| Top Products Campaign | active | automatic | 1,245,890 | 42,156 | 8,542.30 | 42,800.00 | 856 | 3.38 | 2.03 | 19.96 | 20.00 | 5.01 |
| Electronics Promo | active | manual | 892,340 | 31,245 | 4,998.50 | 21,150.00 | 423 | 3.50 | 1.35 | 23.64 | 25.00 | 4.23 |
| Seasonal Sale | paused | automatic | 567,230 | 18,456 | 3,691.20 | 12,670.00 | 289 | 3.25 | 1.57 | 29.13 | 30.00 | 3.43 |
### 2. Item Quality Score Analysis
Monitor the quality scores of your listings to improve visibility and sales performance.
**Business Value:**
* Identify items with low quality scores that need improvement
* Track quality metrics over time
* Prioritize optimization efforts
```sql theme={null}
SELECT
i.id AS item_id,
i.title AS item_title,
i.price,
i.available_quantity,
q.score AS quality_score,
q.level AS quality_level,
q.level_wording AS quality_description,
q.calculated_at AS last_calculated
FROM
nekt_raw.mercadolivre_items i
LEFT JOIN nekt_raw.mercadolivre_quality_score_per_item q
ON i.id = q.entity_id
WHERE
q.score IS NOT NULL
ORDER BY
q.score ASC
LIMIT 50
```
```sql theme={null}
SELECT
i.id AS item_id,
i.title AS item_title,
i.price,
i.available_quantity,
q.score AS quality_score,
q.level AS quality_level,
q.level_wording AS quality_description,
q.calculated_at AS last_calculated
FROM
`nekt_raw.mercadolivre_items` i
LEFT JOIN `nekt_raw.mercadolivre_quality_score_per_item` q
ON i.id = q.entity_id
WHERE
q.score IS NOT NULL
ORDER BY
q.score ASC
LIMIT 50
```
| item\_id | item\_title | price | available\_quantity | quality\_score | quality\_level | quality\_description | last\_calculated |
| - | - | - | - | - | - | - | - |
| MLB12345678 | Smartphone Samsung Galaxy | 1299.00 | 45 | 72 | medium | Bom | 2024-11-27T10:30:00Z |
| MLB23456789 | Notebook Dell Inspiron | 3499.00 | 12 | 85 | high | Muito bom | 2024-11-27T10:25:00Z |
| MLB34567890 | Fone de Ouvido Bluetooth | 89.90 | 230 | 58 | low | Regular | 2024-11-27T10:20:00Z |
### 3. Sales and Revenue Analysis
Analyze order data to understand sales performance and payment methods.
**Business Value:**
* Track revenue trends over time
* Understand payment method preferences
* Identify top-selling items
```sql theme={null}
WITH order_items_exploded AS (
SELECT
o.id AS order_id,
o.date_created,
o.status AS order_status,
o.total_amount,
o.paid_amount,
o.currency_id,
oi.item.id AS item_id,
oi.item.title AS item_title,
oi.quantity,
oi.unit_price,
oi.sale_fee
FROM
nekt_raw.mercadolivre_orders o
CROSS JOIN UNNEST(o.order_items) AS oi
WHERE
o.date_created >= CURRENT_DATE - INTERVAL '30' DAY
)
SELECT
DATE(date_created) AS order_date,
COUNT(DISTINCT order_id) AS total_orders,
SUM(quantity) AS total_units_sold,
ROUND(SUM(total_amount), 2) AS total_revenue,
ROUND(SUM(sale_fee), 2) AS total_fees,
ROUND(AVG(total_amount), 2) AS avg_order_value
FROM
order_items_exploded
WHERE
order_status = 'paid'
GROUP BY
DATE(date_created)
ORDER BY
order_date DESC
```
```sql theme={null}
WITH order_items_exploded AS (
SELECT
o.id AS order_id,
o.date_created,
o.status AS order_status,
o.total_amount,
o.paid_amount,
o.currency_id,
oi.item.id AS item_id,
oi.item.title AS item_title,
oi.quantity,
oi.unit_price,
oi.sale_fee
FROM
`nekt_raw.mercadolivre_orders` o
CROSS JOIN UNNEST(o.order_items) AS oi
WHERE
DATE(o.date_created) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
)
SELECT
DATE(date_created) AS order_date,
COUNT(DISTINCT order_id) AS total_orders,
SUM(quantity) AS total_units_sold,
ROUND(SUM(total_amount), 2) AS total_revenue,
ROUND(SUM(sale_fee), 2) AS total_fees,
ROUND(AVG(total_amount), 2) AS avg_order_value
FROM
order_items_exploded
WHERE
order_status = 'paid'
GROUP BY
DATE(date_created)
ORDER BY
order_date DESC
```
| order\_date | total\_orders | total\_units\_sold | total\_revenue | total\_fees | avg\_order\_value |
| - | - | - | - | - | - |
| 2024-11-27 | 156 | 234 | 45,678.90 | 5,481.47 | 292.81 |
| 2024-11-26 | 142 | 198 | 38,456.20 | 4,614.74 | 270.82 |
| 2024-11-25 | 128 | 176 | 32,890.50 | 3,946.86 | 256.96 |
## Implementation Notes
### Data Quality Considerations
* **Advertising Streams**: Product Ads and Brand Ads streams are only available if your Mercado Livre account has an advertiser ID. Accounts without advertising access still get marketplace streams (orders, items, metrics, fiscal helpers, etc.) but not the advertising campaign tables.
* **Invoices**: Emitted per order when Mercado Livre returns invoice data; orders without an invoice (HTTP 404) are skipped with a log warning. Invoice payloads are Brazil-centric (NF-e style fields and Sefaz metadata).
* **Billing Info**: One request per order; HTTP 403 when the credential cannot read billing data for that order (skipped with a warning). Join child rows back to orders using your pipeline’s parent/child lineage fields if the raw table does not repeat `order_id`.
* **Shipments**: Only orders with a `shipping.id` produce shipment rows; the stream supplies `logistic_type` and tracking fields from the Shipments API.
* **Items List**: Optional catalog table of scan batches; the **Items** stream depends on it for ID discovery. Leave **Items List** off unless you need the intermediate batches as data.
* **Quality Score Availability**: Quality scores are only available for non-catalog items. Items linked to the Mercado Livre product catalog will be skipped during extraction.
* **Metrics Time Window**: Campaign metrics are available for the last 90 days only.
### API Limits & Performance
* **Metrics Granularity Impact**: The more granular the metrics setting (day vs. month), the longer extractions will take due to increased API calls.
* **Order Child Streams**: **Shipments**, **Invoices**, and **Billing Info** each add per-order API calls on top of the base Orders extraction. Disable any you do not need to reduce duration and rate-limit pressure.
* **Visit Data Limit**: Visit statistics are limited to the last 2 years of data.
* **Pagination**: Large seller accounts with many items may experience longer extraction times.
### Regional Considerations
* **Currency**: All financial values are in the local currency of the site (BRL for MLB, ARS for MLA, MXN for MLM).
* **Timestamps**: All timestamps are in UTC.
## Troubleshooting
History windows are set by Mercado Livre. No Nekt setting widens them, and a full sync does not bring back data outside them.
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| `Key is before time window 12 months` | The Mercado Livre billing API only returns 12 months of history. | Move the stream's initial sync date inside the 12-month window. |
| The questions stream fails on high-volume accounts | The API caps how deep offset pagination can go. | Narrow the extraction window for that stream by splitting the period. |
| `atLeastOneFilterProvided` | The claims search requires at least one filter. | Fill in a filter in the stream settings. |
| Product Ads has no history beyond a few months | Backfill for that API is limited to 90 days. | Plan the analysis around the window. A full sync does not retrieve data from before it. |
| Credentials become invalid after a change on the account | The application or the token was revoked on the Mercado Livre side. | Re-authorize the source. |
### History window by stream
| Stream group | Window available from the provider |
| :- | :- |
| Campaign metrics (Product Ads, Brand Ads) | Last 90 days |
| Billing | Last 12 months |
| Visit statistics | Last 2 years |
| Orders and marketplace streams | Not documented by the provider |
## Skills for agents
Mercado Livre connector documentation as plain markdown, for use in AI agent contexts.
# Mercos as a data source
Source: https://docs.nekt.com/sources/mercos
Bring data from Mercos to Nekt.
Mercos is a B2B sales platform that helps companies manage their sales force, orders, and customer relationships. It provides tools for sales teams to manage their catalogs, pricing, and customer orders efficiently.
## Configuring Mercos as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Mercos option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide the following credentials to connect to Mercos:
* **Company Token**: The company token to authenticate against the API service. It can be found on `Minha Conta` -> `Sistema` -> `Integração` -> `Seu Company Token para integração`.
* **Start Date**: The earliest record date to sync (records created or updated after this date will be extracted).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Mercos and their corresponding fields:
Stream for managing customer information.
Primary key: `id`
Replication key: `ultima_alteracao`
* `bairro` - Neighborhood
* `bloqueado` - Whether the client is blocked
* `bloqueado_b2b` - Whether blocked in B2B
* `cep` - Postal code
* `cidade` - City
* `cnpj` - Company registration number
* `complemento` - Address complement
* `contatos` - Contact information containing:
* `cargo` - Position/role
* `emails` - List of email addresses
* `excluido` - Whether deleted
* `id` - Contact ID
* `nome` - Contact name
* `telefones` - List of phone numbers
* `emails` - List of email addresses
* `estado` - State
* `excluido` - Whether the client is deleted
* `extras` - Extra information containing:
* `campo_extra_id` - Extra field ID
* `nome` - Field name
* `nome_arquivo` - File name
* `tipo` - Field type
* `valor` - Field value
* `valor_arquivo` - File value
* `valor_data` - Date value
* `valor_decimal` - Decimal value
* `valor_texto` - Text value
* `id` - The client's system ID
* `inscricao_estadual` - State registration number
* `nome_fantasia` - Company trade name
* `numero` - Street number
* `observacao` - Additional observations
* `razao_social` - Company legal name
* `rua` - Street address
* `suframa` - SUFRAMA registration number
* `telefones` - List of phone numbers
* `tipo` - Client type
* `ultima_alteracao` - Last modification timestamp
Stream for managing sales orders.
Primary key: `id`
Replication key: `ultima_alteracao`
* `cliente_id` - Client ID
* `condicao_pagamento` - Payment terms
* `condicao_pagamento_id` - Payment terms ID
* `criador_id` - Creator ID
* `cupom_de_desconto` - Discount coupon code
* `data_criacao` - Creation date
* `data_emissao` - Issue date
* `endereco_entrega` - Delivery address containing:
* `bairro` - Neighborhood
* `cep` - Postal code
* `cidade` - City
* `complemento` - Address complement
* `endereco` - Street address
* `estado` - State
* `id` - Address ID
* `numero` - Street number
* `extras` - Extra fields containing:
* `campo_extra_id` - Extra field ID
* `nome` - Field name
* `nome_arquivo` - File name
* `tipo` - Field type
* `valor` - Field value
* `valor_arquivo` - File value
* `valor_data` - Date value
* `valor_decimal` - Decimal value
* `valor_texto` - Text value
* `forma_pagamento_id` - Payment method ID
* `id` - Order ID
* `itens` - Order items containing:
* `cotacao_moeda` - Currency quote
* `desconto_de_cupom` - Coupon discount
* `descontos_de_politicas` - Policy discounts (regra\_id, desconto)
* `descontos_de_promocoes` - Promotional discounts (regra\_id, desconto)
* `descontos_do_vendedor` - Seller discounts
* `excluido` - Whether deleted
* `id` - Item ID
* `ipi` - IPI tax value
* `observacoes` - Item observations
* `preco_liquido` - Net price
* `preco_tabela` - List price
* `produto_id` - Product ID
* `quantidade` - Quantity
* `st` - ST tax value
* `subtotal` - Subtotal
* `tabela_preco_id` - Price table ID
* `tipo_ipi` - IPI tax type
* `nome_contato` - Contact name
* `numero` - Order number
* `observacoes` - Order observations
* `status` - Order status
* `status_b2b` - B2B status
* `status_faturamento` - Billing status
* `tipo_pedido_id` - Order type ID
* `total` - Order total amount
* `transportadora_id` - Shipping company ID
* `transportadora_nome` - Shipping company name
* `ultima_alteracao` - Last modification date
Stream for managing product catalog.
Primary key: `id`
Replication key: `ultima_alteracao`
* `altura` - Height
* `ativo` - Whether the product is active
* `codigo` - Product code
* `comissao` - Commission percentage
* `comprimento` - Length
* `excluido` - Whether the product is deleted
* `exibir_no_b2b` - Whether to display in B2B
* `id` - The product's system ID
* `ipi` - IPI tax value
* `largura` - Width
* `moeda` - Currency code
* `multiplo` - Multiple quantity for orders
* `nome` - The product's name
* `observacoes` - Product observations and details
* `peso_bruto` - Gross weight
* `peso_dimensoes_unitario` - Unit weight dimensions
* `preco_minimo` - Minimum price
* `preco_tabela` - List price
* `precos_especificos` - Whether has specific prices
* `saldo_estoque` - Stock balance
* `st` - ST tax value
* `tipo_ipi` - IPI tax type
* `ultima_alteracao` - Last modification timestamp
* `unidade` - Unit of measurement
## Data Model
The following diagram illustrates the relationships between the core data streams in Mercos. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
Orders --> Clients;
Orders --> Products;
Orders -- "cliente_id" --> Clients;
Orders -- "produto_id" --> Products;
```
## Skills for agents
Mercos connector documentation as plain markdown, for use in AI agent contexts.
# Microsoft Ads as a data source
Source: https://docs.nekt.com/sources/microsoft-ads
Bring data from Microsoft Ads to Nekt.
Microsoft Ads (formerly Bing Ads) is Microsoft's advertising platform that allows businesses to create and manage digital advertising campaigns across the Microsoft Search Network, including Bing, Yahoo, and partner sites. It provides tools for targeting audiences, tracking performance, and optimizing ad spend for maximum return on investment.
![]()
## Configuring Microsoft Ads as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Microsoft Ads option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Microsoft Ads data. Click on the `Microsoft Authorization` button and log in with your Microsoft account. Grant the necessary permissions for the ad accounts you want to extract data from.
If you are already logged in on a Microsoft Ads account that is different from the one you want to sync data from, make sure to switch to the correct account before authorizing Nekt.
The following configurations are required:
* **Manager Account ID**: The Manager Account ID identifies the specific manager account you want to sync data from. See the [Finding Your Manager Account ID and Customer ID](#finding-your-manager-account-id-and-customer-id) section below for detailed instructions.
* **Customer ID**: The Customer ID identifies your account in Microsoft Ads. This is different from the Manager Account ID and is required for API authentication. See the [Finding Your Manager Account ID and Customer ID](#finding-your-manager-account-id-and-customer-id) section below for detailed instructions.
* **Report Timezone**: Select the timezone to use for report generation. This determines how dates and times are recorded in your performance reports.
* **Start Date**: (Optional) The earliest date from which records will be synced. Records created or updated after this date will be extracted. For report extractions on full syncs, this date is honored; if left blank, the window will default to one year ago.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Finding Your Manager Account ID and Customer ID
To connect Microsoft Ads to Nekt, you'll need two important identifiers: your **Manager Account ID** and your **Customer ID**. Here's how to find them:
### Step-by-Step Guide
Go to [ads.microsoft.com](https://ads.microsoft.com) and sign in with your Microsoft account.
Once logged in, click on **Settings** in the left menu.
Select **Account settings** in the left menu, in page you'll see your Customer ID. It's a numeric identifier (e.g., `12345678`).
Alternatively, you can find the Customer ID in the URL when you're viewing a specific account. Look for the `cid=` parameter in the URL.
Select **Manager account settings** in the left menu, in page you'll see your Manager account ID. It's a numeric identifier (e.g., `12345678`).
Alternatively, you can find the Account ID in the URL when you're viewing a specific account. Look for the `aid=` parameter in the URL.
# Streams and Fields
Below you'll find all available data streams from Microsoft Ads and their corresponding fields:
Stream for retrieving account information and settings.
**Key Fields:**
* `Id` - Unique identifier for the account
* `Name` - Name of the account
* `Number` - Account number
* `CurrencyCode` - Account currency code
* `AccountLifeCycleStatus` - Lifecycle status of the account
* `AccountFinancialStatus` - Financial status of the account
* `TimeZone` - Account timezone
**Account Details:**
* `ParentCustomerId` - Parent customer ID
* `PrimaryUserId` - The primary user ID for the account
* `BillToCustomerId` - Bill to customer ID
* `PaymentMethodId` - Payment method ID
* `PaymentMethodType` - Type of payment method
**Business Information:**
* `BusinessAddress` - Business address details including city, country, postal code, and state
* `BillingThresholdAmount` - Billing threshold amount
* `Language` - Preferred language of the account
**Metadata:**
* `LastModifiedByUserId` - ID of the user who last modified the account
* `LastModifiedTime` - The last modified timestamp
* `TimeStamp` - The time stamp of the account
* `PauseReason` - Reason the account was paused (if applicable)
Stream for managing ad campaigns and their settings.
**Key Fields:**
* `id` - Unique identifier for the campaign
* `name` - Name of the campaign
* `status` - Campaign status
* `campaign_type` - Type of campaign
* `sub_type` - Campaign sub type
* `parent_id` - Account ID
**Budget Settings:**
* `budget` - Budget amount
* `budget_id` - Budget ID
* `budget_name` - Budget name
* `budget_type` - Budget type
**Bid Strategy Settings:**
* `bid_strategy_type` - Bid strategy type
* `bid_strategy_id` - Bid strategy ID
* `bid_strategy_name` - Bid strategy name
* `bid_strategy_maxcpc` - Max CPC for bid strategy
* `bid_strategy_targetcpa` - Target CPA for bid strategy
* `bid_strategy_targetroas` - Target ROAS for bid strategy
* `bid_strategy_targetadposition` - Target ad position
* `bid_strategy_targetimpressionshare` - Target impression share
* `bid_adjustment` - Bid adjustment
* `multi_media_ad_bid_adjustment` - Multi media ad bid adjustment
**Targeting & Language:**
* `language` - Language
* `target_setting` - Target setting
* `country_code` - Country code
**URLs & Tracking:**
* `tracking_template` - Tracking template
* `custom_parameter` - Custom parameters
* `final_url_suffix` - Final URL suffix
* `url_expansion_opt_out` - URL expansion opt out
**Dynamic Search Ads:**
* `website` - Website domain for dynamic search ads
* `domain_language` - Domain language for dynamic search ads
* `dynamic_description_enabled` - Dynamic description enabled
* `source` - Source for dynamic search ads
* `page_feed_ids` - Page feed IDs
* `feed_label` - Feed label
**Quality Metrics:**
* `keyword_relevance` - Keyword relevance score
* `landing_page_relevance` - Landing page relevance score
* `landing_page_user_experience` - Landing page user experience score
* `quality_score` - Quality score
**Additional Settings:**
* `modified_time` - Last modified time
* `experiment_id` - Experiment ID
* `priority` - Campaign priority
* `store_id` - Store ID
Stream for managing ad groups, which control targeting, budget, and scheduling for a group of ads.
**Key Fields:**
* `id` - Unique identifier for the ad group
* `name` - Name of the ad group
* `parent_id` - Campaign ID
* `campaign` - Campaign name
* `status` - Ad group status
* `ad_group_type` - Ad group type (SearchStandard, SearchDynamic, HotelAds)
**Scheduling:**
* `start_date` - Start date
* `end_date` - End date
* `ad_schedule_use_searcher_time_zone` - Use searcher time zone
**Network & Distribution:**
* `network_distribution` - Network distribution
* `ad_rotation` - Ad rotation
* `privacy_status` - Privacy status
**Bidding:**
* `cpc_bid` - CPC bid amount
* `bid_strategy_type` - Bid strategy type
* `inherited_bid_strategy_type` - Inherited bid strategy type
* `bid_boost_value` - Bid boost value
* `bid_option` - Bid option
* `maximum_bid` - Maximum bid
* `bid_adjustment` - Bid adjustment
**Targeting:**
* `language` - Language
* `target_setting` - Target setting
* `use_predictive_targeting` - Use predictive targeting
**URLs & Tracking:**
* `tracking_template` - Tracking template
* `custom_parameter` - Custom parameters
* `final_url_suffix` - Final URL suffix
* `frequency_cap_settings` - Frequency cap settings
**Quality Metrics:**
* `keyword_relevance` - Keyword relevance score
* `landing_page_relevance` - Landing page relevance score
* `landing_page_user_experience` - Landing page user experience score
* `quality_score` - Quality score
**Metadata:**
* `modified_time` - Last modified time
* `client_id` - Client ID for tracking
Stream for managing individual ads within an ad set. Supports multiple ad types including Responsive Ads, Responsive Search Ads, Expanded Text Ads, Dynamic Search Ads, Product Ads, App Install Ads, and Text Ads.
**Key Fields:**
* `id` - Unique identifier for the ad
* `type` - Ad type (Responsive Ad, Responsive Search Ad, etc.)
* `status` - Ad status
* `parent_id` - Ad group ID
* `campaign` - Campaign name
* `ad_group` - Ad group name
**Editorial Fields:**
* `editorial_status` - Editorial status
* `editorial_appeal_status` - Editorial appeal status
* `editorial_location` - Editorial location
* `editorial_reason_code` - Editorial reason code
* `editorial_term` - Editorial term
* `publisher_countries` - Publisher countries
**URLs & Tracking:**
* `final_url` - Final URL
* `mobile_final_url` - Mobile final URL
* `tracking_template` - Tracking template
* `custom_parameter` - Custom parameters
* `final_url_suffix` - Final URL suffix
* `destination_url` - Destination URL
* `display_url` - Display URL
**Text Ad Fields:**
* `title` - Title (Text Ad, App Install Ad)
* `title_part_1` - Title part 1 (Expanded Text Ad)
* `title_part_2` - Title part 2 (Expanded Text Ad)
* `title_part_3` - Title part 3 (Expanded Text Ad)
* `text` - Ad text/description
* `text_part_2` - Second description
* `path_1` - Path 1
* `path_2` - Path 2
* `domain` - Domain
**Responsive Search Ad Fields:**
* `headlines` - Headlines JSON
* `descriptions` - Descriptions JSON
**Responsive Ad Fields:**
* `business_name` - Business name
* `call_to_action` - Call to action
* `headline` - Headline
* `long_headline` - Long headline
* `long_headlines` - Long headlines JSON
* `images` - Images JSON
* `landscape_image_media_id` - Landscape image media ID
* `square_image_media_id` - Square image media ID
* `landscape_logo_media_id` - Landscape logo media ID
* `square_logo_media_id` - Square logo media ID
**App Install Ad Fields:**
* `app_id` - App ID
* `app_platform` - App platform
**Metadata:**
* `modified_time` - Last modified time
* `device_preference` - Device preference
* `ad_format_preference` - Ad format preference
Stream for managing keywords within ad groups.
**Key Fields:**
* `id` - Unique identifier for the keyword
* `keyword` - Keyword text
* `match_type` - Match type (Exact, Phrase, Broad)
* `status` - Keyword status
* `parent_id` - Ad group ID
* `campaign` - Campaign name
* `ad_group` - Ad group name
**Editorial Fields:**
* `editorial_status` - Editorial status
* `editorial_appeal_status` - Editorial appeal status
* `editorial_location` - Editorial location
* `editorial_reason_code` - Editorial reason code
* `editorial_term` - Editorial term
* `publisher_countries` - Publisher countries
**Bidding:**
* `bid` - Bid amount
* `bid_strategy_type` - Bid strategy type
* `inherited_bid_strategy_type` - Inherited bid strategy type
**Dynamic Text Parameters:**
* `param1` - Dynamic text parameter 1
* `param2` - Dynamic text parameter 2
* `param3` - Dynamic text parameter 3
**URLs & Tracking:**
* `final_url` - Final URL
* `mobile_final_url` - Mobile final URL
* `tracking_template` - Tracking template
* `custom_parameter` - Custom parameters
* `final_url_suffix` - Final URL suffix
* `destination_url` - Destination URL (deprecated)
**Quality Metrics:**
* `quality_score` - Quality score
* `keyword_relevance` - Keyword relevance
* `landing_page_relevance` - Landing page relevance
* `landing_page_user_experience` - Landing page user experience
**Metadata:**
* `modified_time` - Last modified time
* `client_id` - Client ID for tracking
Daily performance report for campaigns, including impressions, clicks, spend, conversions, and quality metrics.
**Primary Keys:** `time_period`, `campaign_id`, `device_type`, `network`
**Time & Identifiers:**
* `time_period` - Time period of the report (YYYY-MM-DD format)
* `account_id` - Account ID
* `account_name` - Account name
* `campaign_id` - Campaign ID
* `campaign_name` - Campaign name
* `campaign_type` - Campaign type
* `campaign_status` - Campaign status
**Dimensions:**
* `device_type` - Device type (Computer, Smartphone, Tablet)
* `network` - Network (Bing and Yahoo! search, Syndicated search partners)
**Performance Metrics:**
* `impressions` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `average_cpc` - Average cost per click
* `spend` - Total spend
* `average_position` - Average ad position
**Conversion Metrics:**
* `conversions` - Number of conversions
* `conversion_rate` - Conversion rate
* `cost_per_conversion` - Cost per conversion
* `revenue` - Revenue from conversions
* `return_on_ad_spend` - Return on ad spend (ROAS)
* `revenue_per_conversion` - Average revenue per conversion
**Quality Metrics:**
* `quality_score` - Quality score (1-10)
* `expected_ctr` - Expected CTR rating
* `ad_relevance` - Ad relevance rating
* `landing_page_experience` - Landing page experience rating
**Additional Metrics:**
* `assists` - Number of assists
* `cost_per_assist` - Cost per assist
* `impression_share_percent` - Impression share percentage
* `exact_match_impression_share_percent` - Exact match impression share percentage
Daily performance report for ad groups, with detailed metrics including impression share and quality scores.
**Primary Keys:** `time_period`, `ad_group_id`, `device_type`, `network`
**Time & Identifiers:**
* `time_period` - Time period of the report (YYYY-MM-DD format)
* `account_id` - Account ID
* `account_name` - Account name
* `campaign_id` - Campaign ID
* `campaign_name` - Campaign name
* `campaign_status` - Campaign status
* `ad_group_id` - Ad Group ID
* `ad_group_name` - Ad Group name
* `status` - Ad Group status
**Dimensions:**
* `device_type` - Device type (Computer, Smartphone, Tablet)
* `network` - Network (Bing and Yahoo! search, Syndicated search partners)
**Performance Metrics:**
* `impressions` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `average_cpc` - Average cost per click
* `spend` - Total spend
**Conversion Metrics:**
* `conversions_qualified` - Number of qualified conversions
* `conversion_rate` - Conversion rate
* `cost_per_conversion` - Cost per conversion
* `revenue` - Revenue from conversions
* `return_on_ad_spend` - Return on ad spend (ROAS)
* `revenue_per_conversion` - Average revenue per conversion
**Quality Metrics:**
* `quality_score` - Quality score (1-10)
* `expected_ctr` - Expected CTR rating
* `ad_relevance` - Ad relevance rating
* `landing_page_experience` - Landing page experience rating
**Additional Metrics:**
* `assists` - Number of assists
* `cost_per_assist` - Cost per assist
* `impression_share_percent` - Impression share percentage
* `exact_match_impression_share_percent` - Exact match impression share percentage
Daily performance report for individual ads, with ad-level metrics and dimensions.
**Primary Keys:** `time_period`, `ad_id`, `device_type`, `network`, `language`
**Time & Identifiers:**
* `time_period` - Time period of the report (YYYY-MM-DD format)
* `account_id` - Account ID
* `account_name` - Account name
* `customer_id` - Customer ID
* `customer_name` - Customer name
**Campaign & Ad Group:**
* `campaign_id` - Campaign ID
* `campaign_name` - Campaign name
* `campaign_type` - Campaign type
* `campaign_status` - Campaign status
* `ad_group_id` - Ad Group ID
* `ad_group_name` - Ad Group name
* `ad_group_status` - Ad Group status
**Ad Details:**
* `ad_id` - Ad ID
* `ad_title` - Ad title
* `ad_description` - Ad description
* `ad_type` - Ad type
* `ad_status` - Ad status
* `destination_url` - Destination URL
* `display_url` - Display URL
**Dimensions:**
* `device_type` - Device type (Computer, Smartphone, Tablet)
* `network` - Network (Bing and Yahoo! search, Syndicated search partners)
* `language` - Language
**Performance Metrics:**
* `impressions` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `average_cpc` - Average cost per click
* `spend` - Total spend
**Conversion Metrics:**
* `conversions_qualified` - Number of qualified conversions
* `conversion_rate` - Conversion rate
* `cost_per_conversion` - Cost per conversion
* `revenue` - Revenue from conversions
* `return_on_ad_spend` - Return on ad spend (ROAS)
* `revenue_per_conversion` - Average revenue per conversion
**Additional Metrics:**
* `assists` - Number of assists
* `cost_per_assist` - Cost per assist
* `top_impression_rate_percent` - Top impression rate percentage
* `absolute_top_impression_rate_percent` - Absolute top impression rate percentage
Daily performance report for keywords, including quality scores, match types, and historical metrics.
**Primary Keys:** `time_period`, `keyword_id`, `device_type`, `network`, `delivered_match_type`
**Time & Identifiers:**
* `time_period` - Time period of the report (YYYY-MM-DD format)
* `account_id` - Account ID
* `account_name` - Account name
* `campaign_id` - Campaign ID
* `campaign_name` - Campaign name
* `campaign_status` - Campaign status
* `ad_group_id` - Ad group ID
* `ad_group_name` - Ad group name
* `ad_group_status` - Ad group status
**Keyword Details:**
* `keyword_id` - Keyword ID
* `keyword` - Keyword text
* `keyword_status` - Keyword status
* `bid_match_type` - Bid match type (Exact, Phrase, Broad)
* `delivered_match_type` - Delivered match type
**Dimensions:**
* `device_type` - Device type (Computer, Smartphone, Tablet)
* `network` - Network (Bing and Yahoo! search, Syndicated search partners)
**Performance Metrics:**
* `impressions` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `average_cpc` - Average cost per click
* `spend` - Total spend
* `average_position` - Average ad position
**Conversion Metrics:**
* `conversions` - Number of conversions
* `conversion_rate` - Conversion rate
* `cost_per_conversion` - Cost per conversion
* `revenue` - Revenue from conversions
* `return_on_ad_spend` - Return on ad spend (ROAS)
**Current Quality Metrics:**
* `quality_score` - Quality score (1-10)
* `expected_ctr` - Expected CTR rating
* `ad_relevance` - Ad relevance rating
* `landing_page_experience` - Landing page experience rating
**Historical Quality Metrics:**
* `historical_quality_score` - Historical quality score
* `historical_expected_ctr` - Historical expected CTR rating
* `historical_ad_relevance` - Historical ad relevance rating
* `historical_landing_page_experience` - Historical landing page experience rating
**Bid Metrics:**
* `current_max_cpc` - Current maximum CPC bid
Daily budget tracking report showing monthly budget, daily spend, and month-to-date expenditure.
**Account Dimensions:**
* `account_name` - Account name
* `account_number` - Account number
* `account_id` - Account ID
**Campaign Dimensions:**
* `campaign_name` - Campaign name
* `campaign_id` - Campaign ID
**Budget Metrics:**
* `date` - Date of the budget data (YYYY-MM-DD format)
* `currency_code` - Currency code
* `monthly_budget` - Monthly budget amount
* `daily_spend` - Daily spend amount
* `month_to_date_spend` - Month-to-date spend amount
Daily geographic performance report with location-based metrics and breakdown by country, state, city, and more.
**Primary Keys:** `time_period`, `account_id`, `campaign_id`, `ad_group_id`, `location_id`, `device_type`, `network`, `language`
**Time & Identifiers:**
* `time_period` - Time period of the report (YYYY-MM-DD format)
* `account_id` - Account ID
* `account_name` - Account name
* `account_number` - Account number
* `campaign_id` - Campaign ID
* `campaign_name` - Campaign name
* `campaign_type` - Campaign type
* `ad_group_id` - Ad group ID
* `ad_group_name` - Ad group name
**Location Dimensions:**
* `location_id` - Location ID
* `country` - Country
* `state` - State/Province
* `metro_area` - Metro area
* `city` - City
* `county` - County
* `postal_code` - Postal code
**Device & Network Dimensions:**
* `device_type` - Device type (Computer, Smartphone, Tablet)
* `network` - Network (Bing and Yahoo! search, Syndicated search partners)
* `language` - Language
**Performance Metrics:**
* `impressions` - Number of impressions
* `clicks` - Number of clicks
* `ctr` - Click-through rate
* `average_cpc` - Average cost per click
* `spend` - Total spend
* `average_position` - Average ad position
**Conversion Metrics:**
* `conversions` - Number of conversions
* `conversion_rate` - Conversion rate
* `cost_per_conversion` - Cost per conversion
* `revenue` - Revenue from conversions
* `return_on_ad_spend` - Return on ad spend (ROAS)
* `revenue_per_conversion` - Average revenue per conversion
**Additional Metrics:**
* `assists` - Number of assists
* `cost_per_assist` - Cost per assist
# Data Model
The following diagram illustrates the relationships between the core data streams in Microsoft Ads. The arrows indicate the join keys that link the different entities, providing a clear overview of the data structure.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Accounts("Accounts");
Campaigns("Campaigns");
AdGroups("Ad Groups");
Ads("Ads");
Keywords("Keywords");
end
subgraph "Performance Reports"
CampaignReport("Campaign Performance Report");
AdGroupReport("Ad Group Performance Report");
AdReport("Ad Performance Report");
KeywordReport("Keyword Performance Report");
LocationReport("User Location Performance Report");
BudgetReport("Budget Summary Report");
end
Campaigns -- "parent_id" --> Accounts;
AdGroups -- "parent_id" --> Campaigns;
Ads -- "parent_id" --> AdGroups;
Keywords -- "parent_id" --> AdGroups;
CampaignReport -- "campaign_id" --> Campaigns;
AdGroupReport -- "ad_group_id" --> AdGroups;
AdGroupReport -- "campaign_id" --> Campaigns;
AdReport -- "ad_id" --> Ads;
AdReport -- "ad_group_id" --> AdGroups;
KeywordReport -- "keyword_id" --> Keywords;
KeywordReport -- "ad_group_id" --> AdGroups;
LocationReport -- "campaign_id" --> Campaigns;
BudgetReport -- "campaign_id" --> Campaigns;
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Microsoft Ads data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
## Campaign Performance Analysis
### 1. Campaign Performance Overview
Track the overall performance of your campaigns, including key metrics and ROI.
**Business Value:**
* Identify which campaigns deliver the best value for money
* Understand the relationship between quality scores and performance
* Optimize budget allocation based on performance metrics
```sql theme={null}
WITH
campaign_metrics AS (
SELECT
c.name AS campaign_name,
c.campaign_type,
c.status,
SUM(cpr.impressions) AS total_impressions,
SUM(cpr.clicks) AS total_clicks,
SUM(cpr.spend) AS total_spend,
SUM(cpr.conversions) AS total_conversions,
SUM(cpr.revenue) AS total_revenue,
AVG(cpr.ctr) AS avg_ctr,
AVG(cpr.average_cpc) AS avg_cpc,
AVG(cpr.quality_score) AS avg_quality_score
FROM
nekt_raw.microsoft_ads_campaigns c
LEFT JOIN nekt_raw.microsoft_ads_campaign_performance_report cpr
ON c.id = cpr.campaign_id
WHERE
date_parse(cpr.time_period, '%Y-%m-%d') >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
c.name,
c.campaign_type,
c.status
)
SELECT
campaign_name,
campaign_type,
status,
total_spend,
total_impressions,
total_clicks,
total_conversions,
total_revenue,
avg_ctr,
avg_cpc,
avg_quality_score,
CASE
WHEN total_spend > 0 THEN total_revenue / total_spend
ELSE 0
END AS roas
FROM
campaign_metrics
ORDER BY
total_spend DESC
```
```sql theme={null}
WITH
campaign_metrics AS (
SELECT
c.name AS campaign_name,
c.campaign_type,
c.status,
SUM(cpr.impressions) AS total_impressions,
SUM(cpr.clicks) AS total_clicks,
SUM(cpr.spend) AS total_spend,
SUM(cpr.conversions) AS total_conversions,
SUM(cpr.revenue) AS total_revenue,
AVG(cpr.ctr) AS avg_ctr,
AVG(cpr.average_cpc) AS avg_cpc,
AVG(cpr.quality_score) AS avg_quality_score
FROM
`nekt_raw.microsoft_ads_campaigns` c
LEFT JOIN `nekt_raw.microsoft_ads_campaign_performance_report` cpr
ON c.id = cpr.campaign_id
WHERE
PARSE_DATE('%Y-%m-%d', cpr.time_period) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
c.name,
c.campaign_type,
c.status
)
SELECT
campaign_name,
campaign_type,
status,
total_spend,
total_impressions,
total_clicks,
total_conversions,
total_revenue,
avg_ctr,
avg_cpc,
avg_quality_score,
SAFE_DIVIDE(total_revenue, total_spend) AS roas
FROM
campaign_metrics
ORDER BY
total_spend DESC
```
| campaign\_name | campaign\_type | status | total\_spend | total\_impressions | total\_clicks | total\_conversions | total\_revenue | avg\_ctr | avg\_cpc | avg\_quality\_score | roas |
| - | - | - | - | - | - | - | - | - | - | - | - |
| Summer Sale 2024 | Search | Active | 8,542.30 | 1,245,890 | 42,156 | 1,245 | 24,890.00 | 3.38 | 0.20 | 7.5 | 2.91 |
| Brand Awareness | Audience | Active | 5,686.00 | 3,892,450 | 28,430 | 567 | 11,340.00 | 0.73 | 0.20 | 6.8 | 1.99 |
| Product Launch | Search | Active | 4,998.50 | 892,340 | 31,245 | 892 | 17,840.00 | 3.50 | 0.16 | 8.2 | 3.57 |
| Retargeting | Audience | Active | 3,784.00 | 2,145,670 | 18,920 | 423 | 8,460.00 | 0.88 | 0.20 | 7.1 | 2.24 |
### 2. Keyword Performance Analysis
Analyze keyword performance to identify top-performing keywords and optimization opportunities.
**Business Value:**
* Identify high-performing keywords for increased investment
* Find underperforming keywords for optimization or removal
* Understand match type effectiveness
```sql theme={null}
SELECT
kpr.keyword,
kpr.bid_match_type,
kpr.campaign_name,
kpr.ad_group_name,
SUM(kpr.impressions) AS total_impressions,
SUM(kpr.clicks) AS total_clicks,
SUM(kpr.spend) AS total_spend,
SUM(kpr.conversions) AS total_conversions,
SUM(kpr.revenue) AS total_revenue,
AVG(kpr.ctr) AS avg_ctr,
AVG(kpr.average_cpc) AS avg_cpc,
AVG(kpr.quality_score) AS avg_quality_score,
AVG(kpr.average_position) AS avg_position,
CASE
WHEN SUM(kpr.clicks) > 0 THEN SUM(kpr.conversions) * 100.0 / SUM(kpr.clicks)
ELSE 0
END AS conversion_rate,
CASE
WHEN SUM(kpr.conversions) > 0 THEN SUM(kpr.spend) / SUM(kpr.conversions)
ELSE 0
END AS cost_per_conversion
FROM
nekt_raw.microsoft_ads_keyword_performance_report kpr
WHERE
date_parse(kpr.time_period, '%Y-%m-%d') >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
kpr.keyword,
kpr.bid_match_type,
kpr.campaign_name,
kpr.ad_group_name
HAVING
SUM(kpr.clicks) > 10
ORDER BY
total_conversions DESC,
total_clicks DESC
LIMIT 50
```
```sql theme={null}
SELECT
kpr.keyword,
kpr.bid_match_type,
kpr.campaign_name,
kpr.ad_group_name,
SUM(kpr.impressions) AS total_impressions,
SUM(kpr.clicks) AS total_clicks,
SUM(kpr.spend) AS total_spend,
SUM(kpr.conversions) AS total_conversions,
SUM(kpr.revenue) AS total_revenue,
AVG(kpr.ctr) AS avg_ctr,
AVG(kpr.average_cpc) AS avg_cpc,
AVG(kpr.quality_score) AS avg_quality_score,
AVG(kpr.average_position) AS avg_position,
SAFE_DIVIDE(SUM(kpr.conversions) * 100.0, SUM(kpr.clicks)) AS conversion_rate,
SAFE_DIVIDE(SUM(kpr.spend), SUM(kpr.conversions)) AS cost_per_conversion
FROM
`nekt_raw.microsoft_ads_keyword_performance_report` kpr
WHERE
PARSE_DATE('%Y-%m-%d', kpr.time_period) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
kpr.keyword,
kpr.bid_match_type,
kpr.campaign_name,
kpr.ad_group_name
HAVING
SUM(kpr.clicks) > 10
ORDER BY
total_conversions DESC,
total_clicks DESC
LIMIT 50
```
| keyword | bid\_match\_type | campaign\_name | ad\_group\_name | total\_impressions | total\_clicks | total\_spend | total\_conversions | total\_revenue | avg\_ctr | avg\_cpc | avg\_quality\_score | avg\_position | conversion\_rate | cost\_per\_conversion |
| - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| buy laptop online | Exact | Electronics Sale | Laptops | 45,670 | 2,340 | 468.00 | 156 | 15,600.00 | 5.12 | 0.20 | 9 | 1.2 | 6.67 | 3.00 |
| best laptop deals | Phrase | Electronics Sale | Laptops | 38,920 | 1,890 | 378.00 | 98 | 9,800.00 | 4.86 | 0.20 | 8 | 1.5 | 5.19 | 3.86 |
| laptop sale | Broad | Electronics Sale | Laptops | 89,340 | 3,120 | 624.00 | 87 | 8,700.00 | 3.49 | 0.20 | 7 | 2.1 | 2.79 | 7.17 |
### 3. Geographic Performance Analysis
Analyze performance by user location to identify high-performing regions and optimize geographic targeting.
**Business Value:**
* Identify top-performing geographic regions
* Optimize location-based bid adjustments
* Discover new market opportunities
```sql theme={null}
SELECT
ulpr.country,
ulpr.state,
ulpr.city,
SUM(ulpr.impressions) AS total_impressions,
SUM(ulpr.clicks) AS total_clicks,
SUM(ulpr.spend) AS total_spend,
SUM(ulpr.conversions) AS total_conversions,
SUM(ulpr.revenue) AS total_revenue,
AVG(ulpr.ctr) AS avg_ctr,
AVG(ulpr.average_cpc) AS avg_cpc,
CASE
WHEN SUM(ulpr.clicks) > 0 THEN SUM(ulpr.conversions) * 100.0 / SUM(ulpr.clicks)
ELSE 0
END AS conversion_rate,
CASE
WHEN SUM(ulpr.spend) > 0 THEN SUM(ulpr.revenue) / SUM(ulpr.spend)
ELSE 0
END AS roas
FROM
nekt_raw.microsoft_ads_user_location_performance_report ulpr
WHERE
date_parse(ulpr.time_period, '%Y-%m-%d') >= CURRENT_DATE - INTERVAL '30' DAY
GROUP BY
ulpr.country,
ulpr.state,
ulpr.city
HAVING
SUM(ulpr.impressions) > 1000
ORDER BY
total_conversions DESC,
total_spend DESC
LIMIT 25
```
```sql theme={null}
SELECT
ulpr.country,
ulpr.state,
ulpr.city,
SUM(ulpr.impressions) AS total_impressions,
SUM(ulpr.clicks) AS total_clicks,
SUM(ulpr.spend) AS total_spend,
SUM(ulpr.conversions) AS total_conversions,
SUM(ulpr.revenue) AS total_revenue,
AVG(ulpr.ctr) AS avg_ctr,
AVG(ulpr.average_cpc) AS avg_cpc,
SAFE_DIVIDE(SUM(ulpr.conversions) * 100.0, SUM(ulpr.clicks)) AS conversion_rate,
SAFE_DIVIDE(SUM(ulpr.revenue), SUM(ulpr.spend)) AS roas
FROM
`nekt_raw.microsoft_ads_user_location_performance_report` ulpr
WHERE
PARSE_DATE('%Y-%m-%d', ulpr.time_period) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY
ulpr.country,
ulpr.state,
ulpr.city
HAVING
SUM(ulpr.impressions) > 1000
ORDER BY
total_conversions DESC,
total_spend DESC
LIMIT 25
```
| country | state | city | total\_impressions | total\_clicks | total\_spend | total\_conversions | total\_revenue | avg\_ctr | avg\_cpc | conversion\_rate | roas |
| - | - | - | - | - | - | - | - | - | - | - | - |
| United States | California | Los Angeles | 156,234 | 5,892 | 1,178.40 | 234 | 4,680.00 | 3.77 | 0.20 | 3.97 | 3.97 |
| United States | New York | New York | 142,890 | 5,234 | 1,046.80 | 198 | 3,960.00 | 3.66 | 0.20 | 3.78 | 3.78 |
| United States | Texas | Houston | 98,450 | 3,890 | 778.00 | 145 | 2,900.00 | 3.95 | 0.20 | 3.73 | 3.73 |
| United States | Florida | Miami | 87,340 | 3,245 | 649.00 | 112 | 2,240.00 | 3.72 | 0.20 | 3.45 | 3.45 |
## Implementation Notes
### Data Quality Considerations
* Performance reports are aggregated daily by default. Use the `time_period` field for trend analysis.
* Quality scores range from 1-10, with higher scores indicating better ad relevance and landing page experience.
* The User Location Performance Report provides granular geographic data down to the postal code level.
### API & Performance
* Microsoft Ads uses the Bulk API for entity data (campaigns, ad groups, ads, keywords) and the Reporting API for performance reports.
* Reports are generated asynchronously, which may result in slight delays during extraction.
* For optimal performance, select only the streams necessary for your analysis.
### Microsoft Search Network
* Microsoft Ads serves ads across the Microsoft Search Network, including Bing, Yahoo, AOL, and partner sites.
* The `network` dimension in reports indicates where your ads appeared (e.g., "Bing and Yahoo! search" or "Syndicated search partners").
## Troubleshooting
| Error or symptom | What it means | What to do |
| :- | :- | :- |
| The account does not appear in the list | The authorizing user has no access to the account in Microsoft Ads. | Grant the user access and re-authorize. See [Finding Your Manager Account ID and Customer ID](#finding-your-manager-account-id-and-customer-id). |
| Campaigns of certain types do not appear in the reports | Not every campaign type is exposed by the Microsoft Ads reporting API. | Check Microsoft's documentation for which campaign types are available for reporting. |
| Provider rate limit | You hit the requests-per-minute cap. | Reduce concurrency by scheduling sources that share the account at different times. |
## Skills for agents
Microsoft Ads connector documentation as plain markdown, for use in AI agent contexts.
# Microsoft Clarity as a data source
Source: https://docs.nekt.com/sources/microsoft-clarity
Bring data from Microsoft Clarity to your Lakehouse.
Microsoft Clarity is a free behavior analytics tool: it records how visitors actually use your website, with session recordings, heatmaps and frustration signals such as rage clicks, dead clicks and quick backs. This connector brings the figures behind the Clarity dashboard into your Lakehouse: traffic, engagement time, scroll depth and the frustration counters, broken down by page, device, browser, operating system, country and campaign.
Clarity's Data Export API only exposes the **last 24, 48 or 72 hours** of a project, allows **10 requests per project per day**, and returns a single total for the whole window rather than one figure per day. There is no way to backfill history: the table is built up one snapshot at a time, so schedule this source to run **once a day** and let it accumulate. Days on which the source does not run are lost permanently.
## 1. Add your Microsoft Clarity access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Microsoft Clarity option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Token:** Your Clarity Data Export token. In [Clarity](https://clarity.microsoft.com/), open the project, go to **Settings** → **Data Export** → **Generate new API token**, give it a name and copy the token immediately. Only project admins can create one, and each token gives access to a single project.
* **Days per Snapshot:** (Optional) How many of the most recent days each snapshot covers: 1, 2 or 3. Keep it at 1 (the default) and run this source once a day, so each snapshot lines up with a calendar day. A larger window returns one combined total for the whole period, which cannot be split back into daily figures.
* **Dimension Breakdowns:** (Optional) Extra breakdowns to pull, on top of the project-wide figures that are always collected. Each entry costs one of the 10 daily API requests, and may combine up to three of `Browser`, `Device`, `Country`, `OS`, `Source`, `Medium`, `Campaign`, `Channel` and `URL`: for example `URL` for per-page figures, or `Source,Medium` for per-campaign ones. Leave it empty to spend a single request per sync.
* **Start Date:** (Optional, advanced) Ignored by this source. The Clarity API has no date filter and no history, so every sync reports the last few days regardless of this value.
A token stops working as soon as the user who created it loses access to the project. If the source starts failing to authenticate, generate a new token under Settings → Data Export and update it here.
3. Click **Next**.
## 2. Select your Microsoft Clarity streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Microsoft Clarity data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Use **INCREMENTAL** for this source. Each run brings a new snapshot of the last few days, and incremental sync is what stacks those snapshots into a history. A full sync would replace the table with the latest snapshot and discard everything collected before it.
2. Click **Next**.
## 4. Configure your Microsoft Clarity data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## How the data is replicated
Clarity's API answers with the figures of the **last 24, 48 or 72 hours counted back from the moment of the request**. It carries no date of its own and cannot be asked for a past period. Every run is therefore a *snapshot*, and the table grows by appending snapshots rather than by updating rows.
Each row is stamped with `extracted_at` (the moment of the sync, which is also the incremental replication key), plus the `window_start` and `window_end` it covers. To read the latest state of a metric, filter on the most recent `extracted_at`. To follow it over time, group by day of `extracted_at`.
Because a snapshot is a fresh reading rather than a correction of the previous one, running the source more than once a day produces overlapping windows for the same period. Running it once a day with **Days per Snapshot** at 1 keeps one clean reading per calendar day.
All timestamps and figures are in UTC, as returned by Clarity.
## How the table is organized
The Clarity API returns its data grouped by metric, and each metric brings its own set of figures. The `project_insights` table keeps that shape: **one row per metric and per dimension value**, rather than one wide row per page or per day.
* `metric_name` says which metric a row belongs to: `Traffic`, `EngagementTime`, `ScrollDepth`, `PopularPages`, `Browser`, `Device`, `OS`, `Country`, `PageTitle`, `ReferrerUrl`, `DeadClickCount`, `RageClickCount`, `ExcessiveScroll`, `QuickbackClick`, `ScriptErrorCount` or `ErrorClickCount`. Only the columns that metric reports are filled; the rest are null.
* `breakdown` says what was requested: `overall` for the project-wide request, or the dimension names you configured, such as `URL` or `Source,Medium`.
* The dimension columns, `url`, `page_title`, `referrer_url`, `os`, `browser`, `device`, `country`, `source`, `medium`, `campaign`, `channel`, carry the value the row refers to.
* `raw` always holds the original row exactly as Clarity returned it, as a JSON string. Clarity documents only part of the fields each metric returns, so anything without a dedicated column is still readable there.
The project-wide request already includes Clarity's own breakdowns: traffic by browser, device, operating system and country, plus popular pages, page titles and referrers, at no extra request. Only add **Dimension Breakdowns** when you need a metric crossed with a dimension Clarity does not break down on its own, such as scroll depth per page.
## Limits and quotas
Clarity applies the following limits to every project, and they cannot be raised; Clarity is a free product with no paid tier for the API:
| Limit | Effect |
| - | - |
| 10 API requests per project per day | The connector plans at most 10 calls per run, one per breakdown. If the quota runs out mid-run, it stops and keeps what it already collected, and the run log says so. |
| Only the last 1 to 3 days | No historical backfill. History is built by running the source daily from now on. |
| 1,000 rows per response, no pagination | A very large `URL` breakdown may be truncated by Clarity itself. |
| Aggregated figures only | Session recordings, heatmaps and per-session data are not available through the API. |
The quota is shared by everything that calls the API with tokens of the same project, including manual tests. If several sources or scripts read the same Clarity project, plan the runs so they fit within the 10 daily requests.
# Streams and Fields
Below you'll find all available data streams from Microsoft Clarity and their corresponding fields.
The dashboard figures of the Clarity project, as one row per metric and dimension value per snapshot.
**Key Fields:**
* `id` - Deterministic identifier of the row, hashed from the snapshot timestamp, the requested breakdown, the metric and the dimension values. Clarity does not ship an identifier of its own
* `extracted_at` - Timestamp of the sync that produced this snapshot, shared by every row of the run and used as the incremental replication key
* `window_start` - Start of the period the figures cover, that is `extracted_at` minus `num_of_days`
* `window_end` - End of the period the figures cover, identical to `extracted_at`
* `num_of_days` - Length of the snapshot window in days: 1, 2 or 3
* `project_id` - Identifier of the Clarity project the data belongs to, read from the API token
* `breakdown` - Which dimensions were requested for this row: `overall` for the project-wide request, or the dimension names joined by commas
* `metric_name` - Clarity metric this row belongs to, which determines the value columns that are filled
* `url` - Page URL the row refers to, on the URL breakdown and on page-level metrics
* `page_title` - Title of the page the row refers to, on the PageTitle metric
* `referrer_url` - Address the visit came from, on the ReferrerUrl metric
* `os` - Operating system of the visitors counted in this row, such as Windows, Android or iOS
* `browser` - Browser of the visitors counted in this row, such as Chrome, Safari or Edge
* `device` - Device category of the visitors counted in this row: PC, Mobile, Tablet or Other
* `country` - Country or region the visitors counted in this row browsed from
* `source` - Traffic source of the visits, taken from the utm\_source parameter
* `medium` - Traffic medium of the visits, taken from the utm\_medium parameter
* `campaign` - Marketing campaign the visits came from, taken from the utm\_campaign parameter
* `channel` - Marketing channel Clarity classified the visits into, such as Direct, OrganicSearch, PaidSearch or Social
* `sessions_count` - Number of sessions this row accounts for, reported by most metrics as the population the metric was measured over
* `total_session_count` - Total number of sessions in the window, reported by the Traffic metric
* `total_bot_session_count` - Number of sessions attributed to bots, reported by the Traffic metric. Subtract it from `total_session_count` for human traffic
* `distant_user_count` - Number of distinct users in the window, reported by the Traffic metric
* `pages_per_session_percentage` - Average number of pages viewed per session, reported by the Traffic metric
* `sub_total` - Count reported by the counting metrics: how many dead clicks, rage clicks, excessive scrolls, quick backs, script errors or error clicks were observed, depending on `metric_name`
* `average_scroll_depth` - Average share of the page visitors scrolled through, in percent, reported by the ScrollDepth metric
* `total_time` - Total time visitors spent, in seconds, reported by the EngagementTime metric. Includes time the page was open but idle
* `active_time` - Time visitors actively interacted, in seconds, reported by the EngagementTime metric
* `visits_count` - Number of visits this row accounts for, when the metric reports it
* `raw` - The untouched row exactly as Clarity returned it, as a JSON string
# Microsoft Fabric as a data source
Source: https://docs.nekt.com/sources/microsoft-fabric
Bring data from Microsoft Fabric Lakehouses and Warehouses to Nekt.
Microsoft Fabric is Microsoft's unified analytics platform, combining data engineering, warehousing and business intelligence on top of OneLake. Nekt reads from Fabric through the **SQL analytics endpoint** that every Lakehouse and Warehouse exposes, so any table or view you can query in Fabric can be brought into your Catalog, along with custom SQL queries of your own.
![]()
## Before you start
Microsoft Fabric accepts **Microsoft Entra ID authentication only**: there is no SQL username and password. Nekt authenticates with a **service principal**, which is an application identity in your own Microsoft Entra tenant. Nekt does not register an application on your behalf, so the four steps below all happen inside your organization.
In the [Azure portal](https://portal.azure.com), go to **Microsoft Entra ID → App registrations → New registration**. Give it a name such as "Nekt connector" and register it.
From the app's **Overview** page, copy the **Directory (tenant) ID** and the **Application (client) ID**. Then go to **Certificates & secrets → New client secret**, create one, and copy its **Value** immediately: it is only shown once.
Client secrets expire. Note the expiry date you chose: when the secret expires, the source stops extracting until you create a new one and update it in Nekt.
A **Fabric administrator** must enable the tenant setting **"Service principals can use Fabric APIs"** in the [Fabric admin portal](https://app.fabric.microsoft.com/admin-portal), under **Tenant settings → Developer settings**.
If your organization restricts that setting to a security group, the application registered in step 1 must be added to that group.
This step is not optional, and skipping it produces confusing symptoms. Microsoft Fabric issues the service principal a security token through its API, not through the SQL connection. Without that token, queries can fail with "permission denied" or "file not found" errors even though the workspace access is configured correctly.
In Fabric, open the workspace holding your Lakehouse or Warehouse, select **Manage access → Add people or groups**, and add the application registered in step 1.
The **Viewer** role is enough for Nekt to read data. You can also share a single Warehouse with the application through item permissions instead of granting workspace-wide access.
Open your Lakehouse or Warehouse in Fabric, go to **Settings → SQL analytics endpoint** (or **SQL endpoint** for a Warehouse) and copy the **SQL connection string**. It looks like `abcd1234efgh.datawarehouse.fabric.microsoft.com`.
Note the name of the Lakehouse or Warehouse as well: that is the Database value Nekt asks for.
If your tenant restricts inbound public access to Fabric, you may also need to allow Nekt's public IP. See [this guide](/sources/pre-work-db-connection) for where to find it.
## Configuring Microsoft Fabric as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Microsoft Fabric** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Fill in the connection settings to authenticate with Microsoft Fabric:
* **SQL connection string**: the value copied in step 4 above.
* **Database**: the name of the Lakehouse or Warehouse to read from.
* **Tenant ID**, **Client ID** and **Client Secret**: the values from your app registration.
* **Login Timeout** (Advanced): Number of seconds to wait before a login attempt to the SQL endpoint times out. Defaults to `60`. Increase it if the Fabric capacity is paused or slow to resume.
* **Cursor Array Size** (Advanced): Maximum number of rows fetched per round trip. Defaults to `10000`. Larger values speed up large tables at the cost of memory; see the note on batch sizing under Good to know.
Click **Next**.
If the Lakehouse or Warehouse contains a large number of tables, you can narrow discovery down with two optional settings.
**Filter schemas** takes a comma-separated list of schema names:
```json theme={null}
"dbo, sales"
```
Naming a schema explicitly also brings back schemas Nekt hides by default, such as Fabric's own `queryinsights`.
**Filter tables** takes a comma-separated list with the following patterns:
* `schema.table`: match a specific table in a specific schema
* `table`: match a table with this name in any schema
* Wildcard patterns using `*` (any sequence) and `?` (a single character)
Examples:
```json theme={null}
"dbo.customers, dbo.orders, products"
```
```json theme={null}
"fact_*, dbo.dim_*, temp_????"
```
If neither setting is filled in, every table and view in the endpoint is available.
Fabric is a warehouse, so the data you want is often a join or an aggregate rather than a raw table. **Query Streams** lets you define SQL that runs inside Fabric and arrives in Nekt as its own table, which also moves far less data than extracting the underlying tables and rebuilding the result afterwards.
Each entry needs a **Stream Name** (lowercase letters, digits and underscores, starting with a letter) and the **SQL Query** itself.
Enable **Query streams only** if you want the source to sync nothing but these queries, skipping table and view discovery entirely.
### 2. Select streams
Choose which data streams you want to sync. You can select all streams or pick specific ones that matter most to you. We recommend selecting only the tables that are useful to you. You can add new tables at any time after the source is created.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
**Most Fabric tables arrive without a primary key**, and this is expected rather than a problem. A Lakehouse SQL analytics endpoint carries no key metadata at all, and a Warehouse only reports keys that were added explicitly; Fabric never enforces them, so even a declared key can contain duplicates.
That means you will usually pick the key columns and the incremental column yourself on this screen. If a table has no reliable column to track changes by, choose FULL\_TABLE.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. Bear in mind that each sync consumes Fabric capacity units, so a schedule matched to how often the data actually changes keeps your Fabric costs down.
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Good to know
* **Fetch batches are sized by declared column width.** The SQL driver reserves memory from each column's *declared* type, not from the data, and a Lakehouse SQL analytics endpoint reports every Delta `STRING` column as `varchar(8000)`. To stay within a safe memory budget, Nekt lowers the number of rows fetched per batch for wide tables and queries; **Cursor Array Size** is only the ceiling. When this happens, a single informational message appears in the run logs; no data is affected.
* **Timestamps have no time zone.** A Fabric Warehouse cannot store `datetimeoffset`, and a Lakehouse endpoint maps Delta `TIMESTAMP` values to `datetime2`. Timestamps arrive exactly as they are stored in Fabric, with no offset applied.
* **Fabric supports a narrower set of data types** than SQL Server does. Columns of types Fabric cannot persist, such as `money`, `datetime`, `nvarchar` or `json`, only appear through views or query streams, and are read as their closest equivalent.
* **Binary columns** (`varbinary`) are delivered base64-encoded, and `uniqueidentifier` columns as text.
* **Both Lakehouses and Warehouses work.** A Lakehouse SQL analytics endpoint is read-only by nature, which is all Nekt needs.
## Skills for agents
Microsoft Fabric connector documentation as plain markdown, for use in AI agent contexts.
# Microsoft Planner as a data source
Source: https://docs.nekt.com/sources/microsoft-planner
Bring data from Microsoft Planner to Nekt.
Microsoft Planner is a task management application included in Microsoft 365. It allows teams to create plans, organize and assign tasks, share files, and communicate progress, providing a visual way to manage work within Microsoft Teams and other Microsoft 365 apps.
![]()
## Configuring Microsoft Planner as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Microsoft Planner option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Microsoft Planner data using OAuth. Click on the **Microsoft Authorization** button and log in with your Microsoft account that has access to the Planner plans you want to sync.
The following configurations are available:
* **Use beta API**: (Default: `false`) Use the Microsoft Graph beta endpoint instead of v1.0. Enables extra fields (recurrence, notes, archival info, etc.) but beta APIs are subject to change and may break without notice.
The authorization flow will request the following permissions:
* Access to Microsoft Graph API
* Read access to Groups and Planner data
Once you're done authorizing the Nekt application, click **Next**.
You may need the approval of an administrator to grant the necessary permissions to the Nekt application.
If you are not an administrator, you can request the approval from an administrator by clicking on the **Request approval** button and providing the reason for the approval.
The administrator will then need to approve the request by clicking on the **Approve** button on the Microsoft Entra ID portal.
Once the approval is granted, the Nekt application will have the necessary permissions to access your Microsoft Planner data.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: FULL\_TABLE sync is used for all streams.
**Incremental sync is not supported.** The Microsoft Planner API does not provide reliable modification timestamps for incremental syncing, so all streams use full table sync.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Microsoft Planner and their corresponding fields:
Stream containing Microsoft 365 groups that can have Planner plans.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the group |
| `displayName` | String | Group display name |
| `description` | String | Group description |
| `createdDateTime` | DateTime | When the group was created |
| `mail` | String | Group email address |
| `mailNickname` | String | Mail nickname/alias |
| `visibility` | String | Group visibility (Public or Private) |
| `classification` | String | Classification level (e.g., low, medium, high business impact) |
| `groupTypes` | Array | Array of group types (e.g., "Unified" for Microsoft 365 groups) |
| `expirationDateTime` | DateTime | When the group is set to expire (if applicable) |
Stream containing Planner plans within Microsoft 365 groups.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the plan |
| `title` | String | Plan title/name |
| `createdDateTime` | DateTime | When the plan was created |
| `owner` | String | (Deprecated) Owner group ID; use `container.containerId` instead |
| `container` | Object | Plan container information (ID, type, and URL) |
| `createdBy` | Object | User and application who created the plan |
| `isArchived` | Boolean | Whether the plan is archived (beta only) |
| `archivalInfo` | String | Who archived/unarchived the plan and why, JSON string (beta only) |
| `contexts` | String | Other UX surfaces using this plan, JSON string (beta only) |
| `creationSource` | String | Origin of the plan, JSON string (beta only) |
| `sharedWithContainers` | String | Containers the plan is shared with, JSON string (beta only) |
Stream containing buckets (columns) within Planner plans for organizing tasks.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the bucket |
| `name` | String | Bucket display name (e.g., "To Do", "In Progress", "Done") |
| `planId` | String | ID of the plan this bucket belongs to |
| `orderHint` | String | Order hint for displaying buckets in list view |
Stream containing plan category labels (up to 25 custom labels per plan).
Labels are stored at the plan level. Tasks reference these labels via `appliedCategories` (e.g., `category1: true` means the task has the first label applied).
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Plan ID (matches `plans.id`) |
| `category1` through `category25` | String | Custom label names defined for the plan |
Stream containing individual tasks within Planner plans.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the task |
| `title` | String | Task title |
| `percentComplete` | Integer | Completion percentage (0, 25, 50, 75, or 100) |
| `createdDateTime` | DateTime | When the task was created |
| `dueDateTime` | DateTime | Task due date |
| `startDateTime` | DateTime | Task start date |
| `completedDateTime` | DateTime | When the task was completed |
| `priority` | Integer | Priority level (0-10, where lower is higher priority) |
| `bucketId` | String | ID of the bucket (column) this task is in |
| `planId` | String | ID of the plan containing this task |
| `orderHint` | String | Order hint for list view |
| `assigneePriority` | String | Order hint for assignee's task list |
| `conversationThreadId` | String | ID of the conversation thread |
| `previewType` | String | Task preview type (automatic, noPreview, checklist, description, reference) |
| `activeChecklistItemCount` | Integer | Number of incomplete checklist items |
| `checklistItemCount` | Integer | Total number of checklist items |
| `referenceCount` | Integer | Number of external references/attachments |
| `hasDescription` | Boolean | Whether the task has a description |
| `assignments` | String | Set of assignees (user ID mapped to details), JSON string |
| `appliedCategories` | String | Categories applied (category1 to category25), JSON string |
| `createdBy` | Object | IdentitySet representing who created the task |
| `completedBy` | Object | IdentitySet representing who completed the task |
| `recurrence` | String | Task recurrence info (seriesId, schedule, etc.), JSON string (beta only) |
| `creationSource` | String | Origin of the task, JSON string (beta only) |
| `specifiedCompletionRequirements` | String | Completion requirements (beta only) |
| `isOnMyDay` | Boolean | Whether task shows in MyDay view (beta only) |
| `isOnMyDayLastModifiedDate` | String | Date task was added/removed from MyDay (beta only) |
| `isArchived` | Boolean | Whether the task is archived (beta only) |
| `hasChat` | Boolean | Whether the task has chat messages (beta only) |
Stream containing additional detailed fields for individual tasks, including notes, references, and checklist items.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `task_id` | String | Task ID (matches `tasks.id`) |
| `description` | String | Task notes/description content |
| `previewType` | String | Preview type |
| `checklist` | String | Checklist items, JSON string |
| `references` | String | External references/links, JSON string |
| `notes` | String | Rich text description, JSON string (beta only) |
| `forms` | String | Forms associated with the task, JSON string (beta only) |
| `completionRequirements` | String | Detailed completion requirements, JSON string (beta only) |
# Data Model
The following diagram illustrates the relationships between the data streams in Microsoft Planner. Groups contain Plans, which in turn contain Buckets, Tasks, and Labels. Tasks have additional relationships with Task Details.
```mermaid theme={null}
erDiagram
Groups ||--o{ Plans : "contains"
Plans ||--o{ Buckets : "has"
Plans ||--o{ Tasks : "has"
Plans ||--|| Labels : "has"
Tasks }o--|| Buckets : "bucketId"
Tasks ||--|| TaskDetails : "has"
Groups {
string id PK
string displayName
string description
string mail
datetime createdDateTime
string visibility
}
Plans {
string id PK
string title
datetime createdDateTime
string owner
object container
object createdBy
}
Buckets {
string id PK
string name
string planId FK
string orderHint
}
Tasks {
string id PK
string title
string planId FK
string bucketId FK
int percentComplete
int priority
datetime dueDateTime
datetime completedDateTime
string assignments
string appliedCategories
}
TaskDetails {
string task_id PK
string description
string checklist
string references
}
Labels {
string id PK
string category1
string category2
string category3
}
```
# Use Cases for Data Analysis
This guide outlines valuable business intelligence use cases when consolidating Microsoft Planner data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Task Completion Overview
Track task completion rates across plans and identify bottlenecks.
**Business Value:**
* Monitor project progress
* Identify overdue tasks
* Track team productivity
```sql theme={null}
WITH task_metrics AS (
SELECT
p.title AS plan_name,
COUNT(*) AS total_tasks,
SUM(CASE WHEN t.percentComplete = 100 THEN 1 ELSE 0 END) AS completed_tasks,
SUM(CASE WHEN t.dueDateTime < CURRENT_TIMESTAMP AND t.percentComplete < 100 THEN 1 ELSE 0 END) AS overdue_tasks,
AVG(t.percentComplete) AS avg_completion
FROM
nekt_raw.microsoft_planner_tasks t
JOIN nekt_raw.microsoft_planner_plans p ON t.planId = p.id
GROUP BY
p.title
)
SELECT
plan_name,
total_tasks,
completed_tasks,
overdue_tasks,
ROUND(avg_completion, 1) AS avg_completion_pct,
ROUND(completed_tasks * 100.0 / NULLIF(total_tasks, 0), 1) AS completion_rate_pct
FROM
task_metrics
ORDER BY
total_tasks DESC
```
```sql theme={null}
WITH task_metrics AS (
SELECT
p.title AS plan_name,
COUNT(*) AS total_tasks,
SUM(CASE WHEN t.percentComplete = 100 THEN 1 ELSE 0 END) AS completed_tasks,
SUM(CASE WHEN t.dueDateTime < CURRENT_TIMESTAMP() AND t.percentComplete < 100 THEN 1 ELSE 0 END) AS overdue_tasks,
AVG(t.percentComplete) AS avg_completion
FROM
`nekt_raw.microsoft_planner_tasks` t
JOIN `nekt_raw.microsoft_planner_plans` p ON t.planId = p.id
GROUP BY
p.title
)
SELECT
plan_name,
total_tasks,
completed_tasks,
overdue_tasks,
ROUND(avg_completion, 1) AS avg_completion_pct,
ROUND(SAFE_DIVIDE(completed_tasks * 100.0, total_tasks), 1) AS completion_rate_pct
FROM
task_metrics
ORDER BY
total_tasks DESC
```
| plan\_name | total\_tasks | completed\_tasks | overdue\_tasks | avg\_completion\_pct | completion\_rate\_pct |
| - | - | - | - | - | - |
| Q1 Marketing Campaign | 45 | 32 | 5 | 71.1 | 71.1 |
| Product Launch 2024 | 38 | 28 | 2 | 73.7 | 73.7 |
| Website Redesign | 52 | 18 | 12 | 34.6 | 34.6 |
| Customer Onboarding | 23 | 20 | 0 | 87.0 | 87.0 |
### 2. Tasks by Bucket (Status Distribution)
Analyze task distribution across buckets to understand workflow status.
**Business Value:**
* Visualize work-in-progress
* Identify workflow bottlenecks
* Balance workload across stages
```sql theme={null}
SELECT
p.title AS plan_name,
b.name AS bucket_name,
COUNT(*) AS task_count,
SUM(CASE WHEN t.percentComplete = 100 THEN 1 ELSE 0 END) AS completed,
SUM(CASE WHEN t.percentComplete > 0 AND t.percentComplete < 100 THEN 1 ELSE 0 END) AS in_progress,
SUM(CASE WHEN t.percentComplete = 0 THEN 1 ELSE 0 END) AS not_started
FROM
nekt_raw.microsoft_planner_tasks t
JOIN nekt_raw.microsoft_planner_plans p ON t.planId = p.id
JOIN nekt_raw.microsoft_planner_buckets b ON t.bucketId = b.id
GROUP BY
p.title,
b.name
ORDER BY
p.title,
task_count DESC
```
```sql theme={null}
SELECT
p.title AS plan_name,
b.name AS bucket_name,
COUNT(*) AS task_count,
SUM(CASE WHEN t.percentComplete = 100 THEN 1 ELSE 0 END) AS completed,
SUM(CASE WHEN t.percentComplete > 0 AND t.percentComplete < 100 THEN 1 ELSE 0 END) AS in_progress,
SUM(CASE WHEN t.percentComplete = 0 THEN 1 ELSE 0 END) AS not_started
FROM
`nekt_raw.microsoft_planner_tasks` t
JOIN `nekt_raw.microsoft_planner_plans` p ON t.planId = p.id
JOIN `nekt_raw.microsoft_planner_buckets` b ON t.bucketId = b.id
GROUP BY
p.title,
b.name
ORDER BY
p.title,
task_count DESC
```
| plan\_name | bucket\_name | task\_count | completed | in\_progress | not\_started |
| - | - | - | - | - | - |
| Q1 Marketing Campaign | To Do | 15 | 0 | 3 | 12 |
| Q1 Marketing Campaign | In Progress | 12 | 0 | 12 | 0 |
| Q1 Marketing Campaign | Done | 18 | 18 | 0 | 0 |
| Product Launch 2024 | Backlog | 8 | 0 | 0 | 8 |
### 3. Overdue Tasks Report
Identify tasks that are past their due date and need attention.
**Business Value:**
* Prioritize late tasks
* Track deadline adherence
* Improve project timeline management
```sql theme={null}
SELECT
t.title AS task_title,
p.title AS plan_name,
b.name AS bucket_name,
t.dueDateTime AS due_date,
DATE_DIFF('day', t.dueDateTime, CURRENT_DATE) AS days_overdue,
t.percentComplete AS percent_complete,
t.priority
FROM
nekt_raw.microsoft_planner_tasks t
JOIN nekt_raw.microsoft_planner_plans p ON t.planId = p.id
JOIN nekt_raw.microsoft_planner_buckets b ON t.bucketId = b.id
WHERE
t.dueDateTime < CURRENT_TIMESTAMP
AND t.percentComplete < 100
ORDER BY
days_overdue DESC,
t.priority ASC
LIMIT 20
```
```sql theme={null}
SELECT
t.title AS task_title,
p.title AS plan_name,
b.name AS bucket_name,
t.dueDateTime AS due_date,
DATE_DIFF(CURRENT_DATE(), DATE(t.dueDateTime), DAY) AS days_overdue,
t.percentComplete AS percent_complete,
t.priority
FROM
`nekt_raw.microsoft_planner_tasks` t
JOIN `nekt_raw.microsoft_planner_plans` p ON t.planId = p.id
JOIN `nekt_raw.microsoft_planner_buckets` b ON t.bucketId = b.id
WHERE
t.dueDateTime < CURRENT_TIMESTAMP()
AND t.percentComplete < 100
ORDER BY
days_overdue DESC,
t.priority ASC
LIMIT 20
```
| task\_title | plan\_name | bucket\_name | due\_date | days\_overdue | percent\_complete | priority |
| - | - | - | - | - | - | - |
| Update documentation | Website Redesign | In Progress | 2024-11-01 | 26 | 50 | 1 |
| Review contracts | Q1 Marketing Campaign | To Do | 2024-11-10 | 17 | 0 | 3 |
| Design mockups | Product Launch 2024 | In Progress | 2024-11-15 | 12 | 75 | 5 |
## Implementation Notes
### Stream Hierarchy
The connector extracts data using a parent-child relationship:
1. **Groups** are extracted first (Microsoft 365 groups with Planner enabled)
2. **Plans** are extracted for each group
3. **Buckets**, **Labels**, and **Tasks** are extracted for each plan
4. **Task Details** are extracted for each task
### JSON Fields
Some fields are stored as JSON strings for compatibility with data lake formats:
* `tasks.assignments` - Contains user assignment details
* `tasks.appliedCategories` - Contains applied label categories
* `task_details.checklist` - Contains checklist items
* `task_details.references` - Contains external references
* Various beta endpoints attributes (`recurrence`, `notes`, `creationSource`, etc.)
Use `JSON_EXTRACT` or equivalent functions to parse these fields in your queries.
### Priority Values
Task priority is stored as an integer (0-10):
* `1` = Urgent
* `3` = Important
* `5` = Medium (default)
* `9` = Low
### Percent Complete Values
Microsoft Planner uses specific values for completion percentage:
* `0` = Not started
* `25` = Started
* `50` = Halfway
* `75` = Almost done
* `100` = Completed
### Permissions Note
The connector only extracts data from Microsoft 365 groups that the authenticated user has access to. To extract data from all groups in an organization, ensure the account has appropriate permissions or use an admin account.
## Skills for agents
Microsoft Planner connector documentation as plain markdown, for use in AI agent contexts.
# Microsoft SQL Server as a data source
Source: https://docs.nekt.com/sources/microsoft-sql-server
Bring data from Microsoft SQL Server to Nekt.
SQL Server is Microsoft's relational database management system designed for enterprise applications and data warehousing. It provides robust data storage, advanced analytics, and business intelligence capabilities with features for high availability, security, and performance optimization.
![]()
## Required pre work
In order to connect Nekt to a database, you have to do some pre work to ensure access is granted in a secure way. So, make sure to check [this guide](/sources/pre-work-db-connection) before proceeding.
## 1. Add account access
1. Once you have done the pre work defined in section 0, you can inform your database accesses. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Postgres option from the list of connectors.
2. Click **Next** and you'll be prompted to add your database access.
You must ensure your database accepts incoming connections from Nekt, and the database user has `SELECT` permissions in order to fetch data.
3. Click **Next**.
If you have a large database with many tables, you can use the `filter_tables` configuration option to speed up discovery and limit which tables are available for sync. This is particularly useful for databases with hundreds or thousands of tables.
**Table Filtering Format:**
Tables can be specified as a comma-separated list with the following patterns:
* `schema.table` - Match a specific table in a specific schema
* `table` - Match a table with this name in any schema
* Wildcard patterns using `*` (matches any sequence) and `?` (matches single character)
**Examples:**
Basic filtering:
```json theme={null}
"dbo.customers,dbo.orders,products"
```
Using wildcards:
```json theme={null}
"user_*,dbo.order_*,temp_????"
```
This would match:
* `user_accounts`, `user_profiles`, etc. (in any schema)
* `dbo.order_items`, `dbo.order_history`, etc. (in dbo schema only)
* `temp_2023`, `temp_test`, etc. (exactly 4 characters after `temp_`)
If `filter_tables` is not specified, all tables in the selected schemas will be available.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you. We recommend selecting only the tables that are useful for you from you. You can add new tables at anytime after the source is created.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Troubleshooting
| Issue | Cause | Solution |
| :- | :- | :- |
| Permission error when listing or reading tables | The connection user has no `SELECT` on the schema or on the selected tables. | Grant `SELECT` to the user on the tables you need, as described in [Required pre work](#required-pre-work). |
| The connection is accepted and closed before login | The server sits behind a firewall allowlist that does not include Nekt. | Allow Nekt's outbound addresses on the firewall, or use an SSH tunnel. Both are covered in the [database connection guide](/sources/pre-work-db-connection). |
| A stream always runs as a full sync | The table has no column usable as a replication key. | Choose an integer column, or a date column updated on every write. Without one, the stream stays on full sync. |
## Skills for agents
Microsoft SQL Server connector documentation as plain markdown, for use in AI agent contexts.
# MicroWork as a data source
Source: https://docs.nekt.com/sources/microwork
Bring data from MicroWork Cloud to Nekt.
MicroWork Cloud is a Brazilian ERP for vehicle dealerships and workshops, covering sales, stock, workshop orders, parts, CRM and finance. This connector brings MicroWork data into your Nekt Catalog through MicroWork's report API.
MicroWork works differently from most connectors, and it is worth understanding before you start. It publishes no per-entity endpoints: there is no "sales" endpoint, no "customers" endpoint. Instead, it exposes a single integration endpoint that runs a **report layout you save inside MicroWork Cloud**. You choose the report, the filters and the columns; MicroWork then gives you a request body that reproduces exactly that report.
So in this connector **you decide what the streams are**. Each report layout you configure becomes one table in your Catalog, with the columns you left on that layout.
## Before you start
You need two things from MicroWork:
1. **An API credential.** Ask your MicroWork representative; it is not self-service. One credential covers every company in your organisation and gives access to all of your MicroWork Cloud data, so treat it as a password.
2. **At least one saved report layout**, prepared as described below.
### Preparing a report in MicroWork Cloud
Open the report that holds the data you want, for example **Relatórios » Veículo » Venda de Veículo**.
Set the filters, pick a period and click **Processar**.
The processed table shows the available columns. Remove the ones you do not need and add the ones you do: **the columns left here are exactly the columns the connector will bring into Nekt**.
Save it under **Meus Leiautes** with a name of your choice.
Define the filters for this layout, including which companies (**Empresas Ativas**) it covers. The period can be left as-is; the connector can move it on every run.
Process the report once more, then click the **``** button at the right of the table (*Parâmetros para MicroWork APIs*). Copy the request body it shows, from the opening brace to the closing brace, braces included.
## Configuring MicroWork as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button in the top-right corner, then select MicroWork from the list of connectors.
Click **Next** and add your access details.
### 1. Add account access
The following configuration fields are available:
* **API credential**: The integration credential MicroWork issued for your organisation. It is sent as an `Authorization: Bearer` header.
* **Reports**: One entry per report layout you want to sync. Each entry has:
* **Report name**: The name of the table this report becomes in your Catalog. Use letters, numbers and underscores, for example `venda_de_veiculo`.
* **Request body**: The text you copied from the report's `` button.
* **Key columns** (optional): Columns of this report that identify a row, for example `chassi`. When you fill this in, re-reading the same period updates rows instead of duplicating them. Leave it empty and the connector keys each row on a hash of the row's own content.
* **Date column** (optional): The column of this report that carries the row's date, for example `datamovimentacao`.
* **Start date filter name** and **End date filter name** (optional): The names of the two filters that bound this report's period, written exactly as they appear inside the `filtros` text of the request body you pasted.
* **API URL** (advanced): Root URL of your MicroWork Cloud instance. Only change it if MicroWork gave your organisation a dedicated address.
* **Start date** (advanced): The earliest date to read on the first run, for reports that name their date filters.
* **Requests per minute** (advanced): Maximum number of requests per minute the connector sends. Defaults to 10.
After configuring access, click **Next**.
### 2. Select streams
Choose which of your configured reports you want to sync. Each report appears here under the name you gave it.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how data should appear in your Catalog, including layer, table naming, and sync type.
* **Layer**: Select the destination layer where extracted tables will be created.
* **Table name**: Use the suggested name or customize it. You can also apply a prefix to all tables.
* **Sync Type**: Choose between `INCREMENTAL` and `FULL_TABLE`.
A report supports `INCREMENTAL` only when you filled in all three of **Date column**, **Start date filter name** and **End date filter name**. Without them the connector has no way to move the report's period, so it re-reads the period pasted in the request body on every run.
When you're done, click **Next**.
### 4. Configure data source
Add a clear description for this source so your team can identify it easily.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), choose how often data should be extracted (for example, daily, weekly, or at fixed times).
Optionally, configure additional settings:
* Delta Log Retention, which controls how long old table states are kept. Learn more [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync), to periodically complement incremental runs with a complete refresh.
When you are ready, click **Next** to finalize setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, trigger a run manually from the source actions.
To see data in the [Catalog](https://app.nekt.ai/catalog), at least one source run must complete successfully.
## Setting up incremental sync
The period of a MicroWork report lives inside the `filtros` text of the request body, as a pair of filters. Their names are **different in every report**: the vehicle sales report calls them `Periododamovimentacaoinicial` and `Periododamovimentacaofinal`, while a CRM report may call them `DataCriacaoInicial` and `DataCriacaoFinal`. Because of that the connector cannot guess them, and you name them yourself.
To find them, look at the `filtros` part of the body you pasted. It is a list of `Name=Value` pairs separated by semicolons, and the two that carry your dates are the ones you want:
```text theme={null}
Estadodoveiculo=1;Cor=null;Periododamovimentacaoinicial=2018-01-01 00:00;Periododamovimentacaofinal=2018-01-31 00:00;Tipodemovimento=11,9
```
Here you would fill in `Periododamovimentacaoinicial` and `Periododamovimentacaofinal`, plus the report column that holds each row's date as the **Date column**.
With all three filled in, every run asks MicroWork only for the period since the last successful run, and writes the dates back in the same format your report already used. Every other filter you configured is left untouched.
## Good to know
* **You choose the columns, in MicroWork.** A table in Nekt has exactly the columns the saved layout returns. If you add a column to the layout later, reload the source's streams in Nekt so the new column is picked up.
* **Every report column arrives as text.** A MicroWork report is a rendered table: the same column can come back formatted differently depending on how the layout is built, and MicroWork documents no types. Rather than guess, and risk a column changing type between runs, the connector stores every report column as text. Cast what you need in a [Query](https://docs.nekt.com/queries/overview) or [Notebook](https://docs.nekt.com/notebooks/overview). The only exception is the **Date column** you name, which is converted to a UTC timestamp.
* **Four extra columns are added to every table.** `_mw_report` (which report the row came from), `_mw_extracted_at` (when it was read), `_mw_row_id` (a hash of the row, used as the key when you name no key columns) and `_mw_row` (the whole row as JSON text, so a column you add to the layout later is still recoverable from past runs).
* **One call per report, per run.** MicroWork returns a whole report in a single response; there is no paging. A report with a very wide period is one large response, so prefer narrow periods and incremental sync over re-reading years on every run.
* **MicroWork limits heavy usage.** The vendor publishes no quota but states it monitors its servers and may limit or block the number of daily queries it considers excessive. That is why the connector defaults to 10 requests per minute. Keep the number of reports and the sync frequency to what you actually need.
* **The credential is organisation-wide.** Which companies a report covers is decided by the layout's **Empresas Ativas** filter, not by the credential.
* **A report that returns no rows teaches the connector nothing.** If a report comes back empty when the source is set up, its table is created with only the four connector columns. Widen the report's period in MicroWork so it returns at least one row, then reload the source's streams.
# Streams and Fields
This connector has no fixed list of streams. Each report layout you configure becomes one stream, named by you, and its fields are the columns of that report.
These four columns are present in every table this connector creates, alongside the report's own columns.
| Field | Type | Description |
| :- | :- | :- |
| `_mw_report` | String | Name of the MicroWork report layout this row was read from. |
| `_mw_extracted_at` | Date | Timestamp at which the connector read this row from MicroWork. |
| `_mw_row_id` | String | Deterministic hash of the row's report columns. Used as the primary key when the report declares no key columns of its own. |
| `_mw_row` | String | The whole report row as JSON text, so a column added to the layout after the table was created is still recoverable. |
An example of what a report stream looks like. These are the columns of the *Relatórios » Veículo » Venda de Veículo* layout used in MicroWork's own integration manual; your reports will have whatever columns you put on them.
| Field | Type | Description |
| :- | :- | :- |
| `chassi` | String | Chassis number of the vehicle, a natural candidate for the report's key column. |
| `cor` | String | Colour of the vehicle. |
| `datamovimentacao` | String | Date the sale was recorded. A good candidate for the report's date column. |
| `docfiscal` | String | Fiscal document issued for the sale. |
| `ficha` | String | Internal record number of the vehicle. |
| `modelo` | String | Model of the vehicle. |
| `proposta` | String | Number of the sales proposal. |
| `tipomovimento` | String | Type of movement recorded, for example a sale. |
| `tipovenda` | String | Type of sale, for example direct consumer credit or cash. |
| `valordocfiscal` | String | Amount on the fiscal document. |
| `vendedor` | String | Salesperson credited with the sale. |
# Mixpanel as a data source
Source: https://docs.nekt.com/sources/mixpanel
Bring event data from Mixpanel to Nekt.
Mixpanel is a product analytics platform that helps teams understand how users interact with their products. It tracks events, user properties, and funnels so you can analyze behavior and improve conversion. The Nekt connector uses the [Mixpanel Data Export API](https://developer.mixpanel.com/reference/raw-export) to replicate raw event data into Nekt.
![]()
## Configuring Mixpanel as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Mixpanel** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You need Mixpanel API credentials with access to the Data Export API. Use a **service account** (username) and its **secret** for authentication.
The following configurations are available:
* **Project ID**: The Mixpanel project ID you want to extract events from. You can find it in your Mixpanel project settings (e.g. **Settings** > **Project Settings** > **Project ID**). This field is required.
* **Username**: The username used to authenticate against the API. This is typically a **service account** username. Mixpanel recommends using a service account for programmatic access. This field is required.
* **Secret**: The secret (password) for the service account or user. This value is stored securely and is never displayed after saving. This field is required.
* **Start date**: The earliest date from which records will be synced. Used for the first full sync and when no previous state exists. Format: `YYYY-MM-DD`. This field is required.
* **Region**: The data residency region of your Mixpanel project: **United States (US)**, **Europe (EU)** or **India (IN)**. Leave empty for US-hosted projects. Projects hosted in the EU or India must set this, otherwise Mixpanel answers the export with `terminated early` and the run fails.
* **Export window (days)** (Advanced): Number of days covered by the first request to the Mixpanel Export API. Smaller windows keep each response short, so a dropped connection costs little to retry. Increase only for low-volume projects. The connector times this first request and sizes the remaining ones from that measurement (see the **Pagination** note under Streams and Fields). Default is `1`.
* **Parallel export windows** (Advanced): How many export windows to download at the same time. Raising this helps when a sync spends most of its time waiting on Mixpanel rather than working, which is the usual case for a large backfill. On a fast connection, extra windows may compete for the same processor instead. Default is `4`, maximum is `16`.
Once you're done, click **Next**.
### 2. Select streams
The Mixpanel connector exposes a single stream: **events**. Choose whether to sync it.
Select the stream and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for the table (which will contain the fetched events), and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted table once the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables from this data source.
* **Table name**: A default name is suggested; you can change it. You can add a **prefix** to all tables at once to speed up configuration.
* **Sync Type**: You can choose between **INCREMENTAL** and **FULL\_TABLE**.
* **Incremental**: Each run fetches only events since the last replicated timestamp. Recommended for ongoing syncs so you keep every event without re-reading full history.
* **Full table**: Each run re-exports events from the configured start date (or from the beginning). Use when you need to backfill or fully refresh the dataset.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted. For event data, common choices are:
* **Daily**: Typical for analytics and reporting.
* **Every 12 hours** or **hourly**: For more up-to-date event pipelines.
* **Weekly**: For lighter reporting needs.
Optionally, you can define:
* **Delta Log Retention**: How long to keep old states of the table. See [Resource control](https://docs.nekt.com/get-started/core-concepts/resource-control).
* **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**: Run a full export periodically in addition to incremental syncs.
When you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once the run completes successfully, data will appear in your Catalog.
You need at least one successful source run to see the table in your [Catalog](https://app.nekt.ai/catalog).
# Streams and Fields
The Mixpanel connector exposes one stream: **events**. It corresponds to the [Mixpanel Export API](https://developer.mixpanel.com/reference/raw-export) and returns one row per event.
Stream of raw analytics events from your Mixpanel project. Each record is a single event (e.g. page view, signup, purchase) with a unique insert ID, event name, timestamp, and a JSON blob of event properties.
**Key fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Stable identifier for the event, derived from the fields Mixpanel uses to identify one: event name, distinct ID, timestamp and insert ID. Deduplicates the same event across re-syncs, which raw exports do not do themselves. |
| `event` | String | The event name (e.g. `"Page View"`, `"Sign Up"`, `"Purchase"`). |
| `time` | Integer | The event timestamp, in milliseconds since the Unix epoch, exactly as the Export API returns it. |
| `event_time` | DateTime | The event timestamp as a UTC timestamp, derived from `time`. |
| `distinct_id` | String | Mixpanel's identifier for the user who triggered the event. |
| `insert_id` | String | The `$insert_id` Mixpanel uses to recognise duplicate events, when the sender supplied one. Empty for events sent without it. |
| `mp_processing_time_ms` | Integer | When Mixpanel's servers processed the event, in milliseconds since the Unix epoch, as opposed to when it occurred. Orders repeat sends of the same event. |
| `properties` | String | Full set of event properties as a JSON string. Includes Mixpanel reserved properties and any custom properties you send. |
**Notes:**
* **Primary keys**: The stream uses `id` and `event_time` to ensure proper deduplication and data integrity.
* **Replication**: The stream uses `event_time` as the replication key. Incremental syncs request only events from the last replicated timestamp onward. Because the Export API works in whole dates, the day containing the bookmark is downloaded again; events strictly before the bookmark are discarded before loading, so nothing is re-written.
* **Pagination**: The connector requests data in short, configurable date ranges (defaulting to **1-day** windows from `start_date` or state to "now") to respect API behavior and avoid timeouts on large projects. The first window is downloaded alone and timed; that measurement sizes the remaining windows (between 1 and 90 days each) so that a quiet project spends fewer of its hourly requests while a busy project keeps every response short. The remaining windows are then downloaded concurrently, up to the **Parallel export windows** setting.
* **Untimed events**: Mixpanel can occasionally return events without a `time` property. Because these cannot be placed in a timeline, deduplicated, or bookmarked, the connector skips them and logs a single warning summary at the end of the sync.
* **Properties**: All event properties (including nested objects) are serialized into the `properties` JSON string. To use them in SQL or BI tools, parse JSON in your transformations (e.g. `JSON_EXTRACT` / `json_extract` or flatten into columns). Many core properties like `distinct_id` and `$insert_id` are already extracted into native columns for convenience.
**Example `properties` (conceptual):**
```json theme={null}
{
"$insert_id": "abc-123",
"time": 1704067200000,
"distinct_id": "user_456",
"$user_id": "user_456",
"$device_id": "device_789",
"utm_source": "google",
"utm_campaign": "winter_sale",
"revenue": 99.99
}
```
In the stream, `id` will be derived from the event's identity fields, `event` will be the event name, `time` will be `1704067200000`, `event_time` will be the corresponding UTC datetime, and the full object above will appear in `properties` as a string.
# Data Model
The connector has a single stream. Events are identified by `id` and `event_time`, and they are ordered by `event_time`.
```mermaid theme={null}
erDiagram
EVENTS {
string id PK
datetime event_time PK
string event
integer time
string distinct_id
string insert_id
integer mp_processing_time_ms
string properties
}
```
# Transformation example: pivoting properties to columns
The `properties` column stores all event attributes as a single JSON string. To analyze dimensions like user ID, UTM fields, or revenue in [Explorer](https://app.nekt.ai/explorer) or downstream models, parse the JSON and expose the keys as separate columns. The example below selects common Mixpanel fields (`$user_id`, `utm_source`, `utm_campaign`, `revenue`) from the properties, alongside native columns like `distinct_id` and `event_time`. Adjust the keys to match your own custom properties.
```sql theme={null}
SELECT
id,
event,
time,
event_time,
distinct_id,
json_extract_scalar(properties, '$.$user_id') AS user_id,
json_extract_scalar(properties, '$.utm_source') AS utm_source,
json_extract_scalar(properties, '$.utm_campaign') AS utm_campaign,
CAST(json_extract_scalar(properties, '$.revenue') AS DOUBLE) AS revenue
FROM
nekt_raw.mixpanel_events
WHERE
event_time >= current_date - interval '30' day
```
Replace `nekt_raw.mixpanel_events` with your actual layer and table name. Use `json_extract_scalar()` for string properties; for numeric properties use `CAST(json_extract_scalar(properties, '$.key') AS DOUBLE)`. Property names with `$` (e.g. `$user_id`) are referenced as `'$.$user_id'` in the JSON path.
```sql theme={null}
SELECT
id,
event,
time,
event_time,
distinct_id,
JSON_VALUE(properties, '$.$user_id') AS user_id,
JSON_VALUE(properties, '$.utm_source') AS utm_source,
JSON_VALUE(properties, '$.utm_campaign') AS utm_campaign,
SAFE_CAST(JSON_VALUE(properties, '$.revenue') AS FLOAT64) AS revenue
FROM
`nekt_raw.mixpanel_events`
WHERE
event_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
```
Replace `nekt_raw.mixpanel_events` with your actual dataset and table name. Use `JSON_VALUE()` for string properties; for numeric properties use `SAFE_CAST(JSON_VALUE(properties, '$.key') AS FLOAT64)`. Property names with `$` (e.g. `$user_id`) are referenced as `'$.$user_id'` in the path.
You can run this as an ad-hoc query in Explorer or turn it into a [transformation](https://docs.nekt.com/get-started/core-concepts/transformations) that writes to a new table so you have a flattened view of Mixpanel events for reporting and joins.
### Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Authentication errors | Wrong username or secret | Verify service account (or user) and secret in Mixpanel (e.g. Project Settings > Service Accounts). Ensure the account has Data Export access. |
| No data / empty table | Start date in the future or no events in range | Check **Start date** and the project's time zone. Confirm that the project has events in the requested date range in Mixpanel. |
| Run fails with "Mixpanel returned 'terminated early'" | Wrong data residency region | The project is hosted in a different region than the one configured. Set **Region** to **EU** or **IN** to match your Mixpanel project. |
| Duplicate or missing events | State, date range issue, or missing timestamps | For incremental syncs, check that state is being saved (e.g. successful run completion). For full syncs, ensure Start date covers the desired period. Events missing a `time` property in Mixpanel are skipped; the run log ends with a warning summarizing how many. |
| Run is slower than usual and the log mentions pacing or the hourly limit | Mixpanel's export rate limit was reached | Expected behavior. Mixpanel allows **60 export requests per hour** per project, so the connector spaces its requests and waits out `429 Too Many Requests` responses automatically. Long backfills take longer for this reason; no action is needed. Other integrations exporting from the same project share this budget. |
| Large extractions slow or timing out | Very high event volume in one project | The connector defaults to 1-day pagination windows to handle large volumes. Ensure the **Export window (days)** setting is kept small (default is 1). For backfills, raising **Parallel export windows** helps when the download from Mixpanel is the bottleneck. If needed, reduce the sync frequency or split the project. |
| Need custom properties in columns | Properties are in JSON string | Add a Query (SQL) or Notebook (Python) that parses `properties` and selects or flattens the fields you need. |
## Implementation Notes
### Rate Limiting and Retries
The Mixpanel Raw Export API enforces strict per-project limits, most notably **60 export requests per hour** and **3 requests per second**. The connector paces its requests to stay within that budget (shared across all parallel windows) and handles `429 Too Many Requests` responses by waiting as long as Mixpanel's `Retry-After` header asks, or by following an escalating wait schedule when the header is absent, before sending the same request again. A download interrupted mid-window is retried as a unit, up to six more times over roughly 34 minutes, and the records already received are re-emitted safely because the target deduplicates on `id`. Because of this pacing and retry logic, historical backfills or full syncs spanning long periods may take significantly longer than usual to complete; this is expected and no user action is required.
### Sync Progress Estimation
During full syncs or large backfills, Mixpanel does not provide a total event count upfront. To provide visibility into the extraction progress, the connector dynamically estimates the total record count based on the average volume of events in completed pagination windows. This estimate updates as the sync progresses, offering a reliable order-of-magnitude projection.
### Performance Optimizations
The connector is specifically tuned to handle high event volumes efficiently. It leverages `msgspec` for rapid message serialization and bypasses redundant type conformance checks (since all extracted properties are mapped to string or integer fields), reducing CPU overhead and maximizing end-to-end throughput. Export windows are downloaded by parallel workers while records are parsed on a single consumer, so the concurrency overlaps network waits rather than competing for the processor.
## Skills for agents
Mixpanel connector documentation as plain markdown, for use in AI agent contexts.
# Mobuss Construção as a data source
Source: https://docs.nekt.com/sources/mobuss
Bring data from Mobuss Construção to your Lakehouse.
Mobuss Construção is a Brazilian construction-management platform covering the whole
construction lifecycle: technical assistance / warranty service (assistência técnica),
inspection and handover (vistoria & entrega), quality forms, workforce time-tracking
(apontamento) and workplace safety (EPI). Nekt extracts this data through the Mobuss
integration API.
## Configuring Mobuss Construção as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Mobuss Construção option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The following configurations are required:
* **API Token**: The access token for the Mobuss integration API. It is issued by the Mobuss team; request it through your Mobuss account manager or support.
* **Company ID**: (Optional) The company identifier issued by the Mobuss team together with the token. Some services require it; leave it empty if the Mobuss team did not provide one.
* **Start Date**: (Optional) The earliest date from which records will be synced for incremental streams (service requests, time entries, PPE deliveries, handover controls). Defaults to 2025-01-01.
* **Quality service IDs**: (Optional, advanced) The legacy-system service IDs (idLegado) used by the quality forms stream. Leave empty to skip that stream.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Mobuss Construção and their corresponding fields.
The Mobuss API only lists service requests closed or resolved on a given date: there is no endpoint listing every service request. Open requests are therefore not extracted.
Construction projects (obras) of the company linked to the token. Full table.
**Key Fields:**
* `id` - Unique identifier of the construction project in Mobuss Construção
* `nome` - Name of the construction project
* `id_legado` - Identifier of the project in the customer's legacy/ERP system
* `locais` - Units/locations of the project (JSON array of objects)
Units/locations (locais) of each construction project. Full table.
**Key Fields:**
* `id_obra` - Identifier of the construction project the location belongs to
* `id` - Unique identifier of the location
* `cod_local` - Code of the location
* `nome` - Name of the location
* `hierarquia` - Hierarchical path of the location within the project
* `ordem` - Order of the location within its hierarchy level
* `data_habitese` - Occupancy-permit (habite-se) date
* `id_legado` - Identifier of the location in the customer's legacy system
Technical-assistance service requests closed (encerradas) on each day. Incremental by closing date.
**Key Fields:**
* `id` - Unique identifier of the service request
* `numero_solicitacao` - Human-readable number of the request
* `nome_solicitante` / `telefone_solicitante` / `email_solicitante` - Requester contact details
* `nome_empreendimento` - Name of the development/project
* `nome_unidade` - Name of the unit/location
* `nome_cliente` - Name of the client
* `area_comum` - Whether the request refers to a common area
* `data_encerramento` - Date/time the request was closed (replication key)
Technical-assistance service requests resolved (resolvidas) on each day. Incremental by resolution date. Same fields as the closed-requests stream, with `data_resolucao` as the replication key.
Full detail of service requests closed or resolved on each day, including visits. Incremental.
**Key Fields:**
* `id` - Unique identifier of the service request
* `numero` - Human-readable number of the request
* `situacao` - Current status
* `data_abertura` - Date/time the request was opened
* `obra` / `unidade` - Project and unit of the request
* `data_resolucao` / `data_encerramento` - Resolution and closing dates
* `diagnostico` - Diagnosis of the request
* `procedencia` - Whether the request was deemed valid
* `descricao` - Description of the request
* `visitas` - Visits (vistoria/reparo) of the request (JSON array of objects)
Comments of each service request extracted by the detail stream.
**Key Fields:**
* `id_solicitacao` - Identifier of the service request
* `id_legado` - Identifier of the comment in the legacy system
* `descricao` - Text of the comment
* `data_inclusao` / `usuario_inclusao` - Creation date and author
* `data_alteracao` / `usuario_alteracao` - Last change date and author
Workforce time entries (apontamentos) per project period, filtered by change date. Incremental in 30-day windows.
**Key Fields:**
* `id_apontamento` - Unique identifier of the time entry
* `id_obra` / `nome_obra` - Project of the time-tracking period
* `id_periodo` / `data_inicio_periodo` / `data_fim_periodo` - Time-tracking period
* `data_inicio_apontamento` / `data_fim_apontamento` - Start/end of the entry
* `nome_servico` / `tipo_servico` - Service worked on
* `nome_local` - Location worked on
* `nome_colaborador` / `cpf_colaborador` / `nome_funcao_colaborador` - Worker details
* `nome_empreiteira_colaborador` - Worker's contractor company
* `valor_funcao_colaborador` - Value of the worker's role
* `retrabalho_apontamento` - Whether the entry refers to rework
* `anexos` - Attachments (JSON array of objects)
PPE (EPI) deliveries and write-offs per worker, by delivery date. Incremental in 30-day windows.
**Key Fields:**
* `record_hash` - Deterministic hash used as primary key (the API exposes no id)
* `dsc_nome_colaborador_recebeu_epi` / `num_cpf_colaborador_recebeu_epi` - Worker who received the PPE
* `dsc_tipo_epi` / `dsc_modelo_epi` / `dsc_categoria_epi` - PPE type, model and category
* `num_ca_epi` - Approval-certificate (CA) number
* `data_entrega_epi` / `num_qtd_entregue_epi` - Delivery date and quantity
* `data_baixa_epi` / `dsc_natureza_baixa_epi` - Write-off date and nature
* `dsc_razao_social_fabricante_epi` / `num_cnpj_fabricante_epi` - Manufacturer details
Inspection & handover controls whose stage was completed on each day, per stage type (Construção, Inspeção Final, Vistoria do Cliente, Entrega das Chaves). Incremental.
**Key Fields:**
* `id` - Unique identifier of the handover control
* `tipo_etapa` / `nome_etapa` - Stage type queried
* `data_conclusao_etapa` - Day the stage was completed (replication key)
* `nome_local` / `codigo_local` - Location of the control
* `nome_cliente` - Client of the location, with phone and e-mail fields
* `nome_obra` / `id_legado_obra` - Project of the control
Quality forms and their non-conformities per configured service. Only extracted when the Quality service IDs setting is filled. Incremental in 30-day windows by change date.
**Key Fields:**
* `id_legado_servico` / `tipo_servico` - Service the forms belong to
* `id_formulario_realizado` - Unique identifier of the filled-in form
* `nome` / `tipo` / `categoria` - Form name, type and category
* `data_realizacao` - Date the form was filled in
* `situacao_formulario` / `situacao_verificacao` - Form and verification status
* `ncs` - Non-conformities linked to the form (JSON array of objects)
* `locais` - Locations linked to the form (JSON array of objects)
# Moderna Sistemas as a data source
Source: https://docs.nekt.com/sources/moderna-sistemas
Bring data from ModernaNet (Moderna Sistemas) to your Lakehouse.
ModernaNet, by Moderna Sistemas, is a management system for clinics and hospitals in Brazil: scheduling, electronic health records, billing, and finance. The connector reads the ModernaNet **integration API** and brings the daily agenda of your clinic, every appointment with its unit, agenda, professional, procedure, patient, and status, to your Lakehouse. It can also extract any other report Moderna enables for your integration.
ModernaNet has no self-service API. Moderna enables the integration API for each installation as a contracted service, and provides the address and the credentials when it does.
## Configuring Moderna Sistemas as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Moderna Sistemas option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The connector authenticates with the **username and password of an integration user** that Moderna creates for you (or a **Client ID** and **Client Secret** for the v2 API).
Before creating the source, ask Moderna (your support or account contact) to enable the **generic integration API** (API Genérica) for Nekt and to send you:
* the **address of the integration API** of your installation;
* the **username and password** (or Client ID/Secret) of the integration user, authorized to run the **DADOSAGENDAMENTO** query (appointments of a day) and any other query you want to extract.
Only Moderna can authorize a user to run a query. If your server restricts access by IP address, also ask Moderna or your IT team to allow Nekt's public IP, shown in the "Helpful resources" section when you connect the source.
The following configurations are available:
* **API URL**: the address of the ModernaNet integration API, provided by Moderna when they enable the integration for your installation (for example `https:///api/Moderna`, or `https:///api/v2` for the Moderna API v2). This is not the address of the ModernaNet login page. Required.
* **Username**: the integration username provided by Moderna. For the Moderna API v2, use the `client_id` generated in ModernaNet (Configurações > TI > Acessos > Cliente API - oAuth). Moderna must also authorize this user to run each query the connector reads. Required.
* **Password**: the integration password provided by Moderna. For the Moderna API v2, use the `client_secret` generated together with the client\_id. Required.
* **Start Date**: the first day to extract on the first sync. Leave it empty to start one year back.
* **Lookback Days**: how many days before the last sync point are read again on every sync, so appointments whose status changed afterwards (confirmed, attended, cancelled) are updated. The default is 7.
* **Future Days**: how many days after today are read on every sync, so upcoming appointments are captured and kept up to date. The default is 60.
* **Appointments Query Name**: the name of the Moderna query that returns the appointments of a day. The default, `DADOSAGENDAMENTO`, is the name in Moderna's documentation; change it only if Moderna enabled it under another name.
* **Custom Queries**: other queries Moderna enabled for your integration (for example billing or financial reports). Each one becomes its own table. For each query, fill in:
* **Table name**: lowercase letters, numbers, and underscores, starting with a letter (for example `billing`).
* **Moderna query name**: the name of the query as Moderna enabled it (for example `DADOSFATURAMENTO`).
* **Date parameter** (optional): the name of the query's date filter (for example `DATAPARAM`). When filled, the query is read one day at a time and synced incrementally; leave it empty to read the whole query on every sync.
* **Date format** (optional): the layout of the date sent in the date parameter. The default is `YYYYMMDD` (for example `20251117`). *Note: for the v2 API, use `YYYY-MM-DD` as it expects standard date formats.*
* **Fixed filters** (optional): other filters sent on every call, as a JSON object, such as `{"ID_UNIDADE": 1}`. *Note: in the v2 API, filters must be equality filters on the view's exact column names.*
* **Key columns** (optional): the columns that identify a row, as Moderna names them (for example `ID_CONTA`). Without them, rows are identified by their full content.
* **Time Zone**: the time zone of the clinic, used to decide which day is today and to build appointment timestamps. The default is `America/Sao_Paulo`.
* **Requests Per Minute**: the maximum number of calls per minute sent to Moderna. Every call runs a query on your ModernaNet database, so the default of 60 is deliberately gentle.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Appointments, and custom queries with a date parameter, support INCREMENTAL. Custom queries without a date parameter are always read in full. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Moderna's v1 API returns a whole day in a single call, while the v2 API uses pages of 1,000 rows. The connector reads your agenda one day at a time in both versions. The first sync of a long history takes at least one call per day (about 6 minutes per year at the default pace).
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Moderna Sistemas and their corresponding fields. Columns keep the names Moderna uses in its queries, in lowercase, so they match Moderna's documentation.
Every appointment of each day, from the `DADOSAGENDAMENTO` query. Each sync reads from the last sync point minus the lookback days up to the configured number of future days.
Primary key: `id_agendamento` · Replication key: `query_date`
| Field | Type | Description |
| :- | :- | :- |
| `id_agendamento` | Integer | Identifier of the appointment. In ModernaNet it identifies the agenda slot the patient was booked into. |
| `query_date` | Date | Day the connector asked Moderna for, used as the incremental replication key. |
| `data_agendamento` | Date | Date of the appointment. |
| `hora_agendamento` | String | Local time of the appointment, as returned by Moderna (for example `13:45`). |
| `scheduled_at` | Datetime | Date and time of the appointment, in the time zone configured on the source. |
| `status_agendamento` | String | Status of the appointment in ModernaNet (for example `Agendado`). |
| `id_unidade` | Integer | Identifier of the clinic unit. |
| `unidade` | String | Name of the clinic unit. |
| `id_grupo_agenda` | Integer | Identifier of the agenda group (for example consultations or diagnostics). |
| `grupo_agenda` | String | Name of the agenda group. |
| `id_agenda` | Integer | Identifier of the agenda. |
| `agenda` | String | Name of the agenda. |
| `id_paciente` | Integer | Identifier of the patient. |
| `paciente` | String | Name of the patient. |
| `cpf_paciente` | String | CPF (tax ID) of the patient. |
| `telefone` | String | Phone number of the patient. |
| `celular` | String | Mobile phone number of the patient. |
| `id_profissional` | Integer | Identifier of the professional (a doctor or an equipment agenda). |
| `profissional` | String | Name of the professional (a doctor or an equipment agenda). |
| `codigo_exame` | Integer | Code of the booked procedure or exam. |
| `nome_exame` | String | Name of the booked procedure or exam. |
| `especialidade_exame` | String | Specialty of the booked procedure or exam. |
| `protocolo_agendamento` | String | Booking protocol number, when ModernaNet issues one. |
| `dataparam` | String | Date parameter echoed back by Moderna, as returned (for example `20251117`). |
| `extra_fields` | String | Fields returned by the query that are not listed above, as a JSON object. Moderna tailors each query per installation, so extra fields land here instead of being dropped. |
One table per entry of **Custom Queries**, named after its table name. The columns of a custom query are defined by Moderna, so each row is stored whole as JSON.
Primary key: `row_key` · Replication key: `query_date` (only when a date parameter is configured)
| Field | Type | Description |
| :- | :- | :- |
| `row_key` | String | Identifier of the row: the values of the key columns joined with `\|`, or a hash of the whole row when no key columns are configured. |
| `query_date` | Date | Day sent in the query's date parameter, used as the incremental replication key. Empty for queries without a date parameter. |
| `record` | String | The row exactly as returned by Moderna, as a JSON object with Moderna's column names. |
## Troubleshooting
| Symptom | Likely cause | What to do |
| :- | :- | :- |
| Username and password refused (HTTP 401/403) | The credentials are wrong, or the user is not authorized to run the query. | Check the credentials with Moderna. **For the v2 API, ensure you use the Client ID and Client Secret generated in ModernaNet, not v1 user credentials.** |
| "The API address answered with a web page" | The API URL points to the ModernaNet login page or to a wrong path. | Use the integration API address Moderna provided, usually ending in `/api/Moderna` or `/api/v2`. |
| A query could not be found (HTTP 404 or HTTP 500 "Invalid object name") | The query name differs from the one enabled for your installation. | Confirm the exact query name with Moderna and update Appointments Query Name or the custom query. |
| Filters rejected (HTTP 400 or HTTP 500 "Invalid column name" / "Conversion failed") | The fixed filters or date formatting sent to a query are invalid. | In v2, body keys are equality filters on the database view columns. Ensure the keys match exactly and dates are well-formatted (e.g., `YYYY-MM-DD`). |
| A cancelled appointment still shows its previous status | ModernaNet frees the slot when an appointment is cancelled, and the query may stop returning it. | Confirm with Moderna whether the query returns cancelled appointments. |
## Implementation Notes
### The v2 API
Moderna supports a v2 API that introduces OAuth token authentication and offset pagination. The connector automatically switches to the v2 standard whenever the **API URL** contains `/api/v2` (e.g., `https:///api/v2/`).
* **Authentication**: Instead of passing headers, it uses the provided `username` (Client ID) and `password` (Client Secret) to fetch a Bearer token via `client_credentials`. The token is cached and reused for the run, getting refreshed automatically if it expires or receives a `401 Unauthorized`.
* **Pagination**: The v2 API limits pages to 1,000 rows. The connector sequentially paginates through the data utilizing the `limit` and `offset` parameters until a short page or the maximum `total` is reached.
### Rate Limiting and Retries
Moderna publishes no official rate limit, and every API call executes a query on the customer's database. To be safe, the connector paces itself based on the configured **Requests Per Minute** (default 60).
* `429 Too Many Requests` and standard `5xx Server Errors` are retried up to 5 times using exponential backoff.
* **SQL Execution Errors** (such as `Invalid object name`, `Invalid column name`, and `Conversion failed`) resulting in `500 Server Error` on the v2 API are known, deterministic failures that fail the run rather than wasting time retrying. Ensure your custom query parameters match your specific database view columns.
# Monday as a data source
Source: https://docs.nekt.com/sources/monday
Bring data from Monday to Nekt.
Monday is a work operating system that enables organizations to build custom workflow apps in minutes to run their processes, projects, and everyday work. It provides tools for project management, task tracking, and team collaboration with a focus on visual and intuitive workflows.
![]()
## Configuring Monday as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Monday option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Monday API Token for this connection. You can find it in your [Monday account settings](https://monday.com/developers/v2/quickstart-view). Once you have it, add the account access and the connector configuration.
The following configurations are available:
* **API token**: The token to authenticate against the API service. It can be generated by clicking on your Profile Picture -> Developers -> API token in your Monday workspace.
* **Board IDs**: Restrict extraction to a specific set of board IDs. When set, only the listed boards and their items are extracted, and boards are returned regardless of state (active, archived, or deleted). Leave empty to extract every active board your token has access to.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Monday and their corresponding fields:
Stream for managing boards and their configuration.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the board |
| `name` | String | Name of the board |
| `board_folder_id` | String | ID of the folder containing the board |
| `board_kind` | String | Kind of board |
| `permissions` | String | Board permissions |
| `state` | String | Board state (all, active, archived, deleted) |
| `type` | String | Board type (board, sub\_items\_board) |
| `updated_at` | String | Last update timestamp |
| `workspace_id` | String | Workspace identifier |
| `description` | String | Board description |
| `items_count` | Integer | Number of items in the board |
| `items_limit` | Integer | Board items limit |
| `item_terminology` | String | Board item terminology |
| `url` | String | Board URL |
| `columns` | Array | Array of column objects (id, title, description, type, archived) |
| `groups` | Array | Array of group objects (id, title) |
| `owners` | Array | Array of owner objects with id and name |
| `creator` | Object | Creator object with id and name |
| `subscribers` | Array | Array of subscriber objects with id and name |
Stream for managing items (tasks/cards) within boards.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the item |
| `name` | String | Item name |
| `created_at` | String | Creation timestamp |
| `creator_id` | String | Creator's ID |
| `email` | String | Associated email |
| `state` | String | Item state |
| `relative_link` | String | Relative link to the item |
| `updated_at` | String | Last update timestamp |
| `url` | String | Item URL |
| `description` | Object | Description object containing blocks with content and metadata |
| `assets` | Array | Array of asset objects (id, name, file\_extension, url, public\_url, uploaded\_by) |
| `board` | Object | Board object with ID |
| `column_values` | Array | Array of column values (id, column, text, type, value) |
| `group` | Object | Group object with id and title |
| `parent_item` | Object | Parent item reference |
| `subitems` | Array | Array of subitem references |
| `subscribers` | Array | Array of subscriber objects |
Stream for managing user accounts and permissions.
**Key Fields:**
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the user |
| `name` | String | User's name |
| `email` | String | User's email |
| `url` | String | User's profile URL |
| `birthday` | String | User's birthday |
| `country_code` | String | User's country code |
| `created_at` | String | Account creation timestamp |
| `current_language` | String | User's language preference |
| `join_date` | String | Join date |
| `last_activity` | String | Last activity timestamp |
| `location` | String | User's location |
| `mobile_phone` | String | Mobile phone number |
| `phone` | String | Phone number |
| `title` | String | User's title |
| `time_zone_identifier` | String | Time zone |
| `utc_hours_diff` | Integer | UTC hour difference |
| `enabled` | Boolean | Whether the user is enabled |
| `is_admin` | Boolean | Admin status |
| `is_guest` | Boolean | Guest status |
| `is_pending` | Boolean | Pending status |
| `is_verified` | Boolean | Verification status |
| `is_view_only` | Boolean | View-only status |
| `photo_original` | String | Original photo URL |
| `photo_small` | String | Small photo URL |
| `photo_thumb` | String | Thumbnail photo URL |
| `photo_thumb_small` | String | Small thumbnail URL |
| `photo_tiny` | String | Tiny photo URL |
| `account` | Object | Account information (id, name, slug) |
| `teams` | Array | Array of team memberships (id, name) |
| `custom_field_metas` | Array | Array of custom field definitions |
| `custom_field_values` | Array | Array of custom field values |
## Implementation Notes
* **Board Filtering**: Setting specific IDs in the configuration allows the connector to efficiently extract only those requested boards, circumventing pagination across irrelevant data, and ignoring the board state constraints (it fetches them whether active, archived, or deleted). Leaving the property empty brings in every accessible active board.
* **Pagination**: Extraction mechanisms utilize standard page-number pagination for parent streams (like Boards) and robust cursor-based pagination for child resources (Items) to ensure performance consistency across large sets of data.
## Skills for agents
Monday connector documentation as plain markdown, for use in AI agent contexts.
# MongoDB as a data source
Source: https://docs.nekt.com/sources/mongo-db
Bring data from MongoDB to Nekt.
MongoDB is a popular NoSQL document database that stores data in flexible, JSON-like documents. It's designed for scalability and performance, making it ideal for applications that need to handle large amounts of unstructured or semi-structured data with high availability and horizontal scaling.
## 1. Add your MongoDB access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the MongoDB option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
**Default configurations**
* **Host:** the full connection URL of your MongoDB database. For detailed instructions, [check MongoDB docs](https://www.mongodb.com/docs/manual/reference/connection-string/). Please make sure your Mongo instance accepts connections coming from your AWS account associated with Nekt.
**Advanced configurations**
* **Include databases:** allows you to filter only the databases of interest inside your Mongo instance. This helps speed up the discovery process, as it tends to be a time-consuming operation for large databases.
* **Batch size**: defines the batch size for fetching records from your Mongo instance. Setting a smaller number helps reduce the overload on the database, however the extraction time will increase as more requests need to be made to fetch the entire collection. We recommend leaving it blank at first, changing it only if needed.
3. Click **Next**.
## 2. Select your MongoDB streams
1. The next step is letting us know which streams you want to bring. A stream represents a collection in your database.
All streams have the following properties in their schema:
* `_id`: a string field corresponding to the document ID.
* `document`: a stringified JSON version of the document.
Additionally, there are some properties that might be included depending on the sync type. For incremental sync, the incremental key is mapped as an additional field in the schema:
* `incremental_key`: a date-time, integer or timestamp field corresponding to the incremental key defined by the user during the stream configuration.
> Tip: The stream can be found more easily by typing its name.
2. Click **Next**.
## 3. Configure your MongoDB data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Important info regarding sync types: incremental sync is allowed for collections
that contain a valid incremental key. The incremental key must be either `Int32`,
`Int64`, `Date` or `Timestamp`; when defining the incremental key, you must
type the exact name of the field that exists in your documents; if an index
doesn't exist for the selected incremental key, a new index will be created
the first time the connector runs.
2. Click **Next**.
## 4. Configure your MongoDB data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how log we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## Troubleshooting
| Issue | Cause | Solution |
| :- | :- | :- |
| Connection failure on a managed cluster | The cluster address is wrong, or its access list does not include Nekt. | Allow Nekt's outbound addresses on the cluster access list and confirm the connection string. |
| Extraction runs without an index and is slow | The expected index does not exist on the source collection. | Create the index on the collection. Without it extraction still works, only slower. |
| Change capture is unavailable on a MongoDB-compatible managed variant | Some managed variants from other providers do not expose the change stream that log-based sync requires. | Use incremental extraction on an updated-at field instead of log-based. |
## Skills for agents
MongoDB connector documentation as plain markdown, for use in AI agent contexts.
# Morada.ai as a data source
Source: https://docs.nekt.com/sources/morada-ai
Bring data from Morada.ai to your Lakehouse.
Morada.ai is an AI sales agent for the real estate market. Its agent, Mia, talks to leads over WhatsApp, qualifies them, and moves them through the sales pipeline. The Data Export API exposes the conversations, messages, contacts, and deals of your workspace, the WhatsApp delivery failures, and the aggregated numbers behind the Conversations and Talk dashboards, so they can be analyzed alongside the rest of your data.
## 1. Add your Morada.ai access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Morada.ai option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Key:** Your workspace API key, in the `mk_...` format. Generate it inside Morada.ai under **Integrações → Configurar API**.
* **Start Date:** (Optional, advanced) The earliest change timestamp to sync on the first run. Leave it empty to export the full history of your workspace: the message errors and analytics streams then cover the last 365 days, since those endpoints always require a time window.
The API key grants read access to every record in the workspace it belongs to. Treat it as a secret.
3. Click **Next**.
## 2. Select your Morada.ai streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Morada.ai data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Morada.ai data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## How the data is replicated
The contacts, conversations, deals, messages, and message errors streams are change feeds keyed by `id`, and every record carries a `cursor_at` timestamp that marks when the record last changed. Incremental syncs ask Morada.ai for everything that changed after the highest `cursor_at` already extracted, so records that were updated in the source reappear and are updated in your table.
Morada.ai does not emit deletion events. When a record is deleted in the source, it simply stops appearing in the feed, and no signal marks the removal. To reconcile deletions, schedule an Additional Full Sync so the table is rebuilt from a complete export.
Some fields hold workspace-specific attributes whose structure varies per account, such as `props`, `metadata`, and `content`. They are delivered as JSON strings, so you can parse them in a Query when you need the individual keys.
### Analytics streams
The analytics streams are different: Morada.ai recalculates them with a job that runs every six hours, and that job can restate numbers that were already extracted. They are therefore replicated as FULL\_TABLE (each run brings the whole period again), and every record carries the freshness of the response that produced it, in the `as_of` and `stale` fields.
Each record is one calendar day (in the `America/Sao_Paulo` timezone, the same one the platform dashboards use) combined with the dimensions of the stream, and all metrics are additive components: counts, and sum and count pairs for the service-level metrics.
No rate, average, or percentage is delivered ready to use. Always rebuild them as `SUM(numerator) / SUM(denominator)` over the period you want: for example, the response rate is `SUM(conversations_answered) / SUM(conversations_started)`. Averaging the daily values gives the wrong number, because it weighs a quiet day the same as a busy one. Weekly and monthly figures are obtained the same way, by summing the days.
The service-level metrics of the tickets stream arrive as a sum in seconds plus a count, so the average is `SUM(first_response_sum_seconds) / SUM(first_response_count)`; divide by 60 for minutes.
## Rate limits
Morada.ai allows around 120 requests per minute per workspace. The connector stays below that limit on its own and automatically waits and retries when the API asks it to, so no configuration is needed on your side. Very large first extractions simply take longer to complete.
# Streams and Fields
Below you'll find all available data streams from Morada.ai and their corresponding fields. The change-feed streams, contacts, conversations, deals, messages, and message errors, share the `id`, `workspace_id`, and `cursor_at` fields.
People your workspace has interacted with.
**Key Fields:**
* `id` - Unique identifier of the contact
* `workspace_id` - Identifier of the Morada.ai workspace the contact belongs to
* `name` - Full name of the contact
* `email` - Email address of the contact
* `phone_number` - Phone number of the contact, typically the WhatsApp number used in conversations
* `banned_at` - Timestamp when the contact was blocked, when the contact has been blocked
* `props` - Additional workspace-specific attributes of the contact, as a JSON string
* `cursor_at` - Change watermark of the record, used as the incremental replication key
Conversations handled by the AI agent (Mia).
**Key Fields:**
* `id` - Unique identifier of the conversation
* `workspace_id` - Identifier of the Morada.ai workspace the conversation belongs to
* `person_id` - Identifier of the contact this conversation is with
* `agent_id` - Identifier of the AI agent (Mia) responsible for the conversation
* `deal_id` - Identifier of the deal associated with the conversation, when there is one
* `target` - Target of the conversation, such as the channel address the contact was reached at
* `started_at` - Timestamp when the conversation started
* `finished_at` - Timestamp when the conversation was closed, when it already ended
* `last_message_at` - Timestamp of the most recent message exchanged in the conversation
* `props` - Additional workspace-specific attributes of the conversation, as a JSON string
* `cursor_at` - Change watermark of the record, used as the incremental replication key
Sales opportunities tracked in your workspace.
**Key Fields:**
* `id` - Unique identifier of the deal
* `workspace_id` - Identifier of the Morada.ai workspace the deal belongs to
* `person_id` - Identifier of the contact related to the deal
* `agent_id` - Identifier of the AI agent (Mia) responsible for the deal
* `owner_id` - Identifier of the user who owns the deal
* `status` - Current status of the deal in the sales pipeline
* `title` - Title of the deal
* `props` - Additional workspace-specific attributes of the deal, as a JSON string
* `cursor_at` - Change watermark of the record, used as the incremental replication key
Individual messages exchanged inside conversations.
**Key Fields:**
* `id` - Unique identifier of the message
* `workspace_id` - Identifier of the Morada.ai workspace the message belongs to
* `conversation_id` - Identifier of the conversation this message belongs to
* `actor` - Author of the message, such as `mia` for the AI agent or `human` for a person
* `type` - Type of the message, such as text or image
* `content` - Body of the message. Non-textual payloads are delivered as a JSON string
* `metadata` - Additional message metadata, as a JSON string
* `created_at` - Timestamp when the message was created
* `delivered_at` - Timestamp when the message was delivered, when delivery was confirmed
* `read_at` - Timestamp when the message was read, when a read receipt was received
* `cursor_at` - Change watermark of the record, used as the incremental replication key
WhatsApp delivery failures reported by Meta for messages sent from your workspace. Useful to investigate why an active message never reached the lead.
**Key Fields:**
* `id` - Unique identifier of this delivery error log entry
* `workspace_id` - Identifier of the Morada.ai workspace the record belongs to
* `message_id` - Identifier of the message the delivery error refers to
* `conversation_id` - Identifier of the conversation the failed message belongs to
* `deal_id` - Identifier of the deal associated with the conversation, when there is one
* `agent_id` - Identifier of the agent responsible for the conversation
* `agent_name` - Name of the agent responsible for the conversation
* `external_id` - Identifier of the message on WhatsApp (wamid)
* `recipient_id` - Phone number of the end customer the message was addressed to, in E.164 format
* `status` - Status of the delivery log entry, always `failed` for this report
* `error_code` - WhatsApp delivery error code reported by Meta, such as `131049`
* `error_title` - Short title of the delivery error
* `error_message` - Message describing the delivery error
* `error_detail` - Additional detail about the delivery error, when available
* `message_created_at` - Timestamp when the failed message was created
* `conversation_started_at` - Timestamp when the conversation of the failed message started
* `conversation_link` - Deep link to the conversation on the Morada.ai platform
* `metadata` - Payload of the WhatsApp delivery status webhook that produced the error, as a JSON string
* `cursor_at` - Timestamp of the error log entry, used as the incremental replication key
Daily conversation counters behind the Conversations dashboard, broken down by agent type, agent, and source. The default scope matches what the dashboard shows: the conversations of your AI agent on WhatsApp.
**Key Fields:**
* `period` - Calendar day of the record, in the timezone reported by the `tz` field
* `agent_type` - Type of the agent that handled the conversations
* `agent_id` - Identifier of the agent that handled the conversations
* `source` - Origin of the conversations
* `conversations_started` - Conversations started in the day, the denominator of the response and appointment rates
* `conversations_answered` - Conversations with at least one reply from the contact, the numerator of the response rate
* `conversations_no_response` - Conversations with no reply at all from the contact
* `appointments` - Conversations with at least one appointment scheduled, the numerator of the appointment rate
* `granularity` - Bucket size of the record, always `day`
* `tz` - Timezone used to group the records by calendar day
* `as_of` - Timestamp of the last successful Morada.ai analytics job that produced these numbers
* `stale` - Whether the analytics data was behind schedule when it was extracted
Daily Talk ticket counters and service-level components behind the Talk dashboard, broken down by queue, attendant, status, and source.
**Key Fields:**
* `period` - Calendar day of the record, in the timezone reported by the `tz` field
* `queue` - Identifier of the distribution queue of the tickets
* `queue_name` - Name of the distribution queue of the tickets
* `agent` - Identifier of the user who attended the tickets
* `agent_name` - Name of the user who attended the tickets
* `status` - Status of the tickets at the time of the snapshot: `waiting`, `in_service`, `redistributed`, or `closed`
* `source` - Origin of the tickets
* `tickets_created_count` - Tickets created in the day
* `tickets_closed` - Tickets closed in the day
* `tickets_in_service` - Tickets under service at the time of the snapshot
* `redistributed_timeout` - Tickets redistributed because the attendant did not answer in time
* `total_messages` - Total number of messages sent by human attendants
* `time_to_assignment_sum_seconds` - Sum, in seconds, of the time between ticket creation and assignment
* `time_to_assignment_count` - Number of tickets contributing to the time to assignment sum
* `first_response_sum_seconds` - Sum, in seconds, of the first response time
* `first_response_count` - Number of tickets contributing to the first response sum
* `handle_time_sum_seconds` - Sum, in seconds, of the ticket handle time
* `handle_time_count` - Number of tickets contributing to the handle time sum
* `granularity` - Bucket size of the record, always `day`
* `tz` - Timezone used to group the records by calendar day
* `as_of` - Timestamp of the last successful Morada.ai analytics job that produced these numbers
* `stale` - Whether the analytics data was behind schedule when it was extracted
When each analytics dataset was last refreshed by Morada.ai. Use it to know how up to date the analytics tables are before reporting on them.
**Key Fields:**
* `metricset` - Name of the analytics dataset, such as `conversations` or `tickets`
* `etl_job` - Name of the Morada.ai job that feeds the dataset
* `as_of` - Timestamp of the last successful run of the job
* `stale` - Whether the dataset is behind schedule or its freshness is unknown
# Moskit as a data source
Source: https://docs.nekt.com/sources/moskit
Bring data from Moskit to Nekt.
Moskit is a customer service and support management platform that helps businesses handle customer inquiries and support requests efficiently. It provides tools for ticket management, customer communication, and service analytics to improve customer satisfaction and support operations.
![]()
## Configuring Moskit as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Moskit** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Moskit data. Check [Moskit documentation](https://moskit.stoplight.io/docs/api-v2/6a456d10b1b93-consideracoes-iniciais#autentica%C3%A7%C3%A3o) to discover where you can generate the API Key for the connection.
The following configurations are available:
* **API Key**: The key to authenticate against the API service. This token can be obtained from the Moskit CRM on Marketplace > Public API.
* **Enable deal notes stream**: (Default: `false`) Enables the `deal_notes` stream: the notes written on each deal, which are not the same as the notes and doneNotes columns of the activities stream. Moskit exposes no field recording when a note was written, so this stream cannot run incrementally and performs an additional API call for every deal that has notes, on every run. Enable only if necessary, since it significantly increases the extraction time.
* **Stage movements lookback days**: (Default: 1) Advanced: number of days re-checked before the saved bookmark when selecting which deals the `deal_stage_movements` stream visits. Movements are deduplicated by primary key, so a wider window costs extraction time but never duplicates rows.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters. You can inform things like what data it brings, to which team it belongs, etc.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description. Streams marked *optional* are only discovered when the corresponding setting is enabled.
| Stream | Slug | Description |
| - | - | - |
| Activities | `activities` | Activities logged against contacts, deals and companies. |
| Activity Types | `activity_types` | Types available for activities. |
| Companies | `companies` | Companies stored in the CRM. |
| Contacts | `contacts` | People stored in the CRM. |
| Custom Field Options | `custom_field_options` | Available options for each custom field. |
| Custom Fields | `custom_fields` | Custom field definitions. |
| Deal Notes | `deal_notes` | *Optional.* Notes written against a deal. |
| Deal Stage Movements | `deal_stage_movements` | Stage changes (movements) of a deal. |
| Deals | `deals` | Sales opportunities tracked in the CRM. |
| Lost Reasons | `lost_reasons` | Reasons registered for lost deals. |
| Pipelines | `pipelines` | Pipelines used to organize deals. |
| Projects | `projects` | Projects registered in the account. |
| Proposals | `proposals` | Proposals generated for deals. |
| Stages | `stages` | Stages within pipelines. |
| Steps | `steps` | Steps configured in the account. |
| Teams | `teams` | Teams defined in the account. |
| Users | `users` | Users of the account. |
## Fields by stream
Below you'll find the available data streams from Moskit and their corresponding fields:
Stream for retrieving activities data.
Stream containing the different types of activities available in Moskit.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the activity type |
| `name` | String | Name of the activity type |
Stream for retrieving companies data.
Stream for retrieving contacts data.
Stream containing the custom fields configured in the Moskit workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the custom field |
| `name` | String | Name of the custom field |
| `active` | Boolean | Indicates if the custom field is currently active |
| `priority` | Integer | Display priority or order |
| `type` | String | Data type of the field |
| `module` | String | The module the field belongs to |
| `pipelines` | Array | List of associated pipeline objects |
Stream containing the available options for dropdown or multiple-choice custom fields.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the option |
| `label` | String | Display label of the option |
| `custom_field_id` | String | Unique identifier of the parent custom field |
Stream containing the notes written against a deal. This stream is only available when "Enable deal notes stream" is turned on in the source configuration.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the note |
| `deal_id` | Integer | Unique identifier for the associated deal |
| `description` | String | Content or description of the note |
| `dateCreated` | String | Date and time when the note was created |
| `user` | Object | Information about the user who created the note |
| `source` | String | Source of the note |
Stream containing the stage changes (movements) of a deal.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the movement |
| `dateCreated` | DateTime | Date and time when the movement occurred |
| `currentStage` | Object | Information about the current stage of the deal |
| `oldStage` | Object | Information about the previous stage of the deal |
| `deal` | Object | Associated deal information |
| `firstNavigation` | Boolean | Indicates if this was the first navigation/opening of the deal |
Stream for retrieving deals (opportunities) data.
Stream containing the reasons why a deal was marked as lost.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the lost reason |
| `name` | String | Description or name of the lost reason |
| `pipelines` | Array | List of associated pipeline objects |
Stream containing the sales pipelines configured in Moskit.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the pipeline |
| `name` | String | Name of the pipeline |
| `priority` | Integer | Display priority or order |
Stream for retrieving projects data.
Stream containing business proposals sent to customers.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the proposal |
| `name` | String | Name of the proposal |
| `pdf` | String | Link or reference to the generated PDF |
| `dateCreated` | String | Date and time when the proposal was created |
| `createdBy` | Object | Information about the user who created the proposal |
| `deal` | Object | Associated deal information |
Stream containing the stages within pipelines.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the stage |
| `name` | String | Name of the stage |
| `priority` | Integer | Display priority or order |
| `description` | String | Detailed description of the stage |
| `pipeline` | Object | Associated pipeline information |
Stream containing step configurations, typically linked to boards.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the step |
| `name` | String | Name of the step |
| `priority` | Integer | Display priority or order |
| `board` | Object | Associated board information |
Stream containing organizational teams.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the team |
| `name` | String | Name of the team |
| `parent` | Object | Information about the parent team, if applicable |
Stream containing the users registered in the Moskit workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the user |
| `name` | String | Full name of the user |
| `username` | String | Username for login |
| `active` | Boolean | Indicates if the user account is active |
| `levelConfig` | Boolean | Configuration access permission |
| `levelExport` | Boolean | Export access permission |
| `levelBulk` | Boolean | Bulk operation access permission |
| `levelDelete` | Boolean | Deletion access permission |
| `levelView` | String | View access level scope |
| `levelEdit` | String | Edit access level scope |
| `jobTitle` | String | User's job title |
| `team` | Object | Associated team information |
| `timezone` | Object | User's timezone settings |
| `picture` | String | URL of the user's profile picture |
| `phones` | Array | List of the user's phone numbers |
| `primaryPhone` | Object | The user's primary phone number |
| `defaultPipeline` | Object | User's default pipeline |
| `defaultBoard` | Object | User's default board |
| `defaultDashboard` | Object | User's default dashboard |
| `dateCreated` | String | Date and time when the user was created |
## Implementation Notes
### Rate Limiting and Retries
The Moskit API enforces a rate limit of 240 requests per minute. The connector paces its requests evenly (around 3.8 requests per second) to stay within this budget and avoid triggering `429 Too Many Requests` errors, ensuring a faster overall extraction. Additionally, the connector handles transient gateway routing failures (a `404 Not Found` with a "no Route matched" message from Moskit's gateway) by transparently retrying the request up to 8 times over roughly three minutes.
### Resilience on Child Streams
For streams that query data per-deal, like **Deal Notes** and **Deal Stage Movements**, if a specific deal consistently fails due to a gateway routing error after all retry attempts, the connector will skip that deal and continue the extraction normally. A warning is logged, and the skipped deal's data may be retrieved on a subsequent run, preventing a single problematic deal from failing the entire extraction.
### Pagination Optimization
Moskit provides a next-page token even on the final page of a list. The connector optimizes API usage by automatically stopping pagination when a returned page has fewer than 50 records (the maximum page size), saving an unnecessary API call that would otherwise return an empty page.
## Skills for agents
Moskit connector documentation as plain markdown, for use in AI agent contexts.
# Mtrix as a data source
Source: https://docs.nekt.com/sources/mtrix
Bring data from Mtrix to Nekt.
Mtrix is a business intelligence platform that provides data and analytics solutions, primarily focused on sales, distribution, and market trends. It helps companies monitor their performance, understand market dynamics, and make data-driven decisions.
![]()
## Configuring Mtrix as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Mtrix option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your Mtrix `Access Key` and `Secret Key` for this connection.
The following configurations are available:
* **Access Key**: The access key to authenticate against the Mtrix API service.
* **Secret Key**: The secret key to authenticate against the Mtrix API service.
* **Start Date**: The earliest date from which you want to start syncing records for historical load.
* **Lookback Window**: The number of months to look back for incremental sync.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Mtrix and their corresponding fields:
Stream for managing sellout facts.
**Primary Keys:** `INVOICE_ID`
**Replication Key:** `SELLOUT_DATE`
**Fields:**
* `ROW_NUMBER` - Row number.
* `DISTRIBUTOR_CODE` - Distributor code.
* `SELLOUT_DATE` - Sellout date.
* `CUSTOMER_ID` - Customer ID.
* `SELLOUT_TYPE` - Sellout type.
* `INVOICE_ID` - Invoice ID.
* `PRODUCT_CODE` - Product code.
* `SALESREP_ID` - Sales representative ID.
* `QTY_UNIT` - Quantity in units.
* `QTY_CONV1` - Converted quantity 1.
* `QTY_CONV2` - Converted quantity 2.
* `QTY_CONV3` - Converted quantity 3.
* `QTY_CONV4` - Converted quantity 4.
* `QTY_CONV5` - Converted quantity 5.
* `QTY_CONV6` - Converted quantity 6.
* `QTY_CONV7` - Converted quantity 7.
* `QTY_CUSTOM1` - Custom quantity 1.
* `QTY_CUSTOM2` - Custom quantity 2.
* `SELLOUT_VALUE_LC` - Sellout value in local currency.
* `SELLOUT_CONV1` - Converted sellout value 1.
## Data Model
The data model for Mtrix is straightforward as it consists of a single primary stream, `facts_sellout`. This stream contains detailed records of sellout transactions and can be used for various sales and performance analyses.
## Additional Notes
* Ensure that the user credentials have the necessary permissions to access the desired endpoints in Mtrix.
* The amount of data can be large. Use the `start_date` parameter to limit the initial sync period.
* The `lookback_window` parameter is useful for incremental syncs to ensure data consistency.
## Skills for agents
Mtrix connector documentation as plain markdown, for use in AI agent contexts.
# Mutual as a data source
Source: https://docs.nekt.com/sources/mutual
Bring data from Mutual to Nekt.
Mutual is a mutual fund and investment management platform that helps individuals and businesses manage their investment portfolios. It provides tools for fund analysis, portfolio tracking, and investment decision-making to support financial planning and wealth management.
![]()
## Configuring Mutual as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Mutual** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Mutual data by providing your API credentials.
The following configurations are available:
* **Access token**: The token required to authenticate against the API service.
* **API base URL**: The base URL to access the API services (e.g., `https://.mutualignit.com.br/public_api/v2`).
* **Start date**: Records created or updated after the start date will be extracted from the source.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: Choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: A folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: We suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster.
* **Sync Type**: You can choose between **INCREMENTAL** and **FULL\_TABLE**.
* **Incremental**: Every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* **Full table**: Every time the extraction happens, we'll get the current state of the data, which is good if you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* **Delta Log Retention**: Determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* **Additional Full Sync**: Determine when to execute an additional full sync. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Streams and Fields
Below you'll find the available data streams from Mutual and their corresponding fields:
| Field | Type | Description |
| :- | :- | :- |
| `address_id` | Integer | Address ID |
| `know_option_id` | Integer | Know option ID |
| `regional_id` | Integer | Regional ID |
| `uuid` | String | Unique identifier |
| `marital_status` | String | Marital status |
| `mother_name` | String | Mother's name |
| `is_referral_eligible` | Boolean | Indicates if the person is eligible for referrals |
| `invoice_unification_type` | String | Invoice unification type |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier |
| `created_at` | DateTime | Creation timestamp |
| `updated_at` | DateTime | Last update timestamp |
| `name` | String | Name |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `can_assign_inspection` | Boolean | Can assign inspection |
| `can_type_accession` | Boolean | Can type accession |
| `status_discount` | Boolean | Status discount |
| `uuid` | String | Unique identifier |
| `person_type` | String | Person type |
| `consultant_origin` | String | Consultant origin |
| `can_generate_accession_monthly_payment` | Boolean | Can generate accession monthly payment |
| `can_generate_accession` | Boolean | Can generate accession |
| `can_generate_product_fee` | Boolean | Can generate product fee |
| `can_edit_pending_contracts_in_analysis` | Boolean | Can edit pending contracts in analysis |
| `classification_id` | Integer | Classification ID |
| Field | Type | Description |
| :- | :- | :- |
| `sms_name` | String | SMS name |
| `cpf_cnpj` | String | CPF or CNPJ |
| `address_id` | Integer | Address ID |
| `uuid` | String | Unique identifier |
| `association_id` | Integer | Association ID |
| Field | Type | Description |
| :- | :- | :- |
| `cpf_cnpj` | String | CPF or CNPJ |
| `regional_id` | Integer | Regional ID |
| `licensed_id` | Integer | Licensed ID |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `bank_id` | Integer | Bank ID |
| `total_installment_value` | Number | Total installment value |
| `total_accession_value` | Number | Total accession value |
| `uuid` | String | Unique identifier |
| `previous_consultant` | Integer | Previous consultant ID |
| `previous_sales_team` | Integer | Previous sales team ID |
| `previous_regional` | Integer | Previous regional ID |
| `contract_negotion_type` | String | Contract negotiation type |
| `referred_by_id` | Integer | Referred by ID |
| `first_contract_active` | DateTime | First contract active timestamp |
| `date_inspection_contract` | Date | Date of inspection contract |
| Field | Type | Description |
| :- | :- | :- |
| `final_total_value` | Number | Final total value |
| `implementos_ids` | Array of Integer | Array of implementos IDs |
| `contract_status` | String | Contract status |
| `uuid` | String | Unique identifier |
| `recommended_by_id` | Integer | Recommended by ID |
| `sub_status_id` | Integer | Sub status ID |
| `is_fixed_cota` | Boolean | Is fixed cota |
| `consultant_id` | Integer | Consultant ID |
| `sales_team_id` | Integer | Sales team ID |
| `regional_id` | Integer | Regional ID |
| `due_day` | String | Due day |
| Field | Type | Description |
| :- | :- | :- |
| `city` | String | City name |
| `number` | String | Address number |
| `complement` | String | Address complement |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `all_consultants` | Boolean | Has access to all consultants |
| `edit_quota` | Boolean | Can edit quota |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `sindicancia_status` | String | Sindicancia status |
| `address_id` | Integer | Address ID |
| `event_status` | String | Event status |
| `value_event_types` | Array of String | Types of events |
| `protocol` | String | Protocol |
| `sub_status_id` | Integer | Sub status ID |
| `type_involvement` | String | Type of involvement |
| Field | Type | Description |
| :- | :- | :- |
| `address_id` | Integer | Address ID |
| `workshop_id` | Integer | Workshop ID |
| `sub_status_id` | Integer | Sub status ID |
| `uuid` | String | Unique identifier |
| `documentation_delivery_date` | Date | Documentation delivery date |
| `date_analysis_documentation` | Date | Date analysis documentation |
| `workshop_direction_date` | Date | Workshop direction date |
| `regulation_schedule_date` | Date | Regulation schedule date |
| `return_date_regulation` | Date | Return date regulation |
| `budget_authorization_date` | Date | Budget authorization date |
| `supervision_inspection_one` | Date | Supervision inspection one date |
| `supervision_inspection_two` | Date | Supervision inspection two date |
| `supervision_inspection_three` | Date | Supervision inspection three date |
| `supervision_inspection_four` | Date | Supervision inspection four date |
| `quality_inspection_request` | Date | Quality inspection request date |
| `repair_completion_date` | Date | Repair completion date |
| `parts_quote_date` | Date | Parts quote date |
| `parts_purchase_date` | Date | Parts purchase date |
| `parts_delivery_forecast` | Date | Parts delivery forecast date |
| `parts_delivery_date` | Date | Parts delivery date |
| `reason` | String | Reason |
| Field | Type | Description |
| :- | :- | :- |
| `deleted` | Boolean | Is deleted |
| `label` | String | Label |
| `hide` | Boolean | Is hidden |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `model_year` | String | Model year |
| `implemento_price` | Number | Implemento price |
| `renavam` | String | Renavam |
| `uuid` | String | Unique identifier |
| `implement_color_id` | Integer | Implement color ID |
| `external_id` | String | External ID |
| Field | Type | Description |
| :- | :- | :- |
| `updated_at` | DateTime | Last update timestamp |
| `deleted` | Boolean | Is deleted |
| `name` | String | Name |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `payment_attempts` | Array of String | Payment attempts |
| `in_serasa` | Boolean | In Serasa |
| `uuid` | String | Unique identifier |
| `billing_simulation_id` | Integer | Billing simulation ID |
| `payment_limit_date` | Date | Payment limit date |
| `days_pro_rata` | Integer | Days pro rata |
| `daily_pro_rata_value` | Number | Daily pro rata value |
| `accession_amount` | Number | Accession amount |
| `monthly_payment_amount` | Number | Monthly payment amount |
| `linha_digitavel` | String | Linha digitável (typeable line for payment) |
| `codigo_de_barra` | String | Código de barra (barcode) |
| `invoice_billet_token` | String | Invoice billet token |
| `invoice_billet_url` | String | Invoice billet URL |
| `checkout_link` | String | Checkout link |
| `pix_qrcode` | String | PIX QR Code |
| Field | Type | Description |
| :- | :- | :- |
| `pix_emv` | String | PIX EMV |
| `pix_qrcode` | String | PIX QR Code |
| `pix_paid` | Boolean | Is PIX paid |
| `uuid` | String | Unique identifier |
| `pix_generated_at` | DateTime | PIX generation timestamp |
| `shorten_url` | String | Shortened URL |
| Field | Type | Description |
| :- | :- | :- |
| `invoice_billet_id` | Integer | Invoice billet ID |
| `contract_object_type` | String | Contract object type |
| `uuid` | String | Unique identifier |
| `plan_name` | String | Plan name |
| `plate` | String | License plate |
| `chassi` | String | Chassi |
| `assembler` | String | Assembler |
| `model` | String | Model |
| `vehicle_type_name` | String | Vehicle type name |
| `vehicle_category_name` | String | Vehicle category name |
| `vehicle_use_type_name` | String | Vehicle use type name |
| `fabrication_year` | String | Fabrication year |
| `model_year` | String | Model year |
| `renavam` | String | Renavam |
| `vehicle_color_name` | String | Vehicle color name |
| `plate_implements` | Array of String | Plate implements |
| `chassi_implements` | Array of String | Chassi implements |
| `model_implements` | Array of String | Model implements |
| `assembler_implements` | Array of String | Assembler implements |
| `fabrication_year_implements` | Array of String | Fabrication year implements |
| `model_year_implements` | Array of String | Model year implements |
| `types_implements` | Array of String | Types implements |
| `renavam_implements` | Array of String | Renavam implements |
| `implement_color_names` | Array of String | Implement color names |
| Field | Type | Description |
| :- | :- | :- |
| `emv` | String | EMV string |
| `qrcode` | String | QR Code |
| `uuid` | String | Unique identifier |
| Field | Type | Description |
| :- | :- | :- |
| `invoice_status` | String | Invoice status |
| `contract_object_type` | String | Contract object type |
| `uuid` | String | Unique identifier |
| `wallet_uid` | String | Wallet UID |
| `implemento_type_id` | Integer | Implemento type ID |
| `vehicle_type_id` | Integer | Vehicle type ID |
| `vehicle_type_name` | String | Vehicle type name |
| Field | Type | Description |
| :- | :- | :- |
| `fuel_type` | String | Fuel type |
| `external_id` | String | External ID |
| `uuid` | String | Unique identifier |
| `km` | Number | Kilometers (mileage) |
| `transmission_type` | String | Transmission type |
## Skills for agents
Mutual connector documentation as plain markdown, for use in AI agent contexts.
# MySQL as a data source
Source: https://docs.nekt.com/sources/mysql
Bring data from MySQL to Nekt.
MySQL is a popular open-source relational database management system (RDBMS) that uses SQL for managing and organizing data. It's widely used for web applications, data warehousing, and enterprise software, providing robust data storage, retrieval, and management capabilities.
![]()
## Configuring MySQL as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **MySQL** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Check MySQL's documentation to discover where you can find the required parameters for the connection. Provide the necessary credentials and connection details to allow Nekt to access your database.
Once you're done, click **Next**.
### 2. Select streams
Choose which tables (streams) you want to sync. For faster extractions, select only the tables that are relevant to your analysis. You can select entire groups of tables or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully. (Note: companies in the Growth plan can choose in which layer the tables will be placed).
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
* **Max batch size for parquet files**: (Default: `100000`) Available under advanced settings, this determines the maximum batch size for target S3 parquet files.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Since MySQL is a relational database, the available streams and fields will correspond exactly to the tables, views, and columns present in your specific database schema. Each selected table will be replicated as a stream, and its columns will map to the fields within that stream.
### System and Metadata Fields
When syncing data using log-based replication (CDC / binlog) for incremental extractions, the connector automatically appends metadata fields to your tables to track row-level changes, operations, and timestamps.
These fields are added to your records when using log-based incremental syncs:
| Field | Type | Description |
| :- | :- | :- |
| `_sdc_operation` | String | The type of operation that occurred on the record in the source (e.g., INSERT, UPDATE, DELETE). |
| `_sdc_event_timestamp` | String | The exact timestamp of when the event occurred in the source database. |
| `_sdc_deleted_at` | String | The timestamp when the record was deleted (if the operation was a deletion). |
| `_sdc_lsn` | String | The Log Sequence Number or binlog position associated with the event. |
## Implementation Notes
### Performance Optimizations
The connector is optimized to handle large databases efficiently by automatically parallelizing the schema discovery phase. It concurrently pre-warms the reflection cache for all tables and views (using up to 16 concurrent workers), significantly reducing the time required to discover and map wide schemas, a crucial optimization when querying hundreds of tables over high-latency networks or SSH tunnels.
### Log-based Replication (CDC) Requirements
When using log-based replication (CDC) for incremental syncs, your MySQL server must be configured with specific binary log settings to ensure all row data and column names are captured correctly.
Ensure the following variables are set on your MySQL server:
* `binlog_row_metadata = FULL`: In this mode, the binary log records column names. If set to `MINIMAL` (the default in some MySQL 8 versions), the connector may not be able to resolve column names and will abort the extraction to prevent data loss.
* `binlog_row_image = FULL`: In this mode, the binary log records all columns for `UPDATE` and `DELETE` operations. If set to `MINIMAL`, the connector receives partial rows which may result in missing identifier columns and rejected records.
### Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Extraction fails with unnamed binlog columns error | `binlog_row_metadata` is not set to `FULL` | The database is not providing column names in the binary log, so changes cannot be matched to the table's columns. Ask your DBA to set `binlog_row_metadata = FULL` on the MySQL server, then run the source again. |
| Missing key properties error during UPDATE/DELETE | `binlog_row_image` is not set to `FULL` | The binary log is only recording changed columns, so updates and deletes arrive without the rest of the row. Ask your DBA to set `binlog_row_image = FULL` on the MySQL server. |
| `Lost connection to MySQL server during query (2013)` | The query ran longer than the server keeps the connection open | Enable **Use batch query** and raise **Chunk size** in the source advanced settings. |
| The connection is accepted and closed before login | The server sits behind a firewall allowlist that does not include Nekt | Allow Nekt's outbound addresses on the firewall, or use an SSH tunnel. Both are covered in the [database connection guide](/sources/pre-work-db-connection). |
| Log-based extraction stops advancing after the source was paused for a while | The server expired the binlog files covering the point where extraction stopped | Increase binlog retention on the server and run a full sync to reset the read position. |
## Skills for agents
MySQL connector documentation as plain markdown, for use in AI agent contexts.
# NectarCRM as a data source
Source: https://docs.nekt.com/sources/nectarcrm
Bring data from NectarCRM to Nekt.
NectarCRM is a Brazilian CRM platform designed for B2B sales teams. It provides tools for managing contacts, opportunities, qualifications, tasks, appointments, and sales pipelines, helping teams track their sales processes from prospecting to closing.
![]()
## Configuring NectarCRM as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the NectarCRM option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to provide your NectarCRM API access token. To find it, go to **Settings > Integrations > Public Token** in your NectarCRM account.
The following configurations are available:
* **API Key**: Your NectarCRM API access token (required).
* **Start Date**: The earliest date from which records will be synced.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.
* **Layer**: choose between the existing layers on your catalog.
* **Folder**: a folder can be created inside the selected layer to group all tables.
* **Table name**: a name is suggested, but feel free to customize it. You can add a **prefix** to all tables at once.
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: only new or updated data is fetched each run.
* Full table: the current state of the data is fetched each run.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## API Limitations
**Pagination and 100-Page Cap**
The NectarCRM API strictly caps paginated listings at 100 pages. With a fixed maximum of 100 records per page, a single query can never return more than 10,000 records. Requesting page 101 results in an API error.
To safely extract large volumes of data while respecting this limit, the connector employs **dynamic date-windowing** for streams that support date filtering (`Appointments`, `Contacts`, `Feedbacks`, `Opportunities`, `Qualifications`, and `Tasks`). The connector automatically divides the extraction period into smaller date chunks, shrinking or expanding the window size based on data density to ensure no single window reaches the 10,000-record cap.
For streams that do not support date filters (such as `Products`, `Users`, `Pipelines`, etc.), pagination behaves normally up to page 100. If an entity listing exceeds 10,000 records, the connector will sync the first 10,000 records and stop cleanly, logging a warning rather than crashing.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Contacts | `contacts` | Contacts (people) stored in the CRM. |
| Opportunities | `opportunities` | Sales opportunities and their pipeline status. |
| Qualifications | `qualifications` | Qualification records used to score opportunities. |
| Appointments | `appointments` | Scheduled appointments and meetings. |
| Tasks | `tasks` | Tasks and follow-up activities. |
| Feedbacks | `feedbacks` | Feedback entries logged in the CRM. |
| Products | `products` | Products available in the catalog. |
| Product Categories | `product_categories` | Categories used to group products. |
| Product Subcategories | `product_subcategories` | Subcategories nested under product categories. |
| Price Tables | `price_tables` | Price tables applied to products and proposals. |
| Proposals | `proposals` | Commercial proposals sent to contacts. |
| Pipelines | `pipelines` | Sales pipelines and their stages. |
| Custom Fields | `custom_fields` | Definitions of custom fields configured in the account. |
| Sources | `sources` | Lead sources (origin channels). |
| Segments | `segments` | Contact segments used for grouping. |
| Categories | `categories` | Generic categories used across the CRM. |
| Users | `users` | Users of the CRM account. |
| Teams | `teams` | Teams that users belong to. |
Fields typed as **String (JSON)** carry nested objects or arrays of objects returned by the API, serialized as JSON strings. This keeps the warehouse schema stable while preserving all data returned by NectarCRM.
## Fields by stream
Below you'll find all available data streams from NectarCRM and their corresponding fields:
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the contact |
| `nome` | String | Name of the contact |
| `razaoSocial` | String | Legal name (Razão Social) |
| `ativo` | Boolean | Indicates if the contact is active |
| `blocked` | Boolean | Indicates if the contact is blocked |
| `empresa` | Boolean | Indicates if the contact is a company |
| `isEmpresa` | Boolean | Company flag |
| `integradoRD` | Boolean | Indicates if the contact is integrated with RD Station |
| `constante` | Integer | Constant identifier |
| `codigo` | String | Contact code |
| `cnpj` | String | CNPJ (Brazilian company registry number) |
| `cpf` | String | CPF (Brazilian individual registry number) |
| `cargo` | String | Job title |
| `inscricaoEstadual` | String | State registration number |
| `inscricaoMunicipal` | String | Municipal registration number |
| `ramal` | String | Extension number |
| `observacao` | String | Notes |
| `foto` | String | Photo URL |
| `site` | String | Website |
| `facebook` | String | Facebook profile |
| `linkedin` | String | LinkedIn profile |
| `skype` | String | Skype handle |
| `twitter` | String | Twitter handle |
| `receitaAnual` | Number | Annual revenue |
| `diasSemInteracao` | Integer | Days without interaction |
| `tarefas` | Integer | Number of tasks |
| `compromissos` | Integer | Number of appointments |
| `oportunidades` | Integer | Number of opportunities |
| `qtdeQualificacoes` | Integer | Number of qualifications |
| `email` | String | Email address |
| `emailPrincipal` | String | Primary email address |
| `emails` | Array of strings | List of email addresses |
| `emailObjects` | String (JSON) | Full email objects |
| `telefone` | String | Phone number |
| `telefonePrincipal` | String | Primary phone number |
| `telefones` | Array of strings | List of phone numbers |
| `categoria` | String | Category label |
| `categoriaObject` | String (JSON) | Full category object |
| `origem` | String | Origin label |
| `origemObject` | String (JSON) | Full origin object |
| `segmento` | String | Segment label |
| `segmentoObject` | String (JSON) | Full segment object |
| `tipoEntidade` | String | Entity type |
| `autor` | String (JSON) | Author of the contact |
| `autorAtualizacao` | String (JSON) | Author of the last update |
| `responsavel` | String (JSON) | Responsible user |
| `corresponsaveis` | String (JSON) | Co-responsible users |
| `contatoPai` | String (JSON) | Parent contact |
| `contatos` | String (JSON) | Related contacts |
| `contatosPai` | String (JSON) | Parent contacts |
| `empresaAtual` | String (JSON) | Current company |
| `empresasAtuais` | String (JSON) | Current companies |
| `listas` | String (JSON) | Lists the contact belongs to |
| `enderecos` | String (JSON) | Address information |
| `regiaoEstado` | String (JSON) | State region |
| `regiaoMunicipio` | String (JSON) | Municipality region |
| `regiaoPais` | String (JSON) | Country region |
| `camposPersonalizados` | String (JSON) | Custom fields |
| `camposPersonalizadosObject` | String (JSON) | Full custom field objects |
| `dataFundacao` | Datetime | Company founding date |
| `dataNascimento` | Datetime | Birth date |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the opportunity |
| `codigo` | String | Code of the opportunity |
| `nome` | String | Name of the opportunity |
| `observacao` | String | Notes |
| `consideracoes` | String | Considerations |
| `pipeline` | String | Pipeline name |
| `etapa` | Integer | Current stage of the opportunity |
| `etapaNome` | String | Name of the current stage |
| `status` | Integer | Status code |
| `temperatura` | String | Temperature or urgency |
| `probabilidade` | Integer | Probability of winning |
| `moeda` | String | Currency |
| `moedaBase` | String | Base currency |
| `vendaBase` | Boolean | Base sale flag |
| `atividadesAtrasadas` | Integer | Number of overdue activities |
| `compromissos` | Integer | Number of appointments |
| `tarefas` | Integer | Number of tasks |
| `qtdPropostas` | Integer | Number of proposals |
| `qtdProrrogada` | Integer | Number of extensions |
| `quantidadeProdutos` | Integer | Number of products |
| `diasEstagnacaoNaEtapa` | Integer | Days stalled in the current stage |
| `diasSemInteracao` | Integer | Days without interaction |
| `valorAvulso` | Number | One-time value |
| `valorAvulsoMoedaBase` | Number | One-time value in base currency |
| `valorLiquido` | Number | Net value |
| `valorMensal` | Number | Monthly value |
| `valorMensalMoedaBase` | Number | Monthly value in base currency |
| `valorPrevistoLiquido` | Number | Expected net value |
| `valorTotal` | Number | Total value |
| `valorTotalDescontos` | Number | Total discount value |
| `valorTotalLiquido` | Number | Total net value |
| `valorTotalMoedaBase` | Number | Total value in base currency |
| `autor` | String (JSON) | Author of the opportunity |
| `autorAtualizacao` | String (JSON) | Author of the last update |
| `autorConclusao` | String (JSON) | Author of the conclusion |
| `responsavel` | String (JSON) | Responsible user |
| `corresponsaveis` | String (JSON) | Co-responsible users |
| `cliente` | String (JSON) | Client information |
| `contato` | String (JSON) | Contact information |
| `contatoPrincipal` | String (JSON) | Primary contact |
| `etapaAtual` | String (JSON) | Current stage object |
| `funilVenda` | String (JSON) | Sales funnel |
| `origem` | String (JSON) | Origin |
| `origemObject` | String (JSON) | Full origin object |
| `oportunidadeOrigem` | String (JSON) | Originating opportunity |
| `qualificacaoOrigem` | String (JSON) | Originating qualification |
| `listas` | String (JSON) | Lists |
| `produtos` | String (JSON) | Products linked to the opportunity |
| `camposListagem` | String (JSON) | Listing fields |
| `camposPersonalizados` | String (JSON) | Custom fields |
| `camposPersonalizadosObject` | String (JSON) | Full custom field objects |
| `justificativas` | Array of strings | Justifications |
| `justificativasOpcoes` | Array of strings | Justification options |
| `dataLimite` | Datetime | Deadline date |
| `dataProrrogacao` | Datetime | Extension date |
| `stageEntryDate` | Datetime | Stage entry date |
| `dataConclusao` | Datetime | Conclusion date |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the qualification |
| `codigo` | String | Code |
| `pipeline` | String | Pipeline name |
| `etapa` | Integer | Stage |
| `status` | Integer | Status code |
| `handRaised` | Boolean | Hand-raised flag |
| `pendingReschedule` | Boolean | Pending reschedule flag |
| `descarteSolicitado` | Boolean | Discard requested flag |
| `reschedulingCount` | Integer | Number of reschedules |
| `compromissos` | Integer | Number of appointments |
| `tarefas` | Integer | Number of tasks |
| `tarefasAbertas` | Integer | Number of open tasks |
| `autor` | String (JSON) | Author of the qualification |
| `autorAtualizacao` | String (JSON) | Author of the last update |
| `autorConclusao` | String (JSON) | Author of the conclusion |
| `responsavel` | String (JSON) | Responsible user |
| `responsavelComercial` | String (JSON) | Commercial responsible user |
| `cliente` | String (JSON) | Client information |
| `compromisso` | String (JSON) | Related appointment |
| `oportunidade` | String (JSON) | Related opportunity |
| `etapaAtual` | String (JSON) | Current stage object |
| `funilVenda` | String (JSON) | Sales funnel |
| `listas` | String (JSON) | Lists |
| `camposListagem` | String (JSON) | Listing fields |
| `camposPersonalizados` | String (JSON) | Custom fields |
| `dataLimite` | Datetime | Deadline date |
| `dataConclusao` | Datetime | Conclusion date |
| `dataInicioEtapa` | Datetime | Stage start date |
| `dataPrimeiraInteracao` | Datetime | First interaction date |
| `dataPrimeiroContato` | Datetime | First contact date |
| `delayedUntil` | Datetime | Delayed-until date |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the appointment |
| `assunto` | String | Subject |
| `descricao` | String | Description |
| `feedback` | String | Feedback text |
| `endereco` | String | Address |
| `local` | String | Location |
| `latitude` | String | Latitude |
| `longitude` | String | Longitude |
| `status` | Integer | Status code |
| `diaInteiro` | Boolean | Indicates if it's an all-day event |
| `gerouInteresse` | Boolean | Indicates if the appointment generated interest |
| `kmRodado` | Boolean | Mileage flag |
| `notificarParticipantes` | Boolean | Notify participants flag |
| `autor` | String (JSON) | Author of the appointment |
| `autorAtualizacao` | String (JSON) | Author of the last update |
| `autorFeedback` | String (JSON) | Author of the feedback |
| `responsavel` | String (JSON) | Responsible user |
| `cliente` | String (JSON) | Client information |
| `oportunidade` | String (JSON) | Related opportunity |
| `qualificacao` | String (JSON) | Related qualification |
| `tipo` | String (JSON) | Appointment type |
| `participantes` | String (JSON) | List of participants |
| `camposPersonalizados` | String (JSON) | Custom fields |
| `camposPersonalizadosObject` | String (JSON) | Full custom field objects |
| `usuariosLocalizacoes` | Array of strings | User locations |
| `dataInicio` | Datetime | Start date and time |
| `dataFim` | Datetime | End date and time |
| `dataFeedback` | Datetime | Feedback date |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the task |
| `titulo` | String | Title |
| `descricao` | String | Description |
| `status` | Integer | Task status (0 = Aberta, 1 = Concluída, 2 = Cancelada, 3 = Em execução) |
| `tipo` | Integer | Type of task |
| `prioridade` | Integer | Priority level |
| `sequencia` | Integer | Sequence number |
| `obrigatoria` | Boolean | Indicates if the task is mandatory |
| `isResponsavelAdmin` | Boolean | Indicates if the responsible user is an admin |
| `autor` | String (JSON) | Author of the task |
| `responsavel` | String (JSON) | Responsible user |
| `cliente` | String (JSON) | Client information |
| `oportunidade` | String (JSON) | Related opportunity |
| `qualificacao` | String (JSON) | Related qualification |
| `checklist` | String (JSON) | Checklist information |
| `modeloTarefa` | String (JSON) | Task template |
| `tarefaPaiSub` | String (JSON) | Parent/sub task |
| `tarefaTipo` | String (JSON) | Task type object |
| `tipoLigacao` | String (JSON) | Call type |
| `camposPersonalizados` | String (JSON) | Custom fields |
| `camposPersonalizadosObject` | String (JSON) | Full custom field objects |
| `dataInicio` | Datetime | Start date |
| `dataFim` | Datetime | End date |
| `dataLimite` | Datetime | Deadline date |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the feedback |
| `assunto` | String | Subject |
| `descricao` | String | Description |
| `tipo` | Integer | Type code |
| `classeId` | Integer | Class identifier |
| `objetoId` | Integer | Object identifier |
| `importante` | Boolean | Indicates if the feedback is marked as important |
| `automatica` | Boolean | Indicates if it was automatically generated |
| `autor` | String (JSON) | Author of the feedback |
| `contato` | String (JSON) | Related contact |
| `oportunidade` | String (JSON) | Related opportunity |
| `compromisso` | String (JSON) | Related appointment |
| `qualificacao` | String (JSON) | Related qualification |
| `tarefa` | String (JSON) | Related task |
| `anexos` | String (JSON) | List of attachments |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the product |
| `nome` | String | Name of the product |
| `descricao` | String | Description |
| `codigo` | String | Product code |
| `alias` | String | Alias |
| `moeda` | String | Currency |
| `ativo` | Boolean | Indicates if the product is active |
| `contador` | Integer | Counter |
| `recorrencia` | Integer | Recurrence type/interval |
| `tipoPrecificacao` | Integer | Pricing type |
| `acrescimoPercentual` | Boolean | Percentage surcharge flag |
| `agruparProdutosDependentes` | Boolean | Group dependent products flag |
| `bloquearEdicaoMoeda` | Boolean | Lock currency editing flag |
| `bloquearEdicaoRecorrencia` | Boolean | Lock recurrence editing flag |
| `comissaoPorcentual` | Boolean | Percentage commission flag |
| `costPriceInPercentage` | Boolean | Cost price in percentage flag |
| `descontoPorcentual` | Boolean | Percentage discount flag |
| `editarProdutosDependentes` | Boolean | Edit dependent products flag |
| `fracionario` | Boolean | Fractional quantity flag |
| `isExpiravel` | Boolean | Expirable flag |
| `isMargemLucro` | Boolean | Profit margin flag |
| `manterValorTabelaPreco` | Boolean | Keep price table value flag |
| `permiteAcrescimo` | Boolean | Allows surcharge flag |
| `permiteDesconto` | Boolean | Allows discount flag |
| `possuiEstoque` | Boolean | Has stock flag |
| `possuiFormula` | Boolean | Has formula flag |
| `precoCompostoProposta` | Boolean | Composite price in proposal flag |
| `valorEditavel` | Boolean | Editable value flag |
| `acrescimoMaximo` | Number | Maximum surcharge |
| `comissao` | Number | Commission |
| `descontoMaximo` | Number | Maximum discount |
| `liquidez` | Number | Liquidity |
| `margemInicial` | Number | Initial margin |
| `quantidadeEstoque` | Number | Stock quantity |
| `valorBase` | Number | Base value/price |
| `valorBaseComposto` | Number | Composite base value |
| `valorCusto` | Number | Cost value |
| `categoria` | String (JSON) | Product category |
| `itensDeCusto` | String (JSON) | Cost items |
| `camposPersonalizados` | String (JSON) | Custom fields |
| `camposPersonalizadosObject` | String (JSON) | Full custom field objects |
| `produtos` | Array of strings | Related products |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the product category |
| `nome` | String | Name |
| `ativo` | Boolean | Indicates if it is active |
| `subCategorias` | Array of strings | Subcategories under this category |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the product subcategory |
| `nome` | String | Name |
| `ativo` | Boolean | Indicates if it is active |
| `categoria` | String (JSON) | Parent category |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the price table |
| `nome` | String | Name |
| `descricao` | String | Description |
| `ativo` | Boolean | Indicates if it is active |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the proposal |
| `numero` | Integer | Proposal number |
| `versao` | Integer | Version number |
| `codigo` | String | Code |
| `codigoFormatado` | String | Formatted code |
| `status` | Integer | Status code |
| `statusLabel` | String | Status label |
| `destinatario` | String | Recipient |
| `destinatarioCargo` | String | Recipient's job title |
| `formaPagamento` | String | Payment method |
| `observacao` | String | Notes |
| `publicUrl` | String | Public URL of the proposal |
| `makeOldProposalObsolete` | Boolean | Make old proposal obsolete flag |
| `autor` | String (JSON) | Author of the proposal |
| `modeloProposta` | String (JSON) | Proposal template |
| `oportunidade` | String (JSON) | Related opportunity |
| `dataProposta` | Datetime | Proposal date |
| `dataCriacao` | Datetime | Creation date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the pipeline |
| `nome` | String | Name |
| `moeda` | String | Currency |
| `ativo` | Boolean | Indicates if it is active |
| `padrao` | Boolean | Indicates if it is the default pipeline |
| `tipo` | Integer | Pipeline type |
| `modeloVenda` | Integer | Sales model |
| `cascataCicloVida` | Integer | Lifecycle cascade |
| `tipoRecorrenciaValorFunil` | Integer | Funnel value recurrence type |
| `conversaoDesejada` | Number | Desired conversion rate |
| `allowMultipleLeadboards` | Boolean | Allow multiple leadboards flag |
| `anyStageWin` | Boolean | Win from any stage flag |
| `consideracoesObrigatorioDescartar` | Boolean | Considerations required to discard |
| `consideracoesObrigatorioGanhar` | Boolean | Considerations required to win |
| `consideracoesObrigatorioPerder` | Boolean | Considerations required to lose |
| `consideracoesObrigatorioProrrogar` | Boolean | Considerations required to extend |
| `corresponsabilidade` | Boolean | Co-responsibility flag |
| `createTasksOnWeekdays` | Boolean | Create tasks on weekdays flag |
| `ignorarRestricaoEquipes` | Boolean | Ignore team restrictions flag |
| `iniciarQualquerEtapa` | Boolean | Start at any stage flag |
| `manageLeadResponseTime` | Boolean | Manage lead response time flag |
| `motivoDescarteObrigatorio` | Boolean | Discard reason required flag |
| `motivoGanhoObrigatorio` | Boolean | Win reason required flag |
| `motivoPerdaObrigatorio` | Boolean | Loss reason required flag |
| `multiplasTabelasPreco` | Boolean | Multiple price tables flag |
| `permitePularEtapa` | Boolean | Allow skipping stages flag |
| `permiteRetornoEtapa` | Boolean | Allow returning to a stage flag |
| `permiteTarefasAvulsas` | Boolean | Allow standalone tasks flag |
| `permitirRecorrenciaVariavel` | Boolean | Allow variable recurrence flag |
| `productRequiredToWin` | Boolean | Product required to win flag |
| `produtoObrigatorio` | Boolean | Product required flag |
| `propostaObrigatoria` | Boolean | Proposal required flag |
| `showCustomCard` | Boolean | Show custom card flag |
| `cardFields` | String (JSON) | Card field configuration |
| `modelosFinalizada` | String (JSON) | Finalization templates |
| `sequencias` | String (JSON) | Stage sequences |
| `pipelinesCascade` | Array of strings | Cascade pipelines |
| `ultimaAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the custom field |
| `nome` | String | Name |
| `alias` | String | Alias |
| `relacionado` | String | Related entity |
| `tipo` | Integer | Type of field |
| `classeId` | Integer | Class identifier |
| `obrigatorio` | Boolean | Indicates if it is mandatory |
| `editavel` | Boolean | Indicates if it is editable |
| `escondido` | Boolean | Indicates if it is hidden |
| `mostrarFiltro` | Boolean | Indicates if it appears in filters |
| `unico` | Boolean | Indicates if values must be unique |
| `opcoes` | Array of strings | Available options for selection |
| `camposPersonalizadosFilhos` | Array of strings | Child custom fields |
| `opcoesPersonalizadas` | String (JSON) | Custom options |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the source |
| `nome` | String | Name |
| `descricao` | String | Description |
| `ativo` | Boolean | Indicates if it is active |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the segment |
| `nome` | String | Name |
| `ativo` | Boolean | Indicates if it is active |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the category |
| `nome` | String | Name |
| `ativo` | Boolean | Indicates if it is active |
| `isCliente` | Boolean | Client category flag |
| `isLead` | Boolean | Lead category flag |
| `isProspect` | Boolean | Prospect category flag |
| `isSuspect` | Boolean | Suspect category flag |
| `isDescarte` | Boolean | Discard category flag |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the user |
| `uuid` | String | UUID of the user |
| `login` | String | Login username |
| `usuarioMasterId` | Integer | Master user ID |
| `usuarioMasterUuid` | String | Master user UUID |
| `nome` | String | Full name |
| `email` | String | Email address |
| `telefone` | String | Phone number |
| `cargo` | String | Job title |
| `ramal` | String | Extension number |
| `sipUsername` | String | SIP username for VoIP |
| `locale` | String | User locale/language |
| `tema` | String | UI theme |
| `ativo` | Boolean | Indicates if the user is active |
| `ausente` | Boolean | Indicates if the user is absent |
| `isOnline` | Boolean | Indicates if the user is online |
| `limitContactExport` | Boolean | Contact export limit flag |
| `sobreporRestricoes` | Boolean | Override restrictions flag |
| `perfilUsuario` | String (JSON) | User profile |
| `atribuicoes` | String (JSON) | User assignments |
| `grupos` | String (JSON) | User groups |
| `gruposResponsaveis` | String (JSON) | Groups where user is responsible |
| `gruposLider` | String (JSON) | Groups where user is leader |
| `dataCriacao` | Datetime | Creation date |
| `dataAtivacao` | Datetime | Activation date |
| `dataUltimoAcesso` | Datetime | Last access date |
| `dataAtualizacao` | Datetime | Last update date |
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the team |
| `nome` | String | Team name |
| `ativo` | Boolean | Indicates if the team is active |
| `padrao` | Boolean | Indicates if the team is the default |
| `historicoCompartilhado` | Boolean | Shared history flag |
| `lider` | String (JSON) | Team leader information |
| `participantes` | String (JSON) | Team participants |
| `responsaveis` | String (JSON) | Team responsible users |
## Implementation Notes
### Tasks Stream Partitioning
The NectarCRM API silently filters the `/tarefas/` (Tasks) endpoint to a single status by default (`0` = Aberta) and does not accept a comma-separated list of statuses. To guarantee that all tasks are synced regardless of their status, the connector uses partition-aware logic.
It sweeps the API individually for each task status:
* `0` (Aberta)
* `1` (Concluída)
* `2` (Cancelada)
* `3` (Em execução)
Since every task has exactly one status, these partitioned requests return disjoint sets of IDs that merge cleanly into a single table on Nekt. Additionally, each status partition maintains its own independent incremental state bookmark to ensure accurate state management across all statuses during incremental syncs.
### Opportunities Status Coverage
The `/oportunidades/` endpoint defaults to returning in-progress opportunities only. The connector explicitly requests all statuses (`status=1,2,3,4`) so that won, lost, and extended opportunities are also synced.
## Skills for agents
NectarCRM connector documentation as plain markdown, for use in AI agent contexts.
# Nekt as a data source
Source: https://docs.nekt.com/sources/nekt
Bring data from Nekt to Nekt.
Nekt is a modern data platform that provides comprehensive data engineering solutions for companies. It enables organizations to extract, transform, and load data from various external sources through custom connectors, manage data pipelines, and create powerful analytics workflows.
![]()
## Configuring Nekt as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Nekt option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need the following credentials from your Nekt account:
* **API Key**: Your personal API key for accessing the Nekt API. You can generate one in your Nekt workspace settings under the "API Keys" section. Make sure the key has the necessary permissions to read the data you want to sync.
* **Start Date**: The earliest record date to sync. Only records created or updated after this date will be synced.
Under **Advanced settings** you can also enable:
* **Include deleted items** (default: `false`): Include deleted records in the sync, alongside the active ones. Applies to the `Sources`, `Transformations` and `Destinations` streams, where deleted records can be identified by the `deleted` field. Requires an API key created by an owner or an admin: if the key was created by a member, the sync will fail while this option is enabled.
Once you have the required credentials, add the account access and click **Next**.
### 2. Select streams
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Nekt and their corresponding fields:
Stream containing all data sources configured in your Nekt workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the source |
| `slug` | String | URL-friendly identifier for the source |
| `active` | Boolean | Whether the source is active |
| `status` | String | Current status of the source |
| `deleted` | Boolean | Whether the source has been deleted. Only returned when the **Include deleted items** option is enabled |
| `description` | String | Description of the source |
| `table_prefix` | String | Prefix applied to tables created by this source |
| `created_at` | DateTime | When the source was created |
| `updated_at` | DateTime | When the source was last updated |
| `connector_config` | String | Configuration settings for the connector (JSON string) |
| `connector_version` | String | Version of the connector being used |
| `last_run` | String | Identifier of the last run |
| `python_execution_cpu` | Integer | CPU allocation for Python execution |
| `python_execution_memory` | Integer | Memory allocation for Python execution |
| `spark_driver_cores` | Integer | Number of cores for Spark driver |
| `spark_driver_memory` | Integer | Memory allocation for Spark driver |
| `spark_executor_cores` | Integer | Number of cores per Spark executor |
| `spark_executor_memory` | Integer | Memory allocation per Spark executor |
| `spark_executor_instances` | Integer | Number of Spark executor instances |
| `spark_executor_disk` | Integer | Disk space allocation per Spark executor |
| `spark_execution_timeout_minutes` | Integer | Timeout for Spark execution |
| `settings_number_of_retries` | Integer | Number of retry attempts on failure |
| `settings_retry_delay_seconds` | Integer | Delay between retry attempts |
| `settings_max_consecutive_failures` | Integer | Maximum consecutive failures before pausing |
| `settings_full_sync_cron` | String | Cron expression for full sync schedule |
| `settings_full_sync_cron_timezone` | String | Timezone for full sync schedule |
| `output_layer` | Object | Target layer information (id, name, slug, description, database\_name) |
| `created_by` | Object | User who created the source (id, name, email, role, permissions) |
| `triggers` | Array | Triggers associated with the source (id, type, cron\_expression, cron\_timezone, events, event\_rule, timestamps) |
| `tags` | Array | Tags associated with the source (id, name, timestamps) |
Stream containing all pipeline execution runs in your Nekt workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the run |
| `number` | Integer | Sequential run number |
| `created_at` | DateTime | When the run was created |
| `updated_at` | DateTime | When the run was last updated |
| `started_at` | DateTime | When the run execution started |
| `ended_at` | DateTime | When the run execution ended |
| `duration_seconds` | Integer | Total execution time in seconds |
| `status` | String | Current status of the run (e.g., running, completed, failed) |
| `full_sync` | Boolean | Whether this was a full synchronization run |
| `credit_charge_status` | String | Status of credit charging for this run |
| `triggered_by_token` | String | Token used to trigger the run (if applicable) |
| `trigger` | Object | Trigger configuration (type, cron\_expression, cron\_timezone, event\_rule) |
| `triggered_by` | Object | User who triggered the run |
| `source` | Object | Source information (id, slug, active, status, description) |
| `destination` | Object | Destination information (id, slug, active, status, description) |
| `transformation` | Object | Transformation information (id, slug, type, active, status, description) |
Stream containing all data destinations configured in your Nekt workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the destination |
| `slug` | String | URL-friendly identifier for the destination |
| `active` | Boolean | Whether the destination is active |
| `status` | String | Current status of the destination |
| `deleted` | Boolean | Whether the destination has been deleted. Only returned when the **Include deleted items** option is enabled |
| `description` | String | Description of the destination |
| `is_ml_ai_connector` | Boolean | Whether this is an ML/AI connector |
| `connector_config` | String | Configuration settings for the connector (JSON string) |
| `connector_version` | String | Version of the connector being used |
| `created_at` | DateTime | When the destination was created |
| `updated_at` | DateTime | When the destination was last updated |
| `last_run` | String | Identifier of the last run |
| `python_execution_cpu` | Integer | CPU allocation for Python execution |
| `python_execution_memory` | Integer | Memory allocation for Python execution |
| `settings_number_of_retries` | Integer | Number of retry attempts on failure |
| `settings_retry_delay_seconds` | Integer | Delay between retry attempts |
| `settings_max_consecutive_failures` | Integer | Maximum consecutive failures before pausing |
| `input_tables` | Array | Array of input table configurations (table, primary\_keys, fields, name, layer) |
| `created_by` | Object | User who created the destination |
| `triggers` | Array | Triggers associated with the destination (id, type, cron\_expression, cron\_timezone, events, event\_rule, timestamps) |
| `tags` | Array | Tags associated with the destination (id, name, timestamps) |
Stream containing all data transformations configured in your Nekt workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the transformation |
| `slug` | String | URL-friendly identifier for the transformation |
| `type` | String | Type of transformation (e.g., pyspark, sql) |
| `active` | Boolean | Whether the transformation is active |
| `status` | String | Current status of the transformation |
| `deleted` | Boolean | Whether the transformation has been deleted. Only returned when the **Include deleted items** option is enabled |
| `description` | String | Description of the transformation |
| `code` | String | Transformation code/script |
| `created_at` | DateTime | When the transformation was created |
| `updated_at` | DateTime | When the transformation was last updated |
| `last_run` | String | Identifier of the last run |
| `dependencies` | Array | Array of Python dependencies |
| `add_apache_sedona` | Boolean | Whether Apache Sedona is enabled |
| `spark_driver_cores` | Integer | Number of cores for Spark driver |
| `spark_driver_memory` | Integer | Memory allocation for Spark driver |
| `spark_executor_cores` | Integer | Number of cores per Spark executor |
| `spark_executor_memory` | Integer | Memory allocation per Spark executor |
| `spark_executor_instances` | Integer | Number of Spark executor instances |
| `spark_executor_disk` | Integer | Disk space allocation per Spark executor |
| `spark_execution_timeout_minutes` | Integer | Timeout for Spark execution |
| `settings_number_of_retries` | Integer | Number of retry attempts on failure |
| `settings_retry_delay_seconds` | Integer | Delay between retry attempts |
| `settings_max_consecutive_failures` | Integer | Maximum consecutive failures before pausing |
| `settings_timezone` | String | Timezone for transformation execution |
| `delta_log_retention_duration` | Integer | Delta log retention period |
| `input_tables` | Array | Array of input tables (id, name\_reference, table, timestamps) |
| `output_tables` | Array | Array of output tables (id, name\_reference, table, timestamps) |
| `created_by` | Object | User who created the transformation |
| `input_volumes` | Array | Array of input volume references |
| `triggers` | Array | Triggers associated with the transformation (id, type, cron\_expression, cron\_timezone, events, event\_rule, timestamps) |
| `tags` | Array | Tags associated with the transformation (id, name, timestamps) |
Stream containing all audit activities and changes in your Nekt workspace.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the activity |
| `activity_type` | String | Type of activity performed |
| `changed_fields` | String | Fields that were changed (JSON string) |
| `created_at` | DateTime | When the activity occurred |
| `created_by_system` | Boolean | Whether the activity was performed by the system |
| `pipeline_automatically_paused_after_x_failures` | Integer | Auto-pause threshold |
| `source` | String | Related source ID (if applicable) |
| `destination` | String | Related destination ID (if applicable) |
| `transformation` | String | Related transformation ID (if applicable) |
| `table` | String | Related table ID (if applicable) |
| `volume` | String | Related volume ID (if applicable) |
| `run` | String | Related run ID (if applicable) |
| `visualization` | String | Related visualization ID (if applicable) |
| `created_by` | Object | User who performed the activity |
Stream containing trigger configurations for data sources. Triggers are fetched per source; when **Include deleted items** is enabled, deleted sources are skipped because their nested endpoints are no longer available.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the trigger |
| `type` | String | Type of trigger (e.g., cron, event) |
| `created_at` | DateTime | When the trigger was created |
| `updated_at` | DateTime | When the trigger was last updated |
| `cron_expression` | String | Cron expression for scheduled triggers |
| `cron_timezone` | String | Timezone for cron execution |
| `events` | Array | Array of events that trigger execution |
| `event_rule` | String | Rule for event-based triggers |
Stream containing trigger configurations for data destinations. Triggers are fetched per destination; when **Include deleted items** is enabled, deleted destinations are skipped because their nested endpoints are no longer available.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the trigger |
| `type` | String | Type of trigger (e.g., cron, event) |
| `created_at` | DateTime | When the trigger was created |
| `updated_at` | DateTime | When the trigger was last updated |
| `cron_expression` | String | Cron expression for scheduled triggers |
| `cron_timezone` | String | Timezone for cron execution |
| `events` | Array | Array of events that trigger execution |
| `event_rule` | String | Rule for event-based triggers |
Stream containing all users configured in your Nekt organization.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier of the user. |
| `first_name` | String | First name of the user. |
| `last_name` | String | Last name of the user. |
| `email` | String | Email address of the user. |
| `picture` | String | URL of the user profile picture. |
| `role` | String | Role of the user. |
| `functional_area` | String | Functional area of the user. |
| `other_functional_area` | String | Other functional area when not in the standard list. |
| `date_joined` | DateTime | Timestamp when the user joined. |
| `last_login` | DateTime | Timestamp of the user's last login. |
| `is_active` | Boolean | Whether the user account is active. |
## Data Model
The following diagram illustrates the relationships between the core data streams in Nekt. The arrows indicate the join keys that link the different entities, providing a clear overview of the data platform structure.
```mermaid theme={null}
graph TD;
Sources("Sources")
Runs("Runs")
Destinations("Destinations")
Transformations("Transformations")
Activities("Activities")
SourceTriggers("Source Triggers")
DestinationTriggers("Destination Triggers")
Users("Users")
Runs -- "source.id" --> Sources
Runs -- "destination.id" --> Destinations
Runs -- "transformation.id" --> Transformations
SourceTriggers -- "source_slug" --> Sources
DestinationTriggers -- "destination_slug" --> Destinations
Activities -- "source" --> Sources
Activities -- "destination" --> Destinations
Activities -- "transformation" --> Transformations
Activities -- "run" --> Runs
```
## Use Cases for Data Analysis
Here are some valuable business intelligence use cases when consolidating Nekt platform data, along with ready-to-use SQL queries that you can run on [Explorer](https://app.nekt.ai/explorer).
### 1. Daily Credits Consumption per Pipeline
Monitor credit consumption for each pipeline in a daily basis.
**Business Value:**
* Track pipeline credit consumption
* Analyze resource utilization and make future projections
* Identify pipelines that can be optmized
```sql theme={null}
SELECT
coalesce(s.slug, t.slug, d.slug) AS identifier,
DATE(r.ended_at) AS date,
count(r.id) AS total_runs,
sum(r.duration_seconds) / 60 AS duration_minutes
FROM
"nekt_raw"."nekt_runs" r
LEFT JOIN "nekt_raw"."nekt_sources" s ON r.source.id = s.id
LEFT JOIN "nekt_raw"."nekt_destinations" d ON r.destination.id = d.id
LEFT JOIN "nekt_raw"."nekt_transformations" t ON r.transformation.id = t.id
WHERE
r.credit_charge_status = 'charged'
GROUP BY
coalesce(s.slug, t.slug, d.slug),
DATE(r.ended_at)
ORDER BY
DATE(r.ended_at) DESC
```
### 2. Data Source Performance and Reliability Analysis
Monitor data pipeline performance, execution times, and failure rates across your data sources.
**Business Value:**
* Track pipeline execution trends and identify performance bottlenecks
* Monitor data pipeline reliability and uptime
* Analyze resource utilization and optimize infrastructure costs
* Identify sources that need attention
```sql theme={null}
WITH
run_metrics AS (
SELECT
r.source.slug AS source_name,
r.status,
r.full_sync,
r.duration_seconds,
r.started_at,
r.ended_at,
r.credit_charge_status,
DATE_TRUNC ('day', r.started_at) AS run_date,
CASE
WHEN r.status = 'success' THEN 1
ELSE 0
END AS success_flag,
CASE
WHEN r.status = 'failed' THEN 1
ELSE 0
END AS failure_flag
FROM
nekt_raw.nekt_runs r
WHERE
r.started_at >= CURRENT_DATE - INTERVAL '30' DAY
AND r.started_at IS NOT NULL
),
daily_summary AS (
SELECT
run_date,
source_name,
COUNT(*) AS total_runs,
SUM(success_flag) AS successful_runs,
SUM(failure_flag) AS failed_runs,
AVG(duration_seconds) AS avg_duration_seconds,
APPROX_PERCENTILE (duration_seconds, 0.5) AS median_duration_seconds,
MAX(duration_seconds) AS max_duration_seconds,
COUNT(
CASE
WHEN full_sync = TRUE THEN 1
END
) AS full_sync_runs
FROM
run_metrics
WHERE
source_name IS NOT NULL
GROUP BY
run_date,
source_name
),
source_reliability AS (
SELECT
source_name,
SUM(total_runs) AS total_runs_30d,
SUM(successful_runs) AS total_successful_runs,
SUM(failed_runs) AS total_failed_runs,
ROUND(
CAST(SUM(successful_runs) AS DOUBLE) / NULLIF(SUM(total_runs), 0) * 100,
2
) AS success_rate_percentage,
ROUND(AVG(avg_duration_seconds) / 60.0, 2) AS avg_duration_minutes,
ROUND(AVG(median_duration_seconds) / 60.0, 2) AS median_duration_minutes,
COUNT(DISTINCT run_date) AS active_days
FROM
daily_summary
GROUP BY
source_name
)
SELECT
source_name,
total_runs_30d,
total_successful_runs,
total_failed_runs,
success_rate_percentage,
avg_duration_minutes,
median_duration_minutes,
active_days,
CASE
WHEN success_rate_percentage >= 95 THEN 'Excellent'
WHEN success_rate_percentage >= 90 THEN 'Good'
WHEN success_rate_percentage >= 80 THEN 'Needs Attention'
ELSE 'Critical'
END AS reliability_status,
CASE
WHEN avg_duration_minutes <= 5 THEN 'Fast'
WHEN avg_duration_minutes <= 15 THEN 'Normal'
WHEN avg_duration_minutes <= 60 THEN 'Slow'
ELSE 'Very Slow'
END AS performance_status
FROM
source_reliability
WHERE
total_runs_30d > 0
ORDER BY
total_runs_30d DESC,
success_rate_percentage DESC
```
### 3. User Activity and Platform Usage Analysis
Track user engagement, platform adoption, and operational activities across your Nekt workspace.
**Business Value:**
* Monitor platform adoption and user engagement
* Identify power users and training needs
* Track operational patterns and peak usage times
* Optimize team collaboration and workflow efficiency
```sql theme={null}
WITH
user_activities AS (
SELECT
a.created_by.email AS user_email,
a.created_by.first_name || ' ' || a.created_by.last_name AS user_name,
a.created_by.role AS user_role,
a.created_by.functional_area AS functional_area,
a.activity_type,
a.created_at,
a.created_by_system,
DATE_TRUNC('day', a.created_at) AS activity_date,
DATE_TRUNC('hour', a.created_at) AS activity_hour,
CASE
WHEN a.source IS NOT NULL THEN 'source'
WHEN a.destination IS NOT NULL THEN 'destination'
WHEN a.transformation IS NOT NULL THEN 'transformation'
WHEN a.visualization IS NOT NULL THEN 'visualization'
WHEN a.run IS NOT NULL THEN 'run'
ELSE 'other'
END AS entity_type
FROM nekt_raw.nekt_activities a
WHERE a.created_at >= CURRENT_DATE - INTERVAL '30' DAY
AND a.created_by_system = false
AND a.created_by.email IS NOT NULL
),
user_summary AS (
SELECT
user_email,
user_name,
user_role,
functional_area,
COUNT(*) AS total_activities,
COUNT(DISTINCT activity_date) AS active_days,
COUNT(DISTINCT activity_type) AS unique_activity_types,
COUNT(DISTINCT entity_type) AS entity_types_used,
MIN(created_at) AS first_activity,
MAX(created_at) AS last_activity,
ARBITRARY(activity_type) AS most_common_activity,
ARBITRARY(entity_type) AS most_used_entity_type
FROM user_activities
GROUP BY user_email, user_name, user_role, functional_area
),
hourly_patterns AS (
SELECT
user_email,
EXTRACT(hour FROM activity_hour) AS hour_of_day,
COUNT(*) AS activities_count
FROM user_activities
GROUP BY user_email, EXTRACT(hour FROM activity_hour)
),
peak_hours AS (
SELECT
user_email,
hour_of_day,
activities_count,
ROW_NUMBER() OVER (PARTITION BY user_email ORDER BY activities_count DESC) AS hour_rank
FROM hourly_patterns
),
team_collaboration AS (
SELECT
functional_area,
COUNT(DISTINCT user_email) AS team_size,
SUM(total_activities) AS team_total_activities,
AVG(total_activities) AS avg_activities_per_user,
AVG(active_days) AS avg_active_days_per_user,
SUM(CASE WHEN active_days >= 20 THEN 1 ELSE 0 END) AS highly_active_users
FROM user_summary
WHERE functional_area IS NOT NULL
GROUP BY functional_area
)
SELECT
us.user_name,
us.user_email,
us.user_role,
us.functional_area,
us.total_activities,
us.active_days,
ROUND(
CAST(us.active_days AS DOUBLE) / 30.0 * 100,
1
) AS engagement_percentage,
us.unique_activity_types,
us.entity_types_used,
us.most_common_activity,
us.most_used_entity_type,
ph.hour_of_day AS peak_activity_hour,
us.first_activity,
us.last_activity,
tc.team_size,
ROUND(tc.avg_activities_per_user, 1) AS team_avg_activities,
CASE
WHEN us.active_days >= 25 THEN 'Highly Active'
WHEN us.active_days >= 15 THEN 'Active'
WHEN us.active_days >= 5 THEN 'Moderate'
ELSE 'Low Activity'
END AS activity_level,
CASE
WHEN us.entity_types_used >= 4 THEN 'Power User'
WHEN us.entity_types_used >= 2 THEN 'Regular User'
ELSE 'Basic User'
END AS user_type
FROM user_summary us
LEFT JOIN peak_hours ph ON us.user_email = ph.user_email AND ph.hour_rank = 1
LEFT JOIN team_collaboration tc ON us.functional_area = tc.functional_area
ORDER BY
us.total_activities DESC,
us.active_days DESC
```
## Skills for agents
Nekt connector documentation as plain markdown, for use in AI agent contexts.
# Netshoes as a data source
Source: https://docs.nekt.com/sources/netshoes
Bring data from the Grupo Netshoes marketplace to Nekt.
Netshoes is one of the largest sports and fashion e-commerce groups in Brazil, and runs the Netshoes and Zattini marketplaces. Its marketplace API lets a seller read the orders placed against their store, the products they have listed together with prices, stock and catalogue approval status, and the customer-service tickets raised about their sales.
![]()
## Configuring Netshoes as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Netshoes option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Netshoes authenticates every request with a pair of credentials: one that identifies the integration and one that identifies your store. Neither expires.
* **Client ID**: identifies the *integration*, not your store. Netshoes issues one to whoever built the integration, after it passes Netshoes' technical validation. Where to find yours depends on how you sell on Netshoes today:
| Your situation | Where the Client ID is |
| - | - |
| You already sell on Netshoes through an **ERP, hub or marketplace platform** (Bling, Tiny, and similar) | It belongs to that provider. Ask their support for the Client ID used on your Netshoes integration. |
| Your own team **built the integration** with Netshoes | It is the Client ID Netshoes gave you when your integration passed technical validation. |
| You **do not have an integration yet** | Register at the [Netshoes developer portal](https://developers.netshoes.com.br/api-portal/), create an app under **APPS**, and ask your Netshoes commercial contact to start the technical validation. The portal issues a sandbox Client ID immediately; the production one comes after validation. |
* **Access Token**: identifies your store. Ask your Netshoes commercial contact for it: it is issued per store, and it is the credential that decides which data this connection can read. In the developer portal, sandbox tokens are listed under **Tokens Associados** on your app.
If your Client ID belongs to an ERP or hub, everything still works, but Netshoes counts API calls per store, so this connection and your ERP's own synchronisation draw on the same allowance. If you see rate-limit warnings in the run log, lower **Requests per minute** in the advanced settings.
Netshoes binds a credential to one environment. A sandbox pair can only read sandbox data and a production pair can only read production data. A production credential pointed at the sandbox is rejected as if it were invalid. Make sure the pair you paste matches the environment you select below.
The following optional and advanced settings are also available:
* **Initial sync date** (Advanced): The earliest order date to read on the first sync. Later runs continue from where the previous one stopped. Leave it empty to read your full order history.
* **Environment** (Advanced): (Default: `production`) Netshoes environment your credentials belong to. Leave it on production: sandbox is only for credentials Netshoes issued for testing, and a production credential pointed at sandbox is rejected as if it were invalid.
* **Orders API version** (Advanced): (Default: `v1`) Which version of the Netshoes API your orders are read from. Netshoes documents orders on v1, which is the default. Only switch to v2 if a sync fails saying Netshoes asked your store to migrate.
* **Seller ID**: Only needed when the same credentials serve more than one store. The codes available to your credentials are listed in the `sellers` table.
* **Lookback window**: (Default: 7 days) How many days before the last sync the order tables re-read on every run. See [Keeping order status up to date](#keeping-order-status-up-to-date) below for why this matters.
* **Storefronts**: (Default: Netshoes and Zattini) Which storefronts to read the department list for. A storefront your store is not listed on is skipped automatically.
* **Page size**: (Default: 50) Records requested per page. Netshoes caps the order listing at 50.
* **Requests per minute**: (Default: 60) How fast this source is allowed to call the API. Netshoes limits how many calls a store may make per minute, and that allowance is shared with every other integration using the same credentials.
* **Base URL**: Only needed if Netshoes told you to use a different address for your store.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync.
The `product_prices`, `product_stocks`, `product_status` and `protocol_messages` streams read one record at a time: one API call per product, or per service ticket. On a large catalogue they take considerably longer than the others, so only select the ones you need.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
| Stream | Slug | Sync | Replication key |
| - | - | - | - |
| Orders | `orders` | INCREMENTAL | `orderDate` |
| Order Shipments | `order_shippings` | FULL\_TABLE | - |
| Order Items | `order_items` | FULL\_TABLE | - |
| Order Returns | `order_devolution_items` | FULL\_TABLE | - |
| Cancellation Reasons | `order_cancellation_reasons` | FULL\_TABLE | - |
| Products | `products` | FULL\_TABLE | - |
| Product Prices | `product_prices` | FULL\_TABLE | - |
| Product Stock | `product_stocks` | FULL\_TABLE | - |
| Product Status | `product_status` | FULL\_TABLE | - |
| Brands | `brands` | FULL\_TABLE | - |
| Colors | `colors` | FULL\_TABLE | - |
| Flavors | `flavors` | FULL\_TABLE | - |
| Sizes | `sizes` | FULL\_TABLE | - |
| Sellers | `sellers` | FULL\_TABLE | - |
| Departments | `departments` | FULL\_TABLE | - |
| Product Types | `product_types` | FULL\_TABLE | - |
| Service Protocols | `protocols` | FULL\_TABLE | - |
| Protocol Messages | `protocol_messages` | FULL\_TABLE | - |
## Fields by stream
| Field | Type | Description |
| - | - | - |
| `orderNumber` | string | Order number as the customer and the Netshoes back office see it. Unique per order and the key of this table. |
| `originNumber` | string | Order number in the storefront that originated the sale, when it differs from the marketplace order number. |
| `businessUnit` | string | Storefront the order was placed on: Netshoes or Zattini. |
| `originSite` | string | Site the order came from, as Netshoes labels it (for example NETSHOES). |
| `orderType` | string | Nature of the order, such as Sale for a regular purchase. |
| `orderStatus` | string | Current status of the order as a whole (for example Approved, Invoiced, Shipped, Delivered, Canceled). Individual shipments carry their own status in the order\_shippings table. |
| `orderDate` | datetime | When the order was placed. Used as the incremental replication key, and the only date the listing can be filtered by. |
| `paymentDate` | datetime | When the payment was captured. |
| `agreedDate` | datetime | Delivery date promised to the customer at checkout. |
| `totalQuantity` | integer | Total number of units across every line of the order. |
| `totalGross` | number | Gross value of the order, before discounts and commission. |
| `totalCommission` | number | Commission Netshoes charges the seller for this order. |
| `totalDiscount` | number | Total discount applied to the order. |
| `totalFreight` | number | Freight charged on the order. |
| `totalNet` | number | Net value of the order after discounts and commission. |
| `devolutionRequested` | boolean | Whether a return has been requested for this order. |
| `exchangeRequested` | boolean | Whether an exchange has been requested for this order. |
| `payments` | string | Payments settling the order as returned by the API (JSON array of objects). |
| `paymentGatewayInfos` | string | Payment gateway records for the order, each with the amount settled and the gateway's registration number (JSON array of objects). |
| `shippings` | string | Shipments of this order exactly as the API returned them (JSON array of objects). The same data is available as rows in order\_shippings, order\_items and order\_devolution\_items. |
| Field | Type | Description |
| - | - | - |
| `orderNumber` | string | Order this shipment belongs to. |
| `shippingCode` | string | Identifier of the shipment within the order. Together with the order number it is the key of this table. |
| `status` | string | Status of this shipment: approved, invoiced, shipped, delivered or canceled. A single order can hold shipments in different states. |
| `shippingEstimate` | datetime | Date the shipment is expected to reach the customer. |
| `deliveryTime` | integer | Delivery time quoted for this shipment, in days. |
| `freightAmount` | number | Freight charged for this shipment. |
| `country` | string | Destination country of the shipment. |
| `cancellationReason` | string | Why the shipment was cancelled, when it was. |
| `platformId` | string | Logistics platform handling the shipment, such as MAGALU\_ENTREGAS for orders fulfilled by Magalu. |
| `serviceId` | string | Identifier of the shipping service used. |
| `serviceName` | string | Name of the shipping service used, such as Normal. |
| `transport_carrier` | string | Carrier responsible for the shipment. 'Magalu Entregas - Fulfillment' means the order is fulfilled by Magalu and its status is advanced automatically rather than by the seller. |
| `transport_carrierId` | string | Carrier code. 906 identifies Magalu Entregas fulfillment. |
| `transport_trackingNumber` | string | Tracking code given by the carrier. |
| `transport_trackingLink` | string | URL where the customer can follow the shipment. |
| `transport_trackingShipDate` | datetime | Date the carrier registered the shipment for tracking. |
| `transport_deliveryDate` | datetime | Date the shipment was delivered to the customer. |
| `transport_shipDate` | datetime | Date the shipment left the seller. |
| `transport_deliveryService` | string | Delivery service chosen at checkout, such as Normal. |
| `transport_deliveryId` | string | Identifier of the delivery service chosen at checkout. |
| `invoice_accessKey` | string | Access key of the electronic invoice (NF-e) issued for this shipment. |
| `invoice_date` | datetime | Issue date of the invoice. |
| `invoice_shipDate` | datetime | Date the invoice records the goods as shipped. |
| `invoice_volume` | integer | Number of packages covered by the invoice. |
| `invoice_url` | string | URL of the invoice document, when the API returns one. |
| `invoiceErrors` | string | Errors and validation errors Netshoes recorded against the invoice (JSON object). Empty when the invoice was accepted. |
| `customer_document` | string | CPF or CNPJ of the customer who placed the order. |
| `customer_stateInscription` | string | State registration number, for business customers. |
| `customer_customerName` | string | Name of the customer who placed the order. |
| `customer_recipientName` | string | Name of the person the shipment is addressed to, which can differ from the buyer. |
| `customer_receiverName` | string | Name of whoever actually took delivery. |
| `customer_tradeName` | string | Trade name, for business customers. |
| `customer_cellPhone` | string | Mobile number registered for the delivery. |
| `customer_landLine` | string | Landline number registered for the delivery. |
| `address_street` | string | Street of the delivery address. |
| `address_number` | string | Street number of the delivery address. |
| `address_complement` | string | Address complement, such as an apartment or block. |
| `address_neighborhood` | string | Neighbourhood of the delivery address. |
| `address_city` | string | City of the delivery address. |
| `address_state` | string | State abbreviation of the delivery address. |
| `address_stateName` | string | Full state name of the delivery address. |
| `address_postalCode` | string | Postal code (CEP) of the delivery address. |
| `address_reference` | string | Free-text reference point for the delivery address. |
| `sender_supplierCnpj` | string | CNPJ of the seller shipping the goods. |
| `sender_sellerCode` | integer | Netshoes code of the seller shipping the goods. |
| `sender_sellerName` | string | Name of the seller shipping the goods. |
| `sender_supplierName` | string | Supplier name registered for the seller. |
| `exchangeOrders` | string | Orders generated by an exchange of this shipment (JSON array). |
| Field | Type | Description |
| - | - | - |
| `orderNumber` | string | Order this line belongs to. |
| `shippingCode` | string | Shipment this line was sent in. |
| `itemId` | integer | Identifier of the line within the shipment. |
| `sku` | string | Seller's SKU for the product sold on this line. |
| `name` | string | Product name as listed on the storefront. |
| `brand` | string | Brand of the product sold. |
| `ean` | string | EAN barcode of the product sold. |
| `manufacturerCode` | string | Manufacturer's reference for the product. |
| `quantity` | integer | Units of this product sold on the line. |
| `status` | string | Status of this individual line, which can differ from the shipment when only part of it was cancelled or returned. |
| `departmentName` | string | Netshoes department the product is catalogued under. |
| `departmentCode` | integer | Code of that department. |
| `color` | string | Colour variation sold. |
| `flavor` | string | Flavour variation sold, for products catalogued by flavour. |
| `size` | string | Size variation sold. |
| `grossUnitValue` | number | Gross price of a single unit. |
| `discountUnitValue` | number | Discount applied to a single unit. |
| `netUnitValue` | number | Net price of a single unit after discount. |
| `totalGross` | number | Gross value of the line. |
| `totalCommission` | number | Commission Netshoes charges on this line. |
| `totalDiscount` | number | Discount applied to the line. |
| `totalFreight` | number | Freight apportioned to the line. |
| `totalNet` | number | Net value of the line after discounts and commission. |
| Field | Type | Description |
| - | - | - |
| `orderNumber` | string | Order the returned line belongs to. |
| `shippingCode` | string | Shipment the returned line was sent in. |
| `itemId` | integer | Identifier of the returned line within the shipment. |
| `sku` | string | Seller's SKU for the returned product. |
| `name` | string | Product name as listed on the storefront. |
| `brand` | string | Brand of the returned product. |
| `ean` | string | EAN barcode of the returned product. |
| `manufacturerCode` | string | Manufacturer's reference for the product. |
| `quantity` | integer | Units returned. |
| `status` | string | Status of the return for this line. |
| `devolutionExchangeStatus` | string | Status of the exchange, when the return was opened as one. |
| `exchangeProcessCode` | integer | Code of the exchange process this return belongs to. |
| `checkInData` | datetime | When the returned goods were checked in at the warehouse. |
| `devolutionData` | datetime | When the return was registered. |
| `departmentName` | string | Netshoes department the product is catalogued under. |
| `departmentCode` | integer | Code of that department. |
| `color` | string | Colour variation returned. |
| `flavor` | string | Flavour variation returned. |
| `size` | string | Size variation returned. |
| `totalGross` | number | Gross value of the returned line. |
| Field | Type | Description |
| - | - | - |
| `code` | string | Code to send when cancelling a shipment for this reason. |
| `description` | string | What the reason means, in Netshoes' own wording. |
| `canBePenalty` | boolean | Whether cancelling for this reason can incur a penalty for the seller. |
| Field | Type | Description |
| - | - | - |
| `sku` | string | Seller's own code for this product variation. It is the key of the catalogue and Netshoes never changes it. |
| `productGroup` | string | Code grouping the size and colour variations of one model. Every SKU of the same model shares it, which is how the storefront offers them together. |
| `name` | string | Product title shown on the storefront, capped at 100 characters by Netshoes. |
| `description` | string | Product description shown on the storefront, capped at 3,000 characters and free of HTML. |
| `department` | string | Netshoes department the product is listed under. |
| `productType` | string | Product type within the department, which determines the attributes Netshoes requires. |
| `brand` | string | Brand of the product, taken from the brands table Netshoes maintains. |
| `color` | string | Colour of this variation, from the colours table. |
| `flavor` | string | Flavour of this variation, for products catalogued by flavour. |
| `size` | string | Size of this variation, from the sizes table. |
| `gender` | string | Audience the product is sold to, such as Masculino, Feminino or Unissex. |
| `manufacturerCode` | string | Manufacturer's own reference for the product. |
| `eanIsbn` | string | EAN or ISBN barcode of the variation. |
| `video` | string | URL of a product video, when one was registered. |
| `height` | number | Packed height of the product, in centimetres. |
| `width` | number | Packed width of the product, in centimetres. |
| `depth` | number | Packed depth of the product, in centimetres. |
| `weight` | number | Packed weight of the product, in grams. |
| `fulfillment_candidate` | boolean | Whether the seller offered this product to Magalu's fulfillment service. Orders for a fulfilled product move through their statuses automatically. |
| `images` | string | Images registered for the product (JSON array of objects, each with a url). |
| `attributes` | string | Attributes required by the product's department and type, as name and values (JSON array of objects). Which attributes apply varies by product type, so they are kept as JSON rather than columns. |
| Field | Type | Description |
| - | - | - |
| `sku` | string | Product this price belongs to. |
| `listPrice` | number | Reference price shown struck through on the storefront. |
| `salePrice` | number | Price the customer actually pays. |
| Field | Type | Description |
| - | - | - |
| `sku` | string | Product this stock position belongs to. |
| `available` | integer | Units available for sale. This is the whole of what the v2 stock resource returns: Netshoes does not expose reserved or total units on this endpoint. |
| Field | Type | Description |
| - | - | - |
| `sku` | string | Product this status belongs to. |
| `status` | string | Moderation status of the product. A product only reaches the storefront once Netshoes' catalogue team has released it. |
| `reviews` | string | Notes the catalogue team left while reviewing the product, each with a message and a date (JSON array of objects). This is where the reason for a rejected product is written. |
| Field | Type | Description |
| - | - | - |
| `code` | string | Netshoes code for the brand. This is the value a product listing must carry to claim it. |
| `name` | string | Name of the brand as Netshoes spells it. A product listing is rejected when its spelling does not match. |
| `externalCode` | string | Code the seller mapped to this brand, when one was registered with Netshoes. |
| Field | Type | Description |
| - | - | - |
| `code` | string | Netshoes code for the colour. This is the value a product listing must carry to claim it. |
| `name` | string | Name of the colour as Netshoes spells it. A product listing is rejected when its spelling does not match. |
| `externalCode` | string | Code the seller mapped to this colour, when one was registered with Netshoes. |
| Field | Type | Description |
| - | - | - |
| `code` | string | Netshoes code for the flavour. This is the value a product listing must carry to claim it. |
| `name` | string | Name of the flavour as Netshoes spells it. A product listing is rejected when its spelling does not match. |
| `externalCode` | string | Code the seller mapped to this flavour, when one was registered with Netshoes. |
| Field | Type | Description |
| - | - | - |
| `code` | string | Netshoes code for the size. This is the value a product listing must carry to claim it. |
| `name` | string | Name of the size as Netshoes spells it. A product listing is rejected when its spelling does not match. |
| `externalCode` | string | Code the seller mapped to this size, when one was registered with Netshoes. |
| Field | Type | Description |
| - | - | - |
| `code` | string | Netshoes code of the seller. This is the value to put in the 'Seller ID' setting when one set of credentials serves several sellers. |
| `name` | string | Registered name of the seller. |
| Field | Type | Description |
| - | - | - |
| `businessUnit` | string | Storefront this department belongs to: netshoes or zattini. |
| `code` | string | Netshoes code of the department. Product types are listed per department using this code. |
| `name` | string | Name of the department as it appears on the storefront. |
| Field | Type | Description |
| - | - | - |
| `departmentCode` | string | Department this product type belongs to. |
| `businessUnit` | string | Storefront the department belongs to. |
| `code` | string | Netshoes code of the product type. A product listing must pair it with the department it was read from. |
| `name` | string | Name of the product type as Netshoes spells it. |
| Field | Type | Description |
| - | - | - |
| `protocolNumber` | string | Number identifying the service ticket. It is what the customer and Netshoes support both quote. |
| `orderNumber` | string | Order the ticket was raised about, linking it to the orders table. |
| `status` | string | Where the ticket stands: open, awaiting the seller, closed. |
| `type` | string | Kind of ticket, as Netshoes classifies it. |
| `cause` | string | Cause Netshoes recorded for the ticket. |
| `channel` | string | Channel the customer used to raise the ticket. |
| `initDate` | datetime | When the ticket was opened. |
| `lastMessageDate` | datetime | When the most recent message was added to the ticket. |
| `sla` | string | Service level the seller has to answer the ticket within. |
| `priority` | string | Priority Netshoes assigned to the ticket. |
| `responsible` | string | Who the ticket is currently waiting on. |
| Field | Type | Description |
| - | - | - |
| `protocolNumber` | string | Ticket this message belongs to. |
| `messageDate` | datetime | When the message was written. Together with the ticket number it identifies the message. |
| `text` | string | Body of the message, in the words of whoever wrote it. The message carries no author of its own; who the ticket is waiting on is on the protocol itself. |
# Data Model
The following diagram illustrates how orders and their nested entities are structured within the connector. A Netshoes order is shipment-centric: the order itself carries the totals and the dates, while the customer, the invoice, the carrier, and the products sold all belong to a **shipment**. One order can hold several shipments, each with its own status.
Nekt splits that into four tables you can easily join:
```mermaid theme={null}
graph TD;
Orders("Orders")
OrderShipments("Order Shipments")
OrderItems("Order Items")
OrderReturns("Order Returns")
OrderShipments -- "orderNumber" --> Orders
OrderItems -- "orderNumber, shippingCode" --> OrderShipments
OrderReturns -- "orderNumber, shippingCode" --> OrderShipments
```
* `orders`: one row per order, with its totals, dates and overall status.
* `order_shippings`: one row per shipment, with the recipient, the delivery address, the invoice and the carrier. Join it to `orders` on `orderNumber`.
* `order_items`: one row per product sold, joined to a shipment by `orderNumber` and `shippingCode`.
* `order_devolution_items`: one row per product the customer returned or exchanged, joined the same way.
All four are filled from the same extraction, so selecting the detail tables costs no additional API calls.
# Implementation Notes
### Keeping order status up to date
Netshoes only lets orders be filtered by **the date they were placed**. An order's status, however, keeps changing afterwards: it is approved, then invoiced, then shipped, then delivered, sometimes weeks later.
That means a plain incremental sync would capture each order once, with whatever status it had at the time, and never see it change again. The **lookback window** exists to prevent that: on every run the connector re-reads the orders placed in the last few days, so their current status replaces what was stored before.
The default of 7 days suits most stores. If your orders regularly take longer than a week to reach the customer, raise it so that deliveries are still within the window when they happen.
### Things to know
* **Orders and products come from different API versions.** Netshoes publishes two APIs: version 1 carries orders, service protocols and the catalogue reference tables, and version 2 carries products, prices and stock. The connector reads each from where it lives, so nothing is required of you beyond the credentials.
* **Catalogue tables change without notice.** Netshoes adds and removes brands, colours, sizes and departments on its own schedule, and a product listing is rejected when it names a value that does not exist. The `brands`, `colors`, `flavors`, `sizes`, `departments` and `product_types` tables are re-read in full on every run so they always reflect the current list.
* **Orders fulfilled by Magalu look different.** When a shipment's carrier is "Magalu Entregas - Fulfillment" (carrier code 906), Netshoes advances the order through invoicing, shipping and delivery automatically, and the seller cannot change those statuses. You can identify these shipments by `transport_carrierId` in `order_shippings`.
* **Rate limiting is expected on large stores.** Netshoes enforces a per-minute call limit and does not publish the number. When it throttles the connection, the sync waits and continues; you'll see a warning in the run log. If it happens on every run, lower **Requests per minute** in the source settings.
* **Not every table is available to every store.** If your credentials are not allowed to read part of the API, or your store is not listed on one of the storefronts, that table is simply not updated and a warning appears in the run log. The rest of the sync carries on.
### Troubleshooting
| Issue | Possible cause | Solution |
| :- | :- | :- |
| Authentication errors (401 Unauthorized) | Mismatched environment or wrong credentials | Netshoes binds a credential to one host. A production pair calling the sandbox is refused exactly as if it were invalid, and the gateway may even claim the `access_token` header is missing. Ensure your **Environment** setting matches where your credentials were issued. Also, Netshoes' own error message always blames the Access Token even when the Client ID is wrong, so carefully verify both values. |
## Skills for agents
Netshoes connector documentation as plain markdown, for use in AI agent contexts.
# NetSuite as a data source
Source: https://docs.nekt.com/sources/netsuite
Bring data from NetSuite to Nekt.
NetSuite is a cloud-based business management software suite that includes ERP, financials, CRM, and e-commerce capabilities. It provides a comprehensive solution for managing business operations, financial data, and customer relationships.
![]()
## Configuring NetSuite as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the NetSuite option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need your NetSuite API credentials for this connection. These can be obtained from your NetSuite account's Setup > Integration > Manage Integrations section. Once you have them, add the following credentials:
* **Consumer Key**: Your NetSuite consumer key for API authentication, obtained from a TBA integration record.
* **Consumer Secret**: Your NetSuite consumer secret for API authentication, obtained from a TBA integration record.
* **Token ID**: Your NetSuite token ID for API authentication, obtained when creating a TBA token associated with the integration record created above.
* **Token Secret**: Your NetSuite token secret for API authentication, obtained when creating a TBA token associated with the integration record created above.
* **Account ID**: Your NetSuite account ID. This can be extracted from your NetSuite URL. For example, if your URL is `https://123456.app.netsuite.com/` your account ID will be `123456`.
* **Date Filter Format**: The date format to use for incremental syncs. Choose from: `DD/MM/YYYY` (Brazilian format), `MM/DD/YYYY` (US format), `YYYY-MM-DD` (ISO format). This must match your NetSuite account configuration.
* **Start Date**: Records created or updated after this date will be extracted from the source.
This connector uses Token Based Authentication (TBA). Follow [this tutorial](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/article_0605103340.html#subsect_0605103551) to know more on how to create these credentials.
### 2. Select streams
Choose which data streams to sync: select all of them or pick only the ones that matter most to you. The available streams are:
* Bill of Materials
* BOM Revision
* BOM Revision Component
* Billing Schedule
* BOM (Bill of Materials)
* BOM Revision
* BOM Revision Component
* Customers
* Department
* Items
* Location
* Previous Transaction Line Link
* Term
* Transaction Lines
* Transactions
* Units Type
* Units Type UOM
* Vendor
We use NetSuiteQL language to query your data, which allows for an efficient extraction process. The only limitation is that the data for each column returns as string, being necessary further processing to parse data to appropriate types (ex. numbers, dates, etc)
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new or modified data since the last sync, which is good for maintaining an efficient data pipeline.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if you want to ensure complete data accuracy.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently your NetSuite data is updated and how current you need your analytics to be.
Optionally, you can determine when to execute a [full sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync). This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from NetSuite and their corresponding fields:
Stream for fetching Bill of Materials records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `links` | Array | Links associated with the BOM |
| `name` | String | BOM name |
| `memo` | String | Memo |
| `subsidiary` | String | Subsidiary ID |
| `externalid` | String | External ID |
| `isinactive` | String | Indicates if the BOM is inactive |
| `includechildren` | String | Indicates if children subsidiaries are included |
| `usecomponentyield` | String | Indicates if component yield is used |
| `availableforallassemblies` | String | Available for all assemblies |
| `availableforalllocations` | String | Available for all locations |
| `availableforlocations` | String | Available for specific locations |
| `availableforsubisidiaries` | String | Available for specific subsidiaries |
| `restricttoassemblies` | String | Restrict to specific assemblies |
| `bomrevision` | String | BOM revision ID |
| `usedonassembly` | String | Assembly items using this BOM |
| `legacybom` | String | Indicates if this is a legacy BOM |
| `createddate` | String | Date created |
| `lastmodifieddate` | String | Last modified date |
Stream for fetching BOM Revision records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `links` | Array | Links associated with the revision |
| `name` | String | Revision name |
| `billofmaterials` | String | Parent BOM ID |
| `externalid` | String | External ID |
| `memo` | String | Memo |
| `isinactive` | String | Indicates if the revision is inactive |
| `effectivestartdate` | String | Effective start date |
| `effectiveenddate` | String | Effective end date |
| `previousrevisionkey` | String | Previous revision key ID |
| `createddate` | String | Date created |
| `lastmodifieddate` | String | Last modified date |
Stream for fetching BOM Revision Component records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `links` | Array | Links associated with the component |
| `bomrevision` | String | Parent BOM Revision ID |
| `item` | String | Item ID |
| `displayname` | String | Display name |
| `description` | String | Component description |
| `quantity` | String | Quantity |
| `units` | String | Units of measure |
| `componentyield` | String | Component yield percentage |
| `bomquantity` | String | BOM quantity |
| `itemsource` | String | Item source |
| `lineid` | String | Line ID |
| `isoutsourcingcharge` | String | Indicates if this is an outsourcing charge |
Stream for fetching billing schedule records from NetSuite. Billing schedules define the recurring billing terms for transactions.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `name` | String | Name of the billing schedule |
| `links` | Array | Links associated with the billing schedule |
| `applytosubtotal` | String | Indicates if the schedule applies to the subtotal |
| `billforactuals` | String | Specifies if only actual time should be invoiced |
| `frequency` | String | Frequency of the billing recurrence |
| `inarrears` | String | Specifies if billing is in arrears |
| `initialamount` | String | The initial amount to be billed |
| `isinactive` | String | Indicates if the billing schedule is inactive |
| `ispublic` | String | Indicates if the billing schedule is public |
| `numberremaining` | String | Number of recurrences remaining |
| `repeatevery` | String | Repeat every X period(s) |
| `scheduletype` | String | Type of schedule |
| `transaction` | String | Transaction ID |
Stream for fetching Bill of Materials records from NetSuite. BOMs define the components and quantities needed to build an assembly item.
**Key Fields:**
* `id` - Internal ID
* `links` - Links associated with the BOM
* `name` - BOM name
* `memo` - Memo
* `subsidiary` - Subsidiary ID
* `externalid` - External ID
* `isinactive` - Indicates if the BOM is inactive
* `includechildren` - Indicates if children subsidiaries are included
* `usecomponentyield` - Indicates if component yield is used
* `availableforallassemblies` - Available for all assemblies
* `availableforalllocations` - Available for all locations
* `availableforlocations` - Available for specific locations
* `availableforsubisidiaries` - Available for specific subsidiaries
* `restricttoassemblies` - Restrict to specific assemblies
* `bomrevision` - BOM revision ID
* `usedonassembly` - Assembly items using this BOM
* `legacybom` - Indicates if this is a legacy BOM
* `createddate` - Date created
* `lastmodifieddate` - Last modified date
Stream for fetching BOM Revision records from NetSuite. Revisions represent versioned configurations of a Bill of Materials, with effective date ranges.
**Key Fields:**
* `id` - Internal ID
* `links` - Links associated with the revision
* `name` - Revision name
* `billofmaterials` - Parent BOM ID (foreign key to BOM)
* `externalid` - External ID
* `memo` - Memo
* `isinactive` - Indicates if the revision is inactive
* `effectivestartdate` - Effective start date
* `effectiveenddate` - Effective end date
* `previousrevisionkey` - Previous revision key ID
* `createddate` - Date created
* `lastmodifieddate` - Last modified date
Stream for fetching the individual component lines of a BOM Revision. Each record represents one item and its quantity within a revision. Full table sync only (no replication key).
**Key Fields:**
* `id` - Internal ID
* `links` - Links associated with the component
* `bomrevision` - Parent BOM Revision ID (composite primary key, foreign key to BOM Revision)
* `item` - Item ID (composite primary key, foreign key to Items)
* `displayname` - Display name
* `description` - Component description
* `quantity` - Quantity
* `units` - Units of measure
* `componentyield` - Component yield percentage
* `bomquantity` - BOM quantity
* `itemsource` - Item source
* `lineid` - Line ID
* `isoutsourcingcharge` - Indicates if this is an outsourcing charge
Stream for fetching customer information from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The customer's system ID |
| `entityid` | String | Entity ID |
| `externalid` | String | External ID |
| `entitytitle` | String | Entity title |
| `links` | Array | Links associated with the customer |
| `companyname` | String | Company name |
| `firstname` | String | First name |
| `lastname` | String | Last name |
| `fullname` | String | Full name |
| `altname` | String | Alternative name for the customer |
| `email` | String | Email address |
| `phone` | String | Phone number |
| `emailpreference` | String | Email preference setting |
| `emailtransactions` | String | Email transactions flag |
| `faxtransactions` | String | Fax transactions flag |
| `printtransactions` | String | Print transactions flag |
| `globalsubscriptionstatus` | String | Global subscription status |
| `unsubscribe` | String | Unsubscribe flag |
| `defaultbillingaddress` | String | Default billing address ID |
| `defaultshippingaddress` | String | Default shipping address ID |
| `shippingcarrier` | String | Shipping carrier |
| `shipcomplete` | String | Ship complete flag |
| `currency` | String | Currency ID |
| `balancesearch` | String | Customer balance |
| `depositbalancesearch` | String | Deposit balance |
| `overduebalancesearch` | String | Overdue balance |
| `unbilledorderssearch` | String | Unbilled orders amount |
| `daysoverduesearch` | String | Days overdue |
| `oncredithold` | String | Credit hold flag |
| `creditholdoverride` | String | Credit hold override status |
| `receivablesaccount` | String | Receivables account |
| `entitystatus` | String | Entity status |
| `isinactive` | String | Inactive flag |
| `isperson` | String | Person flag |
| `isjob` | String | Job flag |
| `isbudgetapproved` | String | Budget approved flag |
| `isautogeneratedrepresentingentity` | String | Auto-generated representing entity flag |
| `duplicate` | String | Duplicate flag |
| `alcoholrecipienttype` | String | Type of alcohol recipient |
| `firstsaledate` | String | Date of first sale |
| `firstsaleperiod` | String | First sale period |
| `lastsaledate` | String | Date of last sale |
| `lastsaleperiod` | String | Last sale period |
| `probability` | String | Probability value |
| `searchstage` | String | Search stage |
| `toplevelparent` | String | Top level parent ID |
| `datecreated` | String | Date customer was created |
| `dateclosed` | String | Date account was closed |
| `lastmodifieddate` | String | Last modified date |
| `language` | String | Language code |
| `custom_fields` | Array | Array of custom fields with name and value pairs |
Stream for fetching department records from NetSuite. Departments are used to track transactions and organize your business structure.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `name` | String | Department name |
| `fullname` | String | Full name of the department |
| `parent` | String | Parent department ID |
| `includechildren` | String | Indicates if department includes children |
| `isinactive` | String | Indicates if the department is inactive |
| `subsidiary` | String | Subsidiary ID |
| `lastmodifieddate` | String | Date the department was last modified |
Stream for fetching item records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The item's system ID |
| `itemid` | String | Item ID code |
| `externalid` | String | External ID |
| `displayname` | String | Display name |
| `fullname` | String | Full name |
| `description` | String | Item description |
| `links` | Array | Links associated with the item |
| `itemtype` | String | Type of item |
| `subtype` | String | Item subtype |
| `class` | String | Class |
| `department` | String | Department |
| `subsidiary` | String | Subsidiaries |
| `cost` | String | Cost |
| `averagecost` | String | Average cost |
| `lastpurchaseprice` | String | Last purchase price |
| `costestimatetype` | String | Cost estimate type |
| `deposit` | String | Deposit flag |
| `expenseaccount` | String | Expense account |
| `incomeaccount` | String | Income account |
| `intercoexpenseaccount` | String | Intercompany expense account |
| `intercoincomeaccount` | String | Intercompany income account |
| `purchaseunit` | String | Purchase unit |
| `saleunit` | String | Sales unit |
| `unitstype` | String | Units type |
| `weightunits` | String | Weight units |
| `minimumquantity` | String | Minimum quantity |
| `enforceminqtyinternally` | String | Enforce minimum quantity internally flag |
| `isdropshipitem` | String | Dropship item flag |
| `isfulfillable` | String | Fulfillable flag |
| `isspecialorderitem` | String | Special order flag |
| `shipindividually` | String | Ship individually flag |
| `matchbilltoreceipt` | String | Match bill to receipt flag |
| `isinactive` | String | Inactive flag |
| `isonline` | String | Online flag |
| `includechildren` | String | Include children flag |
| `copydescription` | String | Copy description flag |
| `printitems` | String | Print items flag |
| `generateaccruals` | String | Generate accruals flag |
| `usemarginalrates` | String | Use marginal rates flag |
| `createddate` | String | Creation date |
| `lastmodifieddate` | String | Last modified date |
| `custom_fields` | Array | Array of custom fields with name and value pairs |
Stream for fetching location records from NetSuite. Locations represent physical or logical places where inventory is stored or business is conducted.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `name` | String | Location name |
| `fullname` | String | Full name of the location |
| `links` | Array | Links associated with the location |
| `mainaddress` | String | Main address of the location |
| `includeincontroltower` | String | Flag for inclusion in control tower |
| `isinactive` | String | Indicates if the location is inactive |
| `makeinventoryavailable` | String | Flag indicating inventory availability |
| `subsidiary` | String | Subsidiary ID |
| `lastmodifieddate` | String | Last modified date |
Stream for fetching links between related transaction lines. This stream is essential for understanding document flow (e.g., linking sales orders to invoices, purchase orders to vendor bills).
| Field | Type | Description |
| :- | :- | :- |
| `linktype` | String | Link type |
| `previousdoc` | String | Transaction ID of previous document |
| `previousline` | String | Line ID of the previous document line |
| `previoustype` | String | Previous document type |
| `previousdate` | String | Date of previous transaction |
| `nextdoc` | String | Transaction ID of next document |
| `nextline` | String | Line ID of the next document line |
| `nexttype` | String | Next document type |
| `nextdate` | String | Date of next transaction |
| `foreignamount` | String | Foreign amount |
| `discount` | String | Discount indicator |
| `lastmodifieddate` | String | Last modified date |
Stream for fetching payment term records from NetSuite. Terms define payment conditions such as due dates and discounts.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `name` | String | Term name |
| `datedriven` | String | Date driven flag |
| `daysuntilexpiry` | String | Days until discount expires |
| `daysuntilnetdue` | String | Days until net due |
| `discountpercent` | String | Discount percentage |
| `installment` | String | Installment flag |
| `isinactive` | String | Indicates if the term is inactive |
| `preferred` | String | Preferred flag |
| `lastmodifieddate` | String | Last modified date |
Stream for fetching transaction line-level details from NetSuite. Each transaction can have multiple lines representing individual items, charges, or accounting entries.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The transaction line's system ID |
| `uniquekey` | String | Unique key for the transaction line |
| `transaction` | String | Transaction ID for this line |
| `linesequencenumber` | String | Line sequence number |
| `documentnumber` | String | Document number for the line |
| `rn` | String | Row number |
| `item` | String | Item associated with this line |
| `itemtype` | String | Type of item |
| `entity` | String | Entity related to the transaction line |
| `quantity` | String | Quantity |
| `quantitybilled` | String | Quantity billed |
| `quantityshiprecv` | String | Quantity shipped/received |
| `quantityrejected` | String | Quantity rejected |
| `units` | String | Units |
| `rate` | String | Rate |
| `rateamount` | String | Rate amount |
| `ratepercent` | String | Rate percent |
| `price` | String | Price for this line |
| `grossamt` | String | Gross amount |
| `netamount` | String | Net amount |
| `taxamount` | String | Tax amount |
| `basegrossamt` | String | Base gross amount |
| `basetaxamount` | String | Base tax amount |
| `foreignamount` | String | Foreign amount |
| `foreignamountpaid` | String | Foreign amount paid |
| `foreignamountpaidnopost` | String | Foreign amount paid (no post) |
| `foreignamountunpaid` | String | Foreign amount unpaid |
| `creditforeignamount` | String | Credit foreign amount |
| `debitforeignamount` | String | Debit foreign amount |
| `amountpending` | String | Amount pending |
| `settlementamount` | String | Settlement amount for the line |
| `foreignpaymentamountunused` | String | Foreign payment amount unused |
| `foreignpaymentamountused` | String | Foreign payment amount used |
| `fxamountlinked` | String | FX amount linked |
| `costestimate` | String | Cost estimate |
| `costestimatebase` | String | Cost estimate base |
| `costestimaterate` | String | Cost estimate rate |
| `costestimatetype` | String | Cost estimate type |
| `estgrossprofit` | String | Estimated gross profit |
| `estgrossprofitpercent` | String | Estimated gross profit percent |
| `accountinglinetype` | String | Accounting line type (e.g. ACCOUNTSRECEIVABLE) |
| `expenseaccount` | String | Expense account |
| `mainline` | String | Main line flag |
| `taxline` | String | Tax line flag |
| `iscogs` | String | Is cost of goods sold flag |
| `isfxvariance` | String | Is FX variance flag |
| `iscustomglline` | String | Is custom GL line flag |
| `isrevrectransaction` | String | Is revenue recognition transaction flag |
| `eliminate` | String | Elimination flag |
| `location` | String | Location |
| `department` | String | Department |
| `class` | String | Class |
| `subsidiary` | String | Subsidiary |
| `isclosed` | String | Is closed flag |
| `isinventoryaffecting` | String | Is inventory affecting flag |
| `isbillable` | String | Is billable flag |
| `isfullyshipped` | String | Is fully shipped flag |
| `fulfillable` | String | Whether the line is fulfillable |
| `hasfulfillableitems` | String | Has fulfillable items flag |
| `hascostline` | String | Has cost line flag |
| `cleared` | String | Cleared flag |
| `periodclosed` | String | Period closed flag |
| `isitemvalueadjustment` | String | Is item value adjustment flag |
| `processedbyrevcommit` | String | Processed by revenue commit flag |
| `dropship` | String | Drop ship flag |
| `specialorder` | String | Special order flag |
| `kitcomponent` | String | Kit component flag |
| `matchbilltoreceipt` | String | Match bill to receipt flag |
| `commitmentfirm` | String | Commitment firm flag |
| `oldcommitmentfirm` | String | Old commitment firm flag |
| `excludefromraterequest` | String | Exclude from rate request flag |
| `transactiondiscount` | String | Transaction discount flag |
| `donotdisplayline` | String | Do not display line flag |
| `donotprintline` | String | Do not print line flag |
| `linecreateddate` | String | Line created date |
| `linelastmodifieddate` | String | Line last modified date |
| `actualshipdate` | String | Actual ship date |
| `expectedreceiptdate` | String | Expected receipt date |
| `duedate` | String | Due date |
| `closedate` | String | Close date |
| `createdfrom` | String | ID of the source transaction that created this line |
| `billingschedule` | String | Billing schedule |
| `memo` | String | Memo text |
| `cseg_nscs_aloca_rec` | String | Custom segment NSCS Allocation Record |
| `cseg_nscs_class_crd` | String | Custom segment NSCS Class CRD |
| `custcol_atlas_promise_date` | String | Custom: Atlas Promise Date |
| `custcol_fte_i_line_sequence` | String | Custom: FTE I Line Sequence |
| `custcol_ftebr_l_cfop_code` | String | Custom: FTEBR L CFOP Code |
| `custcol_nscs_item_code` | String | Custom: NSCS Item Code |
| `custom_fields` | Array | Array of custom fields with name and value pairs |
Stream for fetching transaction records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | The transaction's system ID |
| `tranid` | String | Transaction ID |
| `transactionnumber` | String | Transaction number |
| `number` | String | Transaction number |
| `externalid` | String | External ID |
| `trandisplayname` | String | Transaction display name |
| `typebaseddocumentnumber` | String | Type-based document number |
| `links` | Array | Links associated with the transaction |
| `type` | String | Transaction type |
| `abbrevtype` | String | Abbreviated transaction type |
| `recordtype` | String | Record type |
| `status` | String | Status |
| `approvalstatus` | String | Approval status |
| `customform` | String | Custom form ID |
| `customtype` | String | Custom type |
| `entity` | String | Entity ID |
| `email` | String | Email address |
| `trandate` | String | Transaction date |
| `createddate` | String | Creation date |
| `lastmodifieddate` | String | Last modified date |
| `closedate` | String | Close date |
| `duedate` | String | Due date |
| `startdate` | String | Start date |
| `enddate` | String | End date |
| `prevdate` | String | Previous date |
| `taxpointdate` | String | Tax point date |
| `shipdate` | String | Ship date |
| `foreigntotal` | String | Total in foreign currency |
| `foreignamountpaid` | String | Foreign amount paid |
| `foreignamountpaidnopost` | String | Foreign amount paid (no post) |
| `foreignamountunpaid` | String | Foreign amount unpaid |
| `amountunbilled` | String | Unbilled amount |
| `totalaftertaxes` | String | Total after taxes |
| `basetotalaftertaxes` | String | Base total after taxes |
| `totalcostestimate` | String | Total cost estimate |
| `estgrossprofit` | String | Estimated gross profit |
| `estgrossprofitpercent` | String | Estimated gross profit percentage |
| `taxtotal` | String | Tax total |
| `basetaxtotal` | String | Base tax total |
| `taxdetailsoverride` | String | Tax details override flag |
| `taxpointdateoverride` | String | Tax point date override flag |
| `taxregoverride` | String | Tax registration override flag |
| `legacytax` | String | Legacy tax flag |
| `entitytaxregnum` | String | Entity tax registration number |
| `subsidiarytaxregnum` | String | Subsidiary tax registration number |
| `nexus` | String | Tax nexus |
| `currency` | String | Currency ID |
| `exchangerate` | String | Exchange rate |
| `billingaddress` | String | Billing address ID |
| `billingstatus` | String | Billing status |
| `terms` | String | Payment terms |
| `needsbill` | String | Needs bill flag |
| `paymenthold` | String | Payment hold flag |
| `isfinchrg` | String | Is finance charge flag |
| `balsegstatus` | String | Balance segment status |
| `shipcarrier` | String | Shipping carrier |
| `shipcomplete` | String | Ship complete flag |
| `shippingaddress` | String | Shipping address ID |
| `ordpicked` | String | Order picked flag |
| `printedpickingticket` | String | Printed picking ticket flag |
| `posting` | String | Posting flag |
| `postingperiod` | String | Posting period ID |
| `isreversal` | String | Is reversal flag |
| `void` | String | Void flag |
| `voided` | String | Voided flag |
| `memorized` | String | Memorized flag |
| `includeinforceast` | String | Include in forecast flag |
| `intercoadj` | String | Intercompany adjustment flag |
| `intercostatus` | String | Intercompany status |
| `intercotransaction` | String | Intercompany transaction ID |
| `tobeemailed` | String | To be emailed flag |
| `tobefaxed` | String | To be faxed flag |
| `tobeprinted` | String | To be printed flag |
| `visibletocustomer` | String | Visible to customer flag |
| `daysopen` | String | Days transaction has been open |
| `daysoverduesearch` | String | Days overdue search |
| `source` | String | Source of the transaction |
| `otherrefnum` | String | Other reference number |
| `memo` | String | Memo text |
| `createdby` | String | Created by user ID |
| `lastmodifiedby` | String | Last modified by user ID |
| `userevenuearrangement` | String | Use revenue arrangement flag |
| `custom_fields` | Array | Array of custom fields with name and value pairs |
Stream for fetching Units Type records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `links` | Array | Links associated with the units type |
| `name` | String | Units type name |
| `isinactive` | String | Indicates if the units type is inactive |
| `lastmodifieddate` | String | Last modified date |
Stream for fetching Units Type UOM (Unit of Measure) records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `internalid` | String | Internal ID |
| `links` | Array | Links associated with the UOM |
| `unitname` | String | Unit name |
| `conversionrate` | String | Conversion rate relative to base unit |
| `baseunit` | String | Indicates if this is the base unit |
| `unitstype` | String | Parent Units Type ID |
| `abbreviation` | String | Unit abbreviation |
| `pluralname` | String | Plural unit name |
| `pluralabbreviation` | String | Plural abbreviation |
| `inuse` | String | Indicates if the unit is in use |
Stream for fetching vendor records from NetSuite.
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Internal ID |
| `links` | Array | Links associated with the vendor |
| `entityid` | String | Entity ID |
| `entitynumber` | String | Entity number |
| `entitytitle` | String | Entity title |
| `companyname` | String | Company name |
| `isperson` | String | Indicates if the vendor is a person |
| `firstname` | String | First name |
| `middlename` | String | Middle name |
| `lastname` | String | Last name |
| `fullname` | String | Full name |
| `altname` | String | Alternate name |
| `email` | String | Email address |
| `emailpreference` | String | Email preference |
| `emailtransactions` | String | Email transactions flag |
| `phone` | String | Phone number |
| `altphone` | String | Alternate phone number |
| `fax` | String | Fax number |
| `faxtransactions` | String | Fax transactions flag |
| `printtransactions` | String | Print transactions flag |
| `url` | String | URL |
| `comments` | String | Comments |
| `contactlist` | String | Contact list |
| `globalsubscriptionstatus` | String | Global subscription status |
| `unsubscribe` | String | Unsubscribe flag |
| `subsidiary` | String | Subsidiary ID |
| `subsidiaryedition` | String | Subsidiary edition |
| `terms` | String | Payment terms ID |
| `currency` | String | Currency ID |
| `creditlimit` | String | Credit limit |
| `balance` | String | Balance |
| `balanceprimary` | String | Balance in primary currency |
| `unbilledorders` | String | Unbilled orders amount |
| `unbilledordersprimary` | String | Unbilled orders in primary currency |
| `prepaymentbalance` | String | Prepayment balance |
| `expenseaccount` | String | Expense account |
| `payablesaccount` | String | Payables account |
| `defaultbillingaddress` | String | Default billing address |
| `defaultshippingaddress` | String | Default shipping address |
| `isinactive` | String | Indicates if the vendor is inactive |
| `duplicate` | String | Duplicate flag |
| `isautogeneratedrepresentingentity` | String | Auto-generated representing entity flag |
| `isjobresourcevend` | String | Job resource vendor flag |
| `predconfidence` | String | Prediction confidence |
| `predicteddays` | String | Predicted days |
| `workcalendar` | String | Work calendar |
| `manufacturinglocations` | String | Manufacturing locations |
| `lastmodifieddate` | String | Last modified date |
| `datecreated` | String | Date created |
| `defaultaddress` | String | Default address |
| `externalid` | String | External ID |
| `category` | String | Vendor category |
| `taxidnum` | String | Tax ID number |
| `is1099eligible` | String | Indicates 1099 eligibility |
| `legalname` | String | Legal name |
| `accountnumber` | String | Account number |
| `printoncheckas` | String | Print on checks as |
| `representingsubsidiary` | String | Representing subsidiary ID |
| `custom_fields` | Array | Array of custom fields with name and value pairs |
Stream for fetching unit of measure type records from NetSuite. Units types group related units of measure (e.g., Weight, Length). Full table sync only (no replication key).
**Key Fields:**
* `id` - Internal ID
* `links` - Links associated with the units type
* `name` - Units type name
* `isinactive` - Indicates if the units type is inactive
* `lastmodifieddate` - Last modified date
Stream for fetching individual unit of measure records within a Units Type. Each record represents one unit (e.g., kg, lb) and its conversion rate relative to the base unit. Full table sync only (no replication key).
**Key Fields:**
* `internalid` - Internal ID
* `links` - Links associated with the UOM
* `unitname` - Unit name
* `abbreviation` - Unit abbreviation
* `pluralname` - Plural unit name
* `pluralabbreviation` - Plural abbreviation
* `conversionrate` - Conversion rate relative to base unit
* `baseunit` - Indicates if this is the base unit
* `unitstype` - Parent Units Type ID (foreign key to Units Type)
* `inuse` - Indicates if the unit is in use
Stream for fetching vendor (supplier) records from NetSuite.
**Identification:**
* `id` - Internal ID
* `links` - Links associated with the vendor
* `entityid` - Entity ID
* `entitynumber` - Entity number
* `entitytitle` - Entity title
* `externalid` - External ID
* `companyname` - Company name
* `legalname` - Legal name
* `altname` - Alternate name
* `fullname` - Full name
**Person Fields:**
* `isperson` - Indicates if the vendor is a person
* `firstname` - First name
* `middlename` - Middle name
* `lastname` - Last name
**Contact Information:**
* `email` - Email address
* `emailpreference` - Email preference
* `emailtransactions` - Email transactions flag
* `phone` - Phone number
* `altphone` - Alternate phone number
* `fax` - Fax number
* `faxtransactions` - Fax transactions flag
* `printtransactions` - Print transactions flag
* `url` - URL
* `defaultaddress` - Default address
* `defaultbillingaddress` - Default billing address
* `defaultshippingaddress` - Default shipping address
* `comments` - Comments
* `contactlist` - Contact list
* `globalsubscriptionstatus` - Global subscription status
* `unsubscribe` - Unsubscribe flag
**Financial:**
* `subsidiary` - Subsidiary ID
* `subsidiaryedition` - Subsidiary edition
* `terms` - Payment terms ID
* `currency` - Currency ID
* `creditlimit` - Credit limit
* `balance` - Balance
* `balanceprimary` - Balance in primary currency
* `unbilledorders` - Unbilled orders amount
* `unbilledordersprimary` - Unbilled orders in primary currency
* `prepaymentbalance` - Prepayment balance
* `expenseaccount` - Expense account
* `payablesaccount` - Payables account
* `accountnumber` - Account number
* `printoncheckas` - Print on checks as
**Classification:**
* `category` - Vendor category
* `taxidnum` - Tax ID number
* `is1099eligible` - Indicates 1099 eligibility
* `representingsubsidiary` - Representing subsidiary ID
**Status & Flags:**
* `isinactive` - Indicates if the vendor is inactive
* `duplicate` - Duplicate flag
* `isautogeneratedrepresentingentity` - Auto-generated representing entity flag
* `isjobresourcevend` - Job resource vendor flag
**Predictions:**
* `predconfidence` - Prediction confidence
* `predicteddays` - Predicted days
**Other:**
* `workcalendar` - Work calendar
* `manufacturinglocations` - Manufacturing locations
* `datecreated` - Date created
* `lastmodifieddate` - Last modified date
* `custom_fields` - Array of custom fields with name and value pairs
# Data Model
The following diagram illustrates the relationships between the core data streams in NetSuite. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
subgraph "Core Entities"
Customers("Customers");
Vendors("Vendor");
Items("Items");
Locations("Location");
Departments("Department");
Terms("Term");
BillingSchedule("Billing Schedule");
Vendors("Vendor");
end
subgraph "Transactions"
Transactions("Transactions");
TransactionLines("Transaction Lines");
PrevTranLink("Previous Transaction Line Link");
end
subgraph "Bill of Materials"
BOM("BOM");
BOMRevision("BOM Revision");
BOMRevisionComponent("BOM Revision Component");
end
subgraph "Units of Measure"
UnitsType("Units Type");
UnitsTypeUom("Units Type UOM");
end
TransactionLines -- "transaction" --> Transactions;
TransactionLines -- "item" --> Items;
TransactionLines -- "entity" --> Customers;
TransactionLines -- "location" --> Locations;
TransactionLines -- "department" --> Departments;
TransactionLines -- "billingschedule" --> BillingSchedule;
Transactions -- "entity" --> Customers;
Transactions -- "terms" --> Terms;
PrevTranLink -- "previousdoc" --> Transactions;
PrevTranLink -- "nextdoc" --> Transactions;
BOMRevision -- "billofmaterials" --> BOM;
BOMRevisionComponent -- "bomrevision" --> BOMRevision;
BOMRevisionComponent -- "item" --> Items;
UnitsTypeUom -- "unitstype" --> UnitsType;
Items -- "unitstype" --> UnitsType;
```
# Implementation Notes
Due to NetSuiteQL limitations, all field values are returned as strings. You'll need to convert numeric and date fields to their appropriate types in your data pipeline or queries:
* **Dates**: Parse strings like `2024-01-15` to date types
* **Numbers**: Convert numeric strings to integers or decimals
* **Booleans**: Values typically come as `"T"` (true) or `"F"` (false)
NetSuite supports extensive customization through custom fields. These are captured in the `custom_fields` array for each record, containing objects with `name` and `value` properties.
Custom field names typically start with prefixes like:
* `custentity_` - Custom entity fields
* `custcol_` - Custom column (line-level) fields
* `custbody_` - Custom body (transaction-level) fields
* `cseg_` - Custom segment fields
Most streams support incremental sync using the `lastmodifieddate` field as the replication key. This allows for efficient data extraction by only fetching records that have changed since the last sync.
**Streams with incremental sync:**
* Bill of Materials (`lastmodifieddate`)
* BOM (`lastmodifieddate`)
* BOM Revision (`lastmodifieddate`)
* Customers (`lastmodifieddate`)
* Department (`lastmodifieddate`)
* Items (`lastmodifieddate`)
* Location (`lastmodifieddate`)
* Previous Transaction Line Link (`lastmodifieddate`)
* Term (`lastmodifieddate`)
* Transaction Lines (`linelastmodifieddate`)
* Transactions (`lastmodifieddate`)
* Vendor (`lastmodifieddate`)
**Full table only:**
* Billing Schedule (no replication key)
* BOM Revision Component (no replication key)
* Units Type (no replication key)
* Units Type UOM (no replication key)
The **Previous Transaction Line Link** stream is particularly valuable for understanding document flow in NetSuite. It tracks relationships between:
* Sales Orders → Invoices
* Purchase Orders → Vendor Bills
* Quotes → Sales Orders
* Invoices → Payments
Use `previousdoc`/`nextdoc` and `previousline`/`nextline` to trace the complete lifecycle of a transaction.
## Skills for agents
NetSuite connector documentation as plain markdown, for use in AI agent contexts.
# Nibo as a data source
Source: https://docs.nekt.com/sources/nibo
Bring data from Nibo to your Lakehouse.
Nibo is a Brazilian financial management platform used by companies and their accounting offices to control accounts payable and receivable, bank accounts and reconciliation, service invoices (NFS-e) and boletos. This source brings the data from **Nibo Gestão Financeira** (the "Empresa" API) into your Lakehouse.
## Configuring Nibo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Nibo option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The Nibo API is only available on the **Premium** plan of Nibo Gestão Financeira. If your company is on another plan, the API token will not be available.
To generate your token, open Nibo and go to your company, then **Mais opções → Configurações → API**. Copy the token shown there.
The following configurations are available:
* **API token**: The token you copied from the Nibo API settings screen. It is sent on every request and identifies a single company, so create one source per Nibo company you want to extract.
* **Initial sync date**: (Optional) The earliest date used by the streams that require an explicit date range: the `Account Statement` and the `Budgets`. Every other stream always returns your company's full history. When left empty, it defaults to the beginning of the year two years ago.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
`Schedules` is the union of `Credit Schedules` and `Debit Schedules`, and `Stakeholders` is the union of `Customers`, `Suppliers`, `Employees` and `Partners`. If you select both the union and its parts, the same records will be extracted more than once, into different tables.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Nibo's API does not offer a reliable "changed since" filter, so every Nibo stream reads the full collection on each run. **FULL\_TABLE is the recommended sync type**: it keeps your tables an exact mirror of Nibo, including edits made to old records, which financial data receives often.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## Good to know
* **Nested data is flattened.** Nibo nests objects inside its records (the contact, the category, the cost center, the tax blocks of an invoice). They are extracted as scalar columns with a prefix, such as `stakeholder_name`, `category_id` or `service_iss_value`, so they can be queried directly.
* **Lists become JSON.** When a record carries a list, such as the categories or cost centers an amount was split across, or the payment methods of a boleto, the column holds a JSON string, so the table structure never changes as those lists grow.
* **Missing modules are skipped, not fatal.** If a module such as NFS-e or boletos is not enabled for your company, the corresponding stream is skipped with a warning in the run log and the rest of the extraction proceeds normally.
* **Page size.** Nibo caps every query at 500 records per page, and this source paginates through the whole collection automatically.
# Streams and Fields
Below you'll find all available data streams from Nibo and their corresponding fields:
The company the API token belongs to, with its Nibo plan and the features enabled for it.
**Primary key:** `organization_id`
**Fields:**
* `organization_id` - Unique identifier of the company in Nibo
* `name` - Name of the company
* `cnpj` - Brazilian company tax number (CNPJ)
* `plan` - Nibo plan the company is on. The API requires the Premium plan
* `subscription_plan` - Subscription plan currently billed for the company
* `type` - Numeric code for the kind of Nibo account the company holds
* `is_financial_management` - Whether the company uses Nibo Gestao Financeira
* `invoice_enabled` - Whether the company can issue service invoices (NFS-e) from Nibo
* `accountant_id` - Identifier of the accounting office linked to the company
* `accountant_name` - Name of the accounting office linked to the company
* `users` - Users with access to the company (JSON array with userId, email, name and isOwner)
* `features` - Features enabled for the company (JSON array of feature keys)
* `address_line1` - First line of the company address, usually the street name
* `address_line2` - Second line of the company address, usually the complement
* `address_district` - District or neighbourhood
* `address_city` - City of the company address
* `address_state` - Brazilian state (UF)
* `address_zip_code` - Postal code (CEP)
Users invited to the company, with their roles and whether they have accepted the invitation.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the user
* `email` - Email address the user signs in with
* `create_date` - Timestamp when the user was invited to the company
* `accept_date` - Timestamp when the user accepted the invitation
* `is_organization_owner` - Whether the user owns the company account
* `is_organization_user` - Whether the user belongs to the company itself
* `is_accountant_user` - Whether the user belongs to the accounting office instead of the company
* `roles_text` - Roles and permissions granted to the user, as displayed by Nibo
Bank accounts registered in the company, including whether the statement is imported automatically from the bank.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the bank account
* `name` - Name given to the account inside Nibo
* `type` - Kind of account, such as a checking account, cash or an investment account
* `open_balance` - Opening balance the account started with in Nibo
* `date_of_open_balance` - Date the opening balance refers to
* `is_virtual` - Whether the account is virtual rather than a real bank account
* `is_reconcilable` - Whether entries in this account can be reconciled against a bank statement
* `is_archived` - Whether the account has been archived and is no longer used for new entries
* `bank_agency` - Branch number of the bank account
* `bank_account` - Number of the bank account
* `bank_account_verification_number` - Check digit of the bank account number
* `bank_number` - Brazilian bank code (COMPE) of the institution holding the account
* `scraping_enabled` - Whether Nibo automatically imports this account's statement from the bank
* `scraping_configuration_valid` - Whether the automatic statement import is correctly configured
* `is_automated` - Whether the account is currently automated (connected to the bank)
* `can_be_automated` - Whether this account's bank supports automation in Nibo
* `automation_requires_migration` - Whether the existing automation has to be migrated to keep working
* `automated_account_open_balance_date` - Date from which the automated import starts bringing the balance
* `is_open_finance` - Whether the connection to the bank uses Open Finance
* `has_mfa` - Whether the bank connection uses multi-factor authentication
* `is_mfa_required` - Whether the bank requires multi-factor authentication for this account
* `has_payment_operation` - Whether payments can be sent to the bank directly from this account
* `boleto_enabled` - Whether the account can issue Brazilian bank slips (boletos)
* `cnab_enabled` - Whether the account exchanges CNAB files with the bank
* `update_date` - Timestamp of the last change made to the account
* `update_user` - User who last changed the account
Current balance of every bank account, both as recorded in Nibo and as reported by the bank.
**Primary key:** `account_id`
**Fields:**
* `account_id` - Identifier of the bank account
* `account_name` - Name of the bank account in Nibo
* `balance` - Balance of the account according to the entries recorded in Nibo
* `bank_balance` - Balance reported by the bank itself for automated accounts
* `agency` - Branch number of the bank account
* `account_number` - Number of the bank account
* `account_verification_number` - Check digit of the bank account number
* `is_virtual` - Whether the account is virtual rather than a real bank account
* `is_reconcilable` - Whether entries in this account can be reconciled against a bank statement
* `is_account_automated` - Whether the account statement is imported automatically from the bank
* `pending_reconciliation_count` - Number of entries still waiting to be reconciled in this account
* `total_open_reconciliations` - Total amount of the entries still waiting to be reconciled
* `is_pj_bank_virtual_account_waiting_approve` - Whether a PJBank virtual account is still waiting for approval
Reference list of the banks Nibo supports. Join it with an account's `bank_number`.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the bank in Nibo
* `code` - Short code Nibo uses for the bank
* `name` - Name of the banking institution
* `bank_number` - Brazilian bank code (COMPE) of the institution
* `color` - Brand colour Nibo uses for the bank
* `scraping_enabled` - Whether Nibo can import statements from this bank automatically
* `automation_enabled` - Whether accounts at this bank can be automated in Nibo
Transfers made between two bank accounts of the company, with both legs of the movement.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the transfer
* `origin_entry_entry_id` - Identifier of the entry debited in the origin account
* `origin_entry_account_id` - Identifier of the account the money came from
* `origin_entry_account_name` - Name of the account the money came from
* `origin_entry_date` - Date the money left the origin account
* `origin_entry_identifier` - Identifier Nibo shows for the entry in the origin account
* `origin_entry_value` - Amount debited from the origin account
* `origin_entry_is_transfer` - Always true: the origin entry belongs to a transfer
* `destiny_entry_entry_id` - Identifier of the entry credited in the destination account
* `destiny_entry_account_id` - Identifier of the account the money went to
* `destiny_entry_account_name` - Name of the account the money went to
* `destiny_entry_date` - Date the money arrived in the destination account
* `destiny_entry_identifier` - Identifier Nibo shows for the entry in the destination account
* `destiny_entry_value` - Amount credited to the destination account
* `destiny_entry_is_transfer` - Always true: the destination entry belongs to a transfer
Bank statement of each account: every entry with its running balance. Read month by month from the configured start date.
**Primary key:** `account_id`, `entry_id`
**Fields:**
* `account_id` - Identifier of the bank account this statement line belongs to
* `account_name` - Name of the bank account this statement line belongs to
* `entry_id` - Unique identifier of the entry
* `schedule_id` - Identifier of the schedule (payable or receivable) that produced the entry
* `date` - Date the entry hit the account
* `create_date` - Timestamp when the entry was recorded in Nibo
* `description` - Description of the entry as shown on the statement
* `value` - Amount of the entry: positive for money in, negative for money out
* `current_balance` - Balance of the account right after this entry (running balance)
* `type` - Kind of entry, such as a receipt, a payment or a transfer
* `is_transfer` - Whether the entry is one leg of a transfer between own accounts
* `is_reconciliated` - Whether the entry has already been reconciled against the bank
* `stakeholder_id` - Identifier of the contact involved in the entry
* `stakeholder_name` - Name of the contact involved in the entry
* `stakeholder_type` - Kind of contact involved: customer, supplier, employee or partner
* `stakeholder_is_deleted` - Whether the contact involved in the entry has been deleted in Nibo
Entries of each account still waiting to be reconciled against the bank.
**Primary key:** `account_id`, `account_entry_id`
**Fields:**
* `account_id` - Identifier of the bank account being reconciled
* `account_name` - Name of the bank account being reconciled
* `account_entry_id` - Identifier of the bank entry waiting to be reconciled
* `account_entry_account_id` - Identifier of the account the bank entry belongs to
* `account_entry_value` - Amount of the bank entry
* `account_entry_date` - Date the bank entry hit the account
* `account_entry_description` - Description of the entry as sent by the bank
* `account_entry_rank` - Ranking Nibo assigns to the entry when suggesting matches
* `account_entry_discussions` - Comments exchanged about this entry (JSON array)
* `mapping` - Nibo entries this bank entry can be reconciled against (JSON array)
Every contact of the company in a single table: customers, suppliers, employees and partners. This is the convenient dimension to join financial entries against, since a `stakeholder_id` always resolves here.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the contact in Nibo
* `name` - Display name of the contact
* `initials_name` - Short name or initials shown by Nibo when space is limited
* `email` - Main email address of the contact
* `phone` - Main phone number of the contact
* `type` - Kind of contact as classified by Nibo: customer, supplier, employee or partner
* `person_type` - Numeric code telling whether the contact is an individual or a company
* `is_company` - Whether the contact is a legal entity (company) rather than an individual
* `is_deleted` - Whether the contact has been deleted in Nibo
* `is_archived` - Whether the contact has been archived and is no longer offered for new entries
* `document_number` - Brazilian tax document of the contact (CPF for individuals, CNPJ for companies)
* `document_type` - Which tax document the number refers to (CPF or CNPJ)
* `communication_contact_name` - Name of the person to contact at this customer, supplier or partner
* `communication_email` - Email used for billing and notification messages sent by Nibo
* `communication_phone` - Landline number registered for communication with the contact
* `communication_cell_phone` - Mobile number registered for communication with the contact
* `communication_web_site` - Website of the contact
* `address_line1` - First line of the address, usually the street name
* `address_line2` - Second line of the address, usually the complement
* `address_number` - Street number of the address
* `address_district` - District or neighbourhood
* `address_city` - City of the address
* `address_state` - Brazilian state (UF) of the address
* `address_zip_code` - Postal code (CEP) of the address
* `address_country` - Country of the address
* `address_ibge_code` - IBGE code of the city, used by Brazilian tax documents
* `bank_account_information_bank` - Bank where the contact holds the account used for payments or receipts
* `bank_account_information_agency` - Branch number of the contact's bank account
* `bank_account_information_account_number` - Account number of the contact's bank account
* `company_information_company_name` - Registered legal name (razao social) when the contact is a company
* `company_information_municipal_inscription` - Municipal taxpayer registration of the company, used when issuing invoices
* `update_date` - Timestamp of the last change made to the contact
* `update_user` - User who last changed the contact
Customers registered in the company, the counterparty of every receivable.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the contact in Nibo
* `name` - Display name of the contact
* `initials_name` - Short name or initials shown by Nibo when space is limited
* `email` - Main email address of the contact
* `phone` - Main phone number of the contact
* `type` - Kind of contact as classified by Nibo: customer, supplier, employee or partner
* `person_type` - Numeric code telling whether the contact is an individual or a company
* `is_company` - Whether the contact is a legal entity (company) rather than an individual
* `is_deleted` - Whether the contact has been deleted in Nibo
* `is_archived` - Whether the contact has been archived and is no longer offered for new entries
* `document_number` - Brazilian tax document of the contact (CPF for individuals, CNPJ for companies)
* `document_type` - Which tax document the number refers to (CPF or CNPJ)
* `communication_contact_name` - Name of the person to contact at this customer, supplier or partner
* `communication_email` - Email used for billing and notification messages sent by Nibo
* `communication_phone` - Landline number registered for communication with the contact
* `communication_cell_phone` - Mobile number registered for communication with the contact
* `communication_web_site` - Website of the contact
* `address_line1` - First line of the address, usually the street name
* `address_line2` - Second line of the address, usually the complement
* `address_number` - Street number of the address
* `address_district` - District or neighbourhood
* `address_city` - City of the address
* `address_state` - Brazilian state (UF) of the address
* `address_zip_code` - Postal code (CEP) of the address
* `address_country` - Country of the address
* `address_ibge_code` - IBGE code of the city, used by Brazilian tax documents
* `bank_account_information_bank` - Bank where the contact holds the account used for payments or receipts
* `bank_account_information_agency` - Branch number of the contact's bank account
* `bank_account_information_account_number` - Account number of the contact's bank account
* `company_information_company_name` - Registered legal name (razao social) when the contact is a company
* `company_information_municipal_inscription` - Municipal taxpayer registration of the company, used when issuing invoices
* `update_date` - Timestamp of the last change made to the contact
* `update_user` - User who last changed the contact
Suppliers registered in the company, the counterparty of most payables.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the contact in Nibo
* `name` - Display name of the contact
* `initials_name` - Short name or initials shown by Nibo when space is limited
* `email` - Main email address of the contact
* `phone` - Main phone number of the contact
* `type` - Kind of contact as classified by Nibo: customer, supplier, employee or partner
* `person_type` - Numeric code telling whether the contact is an individual or a company
* `is_company` - Whether the contact is a legal entity (company) rather than an individual
* `is_deleted` - Whether the contact has been deleted in Nibo
* `is_archived` - Whether the contact has been archived and is no longer offered for new entries
* `document_number` - Brazilian tax document of the contact (CPF for individuals, CNPJ for companies)
* `document_type` - Which tax document the number refers to (CPF or CNPJ)
* `communication_contact_name` - Name of the person to contact at this customer, supplier or partner
* `communication_email` - Email used for billing and notification messages sent by Nibo
* `communication_phone` - Landline number registered for communication with the contact
* `communication_cell_phone` - Mobile number registered for communication with the contact
* `communication_web_site` - Website of the contact
* `address_line1` - First line of the address, usually the street name
* `address_line2` - Second line of the address, usually the complement
* `address_number` - Street number of the address
* `address_district` - District or neighbourhood
* `address_city` - City of the address
* `address_state` - Brazilian state (UF) of the address
* `address_zip_code` - Postal code (CEP) of the address
* `address_country` - Country of the address
* `address_ibge_code` - IBGE code of the city, used by Brazilian tax documents
* `bank_account_information_bank` - Bank where the contact holds the account used for payments or receipts
* `bank_account_information_agency` - Branch number of the contact's bank account
* `bank_account_information_account_number` - Account number of the contact's bank account
* `company_information_company_name` - Registered legal name (razao social) when the contact is a company
* `company_information_municipal_inscription` - Municipal taxpayer registration of the company, used when issuing invoices
* `update_date` - Timestamp of the last change made to the contact
* `update_user` - User who last changed the contact
Employees registered in the company, used on payroll-related payables.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the contact in Nibo
* `name` - Display name of the contact
* `initials_name` - Short name or initials shown by Nibo when space is limited
* `email` - Main email address of the contact
* `phone` - Main phone number of the contact
* `type` - Kind of contact as classified by Nibo: customer, supplier, employee or partner
* `person_type` - Numeric code telling whether the contact is an individual or a company
* `is_company` - Whether the contact is a legal entity (company) rather than an individual
* `is_deleted` - Whether the contact has been deleted in Nibo
* `is_archived` - Whether the contact has been archived and is no longer offered for new entries
* `document_number` - Brazilian tax document of the contact (CPF for individuals, CNPJ for companies)
* `document_type` - Which tax document the number refers to (CPF or CNPJ)
* `communication_contact_name` - Name of the person to contact at this customer, supplier or partner
* `communication_email` - Email used for billing and notification messages sent by Nibo
* `communication_phone` - Landline number registered for communication with the contact
* `communication_cell_phone` - Mobile number registered for communication with the contact
* `communication_web_site` - Website of the contact
* `address_line1` - First line of the address, usually the street name
* `address_line2` - Second line of the address, usually the complement
* `address_number` - Street number of the address
* `address_district` - District or neighbourhood
* `address_city` - City of the address
* `address_state` - Brazilian state (UF) of the address
* `address_zip_code` - Postal code (CEP) of the address
* `address_country` - Country of the address
* `address_ibge_code` - IBGE code of the city, used by Brazilian tax documents
* `bank_account_information_bank` - Bank where the contact holds the account used for payments or receipts
* `bank_account_information_agency` - Branch number of the contact's bank account
* `bank_account_information_account_number` - Account number of the contact's bank account
* `company_information_company_name` - Registered legal name (razao social) when the contact is a company
* `company_information_municipal_inscription` - Municipal taxpayer registration of the company, used when issuing invoices
* `update_date` - Timestamp of the last change made to the contact
* `update_user` - User who last changed the contact
Partners (socios) of the company, used on profit distribution entries.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the contact in Nibo
* `name` - Display name of the contact
* `initials_name` - Short name or initials shown by Nibo when space is limited
* `email` - Main email address of the contact
* `phone` - Main phone number of the contact
* `type` - Kind of contact as classified by Nibo: customer, supplier, employee or partner
* `person_type` - Numeric code telling whether the contact is an individual or a company
* `is_company` - Whether the contact is a legal entity (company) rather than an individual
* `is_deleted` - Whether the contact has been deleted in Nibo
* `is_archived` - Whether the contact has been archived and is no longer offered for new entries
* `document_number` - Brazilian tax document of the contact (CPF for individuals, CNPJ for companies)
* `document_type` - Which tax document the number refers to (CPF or CNPJ)
* `communication_contact_name` - Name of the person to contact at this customer, supplier or partner
* `communication_email` - Email used for billing and notification messages sent by Nibo
* `communication_phone` - Landline number registered for communication with the contact
* `communication_cell_phone` - Mobile number registered for communication with the contact
* `communication_web_site` - Website of the contact
* `address_line1` - First line of the address, usually the street name
* `address_line2` - Second line of the address, usually the complement
* `address_number` - Street number of the address
* `address_district` - District or neighbourhood
* `address_city` - City of the address
* `address_state` - Brazilian state (UF) of the address
* `address_zip_code` - Postal code (CEP) of the address
* `address_country` - Country of the address
* `address_ibge_code` - IBGE code of the city, used by Brazilian tax documents
* `bank_account_information_bank` - Bank where the contact holds the account used for payments or receipts
* `bank_account_information_agency` - Branch number of the contact's bank account
* `bank_account_information_account_number` - Account number of the contact's bank account
* `company_information_company_name` - Registered legal name (razao social) when the contact is a company
* `company_information_municipal_inscription` - Municipal taxpayer registration of the company, used when issuing invoices
* `update_date` - Timestamp of the last change made to the contact
* `update_user` - User who last changed the contact
Categories used to classify receivables and payables. This is the dimension a profit and loss report groups by.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the category
* `name` - Name of the category
* `type` - Whether the category classifies money in (in) or money out (out)
* `group_id` - Identifier of the group the category belongs to
* `group_name` - Name of the group the category belongs to
* `subgroup_id` - Identifier of the subgroup the category belongs to
* `subgroup_name` - Name of the subgroup the category belongs to
* `is_editable` - Whether the category was created by the company and can be edited
Top level groups the categories are organized into.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the category group
* `name` - Name of the category group
* `reference_code` - Code Nibo uses to reference the group in reports
Full hierarchy of groups, subgroups and categories, with each group's children as a JSON array.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the group
* `name` - Name of the group
* `reference_code` - Code Nibo uses to reference the group in reports
* `order` - Position of the group in the hierarchy shown in Nibo
* `children` - Categories and subgroups under this group (JSON array with id, name, type, isSubgroup, isEditable, order and isDeleted for each child)
Cost centers used to split an entry across areas of the company.
**Primary key:** `cost_center_id`
**Fields:**
* `cost_center_id` - Unique identifier of the cost centre
* `description` - Name of the cost centre as shown in Nibo
* `update_date` - Timestamp of the last change made to the cost centre
* `update_user` - User who last changed the cost centre
Every schedule of the company, receivables and payables together. This is the only endpoint that also returns the recurrence and the boleto linked to each schedule.
**Primary key:** `schedule_id`
**Fields:**
* `schedule_id` - Unique identifier of the schedule
* `type` - Kind of schedule: a receivable (credit) or a payable (debit)
* `description` - Description given to the schedule in Nibo
* `value` - Full amount of the schedule
* `paid_value` - Amount already settled, whether received or paid
* `open_value` - Amount still outstanding on the schedule
* `due_date` - Date the schedule is due to be received or paid
* `schedule_date` - Date the schedule was recorded for, used as the cash-basis date
* `accrual_date` - Date the schedule is recognised on, used as the accrual-basis date
* `is_paid` - Whether the schedule has been fully settled
* `is_dued` - Whether the schedule is past its due date and still open
* `is_entry` - Whether the schedule represents money coming in
* `is_bill` - Whether the schedule is billed with a Brazilian bank slip (boleto)
* `is_debit_note` - Whether the schedule is a debit note
* `is_flagged` - Whether the schedule was flagged for attention in Nibo
* `is_payment_scheduled` - Whether a payment has already been scheduled with the bank
* `stakeholder_id` - Identifier of the contact the schedule refers to
* `stakeholder_name` - Name of the contact the schedule refers to
* `stakeholder_type` - Kind of contact: customer, supplier, employee or partner
* `stakeholder_cpf_cnpj` - Tax document (CPF or CNPJ) of the contact
* `stakeholder_is_deleted` - Whether the contact has been deleted in Nibo
* `category_id` - Identifier of the category the schedule is classified under
* `category_name` - Name of the category the schedule is classified under
* `category_type` - Whether the category classifies money in (in) or money out (out)
* `category_is_deleted` - Whether the category has been deleted in Nibo
* `categories` - Every category the amount was split across (JSON array with categoryId, categoryName, value, type and the parent category)
* `cost_center_id` - Identifier of the main cost centre of the schedule
* `cost_center_description` - Name of the main cost centre of the schedule
* `cost_center_is_deleted` - Whether the cost centre has been deleted in Nibo
* `cost_centers` - Every cost centre the amount was split across (JSON array with costCenterId, percent and value)
* `cost_center_value_type` - Whether the cost centre split is expressed as a percentage or an amount
* `has_installment` - Whether the schedule is one instalment of a larger agreement
* `installment_id` - Identifier shared by every schedule of the same instalment plan
* `has_recurrence` - Whether the schedule repeats on a recurring basis
* `recurrence_id` - Identifier shared by every schedule of the same recurrence
* `recurrence_interval` - How many periods there are between two occurrences
* `recurrence_interval_type` - Numeric code for the period the interval is counted in (day, month, year)
* `recurrence_end_type` - Numeric code for how the recurrence ends (never, on a date, after N times)
* `recurrence_provision_in_advance` - How many future occurrences Nibo creates in advance
* `recurrence_base_day` - Day of the month the recurrence falls on
* `has_invoice` - Whether a service invoice (NFS-e) has been issued for the schedule
* `has_pending_invoice` - Whether a service invoice is still waiting to be issued
* `has_schedule_invoice` - Whether the invoice is scheduled to be issued automatically
* `auto_generate_nf_se_type` - Numeric code for when Nibo issues the service invoice automatically
* `auto_generate_collection_type` - Numeric code for when Nibo issues the bank slip automatically
* `collection_id` - Identifier of the bank slip issued for the schedule
* `collection_status_code` - Numeric status of the bank slip issued for the schedule
* `collection_status_description` - Human readable status of the bank slip issued for the schedule
* `has_entry_promise` - Whether a payment promise was recorded for the schedule
* `has_open_entry_promise` - Whether a payment promise is still open for the schedule
* `auto_generate_entry_promise` - Whether Nibo records the payment promise automatically
* `custom_attributes` - Custom fields defined by the company (JSON object). The keys differ from company to company, so the value is kept as JSON
* `create_date` - Timestamp when the schedule was created in Nibo
* `create_user` - User who created the schedule
* `update_date` - Timestamp of the last change made to the schedule
* `update_user` - User who last changed the schedule
Accounts receivable: everything the company is scheduled to receive.
**Primary key:** `schedule_id`
**Fields:**
* `schedule_id` - Unique identifier of the schedule
* `type` - Kind of schedule: a receivable (credit) or a payable (debit)
* `description` - Description given to the schedule in Nibo
* `value` - Full amount of the schedule
* `paid_value` - Amount already settled, whether received or paid
* `open_value` - Amount still outstanding on the schedule
* `due_date` - Date the schedule is due to be received or paid
* `schedule_date` - Date the schedule was recorded for, used as the cash-basis date
* `accrual_date` - Date the schedule is recognised on, used as the accrual-basis date
* `is_paid` - Whether the schedule has been fully settled
* `is_dued` - Whether the schedule is past its due date and still open
* `is_entry` - Whether the schedule represents money coming in
* `is_bill` - Whether the schedule is billed with a Brazilian bank slip (boleto)
* `is_debit_note` - Whether the schedule is a debit note
* `is_flagged` - Whether the schedule was flagged for attention in Nibo
* `is_payment_scheduled` - Whether a payment has already been scheduled with the bank
* `stakeholder_id` - Identifier of the contact the schedule refers to
* `stakeholder_name` - Name of the contact the schedule refers to
* `stakeholder_type` - Kind of contact: customer, supplier, employee or partner
* `stakeholder_cpf_cnpj` - Tax document (CPF or CNPJ) of the contact
* `stakeholder_is_deleted` - Whether the contact has been deleted in Nibo
* `category_id` - Identifier of the category the schedule is classified under
* `category_name` - Name of the category the schedule is classified under
* `category_type` - Whether the category classifies money in (in) or money out (out)
* `category_is_deleted` - Whether the category has been deleted in Nibo
* `categories` - Every category the amount was split across (JSON array with categoryId, categoryName, value, type and the parent category)
* `cost_center_id` - Identifier of the main cost centre of the schedule
* `cost_center_description` - Name of the main cost centre of the schedule
* `cost_center_is_deleted` - Whether the cost centre has been deleted in Nibo
* `cost_centers` - Every cost centre the amount was split across (JSON array with costCenterId, percent and value)
* `cost_center_value_type` - Whether the cost centre split is expressed as a percentage or an amount
* `has_installment` - Whether the schedule is one instalment of a larger agreement
* `installment_id` - Identifier shared by every schedule of the same instalment plan
* `has_recurrence` - Whether the schedule repeats on a recurring basis
* `recurrence_id` - Identifier shared by every schedule of the same recurrence
* `recurrence_interval` - How many periods there are between two occurrences
* `recurrence_interval_type` - Numeric code for the period the interval is counted in (day, month, year)
* `recurrence_end_type` - Numeric code for how the recurrence ends (never, on a date, after N times)
* `recurrence_provision_in_advance` - How many future occurrences Nibo creates in advance
* `recurrence_base_day` - Day of the month the recurrence falls on
* `has_invoice` - Whether a service invoice (NFS-e) has been issued for the schedule
* `has_pending_invoice` - Whether a service invoice is still waiting to be issued
* `has_schedule_invoice` - Whether the invoice is scheduled to be issued automatically
* `auto_generate_nf_se_type` - Numeric code for when Nibo issues the service invoice automatically
* `auto_generate_collection_type` - Numeric code for when Nibo issues the bank slip automatically
* `collection_id` - Identifier of the bank slip issued for the schedule
* `collection_status_code` - Numeric status of the bank slip issued for the schedule
* `collection_status_description` - Human readable status of the bank slip issued for the schedule
* `has_entry_promise` - Whether a payment promise was recorded for the schedule
* `has_open_entry_promise` - Whether a payment promise is still open for the schedule
* `auto_generate_entry_promise` - Whether Nibo records the payment promise automatically
* `custom_attributes` - Custom fields defined by the company (JSON object). The keys differ from company to company, so the value is kept as JSON
* `create_date` - Timestamp when the schedule was created in Nibo
* `create_user` - User who created the schedule
* `update_date` - Timestamp of the last change made to the schedule
* `update_user` - User who last changed the schedule
Accounts payable: everything the company is scheduled to pay.
**Primary key:** `schedule_id`
**Fields:**
* `schedule_id` - Unique identifier of the schedule
* `type` - Kind of schedule: a receivable (credit) or a payable (debit)
* `description` - Description given to the schedule in Nibo
* `value` - Full amount of the schedule
* `paid_value` - Amount already settled, whether received or paid
* `open_value` - Amount still outstanding on the schedule
* `due_date` - Date the schedule is due to be received or paid
* `schedule_date` - Date the schedule was recorded for, used as the cash-basis date
* `accrual_date` - Date the schedule is recognised on, used as the accrual-basis date
* `is_paid` - Whether the schedule has been fully settled
* `is_dued` - Whether the schedule is past its due date and still open
* `is_entry` - Whether the schedule represents money coming in
* `is_bill` - Whether the schedule is billed with a Brazilian bank slip (boleto)
* `is_debit_note` - Whether the schedule is a debit note
* `is_flagged` - Whether the schedule was flagged for attention in Nibo
* `is_payment_scheduled` - Whether a payment has already been scheduled with the bank
* `stakeholder_id` - Identifier of the contact the schedule refers to
* `stakeholder_name` - Name of the contact the schedule refers to
* `stakeholder_type` - Kind of contact: customer, supplier, employee or partner
* `stakeholder_cpf_cnpj` - Tax document (CPF or CNPJ) of the contact
* `stakeholder_is_deleted` - Whether the contact has been deleted in Nibo
* `category_id` - Identifier of the category the schedule is classified under
* `category_name` - Name of the category the schedule is classified under
* `category_type` - Whether the category classifies money in (in) or money out (out)
* `category_is_deleted` - Whether the category has been deleted in Nibo
* `categories` - Every category the amount was split across (JSON array with categoryId, categoryName, value, type and the parent category)
* `cost_center_id` - Identifier of the main cost centre of the schedule
* `cost_center_description` - Name of the main cost centre of the schedule
* `cost_center_is_deleted` - Whether the cost centre has been deleted in Nibo
* `cost_centers` - Every cost centre the amount was split across (JSON array with costCenterId, percent and value)
* `cost_center_value_type` - Whether the cost centre split is expressed as a percentage or an amount
* `has_installment` - Whether the schedule is one instalment of a larger agreement
* `installment_id` - Identifier shared by every schedule of the same instalment plan
* `has_recurrence` - Whether the schedule repeats on a recurring basis
* `recurrence_id` - Identifier shared by every schedule of the same recurrence
* `recurrence_interval` - How many periods there are between two occurrences
* `recurrence_interval_type` - Numeric code for the period the interval is counted in (day, month, year)
* `recurrence_end_type` - Numeric code for how the recurrence ends (never, on a date, after N times)
* `recurrence_provision_in_advance` - How many future occurrences Nibo creates in advance
* `recurrence_base_day` - Day of the month the recurrence falls on
* `has_invoice` - Whether a service invoice (NFS-e) has been issued for the schedule
* `has_pending_invoice` - Whether a service invoice is still waiting to be issued
* `has_schedule_invoice` - Whether the invoice is scheduled to be issued automatically
* `auto_generate_nf_se_type` - Numeric code for when Nibo issues the service invoice automatically
* `auto_generate_collection_type` - Numeric code for when Nibo issues the bank slip automatically
* `collection_id` - Identifier of the bank slip issued for the schedule
* `collection_status_code` - Numeric status of the bank slip issued for the schedule
* `collection_status_description` - Human readable status of the bank slip issued for the schedule
* `has_entry_promise` - Whether a payment promise was recorded for the schedule
* `has_open_entry_promise` - Whether a payment promise is still open for the schedule
* `auto_generate_entry_promise` - Whether Nibo records the payment promise automatically
* `custom_attributes` - Custom fields defined by the company (JSON object). The keys differ from company to company, so the value is kept as JSON
* `create_date` - Timestamp when the schedule was created in Nibo
* `create_user` - User who created the schedule
* `update_date` - Timestamp of the last change made to the schedule
* `update_user` - User who last changed the schedule
Receivables that were actually received, already linked to the bank account they hit.
**Primary key:** `entry_id`
**Fields:**
* `entry_id` - Unique identifier of the entry
* `schedule_id` - Identifier of the schedule this entry settles
* `date` - Date the money moved, used as the cash-basis date
* `accrual_date` - Date the entry is recognised on, used as the accrual-basis date
* `value` - Amount received or paid
* `description` - Description recorded for the entry
* `identifier` - Identifier Nibo shows for the entry on the account statement
* `reference` - Free reference recorded with the entry, such as a document number
* `check_num` - Cheque number, when paid by cheque
* `is_virtual` - Whether the entry sits in a virtual account rather than a real one
* `is_reconciliated` - Whether the entry has been reconciled against the bank statement
* `is_transfer` - Whether the entry is one leg of a transfer between own accounts
* `is_flagged` - Whether the entry was flagged for attention in Nibo
* `bank_balance_date_is_greater_than_entry_date` - Whether the bank balance is already more recent than this entry, which means the entry is covered by the imported statement
* `account_id` - Identifier of the bank account the money moved through
* `account_name` - Name of the bank account the money moved through
* `account_is_deleted` - Whether that bank account has been deleted in Nibo
* `stakeholder_id` - Identifier of the contact the money was received from or paid to
* `stakeholder_name` - Name of the contact the money was received from or paid to
* `stakeholder_is_deleted` - Whether the contact has been deleted in Nibo
* `category_id` - Identifier of the category the entry is classified under
* `category_name` - Name of the category the entry is classified under
* `category_type` - Whether the category classifies money in (in) or money out (out)
* `category_is_deleted` - Whether the category has been deleted in Nibo
* `categories` - Every category the amount was split across (JSON array with categoryId, categoryName, value, type and the parent category)
* `cost_center_id` - Identifier of the main cost centre of the entry
* `cost_center_description` - Name of the main cost centre of the entry
* `cost_centers` - Every cost centre the amount was split across (JSON array with costCenterId, percent and value)
Payables that were actually paid, already linked to the bank account they hit.
**Primary key:** `entry_id`
**Fields:**
* `entry_id` - Unique identifier of the entry
* `schedule_id` - Identifier of the schedule this entry settles
* `date` - Date the money moved, used as the cash-basis date
* `accrual_date` - Date the entry is recognised on, used as the accrual-basis date
* `value` - Amount received or paid
* `description` - Description recorded for the entry
* `identifier` - Identifier Nibo shows for the entry on the account statement
* `reference` - Free reference recorded with the entry, such as a document number
* `check_num` - Cheque number, when paid by cheque
* `is_virtual` - Whether the entry sits in a virtual account rather than a real one
* `is_reconciliated` - Whether the entry has been reconciled against the bank statement
* `is_transfer` - Whether the entry is one leg of a transfer between own accounts
* `is_flagged` - Whether the entry was flagged for attention in Nibo
* `bank_balance_date_is_greater_than_entry_date` - Whether the bank balance is already more recent than this entry, which means the entry is covered by the imported statement
* `account_id` - Identifier of the bank account the money moved through
* `account_name` - Name of the bank account the money moved through
* `account_is_deleted` - Whether that bank account has been deleted in Nibo
* `stakeholder_id` - Identifier of the contact the money was received from or paid to
* `stakeholder_name` - Name of the contact the money was received from or paid to
* `stakeholder_is_deleted` - Whether the contact has been deleted in Nibo
* `category_id` - Identifier of the category the entry is classified under
* `category_name` - Name of the category the entry is classified under
* `category_type` - Whether the category classifies money in (in) or money out (out)
* `category_is_deleted` - Whether the category has been deleted in Nibo
* `categories` - Every category the amount was split across (JSON array with categoryId, categoryName, value, type and the parent category)
* `cost_center_id` - Identifier of the main cost centre of the entry
* `cost_center_description` - Name of the main cost centre of the entry
* `cost_centers` - Every cost centre the amount was split across (JSON array with costCenterId, percent and value)
Brazilian service invoices (NFS-e) issued through Nibo, with every tax block flattened into its own column.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the service invoice
* `status_code` - Numeric status of the invoice in Nibo
* `status_description` - Human readable status of the invoice, such as issued or cancelled
* `last_message` - Last message Nibo recorded about the invoice
* `last_status_message` - Last message returned by the city hall about the invoice
* `last_status_change` - Timestamp of the last status change of the invoice
* `value` - Total amount of the invoice
* `deductions` - Amount deducted from the base used to calculate the taxes
* `discounts` - Discount granted on the invoice
* `remarks` - Free remarks printed on the invoice
* `schedule_id` - Identifier of the receivable the invoice was issued for
* `schedule_due_date` - Due date of the receivable the invoice was issued for
* `schedule_date` - Date recorded on the receivable the invoice was issued for
* `schedule_value` - Amount of the receivable the invoice was issued for
* `schedule_description` - Description of the receivable the invoice was issued for
* `schedule_is_deleted` - Whether that receivable has been deleted in Nibo
* `service_profile_id` - Identifier of the service profile used to issue the invoice
* `service_profile_name` - Name of the service profile used to issue the invoice
* `service_profile_is_deleted` - Whether that service profile has been deleted in Nibo
* `stakeholder_id` - Identifier of the contact the invoice was issued to
* `stakeholder_name` - Name of the contact the invoice was issued to
* `stakeholder_type` - Kind of contact the invoice was issued to
* `stakeholder_is_deleted` - Whether that contact has been deleted in Nibo
* `customer_name` - Name of the customer as printed on the invoice
* `customer_cpf_cnpj` - Tax document (CPF or CNPJ) of the customer on the invoice
* `customer_email` - Email the invoice was sent to
* `customer_phone` - Phone number of the customer on the invoice
* `customer_legal_type` - Legal nature of the customer, individual or company
* `customer_address_address` - Street of the customer address printed on the invoice
* `customer_address_number` - Street number of the customer address
* `customer_address_complement` - Complement of the customer address
* `customer_address_district` - District of the customer address
* `customer_address_city` - City of the customer address
* `customer_address_uf` - Brazilian state (UF) of the customer address
* `customer_address_postal_code` - Postal code (CEP) of the customer address
* `service_city_service_code` - Service code defined by the city hall for the service provided
* `service_cnae_code` - CNAE code of the economic activity the service belongs to
* `service_list_item_lc116` - Item of the LC 116 national service list the service falls under
* `service_remove_iss_value_from_base_value_to_calculate_pis_and_cofins` - Whether the ISS is removed from the base used to calculate PIS and COFINS
* `service_iss_aliquot` - ISS rate applied
* `service_iss_value` - ISS amount of the invoice
* `service_iss_retained` - Whether the ISS is withheld by the customer
* `service_iss_retained_description` - Reason recorded for withholding the ISS
* `service_pis_aliquot` - PIS rate applied
* `service_pis_value` - PIS amount of the invoice
* `service_pis_retained` - Whether the PIS is withheld by the customer
* `service_pis_reduction_base` - Reduction applied to the PIS calculation base
* `service_cofins_aliquot` - COFINS rate applied
* `service_cofins_value` - COFINS amount of the invoice
* `service_cofins_retained` - Whether the COFINS is withheld by the customer
* `service_cofins_reduction_base` - Reduction applied to the COFINS calculation base
* `service_csll_aliquot` - CSLL rate applied
* `service_csll_value` - CSLL amount of the invoice
* `service_csll_retained` - Whether the CSLL is withheld by the customer
* `service_ir_aliquot` - Income tax (IR) rate applied
* `service_ir_value` - Income tax (IR) amount of the invoice
* `service_ir_retained` - Whether the income tax is withheld by the customer
* `service_inss_aliquot` - INSS rate applied
* `service_inss_value` - INSS amount of the invoice
* `service_inss_retained` - Whether the INSS is withheld by the customer
* `service_inss_reduction_base` - Reduction applied to the INSS calculation base
* `service_federal_tax_aliquot` - Aggregated federal tax rate printed on the invoice
* `service_federal_tax_value` - Aggregated federal tax amount printed on the invoice
* `create_date` - Timestamp when the invoice was created in Nibo
* `create_user` - User who created the invoice
* `update_date` - Timestamp of the last change made to the invoice
* `update_user` - User who last changed the invoice
Service profiles: the reusable tax templates a service invoice is issued from.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the service profile
* `name` - Name of the service profile
* `description` - Description of the service printed on the invoice
* `additional_remarks` - Additional remarks printed on every invoice issued with this profile
* `is_suggestion` - Whether the profile was suggested by Nibo rather than created by the company
* `city_service_code` - Service code defined by the city hall for this profile
* `cnae_code` - CNAE code of the economic activity of this profile
* `service_list_item_lc116` - Item of the LC 116 national service list this profile falls under
* `remove_iss_value_from_base_value_to_calculate_pis_and_cofins` - Whether the ISS is removed from the base used to calculate PIS and COFINS
* `iss_aliquot` - ISS rate configured on the profile
* `iss_retained` - Whether invoices issued with this profile withhold the ISS
* `pis_aliquot` - PIS rate configured on the profile
* `pis_retained` - Whether invoices issued with this profile withhold the PIS
* `pis_reduction_base` - Reduction applied to the PIS calculation base
* `cofins_aliquot` - COFINS rate configured on the profile
* `cofins_retained` - Whether invoices issued with this profile withhold the COFINS
* `cofins_reduction_base` - Reduction applied to the COFINS calculation base
* `csll_aliquot` - CSLL rate configured on the profile
* `csll_retained` - Whether invoices issued with this profile withhold the CSLL
* `ir_aliquot` - Income tax (IR) rate configured on the profile
* `ir_retained` - Whether invoices issued with this profile withhold the income tax
* `inss_aliquot` - INSS rate configured on the profile
* `inss_retained` - Whether invoices issued with this profile withhold the INSS
* `inss_reduction_base` - Reduction applied to the INSS calculation base
* `federal_tax_aliquot` - Aggregated federal tax rate configured on the profile
* `construction_enabled` - Whether the profile supports construction specific invoice fields
* `event_enabled` - Whether the profile supports event specific invoice fields
Brazilian bank slips (boletos) issued to customers through Nibo.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the bank slip
* `schedule_id` - Identifier of the receivable the bank slip was issued for
* `stakeholder_id` - Identifier of the customer the bank slip was issued to
* `collection_profile_id` - Identifier of the profile whose fine, interest and reminder rules were applied
* `description` - Description printed on the bank slip
* `due_date` - Date the bank slip is due
* `value` - Amount charged on the bank slip
* `paid_value` - Amount actually paid by the customer
* `status_code` - Numeric status of the bank slip
* `status_description` - Human readable status of the bank slip, such as open, paid or cancelled
* `las_status_change_date` - Timestamp of the last status change of the bank slip. The field name carries the typo Nibo uses in its API
* `debtor_name` - Name of the customer being charged
* `debtor_document` - Tax document (CPF or CNPJ) of the customer being charged
* `beneficiary_name` - Name of the company receiving the payment
* `beneficiary_document` - Tax document (CNPJ) of the company receiving the payment
* `url` - Link where the customer can pay the bank slip
* `pdf` - Link to the PDF version of the bank slip
* `create_date` - Timestamp when the bank slip was issued
* `create_user` - User who issued the bank slip
Profiles that define the fine, interest and reminder rules applied to boletos.
**Primary key:** `id`
**Fields:**
* `id` - Unique identifier of the collection profile
* `organization_id` - Identifier of the company the profile belongs to
* `name` - Name of the collection profile
* `fine_percentage` - Fine charged, as a percentage, once the bank slip is overdue
* `interest_percentage` - Interest charged, as a percentage, while the bank slip stays overdue
* `interest_period_type_code` - Numeric code for the period the interest rate refers to
* `interest_period_type_description` - Period the interest rate refers to, such as per day or per month
* `display_beneficiary_business_name` - Whether the legal name of the company is printed on the bank slip
* `notification_strategy_json` - Rules for the reminders sent to the customer, as the JSON string Nibo returns
* `payment_methods` - Payment methods accepted by the profile (JSON array with paymentForm and provider)
* `create_date` - Timestamp when the profile was created
* `create_user` - User who created the profile
* `update_date` - Timestamp of the last change made to the profile
* `update_user` - User who last changed the profile
Budget planned per category, exploded into one row per category and month.
**Primary key:** `year`, `category_id`, `month`
**Fields:**
* `organization_id` - Identifier of the company the budget belongs to
* `year` - Year the budget was planned for
* `month` - Month of the year the planned amount refers to, from 1 to 12
* `category_id` - Identifier of the category the amount was budgeted for
* `value` - Amount planned for this category in this month
# Notion as a data source
Source: https://docs.nekt.com/sources/notion
Bring data from Notion to Nekt.
Notion is the AI workspace that works for you, a productivity platform where teams can create docs, wikis, projects, and manage knowledge in one place. It's widely used by teams for documentation, project management, and collaboration.
![]()
## Configuring Notion as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **Notion** option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Nekt supports Notion OAuth, making the setup process simple and secure. You just need to authenticate with your Notion account.
1. Click **"Notion Authorization"** when prompted
2. You'll be redirected to Notion's authorization page
3. Sign in to your Notion account if you haven't already
4. Review the permissions that Nekt is requesting
5. Click **"Allow"** to grant access to your Notion workspace
6. You'll be redirected back to Nekt with your account connected
The OAuth process automatically handles authentication, token refreshes, and gives Nekt access to the databases you have permission to view in your Notion workspace.
**Important**
* The OAuth connection will give Nekt access to databases based on your user permissions in the Notion workspace.
* You can revoke access at any time from your Notion workspace settings.
* Only connect workspaces that contain the data you need for your analytics.
After you authorize Nekt, define from when you want to extract records by setting the **Start date** parameter and click **Next**.
### 2. Select streams
The Notion connector creates four types of data streams:
**Databases Stream**
* **Stream name**: `databases`
* **Purpose**: Lists all databases accessible to your Notion integration
* **Use case**: Discover what databases are available in your Notion workspace
**Database Content Streams**
* **Stream pattern**: `database_{database_id}`
* **Purpose**: Extracts the actual content (pages/rows) from each individual database
* **Use case**: Get the data stored within your Notion databases
**Pages Stream**
* **Stream name**: `pages`
* **Purpose**: Fetches all pages via the Notion Search API, including their full body content (blocks)
* **Use case**: Extract full documentation, wiki entries, and page contents from your workspace
**Comments Stream**
* **Stream name**: `comments`
* **Purpose**: Fetches all comments associated with your pages
* **Use case**: Extract discussions, feedback, and page comments from your workspace
Choose which data streams you want to sync: you can select all streams or pick specific ones that matter most to you.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* **Incremental sync**: The connector supports incremental synchronization for database content streams and the pages stream using Notion's `last_edited_time`. Only extracts pages that have been created or modified since the last sync.
* **Full sync**: Extracts all pages from the database every time.
The `databases` stream always performs a full sync since it's just listing available databases.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure **Delta Log Retention** and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Notion and their corresponding fields:
The `databases` stream includes the following properties for each database:
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the database |
| `title` | Array | Array of rich text objects containing the database title information |
| `object` | String | Always "database" for database objects |
| `properties` | String | Stringified JSON of the database schema/properties |
| `created_time` | String | When the database was created |
| `created_by` | Object | User who created the database |
| `last_edited_by` | Object | User who last edited the database |
| `last_edited_time` | String | When the database was last modified |
| `cover` | String | Stringified JSON of the database cover image information |
| `icon` | String | Stringified JSON of the database icon (emoji or image) |
| `parent` | Object | Information about where the database is located |
| `url` | String | Internal Notion URL for the database |
| `public_url` | String | Public URL if the database is shared |
| `archived` | Boolean | Whether the database is archived |
| `in_trash` | Boolean | Whether the database is in trash |
Each `database_{database_id}` content stream includes:
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier for the page/row |
| `object` | String | Always "page" for database entries |
| `last_edited_time` | String | When the page was last modified (used for incremental sync) |
| `properties` | String | Stringified JSON containing all the page properties based on the database schema |
| `created_time` | String | When the page was created |
| `created_by` | Object | User who created the page |
| `last_edited_by` | Object | User who last edited the page |
| `cover` | Object | Page cover image information |
| `icon` | Object | Page icon (emoji or image) |
| `parent` | Object | Reference to the parent database |
| `archived` | Boolean | Whether the page is archived |
| `in_trash` | Boolean | Whether the page is in trash |
| `url` | String | Internal Notion URL for the page |
| `public_url` | String | Public URL if the page is shared |
The pages stream fetches all pages via the Notion Search API with full content retrieval. It includes:
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the page. |
| `object` | String | Type classification of the record. |
| `is_archived` | Boolean | Whether the page is archived. |
| `created_time` | String | Timestamp when the page was created. |
| `last_edited_time` | String | Timestamp when the page was last updated. |
| `created_by` | String | Stringified JSON of the user who created the page. |
| `last_edited_by` | String | Stringified JSON of the user who last edited the page. |
| `cover` | String | Stringified JSON of the cover image for the page. |
| `icon` | String | Stringified JSON of the icon for the page. |
| `parent` | String | Stringified JSON of the parent of the page. |
| `properties` | String | Property values of the page represented as a JSON string. |
| `content` | String | Page body content (block children) represented as a JSON string. |
| `archived` | Boolean | Whether the page is archived (deprecated in favor of `in_trash`). |
| `in_trash` | Boolean | Whether the page is in trash. |
| `url` | String | URL of the page. |
| `public_url` | String | Public URL of the page (if published publicly). |
The comments stream fetches discussions and comments linked to the fetched pages. It includes:
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the comment. |
| `parent` | String | Stringified JSON containing parent reference data. |
| `discussion_id` | String | Identifier for the discussion thread. |
| `rich_text` | String | Stringified JSON array of rich text objects representing the comment body. |
| `created_time` | String | Timestamp when the comment was created. |
| `last_edited_time` | String | Timestamp when the comment was last modified. |
| `created_by` | String | Stringified JSON of the user who created the comment. |
| `page_id` | String | The ID of the page the comment belongs to (mapped via parent context). |
# Data Model
The following diagram illustrates the relationships between the core data streams in Notion.
```mermaid theme={null}
erDiagram
DATABASES {
string id PK
}
DATABASE_CONTENT {
string id PK
string database_id FK
}
PAGES {
string id PK
}
COMMENTS {
string id PK
string page_id FK
}
DATABASES ||--o{ DATABASE_CONTENT : contains
PAGES ||--o{ COMMENTS : has
```
## Implementation Notes
### API Limits & Performance
* **Connection Pooling**: The connector uses connection pooling to prevent socket exhaustion during large data extractions. This ensures stable and reliable syncs without connection drops when fetching high volumes of records, particularly from heavily nested structures like the `pages` and `comments` streams.
### Transformation Requirements
* **Properties parsing**: All page properties (including custom columns defined in your Notion databases) are serialized into the `properties` JSON string. To use these fields in your analysis, you will need to parse the JSON in your SQL transformations (e.g., using `JSON_EXTRACT` in Athena or `JSON_VALUE` in BigQuery) to expose them as individual columns.
* **Titles and text**: Nested structures like titles, user objects, or rich text are exported as stringified JSON blocks and might require further extraction depending on your database engine.
### Authentication Limits
* Nekt securely manages the Notion OAuth integration natively in the platform. Refresh tokens are seamlessly updated and evaluated during internal system validations to guarantee uninterrupted extractions. Ensure your connected user retains the necessary read permissions over the source Notion databases.
## Skills for agents
Notion connector documentation as plain markdown, for use in AI agent contexts.
# Nuvemshop as a data source
Source: https://docs.nekt.com/sources/nuvemshop
Bring data from Nuvemshop (Tiendanube) to Nekt.
Nuvemshop (Tiendanube in Spanish-speaking countries) is Latin America's leading e-commerce platform, letting merchants build online stores, manage catalogs and inventory, process orders and payments, and sell across channels. Its REST API exposes the store's products, orders, customers, coupons and more.
![]()
## Configuring Nuvemshop as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Nuvemshop option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
The simplest way to connect is a **Custom App** ("Aplicativo sob medida"), created directly in your store admin, no partner account needed (available on the Escala and Next plans):
1. In the store admin, go to **Apps > Custom apps** and click **Create custom app**.
2. Give it a name (e.g. "Nekt") and grant **read** permissions for the resources you want to sync (or full access).
3. Save, open the app and click **Reveal** to see the token, then **Copy token**. The full token is shown **only once**, so copy it right away.
4. The **Store ID** is displayed on the same page.
See [Nuvemshop's guide on custom apps](https://atendimento.nuvemshop.com.br/pt_BR/aplicativos/como-criar-aplicativos-sob-medida-e-gerar-tokens-para-minha-loja-nuvemshop). Tokens never expire; they only stop working when the app is deleted or deactivated, or when the token is renewed. If that happens, copy the new token into the source. Partner apps using [Nuvemshop's OAuth flow](https://dev.nuvemshop.com.br/en/docs/applications/authentication) work the same way (the `access_token` and `user_id` from the token endpoint).
The following configurations are available:
* **Access Token**: The token revealed on the custom app page (or the OAuth `access_token`).
* **Store ID**: The numeric identifier of your store, shown on the custom app page (also the `user_id` returned with OAuth tokens and the number in `https://api.nuvemshop.com.br/v1/{store_id}`).
* **Initial sync date**: The earliest date from which records will be synced. Incremental streams (orders, products, customers, categories, abandoned checkouts) only fetch records updated after this date.
The app must have **read** permission on the resources you want to sync (products, orders, customers, coupons, etc.). Nekt checks the credentials when you validate the source, so a wrong token or Store ID is reported immediately.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
## Available streams
The table below lists every stream, its **slug** (the exact identifier to pass when creating the source via API) and a short description.
| Stream | Slug | Description |
| - | - | - |
| Store | `store` | Profile of the store (name, plan, country, currency, languages, domains). Single record. |
| Products | `products` | Catalog products, incremental on `updated_at`. |
| Product Variants | `product_variants` | One row per product variant (price, stock, SKU, attributes), flattened from products. |
| Categories | `categories` | Product categories and their hierarchy, incremental on `updated_at`. |
| Orders | `orders` | Orders of every status (open, closed, cancelled), incremental on `updated_at`. |
| Order Products | `order_products` | One row per order line item, flattened from orders. |
| Transactions | `transactions` | Payment transactions of each order (captures, refunds, failures). *Not selected by default:* requires one request per order, so enable it only when payment-level detail is needed. |
| Customers | `customers` | Store customers with contact, billing and marketing opt-in data, incremental on `updated_at`. |
| Abandoned Checkouts | `abandoned_checkouts` | Checkouts started but not completed, incremental on `updated_at`. |
| Coupons | `coupons` | Discount coupons and their usage. Full table (the API exposes no timestamps). |
| Locations | `locations` | Stock locations (warehouses / branches). Full table. |
## Fields by stream
Profile of the connected store.
* `id` - Unique numeric identifier of the store
* `name` - Store name per language (JSON object keyed by language code)
* `description` - Store description per language (JSON object)
* `type` - Store type/segment (e.g. clothing, sports)
* `email` - Email address of the store owner
* `logo` - Store logo URL
* `contact_email` - Public contact email
* `facebook`, `twitter`, `google_plus`, `instagram`, `pinterest`, `youtube`, `tiktok`, `blog` - Social/blog URLs
* `url_with_protocol` - Primary storefront URL
* `address` - Physical address of the store
* `phone` - Store phone number
* `whatsapp_phone_number` - WhatsApp contact number
* `business_id` - Business tax identifier (CPF/CNPJ, DNI/CUIT)
* `business_name` - Legal name of the company
* `business_address` - Legal address of the company
* `customer_accounts` - Whether customer accounts are optional or mandatory
* `plan_name` - Nuvemshop subscription plan
* `country` - Country in ISO 3166-1 format
* `languages` - Enabled languages with currency and active flag (JSON object)
* `domains` - Domain names attached to the store
* `original_domain` - Original nuvemshop.com.br / tiendanube.com subdomain
* `current_theme` - Active storefront theme
* `main_language` - Primary language
* `main_currency` - Primary currency (ISO 4217)
* `admin_language` - Admin panel language
* `features` - API features enabled for the store
* `created_at` - Timestamp when the store was created
* `updated_at` - Timestamp of the last update of the store profile
Catalog products. Incremental on `updated_at`.
* `id` - Unique numeric identifier of the product
* `name` - Product name per language (JSON object)
* `description` - Product description as HTML per language (JSON object)
* `handle` - URL-friendly slug per language (JSON object)
* `variants` - Product variants (JSON array); also available flattened in `product_variants`
* `images` - Product images with URL, position and alt text (JSON array)
* `categories` - Identifiers of the categories the product belongs to
* `brand` - Brand of the product
* `published` - Whether the product is visible in the storefront
* `visibility` - visible, unlisted or hidden
* `free_shipping` - Whether the product is eligible for free shipping
* `video_url` - URL of the product video
* `seo_title` - SEO title per language (JSON object)
* `seo_description` - SEO description per language (JSON object)
* `attributes` - Names of the attributes that define the variants (JSON array)
* `tags` - Comma-separated tags
* `requires_shipping` - Whether the product is physical (true) or digital (false)
* `canonical_url` - Canonical URL of the product page
* `has_stock`, `is_kit` - Stock availability and kit flags
* `invalid_at` - Timestamp when the product became invalid, if applicable
* `videos` - Videos attached to the product (JSON array)
* `created_at` - Timestamp when the product was created
* `updated_at` - Timestamp of the last update (replication key)
One row per variant, flattened from the products payload. Incremental on `product_updated_at`.
* `id` - Unique numeric identifier of the variant
* `product_id` - Identifier of the parent product
* `product_updated_at` - Last update of the parent product (replication key)
* `image_id` - Identifier of the image associated with the variant
* `position` - Position of the variant within the product
* `price` - Selling price
* `compare_at_price` - List price shown as comparison when a promotion is active
* `promotional_price` - Promotional price, when set
* `cost` - Cost of the variant for the merchant
* `stock_management` - Whether stock is tracked
* `stock` - Available stock across locations; null when unlimited
* `weight`, `width`, `height`, `depth` - Dimensions (kg / cm)
* `sku` - Stock keeping unit
* `barcode` - Barcode (EAN/UPC)
* `mpn` - Manufacturer part number
* `price_without_taxes` - Selling price before taxes
* `visible` - Whether the variant is visible in the storefront
* `age_group`, `gender`, `size_type`, `size_system` - Google Shopping attributes
* `values` - Attribute values that identify the variant, per language (JSON array)
* `inventory_levels` - Stock per location (JSON array)
* `created_at` - Timestamp when the variant was created
* `updated_at` - Timestamp of the last update of the variant
Product categories. Incremental on `updated_at`.
* `id` - Unique numeric identifier of the category
* `name` - Category name per language (JSON object)
* `description` - Category description as HTML per language (JSON object)
* `handle` - URL-friendly slug per language (JSON object)
* `parent` - Identifier of the parent category, or null
* `visibility` - visible, hidden or soft-hidden
* `visibility_updated_at` - Last change to the visibility field
* `subcategories` - Identifiers of the first-level subcategories
* `seo_title` - SEO title per language (JSON object)
* `seo_description` - SEO description per language (JSON object)
* `google_shopping_category` - Google product taxonomy category
* `created_at` - Timestamp when the category was created
* `updated_at` - Timestamp of the last update (replication key)
Orders of every status. Incremental on `updated_at`.
**Identification**
* `id` - Unique numeric identifier of the order
* `token` - Token that identifies the order
* `number` - Sequential order number shown to merchants and customers
* `store_id` - Identifier of the store
* `customer_id` - Identifier of the customer (flattened from `customer`)
* `customer` - Customer that placed the order (JSON object)
* `contact_email`, `contact_name`, `contact_phone`, `contact_identification` - Buyer contact data
**Billing**
* `billing_name`, `billing_phone`, `billing_address`, `billing_number`, `billing_floor`, `billing_locality`, `billing_zipcode`, `billing_city`, `billing_province`, `billing_country` - Billing address
* `billing_customer_type` - person or company
* `billing_business_name`, `billing_trade_name`, `billing_state_registration`, `billing_document_type` - Company billing data
* `billing_fiscal_regime`, `billing_invoice_use` - Mexico-specific invoicing fields
**Shipping**
* `shipping_address` - Shipping address (JSON object)
* `shipping_status` - unpacked, unfulfilled, fulfilled, shipped, delivered, partially\_packed or partially\_fulfilled
* `shipping`, `shipping_option`, `shipping_option_code`, `shipping_option_reference` - Shipping method and option
* `shipping_pickup_type` - ship or pickup
* `shipping_pickup_details`, `shipping_store_branch_name`, `shipping_store_branch_extra` - Pickup point data
* `shipping_carrier_name`, `shipping_tracking_number`, `shipping_tracking_url` - Carrier and tracking
* `shipping_suboption` - Sub-options chosen for the shipping method (JSON)
* `shipping_cost_customer` - Shipping cost charged to the customer
* `shipping_cost_owner` - Shipping cost paid by the store owner
* `shipping_min_days`, `shipping_max_days` - Estimated delivery window in weekdays
* `shipped_at` - Timestamp when the order was shipped
* `weight` - Total weight in kilograms
**Amounts & payment**
* `subtotal` - Total of the products before shipping and discounts
* `discount`, `discount_coupon`, `discount_gateway` - Discounts applied
* `total` - Total including shipping and discounts
* `total_usd` - Total converted to US dollars
* `total_paid_by_customer`, `total_paid_by_customer_including_fees` - Amounts actually paid
* `currency` - Currency in ISO 4217 format
* `gateway`, `gateway_id`, `gateway_name`, `gateway_link` - Payment gateway data
* `payment_status` - authorized, pending, paid, partially\_paid, abandoned, refunded, partially\_refunded or voided
* `payment_method`, `payment_credit_card_company`, `payment_installments` - Flattened from `payment_details`
* `payment_details` - Payment details (JSON object)
* `payment_count` - Number of payments made
* `paid_at` - Timestamp when the order was paid
**Status & timestamps**
* `status` - open, closed or cancelled
* `cancelled_at`, `cancel_reason` - Cancellation data
* `closed_at` - Timestamp when the order was archived
* `read_at` - Timestamp when the order was marked as read
* `completed_at` - Completion date with timezone information (JSON object)
* `created_at` - Timestamp when the order was created
* `updated_at` - Timestamp of the last update (replication key)
**Other**
* `products` - Line items (JSON array); also available flattened in `order_products`
* `attributes` - Custom attributes (JSON array)
* `coupon` - Coupons applied (JSON array)
* `promotional_discount` - Promotional discount applied (JSON object)
* `fulfillments` - Fulfillment orders (JSON array)
* `note`, `owner_note` - Customer and store owner notes
* `extra` - Custom information (JSON object)
* `storefront` - store, meli, api, form or pos
* `app_id` - Identifier of the app that created the order
* `client_details` - Browser IP and user agent (JSON object)
* `same_billing_and_shipping_address` - Whether billing and shipping addresses match
* `is_subscription`, `is_subscription_initial` - Subscription flags
* `landing_url` - Landing URL of the session that originated the order
* `has_shippable_products` - Whether the order contains products that require shipping
* `free_shipping_config` - Free shipping configuration applied (JSON)
* `next_action` - Next available operation in the order flow
* `subtotal_without_taxes`, `total_paid` - Additional amounts
* `coupon_id` - Identifier of the coupon applied
* `order_origin` - Origin of the order as reported by the platform
* `checkout_enabled`, `checkout_enabled_validation_messages`, `checkout_bot_suspect`, `checkout_app_blocked` - Checkout status flags
* `customer_visit` - Attribution data of the visit that led to the order (JSON object)
* `billing_business_activity` - Business activity of the billing company
One row per order line item, flattened from the orders payload. Incremental on `order_updated_at`.
* `id` - Unique numeric identifier of the line item
* `order_id` - Identifier of the parent order
* `order_updated_at` - Last update of the parent order (replication key)
* `product_id` - Identifier of the product purchased
* `variant_id` - Identifier of the variant purchased
* `name` - Product name at the time of purchase
* `price` - Unit price at the time of purchase
* `compare_at_price` - List price at the time of purchase
* `quantity` - Units purchased
* `weight`, `width`, `height`, `depth` - Unit dimensions
* `free_shipping` - Whether the product had free shipping
* `sku` - SKU of the variant purchased
* `barcode` - Barcode of the variant purchased
* `variant_values` - Attribute values of the variant purchased
* `catalog_kit_id`, `order_kit_id`, `kit` - Kit data, when the item belongs to a kit
* `properties` - Custom field values of the line item (JSON array)
* `image` - Image of the product at the time of purchase (JSON object)
* `issues` - Issues detected on the line item
* `cost` - Unit cost for the merchant
* `name_without_variants` - Product name without variant attributes
* `has_promotional_price`, `is_gift` - Item flags
* `promotions` - Promotions applied to the item (JSON object)
Payment transactions of each order. Child of `orders`; not selected by default.
* `id` - Unique identifier of the transaction
* `order_id` - Identifier of the parent order
* `order_updated_at` - Last update of the parent order (replication key)
* `payment_attempt_id` - Identifier of the payment attempt
* `payment_provider_id` - Identifier of the payment provider
* `payment_provider_tax_id` - Tax identifier of the payment provider (Brazil)
* `payment_method` - Payment method type and identifier (JSON object)
* `info` - Provider-specific payment details (JSON object)
* `status` - authorized, paid, failed, pending, refunded, voided...
* `events` - Events that changed the transaction status (JSON array)
* `captured_amount`, `refunded_amount`, `authorized_amount`, `voided_amount`, `discount_amount` - Amounts with currency (JSON objects)
* `failure_code` - Code describing why the transaction failed
* `created_at` - Timestamp when the transaction was created
Store customers. Incremental on `updated_at`.
* `id` - Unique numeric identifier of the customer
* `name` - Full name
* `email` - Email address
* `phone` - Phone number
* `identification` - Personal/tax identification (CPF/CNPJ, DNI)
* `note` - Store owner's notes
* `default_address` - Default shipping address (JSON object)
* `addresses` - All shipping addresses (JSON array)
* `billing_name`, `billing_phone`, `billing_address`, `billing_number`, `billing_floor`, `billing_locality`, `billing_zipcode`, `billing_city`, `billing_province`, `billing_country` - Billing address
* `customer_type`, `business_name`, `trade_name`, `state_registration`, `business_activity`, `document_type` - Company data, when the customer is a company
* `fiscal_regime`, `invoice_use` - Mexico-specific invoicing fields
* `extra` - Custom information (JSON object)
* `total_spent` - Total amount spent in the store
* `total_spent_currency` - Currency of `total_spent`
* `last_order_id` - Identifier of the most recent order
* `active` - Whether the account is active
* `first_interaction` - First interaction with the store
* `accepts_marketing` - Whether the customer opted in to email marketing
* `accepts_marketing_updated_at` - Last change to the marketing opt-in
* `created_at` - Timestamp when the customer was created
* `updated_at` - Timestamp of the last update (replication key)
Checkouts started but not completed. Incremental on `updated_at`.
* `id` - Unique numeric identifier of the checkout
* `token` - Token that identifies the checkout
* `store_id` - Identifier of the store
* `abandoned_checkout_url` - URL to recover the checkout
* `contact_email`, `contact_name`, `contact_phone`, `contact_identification` - Customer contact data
* `customer` - Customer that started the checkout, when identified (JSON object)
* `shipping_name`, `shipping_phone`, `shipping_address`, `shipping_number`, `shipping_floor`, `shipping_locality`, `shipping_zipcode`, `shipping_city`, `shipping_province`, `shipping_country` - Shipping address
* `shipping_min_days`, `shipping_max_days` - Estimated delivery window in weekdays
* `billing_name`, `billing_phone`, `billing_address`, `billing_number`, `billing_floor`, `billing_locality`, `billing_zipcode`, `billing_city`, `billing_province`, `billing_country` - Billing address
* `shipping_cost_owner`, `shipping_cost_customer` - Shipping costs
* `coupon` - Coupons applied (JSON array)
* `promotional_discount` - Promotional discount applied (JSON object)
* `subtotal`, `discount`, `discount_coupon`, `discount_gateway`, `total`, `total_usd` - Amounts
* `checkout_enabled` - Whether the checkout can still be resumed
* `weight` - Total weight in kilograms
* `currency` - Currency in ISO 4217 format
* `language` - Language used during checkout
* `gateway`, `gateway_id` - Payment gateway data
* `shipping`, `shipping_option`, `shipping_option_code`, `shipping_option_reference`, `shipping_pickup_details`, `shipping_tracking_number`, `shipping_tracking_url`, `shipping_store_branch_name`, `shipping_pickup_type`, `shipping_suboption` - Shipping method data
* `extra` - Custom information (JSON object)
* `storefront` - store, meli, api, form or pos
* `note` - Note left by the customer
* `next_action` - noop, close, waiting\_ipn or waiting\_manual\_confirmation
* `payment_status`, `payment_count`, `gateway_name` - Payment data
* `billing_customer_type`, `billing_business_name`, `billing_trade_name`, `billing_state_registration`, `billing_business_activity`, `billing_document_type`, `billing_fiscal_regime`, `billing_invoice_use` - Company billing data
* `subtotal_without_taxes`, `total_paid` - Additional amounts
* `shipping_store_branch_extra`, `owner_note`, `order_origin` - Additional order data
* `same_billing_and_shipping_address`, `is_subscription`, `is_subscription_initial` - Flags
* `contact_accepts_marketing`, `contact_accepts_marketing_updated_at` - Marketing opt-in
* `checkout_enabled_validation_messages`, `checkout_bot_suspect`, `checkout_app_blocked`, `was_notified`, `has_stock_available` - Checkout status flags
* `customer_visit`, `cart_flags` - Attribution and cart flags (JSON objects)
* `payment_details` - Payment details (JSON object)
* `attributes` - Line-item properties (JSON array)
* `products` - Products in the cart (JSON array)
* `created_at` - Timestamp when the checkout was created
* `updated_at` - Timestamp of the last update (replication key)
* `completed_at` - Timestamp when the checkout was completed, if it was
Discount coupons. Full table.
* `id` - Unique numeric identifier of the coupon
* `code` - Code the customer types to apply the coupon
* `type` - percentage, absolute or shipping
* `valid` - Whether the coupon is currently valid
* `start_date`, `end_date` - Validity window
* `deleted_at` - Timestamp when the coupon was deleted, or null
* `max_uses` - Maximum number of uses; null for unlimited
* `used` - Number of times the coupon has been used
* `value` - Discount amount or percentage
* `includes_shipping` - Whether the discount also applies to shipping
* `first_consumer_purchase` - Whether it applies only to the first purchase
* `min_price` - Minimum cart value required
* `categories` - Categories the coupon is restricted to (JSON array)
* `products` - Products the coupon is restricted to (JSON array)
* `combines_with_other_discounts` - Whether it can be combined with other promotions
* `only_cheapest_shipping` - Whether it applies only to the cheapest eligible shipping option
* `start_time`, `end_time` - Time of day bounds of the validity window
* `max_uses_per_client` - Maximum uses per customer; null for unlimited
* `max_discount_amount` - Cap on the discount amount for percentage coupons
* `is_deleted` - Whether the coupon has been deleted
Stock locations (warehouses / branches). Full table.
* `id` - Unique identifier of the location
* `store_id` - Identifier of the store
* `name` - Location name per language (JSON object)
* `priority` - Priority used to assign stock at checkout (lower = higher priority)
* `address` - Full address including province, region and country (JSON object)
* `is_default` - Whether this is the default location
* `tags` - Sales-channel tags (e.g. online, offline)
* `additional_days`, `weekends_and_holidays` - Handling-time settings
* `deleted_at` - Timestamp when the location was deleted, or null
* `created_at` - Timestamp when the location was created
* `updated_at` - Timestamp of the last update
# Data Model
The following diagram illustrates the relationships between the core data streams in Nuvemshop. The arrows indicate the join keys that link the different entities.
```mermaid theme={null}
graph TD;
Store("Store")
Categories("Categories")
Products("Products")
ProductVariants("Product Variants")
Orders("Orders")
OrderProducts("Order Products")
Transactions("Transactions")
Customers("Customers")
AbandonedCheckouts("Abandoned Checkouts")
Coupons("Coupons")
Locations("Locations")
ProductVariants -- "product_id" --> Products
OrderProducts -- "order_id" --> Orders
OrderProducts -- "product_id" --> Products
OrderProducts -- "variant_id" --> ProductVariants
Transactions -- "order_id" --> Orders
Orders -- "customer_id" --> Customers
```
## Implementation Notes
### Data Quality Considerations
* **Nested Objects**: Objects returned by the API (multi-language names, addresses, payment details, amounts with currency) are delivered as **JSON strings** so the tables keep a stable, flat schema. Parse them in a Query when you need individual attributes.
* **Malformed Payloads**: If the Nuvemshop API occasionally returns malformed nested entities (for example, non-list structures for `variants` in a product or `products` in an order), the connector will gracefully skip those specific child records to avoid failing the entire sync.
### API Limits & Performance
Nuvemshop enforces a leaky-bucket limit of 40 requests with a refill rate of 2 requests per second (10× higher on Next/Evolution plans). The connector throttles itself when the bucket is nearly full and retries `429` responses honouring the API's reset header. The sync pauses and continues whenever this happens, so a large first sync may take a while but will complete without manual intervention or data loss.
## Skills for agents
Nuvemshop connector documentation as plain markdown, for use in AI agent contexts.
# Octadesk as a data source
Source: https://docs.nekt.com/sources/octadesk
Bring data from Octadesk to your Lakehouse.
Octadesk is a Brazilian customer service platform. The connector extracts your help desk tickets and every interaction on them (comments, replies, status changes, and automations), along with the people and structure around them: requesters (customers), agents, organizations, groups and their members, products, subjects, tags, custom fields, and forms. With it you can analyze your support volume, response and resolution times, and your team's workload in your Lakehouse.
The connector reads the Octadesk API documented at [api-docs.octadesk.services](https://api-docs.octadesk.services/docs/), which covers the ticket help desk. WhatsApp and chat conversations are not included.
## Configuring Octadesk as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Octadesk option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Octadesk authenticates with an API token, the e-mail of the user who uses it, and your account's subdomain. To create the token, in Octadesk go to **Tickets > Geral > Código de integração**, create a token, and copy it (it starts with `octa.`).
Use the e-mail and token of an administrator. Agents, groups, custom fields, and other settings are only visible to users whose profile can see them; streams the user cannot see are skipped with a warning.
The following configurations are available:
* **Subdomain**: the first part of your Octadesk address. For `https://mycompany.octadesk.com`, enter `mycompany`. Required.
* **User E-mail**: the e-mail of an active Octadesk user. Required.
* **API Token**: the token created in **Código de integração**. Required.
* **Start Date**: on the first sync, tickets last updated from this date on are extracted, with their interactions. Later syncs only read tickets updated since the previous one. Required.
* **Requests Per Minute**: how fast the connector may call Octadesk. The default is 120.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: Tickets and Ticket Interactions support INCREMENTAL. The other streams are FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Octadesk and their corresponding fields. API reference: [Octadesk API](https://api-docs.octadesk.services/docs/).
Timestamps are stored in UTC. Objects and lists (custom fields, interactions, comments, attachments, phone numbers) are stored as JSON text, and the most used values inside them also get their own columns (for example `product_id`, `organization_name`, `person_email`). Every table also has a `raw_payload` column with the whole record as Octadesk returned it.
Ticket Interactions are read one ticket at a time, for the tickets updated since the previous sync. The first sync of an account with many tickets can take a while; the following ones only read what changed.
Tickets of the account (`GET /tickets/search`), with requester, assignee, group, status, subject, product, and the first, last, and last human interaction. Read incrementally by the day of the last update: each sync reads the tickets updated since the previous one (the day before the last sync point is re-read).
Primary key: `id` · Incremental key: `last_date_update`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the ticket. |
| `number` | Integer | Ticket number shown in Octadesk (sequential within the account). |
| `summary` | String | Subject line of the ticket. |
| `description` | String | Description of the ticket (HTML). |
| `description_as_text` | String | Description of the ticket as plain text. |
| `id_type` | String | Identifier of the ticket type. |
| `type_name` | String | Ticket type (problema, tarefa, dúvida, incidente, não especificado). |
| `id_priority` | String | Identifier of the priority. |
| `priority_name` | String | Priority of the ticket (alta, média, baixa). |
| `id_channel` | String | Identifier of the channel the ticket came in. |
| `number_channel` | Integer | Code of the channel the ticket came in (Octadesk NumberChannel enum). |
| `channel_name` | String | Name of the ticket's channel. |
| `inbox_mail` | String | E-mail inbox that received the ticket. |
| `inbox_domain` | String | Domain of the inbox that received it. |
| `id_requester` | String | Identifier of the requester (customer). |
| `requester_name` | String | Name of the requester. |
| `requester_mail` | String | E-mail of the requester. |
| `requester_customer_code` | String | Customer code of the requester, when registered. |
| `requester_multi_channel_id` | String | Identifier of the requester on the multichannel integration. |
| `id_organization` | String | Identifier of the requester's organization. |
| `organization_name` | String | Name of the requester's organization. |
| `id_assigned` | String | Identifier of the agent assigned to it. |
| `assigned_name` | String | Name of the assigned agent. |
| `assigned_mail` | String | E-mail of the assigned agent. |
| `id_group_assigned` | String | Identifier of the group the ticket is assigned to. |
| `group_assigned_name` | String | Name of the assigned group. |
| `id_subject` | String | Identifier of the subject (assunto). |
| `topic_group_name` | String | Name of the subject's topic group. |
| `topic_name` | String | Name of the ticket's subject topic. |
| `id_current_status` | String | Identifier of the current status. |
| `current_status_name` | String | Current status (novo, pendente, em andamento, resolvido, rejeitado...). |
| `state_progress` | Integer | Code of the ticket's progress state (Octadesk StateProgress enum). |
| `state_progress_name` | String | Name of the progress state. |
| `number_ticket_created_from` | Integer | Number of the ticket this one was created from, if any. |
| `tags` | Array | Tags of the ticket. |
| `tags_join` | String | Tags of the ticket, comma-joined. |
| `product_members_join` | String | Values of the related product's fields, joined. |
| `product_id` | String | Identifier of the related product. |
| `product_sku` | String | SKU of the related product. |
| `product` | String | Related product (JSON object: id, sku, fields). |
| `open_date` | Datetime | When the ticket was opened (UTC). |
| `due_date` | Datetime | SLA due date of the ticket (UTC). |
| `done_date` | Datetime | When the ticket was solved (UTC). |
| `last_date_update` | Datetime | When the ticket was last updated (UTC); incremental replication key. |
| `last_person_update_id` | String | Identifier of the person who last updated the ticket. |
| `last_person_update_name` | String | Name of the person who last updated the ticket. |
| `last_person_update_email` | String | E-mail of the person who last updated the ticket. |
| `last_person_update` | String | Person who last updated the ticket (JSON object). |
| `attachments` | String | Files attached to the ticket (JSON array). |
| `first_interaction` | String | First interaction of the ticket (JSON object). |
| `last_interaction` | String | Latest interaction of the ticket (JSON object). |
| `last_human_interaction` | String | Latest interaction made by a person, not an automation (JSON object). |
| `cc` | Array | E-mails copied on the ticket. |
| `cco` | Array | E-mails blind-copied on the ticket. |
| `merge` | String | Merge information: parent and child ticket numbers (JSON object). |
| `custom_field` | String | Custom fields of the ticket, keyed by field code (JSON object). |
| `requester_custom_field` | String | Custom fields of the requester (JSON object). |
| `organization_custom_field` | String | Custom fields of the requester's organization (JSON object). |
| `has_custom_work_flow` | Boolean | Whether the ticket follows a custom workflow. |
| `current_step_flow_type` | Integer | Code of the current workflow step type (Octadesk TypeStepFlow enum). |
| `current_step_flow_name` | String | Name of the current workflow step. |
| `survey` | String | Satisfaction survey of the ticket (JSON object). |
| `is_survey_applied` | Boolean | Whether a satisfaction survey was sent for the ticket. |
| `multi_channel_field` | String | Multichannel integration data of the ticket (JSON object). |
| `id_company_ticket_share` | String | Identifier of the company the ticket is shared with. |
| `company_ticket_share` | String | Company the ticket is shared with (JSON object). |
| `company_ticket_share_active` | Boolean | Whether the ticket sharing is active. |
| `company_ticket_share_number` | Integer | Number of the ticket on the company it is shared with. |
| `reported_as_spam` | Boolean | Whether the ticket is marked as spam. |
| `total_human_interactions` | Integer | Number of interactions made by people. |
| `total_interactions` | Integer | Number of interactions, automations included. |
| `total_employee_interactions` | Integer | Number of interactions made by agents. |
| `id_form` | String | Identifier of the form the ticket was opened with. |
| `octadesk_number_url` | String | Link to the ticket in Octadesk. |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Every interaction of the tickets read in the sync (`GET /tickets/{number}/interactions`, one call per ticket): public and internal comments, e-mail replies, status and property changes, and automation actions.
Primary key: `id` · Incremental key: `date_creation`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the interaction. |
| `ticket_number` | Integer | Number of the ticket it belongs to. |
| `ticket_id` | String | Identifier (UUID) of the ticket it belongs to. |
| `date_creation` | Datetime | When the interaction happened (UTC); incremental replication key. |
| `person_id` | String | Identifier of the person who interacted. |
| `person_name` | String | Name of the person who interacted. |
| `person_email` | String | E-mail of the person who interacted. |
| `person_type` | Integer | Kind of person who interacted: 1 agent, 2 customer, 3 trigger, 4 system, 5 sharing agent. |
| `person` | String | Person who interacted (JSON object). |
| `is_human_interaction` | Boolean | Whether a person (not an automation) made the interaction. |
| `comments` | String | Comments of the interaction: public, internal, e-mail (JSON array). |
| `attachments` | String | Files attached to the interaction (JSON array). |
| `properties_changes` | String | Ticket properties changed by the interaction (JSON object). |
| `multi_channel_field` | String | Multichannel integration data of the interaction (JSON object). |
| `has_multi_channel_comment` | Boolean | Whether the interaction carries a multichannel comment. |
| `forward_reply_comment` | String | Forwarded reply of the interaction (JSON object). |
| `forward_comment` | String | Forwarded comment of the interaction (JSON object). |
| `multi_channel_comment` | String | Multichannel comment of the interaction (JSON object). |
| `internal_comment` | String | Internal comment of the interaction (JSON object). |
| `spam_status` | String | Whether this interaction marked or unmarked the ticket as spam. |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Customers (requesters) registered in the account (`GET /persons/requesters`), with their organization, phone numbers, custom fields, and responsible agent.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the person. |
| `name` | String | Name of the person. |
| `email` | String | Main e-mail of the person. |
| `date_creation` | Datetime | When the person was registered (UTC). |
| `is_enabled` | Boolean | Whether the person is active in Octadesk. |
| `type` | Integer | Kind of person: 0 none, 1 agent, 2 customer, 3 trigger, 4 system, 5 sharing agent, 6 sharing customer. |
| `role_type` | Integer | Role of the person: 0 none, 1 account owner, 2 admin, 3 master agent, 4 agent, 5 client, 6 corporate. |
| `permission_type` | Integer | Permission level of the person (Octadesk PermissionType enum). |
| `permission_view` | Integer | Which tickets the person can see (Octadesk PermissionViewType enum). |
| `participant_permission` | Integer | Permission on tickets the person took part in (Octadesk enum). |
| `is_locked` | Boolean | Whether the person's access is locked. |
| `customer_code` | String | Customer code registered for the person. |
| `others_email` | Array | Other e-mails of the person. |
| `others_customer_code` | String | Other customer codes of the person (JSON array of key/value pairs). |
| `id_contact_status` | String | Identifier of the contact status. |
| `contact_status` | String | Name of the contact status. |
| `phone_contacts` | String | Phone numbers of the person: number, country code, type (JSON array). |
| `custom_field` | String | Custom fields of the person, keyed by field code (JSON object). |
| `organization_id` | String | Identifier of the person's main organization. |
| `organization_name` | String | Name of the person's main organization. |
| `organization` | String | Main organization of the person (JSON object). |
| `organizations` | String | Every organization the person belongs to (JSON array). |
| `groups` | String | Groups the person belongs to (JSON array). |
| `responsible_id` | String | Identifier of the agent responsible for this customer. |
| `responsible_name` | String | Name of the agent responsible for this customer. |
| `responsible_email` | String | E-mail of the agent responsible for this customer. |
| `products` | String | Products linked to the person (JSON array). |
| `apps` | String | Octadesk apps linked to the person and their fields (JSON array). |
| `my_apps` | Array | Identifiers of the apps the person uses. |
| `my_ticket_views` | Array | Identifiers of the ticket views the person uses. |
| `thumb_url` | String | Link to the person's picture. |
| `facebook_id` | String | Facebook identifier of the person. |
| `facebook_user_name` | String | Facebook user name of the person. |
| `home_page` | String | Home page settings of the person (JSON object). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Agents (employees) of the account (`GET /persons/agents`), with their role, permissions, and groups.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the person. |
| `name` | String | Name of the person. |
| `email` | String | Main e-mail of the person. |
| `date_creation` | Datetime | When the person was registered (UTC). |
| `is_enabled` | Boolean | Whether the person is active in Octadesk. |
| `type` | Integer | Kind of person: 0 none, 1 agent, 2 customer, 3 trigger, 4 system, 5 sharing agent, 6 sharing customer. |
| `role_type` | Integer | Role of the person: 0 none, 1 account owner, 2 admin, 3 master agent, 4 agent, 5 client, 6 corporate. |
| `permission_type` | Integer | Permission level of the person (Octadesk PermissionType enum). |
| `permission_view` | Integer | Which tickets the person can see (Octadesk PermissionViewType enum). |
| `participant_permission` | Integer | Permission on tickets the person took part in (Octadesk enum). |
| `is_locked` | Boolean | Whether the person's access is locked. |
| `customer_code` | String | Customer code registered for the person. |
| `others_email` | Array | Other e-mails of the person. |
| `others_customer_code` | String | Other customer codes of the person (JSON array of key/value pairs). |
| `id_contact_status` | String | Identifier of the contact status. |
| `contact_status` | String | Name of the contact status. |
| `phone_contacts` | String | Phone numbers of the person: number, country code, type (JSON array). |
| `custom_field` | String | Custom fields of the person, keyed by field code (JSON object). |
| `organization_id` | String | Identifier of the person's main organization. |
| `organization_name` | String | Name of the person's main organization. |
| `organization` | String | Main organization of the person (JSON object). |
| `organizations` | String | Every organization the person belongs to (JSON array). |
| `groups` | String | Groups the person belongs to (JSON array). |
| `responsible_id` | String | Identifier of the agent responsible for this customer. |
| `responsible_name` | String | Name of the agent responsible for this customer. |
| `responsible_email` | String | E-mail of the agent responsible for this customer. |
| `products` | String | Products linked to the person (JSON array). |
| `apps` | String | Octadesk apps linked to the person and their fields (JSON array). |
| `my_apps` | Array | Identifiers of the apps the person uses. |
| `my_ticket_views` | Array | Identifiers of the ticket views the person uses. |
| `thumb_url` | String | Link to the person's picture. |
| `facebook_id` | String | Facebook identifier of the person. |
| `facebook_user_name` | String | Facebook user name of the person. |
| `home_page` | String | Home page settings of the person (JSON object). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Organizations (companies) your customers belong to, disabled ones included (`GET /organizations`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the organization. |
| `name` | String | Name of the organization. |
| `description` | String | Description of the organization. |
| `date_creation` | Datetime | When the organization was registered (UTC). |
| `is_enabled` | Boolean | Whether the organization is active. |
| `domains` | Array | E-mail domains that identify the organization's people. |
| `group_id` | String | Identifier of the group that serves the organization. |
| `group_name` | String | Name of the group that serves it. |
| `group` | String | Group that serves the organization (JSON object). |
| `products` | String | Products linked to the organization (JSON array). |
| `custom_field` | String | Custom fields of the organization, keyed by field code (JSON object). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Agent groups (teams) of the account (`GET /groups`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the group. |
| `name` | String | Name of the group. |
| `description` | String | Description of the group. |
| `date_creation` | Datetime | When the group was created (UTC). |
| `is_enabled` | Boolean | Whether the group is active. |
| `standard` | Boolean | Whether this is the account's default group. |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Agents that belong to each group, one row per group and agent (`GET /groups/{id}/persons`).
Primary key: `group_id`, `id`
| Field | Type | Description |
| :- | :- | :- |
| `group_id` | String | Identifier of the group. |
| `id` | String | Unique identifier (UUID) of the person. |
| `name` | String | Name of the person. |
| `email` | String | Main e-mail of the person. |
| `date_creation` | Datetime | When the person was registered (UTC). |
| `is_enabled` | Boolean | Whether the person is active in Octadesk. |
| `type` | Integer | Kind of person: 0 none, 1 agent, 2 customer, 3 trigger, 4 system, 5 sharing agent, 6 sharing customer. |
| `role_type` | Integer | Role of the person: 0 none, 1 account owner, 2 admin, 3 master agent, 4 agent, 5 client, 6 corporate. |
| `permission_type` | Integer | Permission level of the person (Octadesk PermissionType enum). |
| `permission_view` | Integer | Which tickets the person can see (Octadesk PermissionViewType enum). |
| `participant_permission` | Integer | Permission on tickets the person took part in (Octadesk enum). |
| `is_locked` | Boolean | Whether the person's access is locked. |
| `customer_code` | String | Customer code registered for the person. |
| `others_email` | Array | Other e-mails of the person. |
| `others_customer_code` | String | Other customer codes of the person (JSON array of key/value pairs). |
| `id_contact_status` | String | Identifier of the contact status. |
| `contact_status` | String | Name of the contact status. |
| `phone_contacts` | String | Phone numbers of the person: number, country code, type (JSON array). |
| `custom_field` | String | Custom fields of the person, keyed by field code (JSON object). |
| `organization_id` | String | Identifier of the person's main organization. |
| `organization_name` | String | Name of the person's main organization. |
| `organization` | String | Main organization of the person (JSON object). |
| `organizations` | String | Every organization the person belongs to (JSON array). |
| `groups` | String | Groups the person belongs to (JSON array). |
| `responsible_id` | String | Identifier of the agent responsible for this customer. |
| `responsible_name` | String | Name of the agent responsible for this customer. |
| `responsible_email` | String | E-mail of the agent responsible for this customer. |
| `products` | String | Products linked to the person (JSON array). |
| `apps` | String | Octadesk apps linked to the person and their fields (JSON array). |
| `my_apps` | Array | Identifiers of the apps the person uses. |
| `my_ticket_views` | Array | Identifiers of the ticket views the person uses. |
| `thumb_url` | String | Link to the person's picture. |
| `facebook_id` | String | Facebook identifier of the person. |
| `facebook_user_name` | String | Facebook user name of the person. |
| `home_page` | String | Home page settings of the person (JSON object). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Statuses customers can be tagged with (`GET /persons/contact-status`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the contact status. |
| `name` | String | Name of the contact status. |
| `default` | Boolean | Whether this is the default contact status. |
| `created_by_id` | String | Identifier of the person who created the status. |
| `created_by_name` | String | Name of the person who created it. |
| `created_by` | String | Person who created the status (JSON object). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Products registered in the account, disabled ones included (`GET /products/search`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the product. |
| `members_join` | String | Values of the product's fields, joined (how Octadesk names the product). |
| `product_group_id` | String | Identifier of the product group the product belongs to. |
| `product_group_name` | String | Name of the product group. |
| `product_group` | String | Product group of the product (JSON object). |
| `thumb_url` | String | Link to the product's picture. |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Product groups (catalogs) and the fields their products are described with (`GET /products/groups`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the product group. |
| `name` | String | Name of the product group. |
| `description` | String | Description of the product group. |
| `date_creation` | Datetime | When the product group was created (UTC). |
| `is_enabled` | Boolean | Whether the product group is active. |
| `fields` | Array | Fields the group's products are described with. |
| `products` | String | Products of the group, when the listing includes them (JSON array). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Ticket subjects (assuntos), with their topic and topic group (`GET /subjects`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the subject. |
| `name` | String | Name of the subject. |
| `is_enabled` | Boolean | Whether the subject is active. |
| `invisible_to_client` | Boolean | Whether customers cannot pick this subject. |
| `id_topic` | String | Identifier of the subject's topic. |
| `topic_name` | String | Name of the subject's topic. |
| `topic` | String | Topic of the subject (JSON object). |
| `id_topic_group` | String | Identifier of the topic group. |
| `topic_group_name` | String | Name of the topic group. |
| `topic_group` | String | Topic group of the subject (JSON object). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Tags available to label tickets (`GET /tags`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier of the tag. |
| `name` | String | Name of the tag. |
| `text` | String | Text of the tag as shown in Octadesk. |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Custom fields configured for tickets, people, and organizations (`GET /custom-fields`). The `field_id` is the key each field appears under in the `custom_field` columns of the other streams.
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the custom field. |
| `field_id` | String | Code of the field, the key it appears under in records' custom\_field. |
| `title` | String | Label of the field. |
| `description` | String | Description of the field. |
| `type` | Integer | Kind of field: text, list, date, number... (Octadesk FieldType enum). |
| `domain_type` | Integer | Octadesk domain the field applies to (DomainType enum: tickets, people...). |
| `required` | Boolean | Whether the field is mandatory. |
| `is_enabled` | Boolean | Whether the field is active. |
| `is_customer_visible` | Boolean | Whether customers can see the field. |
| `is_customer_editable` | Boolean | Whether customers can fill the field. |
| `is_searchable` | Boolean | Whether the field can be searched. |
| `is_show_at_close` | Boolean | Whether the field is asked when a conversation is closed. |
| `is_show_in_conversation_details` | Boolean | Whether the field shows in the conversation details. |
| `order` | Integer | Display position of the field. |
| `alphabetically_ordered` | Boolean | Whether the field's options are sorted alphabetically. |
| `hint` | String | Hint shown next to the field. |
| `place_holder` | String | Placeholder of the field. |
| `list_item` | String | Options of a list field: order and text (JSON array). |
| `levels` | String | Options of a multi-level field (JSON array). |
| `reflected_fields` | String | Fields this one is mirrored to (JSON array). |
| `new_reflection_fields` | String | New mirrored fields of this one (JSON array). |
| `date_creation` | Datetime | When the field was created (UTC). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
Forms used to open tickets and register people (`GET /forms`).
Primary key: `id`
| Field | Type | Description |
| :- | :- | :- |
| `id` | String | Unique identifier (UUID) of the form. |
| `name` | String | Name of the form. |
| `description` | String | Description of the form. |
| `domain` | Integer | Octadesk domain the field applies to (DomainType enum: tickets, people...). |
| `is_default` | Boolean | Whether this is the domain's default form. |
| `is_enabled` | Boolean | Whether the form is active. |
| `fields` | String | Fields of the form and their rules (JSON array). |
| `smart_form_events` | String | Smart-form rules attached to the form (JSON array). |
| `raw_payload` | String | The whole record exactly as Octadesk returned it (JSON object), including fields that have no column of their own. |
# Octofood as a data source
Source: https://docs.nekt.com/sources/octofood
Bring data from your Octofood store to your Lakehouse.
Octofood is a Brazilian e-commerce platform built for food and beverage stores, covering online ordering, scheduled delivery, meal kits and subscriptions. Every Octofood store serves its own integration API from its own domain, and the connector reads the store's customers, orders, products, subscriptions, store credits, abandoned carts and custom shipping ranges.
Access to the API is issued per store by Octofood support and depends on your plan: it is not included in the Basic plan. If you do not have API credentials yet, ask Octofood support to enable the API for your store.
## Configuring Octofood as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Octofood option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
Octofood authenticates every request with a **webservice user and password** issued for your store. These are not the same as your store panel login. To get them, contact Octofood support and ask them to enable the API and send you the webservice credentials.
You will need three values:
* **Store address**: the address of your own store, including https, for example [https://minhaloja.com.br](https://minhaloja.com.br). Each Octofood store serves its API from its own domain, so this is your store's address and not a shared Octofood address.
* **Webservice user**: the API user issued by Octofood support.
* **Webservice password**: the password issued together with the user.
If the connection is refused, check the store address first. It has to be the domain customers use to reach your store, with no path after it.
The following configurations are also available:
* **Initial sync date**: The earliest date from which orders, customers, subscriptions and abandoned carts will be synced. Products, store credits and shipping ranges are always read in full.
* **Lookback (days)**: How many days before the last synced point each run re-reads, 30 by default. See the note on changed records below for why this matters.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
## How the extraction works
* **Orders, customers, subscriptions and abandoned carts** are read through the API's date-range filters and synced incrementally.
* **Products, store credits and shipping ranges** have no date filter in the API and are read in full on every run. Store credits are read in full on purpose, because the remaining balance falls as the customer spends it.
* **Timestamps** are returned by Octofood in the store's local time, with no time zone attached. The connector keeps them exactly as the store sent them rather than assuming an offset. Calendar dates such as birthdays and delivery dates are kept as dates.
* **Numbers** arrive as text, because the API answers in XML. Prices, quantities and identifiers are converted to numeric columns, and empty values become null. Nested structures such as order line items, extra addresses, subscription cycles and nutritional tables are stored as JSON text columns.
**Changed records are only re-read inside the lookback window.** The Octofood API can only filter by the date a record was *created*, not by when it last changed. An order's status, a subscription's cycles and a customer's address all change after that date. Each run therefore re-reads the last few days again, controlled by the **Lookback (days)** setting. If your orders often change status long after they are placed, raise it so those changes are still picked up.
**Records marked as imported disappear from the Octofood listings.** Octofood lets an integration mark an order or a customer as imported, and a marked record stops being returned by the API. This connector never marks anything, it only reads. But if another integration on your store marks records as imported, those records become invisible to Nekt as well. If you already run an ERP integration over this same API, check this before treating the tables as complete.
## Available streams
| Stream | Slug | Replication | Description |
| - | - | - | - |
| Customers | `customers` | Incremental | Customers registered in the store, with their main address; extra addresses as a JSON column. |
| Customer Credits | `customer_credits` | Full table | Store credit balances held by customers, with the remaining and original value. |
| Abandoned Carts | `abandoned_carts` | Incremental | One row per product left in a cart without an order being placed. |
| Products | `products` | Full table | Product catalogue, one row per price variation, with nutritional table and kit composition as JSON columns. |
| Orders | `orders` | Incremental | Orders with customer, delivery address, payment, totals and status; line items as a JSON column. |
| Subscriptions | `subscriptions` | Incremental | Recurring subscriptions with plan, delivery settings and totals; billing cycles as a JSON column. |
| Shipping Zip Codes | `shipping_zip_codes` | Full table | Postal code ranges configured for custom shipping rates. |
## Fields by stream
Below you'll find all available data streams from Octofood and their corresponding fields:
One row per customer registered in the store. Extra delivery addresses travel as a JSON column.
**Slug:** `customers`\
**Primary key:** `cod_cliente`\
**Replication:** Incremental (`data_cadastro`)
**Fields:**
* `cod_cliente` - Unique identifier of the customer in the store
* `importado` - Whether the record was already marked as imported by an integration: 0 = not imported, 1 = imported. Customers marked as imported stop appearing in this listing
* `status` - Registration status of the customer, either 'Ativo' (active) or 'Inativo' (inactive)
* `nome` - First name of the customer
* `sobrenome` - Last name of the customer
* `responsavel` - Name of the person responsible for the company, when the registration is a legal entity (CNPJ)
* `sexo` - Gender of the customer: 'f' for female and 'm' for male
* `email` - Email address of the customer
* `cnpj` - Brazilian company tax identifier (CNPJ), present when the customer is a legal entity
* `cpf` - Brazilian individual tax identifier (CPF), present when the customer is an individual
* `inscricao_estadual` - State tax registration of the company
* `endereco` - Street of the customer's main address
* `numero` - Street number of the main address
* `complemento` - Address complement of the main address, such as apartment or block
* `bairro` - Neighbourhood of the main address
* `cidade` - City of the main address
* `estado` - Two-letter Brazilian state code of the main address
* `cep` - Postal code (CEP) of the main address, formatted as '99999-999'
* `telefone` - Landline phone number of the customer
* `celular` - Mobile phone number of the customer
* `como_conheceu` - Answer to the store's 'how did you hear about us' question
* `como_conheceu_texto` - Free-text detail for the 'how did you hear about us' answer
* `data_cadastro` - Date the customer registered in the store, used as the incremental replication key
* `data_nascimento` - Date of birth of the customer
* `data_ultimo_pedido` - Date of the customer's most recent order
* `enderecos` - Additional delivery addresses registered by the customer (JSON object holding a list of addresses, each with cod\_endereco, titulo, responsavel, endereco, numero, complemento, bairro, cidade, estado and cep)
Store credit balances held by customers. Read in full on every run, because the remaining balance falls as the customer spends it.
**Slug:** `customer_credits`\
**Primary key:** `cod_cliente`, `cod_pedido`, `data_inserido`\
**Replication:** Full table
**Fields:**
* `cod_cliente` - Unique identifier of the customer who holds the credit
* `cod_pedido` - Identifier of the order that generated the credit. Documented as always present, but absent from some responses
* `nome` - Full name of the customer who holds the credit
* `email` - Email address of the customer who holds the credit
* `data_inserido` - Date the credit was granted
* `data_validade` - Date the credit expires
* `valor_credito` - Remaining balance of the credit. This value decreases as the customer spends it
* `valor_credito_inicial` - Original value of the credit when it was granted
One row per product left in a cart without an order being placed, not one row per cart.
**Slug:** `abandoned_carts`\
**Primary key:** `cod_carrinho_abandonado`\
**Replication:** Incremental (`data_inserido`)
**Fields:**
* `cod_carrinho_abandonado` - Unique identifier of the abandoned cart entry
* `cod_cliente` - Unique identifier of the customer who left the product in the cart
* `data_inserido` - Timestamp the product was added to the cart, used as the incremental replication key. Expressed in the store's local time, without a UTC offset
* `data_ultima_compra` - Timestamp of the customer's most recent order in the store, in the store's local time
* `nome_cliente` - Full name of the customer, as named in the specification's field table
* `email_cliente` - Email address of the customer, as named in the specification's field table
* `nome` - Full name of the customer, as returned by the specification's own response example. Populated instead of nome\_cliente on stores that follow the example
* `email` - Email address of the customer, as returned by the specification's own response example. Populated instead of email\_cliente on stores that follow the example
* `descricao` - Description of the product as it was added to the cart, including the chosen size or variation
* `produto_nome` - Name of the product in the store catalogue
* `quantidade` - Quantity of the product left in the cart
* `valor` - Unit price of the product at the time it was added to the cart
* `codigo` - Code of the product variation, also used as the product code when the product has a single variation
* `imagem` - URL of the product image
One row per price variation of a product. A product sold in three sizes has three rows sharing the same product id, which is why the key is the pair. The nutritional table and the components of kits and dishes travel as JSON columns.
**Slug:** `products`\
**Primary key:** `cod_produto`, `cod_valor`\
**Replication:** Full table
**Fields:**
* `cod_produto` - Unique identifier of the product in the store
* `cod_valor` - Unique identifier of the product's price variation. A product with several sizes or weights has one entry per variation
* `publicar` - Whether the product is visible in the store: 0 = hidden, 1 = published
* `tipo` - Type of the product: 'produto' (single product), 'kit' or 'prato' (dish)
* `nome` - Name of the product
* `codigo` - Store code or SKU of the product variation
* `url` - URL of the product page in the store
* `url_imagem` - URL of the product's main image
* `desconto` - Price of the product after discount
* `valor` - Price of the product before discount
* `unidade` - Unit the product is sold in: g (grams), ml (millilitre), l (litre), cx (box), pct (pack), um (unit) or tam (size)
* `unidade_valor` - Amount of the unit the price refers to, for example 100 for a price per 100 grams
* `calorias` - Calorie count of the kit or dish
* `descricao` - Short description of the product. Returned only because this connector always requests the description fields
* `detalhamento` - Full description of the product
* `descricao_nutricional` - Free-text nutritional description of the product
* `seo_title` - Page title used for search engine optimisation
* `seo_description` - Meta description used for search engine optimisation
* `seo_robots` - Robots directive for the product page, for example 'noindex, nofollow'
* `seo_keywords` - Keywords used for search engine optimisation
* `tabela_nutricional` - Nutritional table of the product (JSON object with quant\_por\_porcao, the amount each value refers to, and the list of nutritional values, each with nome, quantidade and valor\_diario)
* `produtos` - Products that make up a kit or dish (JSON object holding a list, each entry with periodo, ordem, dia, nome, unidade, unidade\_valor, codigo, cod\_produto and cod\_valor). Empty for a single product
One row per order, with the customer, delivery address, payment method and totals. Line items travel as a JSON column, including the products inside kits and dishes.
**Slug:** `orders`\
**Primary key:** `cod_pedido`\
**Replication:** Incremental (`data_pedido`)
**Fields:**
* `cod_pedido` - Unique identifier of the order in the store, also used as the order number shown to the customer
* `cod_cliente` - Unique identifier of the customer who placed the order
* `cpf` - Brazilian individual tax identifier (CPF) of the customer
* `cnpj` - Brazilian company tax identifier (CNPJ), when the order was placed by a legal entity
* `status` - Current status of the order: Novo, Pendente, Pago, Enviado, Entregue, Impresso or Cancelado
* `status_data` - Timestamp the current status was set, in the store's local time without a UTC offset
* `status_descricao` - Free-text description attached to the current status
* `data_programada` - Date the order is scheduled to be delivered
* `hora_programada` - Time or delivery window the order is scheduled for, as free text such as '08:00 / 18:00'
* `data_pedido` - Timestamp the order was placed, used as the incremental replication key. Expressed in the store's local time, without a UTC offset
* `portaria` - Whether the order may be left at the building's reception: 'S' for yes and 'N' for no. Only meaningful when the store enables the option at checkout
* `ent_responsavel` - Name of the person who will receive the order at the delivery address
* `ent_endereco` - Street of the delivery address
* `ent_numero` - Street number of the delivery address
* `ent_complemento` - Address complement of the delivery address
* `ent_bairro` - Neighbourhood of the delivery address
* `ent_cidade` - City of the delivery address
* `ent_estado` - Two-letter Brazilian state code of the delivery address
* `ent_cep` - Postal code (CEP) of the delivery address, formatted as '99999-999'
* `forma_entrega` - Delivery method chosen for the order, such as 'Correios PAC', 'Correios Sedex', 'Transportadora', 'Retirada no local' or 'Loggi'
* `codigo_pagamento` - Numeric code of the payment method. For example 1 = Gateway Boleto, 2 = cash at the store, 7 = PayPal, 10 = Gateway Visa, 11 = Gateway Mastercard, 16 = Sodexo meal voucher
* `forma_pagamento` - Human-readable description of the payment method
* `observacoes` - Notes attached to the order by the customer
* `certificado` - Code of the discount certificate applied to the order
* `valor_total` - Value of the order before discounts and before shipping
* `valor_frete` - Shipping charged on the order
* `valor_certificado` - Value of the discount certificate applied, expressed as a negative number
* `valor_desconto` - Discount applied to the order, expressed as a negative number
* `valor_troco` - Change the customer expects to receive, for orders paid in cash on delivery
* `itens` - Line items of the order (JSON object holding a list, each item with cod\_item, cod\_valor, codigo, descricao, quantidade, valor and valor\_real; items that are kits or dishes also carry 'produtos', the products they contain, and 'cardapio', the menu breakdown by period and day)
One row per recurring subscription, with its plan, delivery settings and totals. Billing cycles travel as a JSON column.
**Slug:** `subscriptions`\
**Primary key:** `cod_assinatura`\
**Replication:** Incremental (`data_assinatura`)
**Fields:**
* `cod_assinatura` - Unique identifier of the subscription in the store, also used as the subscription number
* `descricao` - Description of the subscription plan, such as 'Assinatura / Por mês / Entrega Quinzenal'
* `cod_cliente` - Unique identifier of the subscribing customer
* `cpf` - Brazilian individual tax identifier (CPF) of the customer
* `cnpj` - Brazilian company tax identifier (CNPJ), when the subscriber is a legal entity
* `status` - Current status of the subscription. Documented as Ativa, Pendente or Cancelada, but some stores return a numeric status code instead, so the value is kept as text
* `intervalo` - Billing interval unit of the subscription: 'semana' (week), 'mes' (month) or 'ano' (year)
* `intervalo_contador` - Number of interval units between charges. A monthly plan is intervalo 'mes' with intervalo\_contador 1
* `data_assinatura` - Timestamp the subscription was created, used as the incremental replication key. Expressed in the store's local time, without a UTC offset
* `data_cancelamento` - Timestamp the subscription was cancelled, in the store's local time
* `data_proximo_cobranca` - Timestamp of the next scheduled charge, in the store's local time
* `entrega_dia` - Day of the week the subscription is delivered. Only set for subscriptions configured with a physical delivery
* `entrega_frequencia` - Delivery frequency of the subscription, such as 'quinzenal'. Only set for subscriptions with a physical delivery
* `entrega_horario` - Delivery time window of the subscription. Only set for subscriptions with a physical delivery
* `entrega_responsavel` - Name of the person who will receive the delivery
* `entrega_endereco` - Street of the delivery address
* `entrega_numero` - Street number of the delivery address
* `entrega_complemento` - Address complement of the delivery address
* `entrega_bairro` - Neighbourhood of the delivery address
* `entrega_cidade` - City of the delivery address
* `entrega_estado` - Two-letter Brazilian state code of the delivery address
* `entrega_cep` - Postal code (CEP) of the delivery address, formatted as '99999-999'
* `observacoes` - Notes attached to the subscription
* `desconto_ciclos` - Number of billing cycles that receive the discount
* `certificado` - Code of the discount voucher applied to the subscription
* `valor_total` - Value of the subscription before discounts and shipping
* `valor_certificado` - Value of the discount voucher applied, expressed as a negative number
* `ciclos` - Billing cycles of the subscription (JSON object holding a list, each cycle with ciclo, its status, valor, data\_inicio, data\_fim, data\_pagamento and data\_cobranca). Cycle status values are Pendente, Pago, Cancelado or a failed charge, and some stores return them as numeric codes
Postal code ranges configured for custom shipping rates.
**Slug:** `shipping_zip_codes`\
**Primary key:** `cod_regiao`\
**Replication:** Full table
**Fields:**
* `cod_regiao` - Unique identifier of the shipping region in the store
* `local` - Name of the location or city the postal code range covers
* `estado` - Two-letter Brazilian state code of the range
* `cep_inicial` - First postal code (CEP) of the range, formatted as '99999-999'
* `cep_final` - Last postal code (CEP) of the range, formatted as '99999-999'
* `valor` - Shipping price charged for this postal code range
* `valor_minimo` - Minimum order value required for this postal code range
# OData as a data source
Source: https://docs.nekt.com/sources/odata
Bring data from any OData service (versions 2, 3 and 4) to Nekt.
OData (Open Data Protocol) is a standard for exposing data over REST that many business systems implement, including SAP Gateway services, Microsoft Dataverse and Business Central, and custom services built on .NET, Java or Node.js. This connector works with any service that speaks **OData version 2, 3 or 4**: it reads the service's `$metadata` document and turns each **entity set** into a table, with columns and types taken from the service itself.
For Microsoft Dynamics 365 Finance and Operations, use the dedicated [Microsoft Dynamics Finance and Operations](/sources/dynamics-fo) connector, which only asks for your environment URL and Microsoft Entra app.
## Before you start
You need:
* The **service URL**: the root address of the OData service, the one that lists its entity sets. Its metadata document is at `/$metadata`. For example `https://services.odata.org/V4/Northwind/Northwind.svc` or `https://my-sap-host/sap/opu/odata/sap/API_BUSINESS_PARTNER`.
* Credentials for one of the supported authentication methods, from the administrator of the service.
* The service must be reachable from the internet.
## Configuring OData as a source
In the [Sources](https://app.nekt.ai/sources) tab, click the "Add source" button on the top right, then select OData from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
* **Service URL**: The root URL of the OData service. If the service needs a parameter on every request (such as SAP's `sap-client=100`), you can include it in the URL or add it under **Query parameters**.
* **Authentication**: How the connector signs in:
* **No authentication**: for public services.
* **Basic**: a **Username** and **Password**.
* **Bearer token**: a **Token** sent as `Authorization: Bearer `.
* **API key**: the **API key name** (header or query parameter name, for example `APIKey`), the **API key**, and whether it goes in a **header** or the **query string**.
* **OAuth 2.0 client credentials**: the **Access token URL**, **Client ID**, **Client secret** and, if the server requires them, a **Scope** and extra **Token request parameters** (for example `audience`).
* **Microsoft Entra ID (Azure AD) app**: the **Tenant ID**, **Client ID** and **Client secret** of an app registration. The scope defaults to `/.default`.
* **Entities**: The entity sets to sync. Each becomes a stream. Use the collection name that appears in the service URL (for example `Customers`). For each entity set you can optionally set:
* **Incremental field**: a date/time (or ever-increasing number) field the service updates when a record changes. With it, each run reads only records changed since the previous one.
* **Filter**: an OData `$filter` expression, for example `Country eq 'Brazil'`.
* **Fields**: the fields to read. Key fields and the incremental field are always included.
Leave **Entities** empty to sync every entity set of the service.
Advanced settings:
* **Initial sync date**: For incremental entity sets, only records changed after this date are read on the first run.
* **Default incremental fields**: Field names to use as the incremental field of any entity set that has them, in order of preference.
* **OData version**: Detected automatically from the metadata; set it only if the service declares the wrong one.
* **Query parameters** and **Headers**: Added to every request.
* **Page size**, **Pagination** and **Order by keys**: See [Paging](#paging).
* **Incremental lookback (minutes)**: Each incremental sync re-reads records changed this many minutes before the previous position, to catch changes the service saves late. Off by default.
* **Request timeout**: How long to wait for each page before retrying (default 300 seconds).
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. Each stream corresponds to one entity set. For faster extractions, select only the streams relevant to your analysis.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired Layer where the data will be placed, a Folder to organize it, a name for each table, and the type of sync.
* **Layer**: choose between the existing Layers on your catalog. This is where your extracted tables appear once the extraction runs successfully.
* **Folder**: a Folder can be created inside the selected Layer to group all tables created from this source.
* **Table name**: a name is suggested, but you can customize it. You can also add a **prefix** to all tables at once.
* **Sync Type**: choose between INCREMENTAL and FULL\_TABLE.
* Incremental: available for entity sets with an incremental field; each extraction fetches only records changed since the last run.
* Full table: each extraction fetches the whole entity set.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source.
Optionally, you can define additional settings such as Delta Log Retention and an additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync) to complement the incremental extractions.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking the arrow button. Once executed, your data appears in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Streams are discovered from the service's `$metadata`: one stream per entity set, named in snake\_case (`SalesOrders` becomes `sales_orders`). Columns keep the names the service publishes, and the entity key becomes the table's primary key. When an entity set holds several record types (derived types), their extra fields are included too.
## How columns are represented
| OData type | Column type |
| - | - |
| `Edm.String`, `Edm.Guid`, enumerations, `Edm.TimeOfDay`, `Edm.Duration` | Text |
| `Edm.Binary` | Text (base64) |
| `Edm.Int16`, `Edm.Int32`, `Edm.Int64`, `Edm.Byte` | Integer |
| `Edm.Decimal`, `Edm.Double`, `Edm.Single` | Number |
| `Edm.DateTimeOffset`, `Edm.DateTime` | Timestamp (UTC) |
| `Edm.Date` | Date |
| Complex types, collections, geographic types | Text containing JSON |
`Edm.DateTime` values carry no time zone in OData versions 2 and 3; they are stored as UTC.
Navigation properties (links to related entities) are not expanded. Sync the related entity set as its own stream instead.
## Paging
The **Pagination** setting chooses how each entity set is read page by page:
| Option | How it reads | When to use |
| - | - | - |
| **Server** (default) | Follows the address of the next page that the service returns. | Most services. |
| **Keyset** | Asks for records after the last key read (`$filter` on the key, **Page size** records per request). Stays correct while records are added, changed or deleted during the sync. | Entity sets with a single-field key, especially large ones. Others fall back to Offset. |
| **Offset** | Reads pages with `$top`/`$skip`. | Services that never page on their own (they return everything, or cut the result silently). |
| **Automatic** | Keyset for entity sets with a single numeric key, Server for the others. | |
Records are always sorted by the entity key (**Order by keys**, on by default), so pages stay stable. Turn it off only if the service rejects sorting. If a service ignores the paging parameters and keeps returning the same records, the sync stops with an error instead of looping; choose another **Pagination** option.
## Incremental sync
For an entity set with an incremental field, each run asks only for records whose field is equal to or later than the last value seen (minus the **Incremental lookback**, if set). Records whose incremental field does not change on update, records whose incremental field is empty, and deleted records are not detected: use FULL\_TABLE (or an additional Full Sync) for those entity sets.
If you change an entity set's **Filter** or **Incremental field**, its next sync starts again from the **Initial sync date**, so records that the new filter now includes are loaded too.
Only text, number and date/time fields can be incremental fields. Enumerations, GUIDs and durations cannot.
## Troubleshooting
* **"The service URL did not return an OData metadata document"**: the URL does not point to the service root. Open `/$metadata` in a browser; it must show an XML document starting with `edmx:Edmx`.
* **"These entity sets were not found"**: the sync stops when an entity set listed in **Entities** does not exist on the service, so its table is never left silently outdated. Check the names (the collection names listed at the service URL), or remove the ones that no longer exist.
* **Access denied for an entity set**: the account used by the connector lacks read permission on it. Grant it, or remove the entity set from **Entities**.
* **The service is throttling**: the connector waits the time the service asks for and continues; no data is lost.
# Odoo as a data source
Source: https://docs.nekt.com/sources/odoo
Bring data from Odoo to Nekt.
Odoo is an open-source enterprise resource planning (ERP) system that helps businesses manage their operations, including sales, inventory, accounting, manufacturing, and more. It provides a comprehensive suite of integrated business applications designed to streamline workflows and improve efficiency.
## Configuring Odoo as a Source
In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Odoo option from the list of connectors.
Click **Next** and you'll be prompted to add your access.
### 1. Add account access
You'll need to authorize Nekt to access your Odoo instance. Provide the necessary credentials and connection details for your Odoo installation.
The following configurations are available:
* **API URL**: The base URL of your Odoo instance (e.g., `https://mycompany.odoo.com`). This is the root URL where your Odoo instance is hosted.
* **API Key**: The API key (bearer token) to authenticate against the Odoo API. This token grants access to your Odoo instance and must have the necessary permissions to read the data you want to extract.
* **Database**: (Optional) The name of the database to connect to. This is only required when a single Odoo server hosts multiple databases. If your instance uses a single database or the database name is determined automatically, you can leave this field empty.
* **Include archived records** (Advanced): (Default: `false`) Also extract records archived in Odoo (`active = false`), such as archived leads, customers or users. By default Odoo only returns active records. After enabling it, run a Full sync to bring records archived before the change.
Once you're done, click **Next**.
### 2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
> Tip: The stream can be found more easily by typing its name.
Select the streams and click **Next**.
### 3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Folder**: a folder can be created inside the selected layer to group all tables being created from this new data source.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: you can choose between INCREMENTAL and FULL\_TABLE.
* Incremental: every time the extraction happens, we'll get only the new data, which is good if, for example, you want to keep every record ever fetched.
* Full table: every time the extraction happens, we'll get the current state of the data, which is good if, for example, you don't want to have deleted data in your catalog.
Once you are done configuring, click **Next**.
### 4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters.
To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Optionally, you can define some additional settings:
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
Once you are ready, click **Next** to finalize the setup.
### 5. Check your new source
You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you need at least one successful source run.
# Streams and Fields
Below you'll find all available data streams from Odoo and their corresponding fields:
Stream for retrieving payment records from Odoo's `account.payment` model. This stream contains payment transactions and their associated details.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the payment record |
| `name` | String | Payment number/reference |
| `amount` | Number | Payment amount |
| `date` | DateTime | Payment date |
| `state` | String | Payment state (draft, posted, sent, reconciled, cancelled) |
| `payment_type` | String | Type of payment (inbound, outbound) |
| `partner_type` | String | Partner type (customer, supplier) |
| `create_date` | DateTime | Timestamp |
| `write_date` | DateTime | Timestamp |
| `amount_signed` | Number | Signed payment amount |
| `amount_company_currency_signed` | Number | Amount in company currency |
| `amount_available_for_refund` | Number | Available refund amount |
| `currency_id` | Integer | Currency identifier |
| `company_currency_id` | Integer | Company currency identifier |
| `effective_date` | DateTime | Effective payment date |
| `payment_method_id` | Integer | Payment method identifier |
| `payment_method_code` | String | Payment method code |
| `payment_method_line_id` | Integer | Payment method line identifier |
| `payment_token_id` | Integer | Saved payment token identifier |
| `payment_transaction_id` | Integer | Payment transaction identifier |
| `use_electronic_payment_method` | Boolean | Whether electronic payment method is used |
| `partner_id` | Integer | Customer/Vendor identifier |
| `partner_bank_id` | Integer | Recipient bank account identifier |
| `require_partner_bank_account` | Boolean | Whether partner bank account is required |
| `show_partner_bank_account` | Boolean | Whether to show partner bank account |
| `destination_account_id` | Integer | Destination account identifier |
| `outstanding_account_id` | Integer | Outstanding account identifier |
| `journal_id` | Integer | Journal identifier |
| `company_id` | Integer | Company identifier |
| `available_journal_ids` | Array | Available journal identifiers |
| `available_partner_bank_ids` | Array | Available partner bank identifiers |
| `available_payment_method_line_ids` | Array | Available payment method line identifiers |
| `invoice_ids` | Array | Related invoice identifiers |
| `reconciled_invoice_ids` | Array | Reconciled invoice identifiers |
| `reconciled_invoices_count` | Integer | Number of reconciled invoices |
| `reconciled_invoices_type` | String | Type of reconciled invoices |
| `reconciled_bill_ids` | Array | Reconciled bill identifiers |
| `reconciled_bills_count` | Integer | Number of reconciled bills |
| `reconciled_statement_line_ids` | Array | Reconciled statement line identifiers |
| `reconciled_statement_lines_count` | Integer | Number of reconciled statement lines |
| `is_reconciled` | Boolean | Whether payment is reconciled |
| `is_matched` | Boolean | Whether matched with bank statement |
| `payment_reference` | String | Payment reference |
| `bank_reference` | String | Bank reference |
| `cheque_reference` | String | Cheque reference |
| `check_number` | String | Check number |
| `check_amount_in_words` | String | Amount in words |
| `check_layout_available` | Boolean | Whether check layout is available |
| `check_manual_sequencing` | Boolean | Whether manual numbering is used |
| `show_check_number` | Boolean | Whether to show check number |
| `qr_code` | String | QR Code URL |
| `payment_receipt_title` | String | Payment receipt title |
| `move_id` | Integer | Journal entry identifier |
| `source_payment_id` | Integer | Source payment identifier |
| `paired_internal_transfer_payment_id` | Integer | Paired internal transfer payment identifier |
| `duplicate_payment_ids` | Array | Duplicate payment identifiers |
| `refunds_count` | Integer | Number of refunds |
| `activity_ids` | Array | Activity identifiers |
| `activity_calendar_event_id` | Integer | Next activity calendar event identifier |
| `activity_date_deadline` | DateTime | Next activity deadline |
| `activity_state` | String | Activity state |
| `activity_summary` | String | Next activity summary |
| `activity_type_id` | Integer | Next activity type identifier |
| `activity_user_id` | Integer | Responsible user identifier |
| `activity_exception_decoration` | String | Activity exception decoration |
| `activity_exception_icon` | String | Activity exception icon |
| `activity_type_icon` | String | Activity type icon |
| `my_activity_date_deadline` | DateTime | My activity deadline |
| `message_ids` | Array | Message identifiers |
| `message_follower_ids` | Array | Follower identifiers |
| `message_partner_ids` | Array | Follower partner identifiers |
| `message_attachment_count` | Integer | Attachment count |
| `message_main_attachment_id` | Integer | Main attachment identifier |
| `message_has_error` | Boolean | Whether message has error |
| `message_has_error_counter` | Integer | Number of message errors |
| `message_has_sms_error` | Boolean | Whether message has SMS error |
| `message_is_follower` | Boolean | Whether user is follower |
| `message_needaction` | Boolean | Whether action is needed |
| `message_needaction_counter` | Integer | Number of actions needed |
| `has_message` | Boolean | Whether record has messages |
| `website_message_ids` | Array | Website message identifiers |
| `memo` | String | Payment memo/notes |
| `display_name` | String | Display name |
| `is_donation` | Boolean | Whether payment is donation |
| `is_sent` | Boolean | Whether payment is sent |
| `country_code` | String | Country code |
| `need_cancel_request` | Boolean | Whether cancel request is needed |
| `suitable_payment_token_ids` | Array | Suitable payment token identifiers |
| `attachment_ids` | Array | Attachment identifiers |
| `create_uid` | Integer | Created by user identifier |
| `write_uid` | Integer | Last updated by user identifier |
Stream for retrieving insurance quotation records from Odoo's `mut.insurance.quotation` model. This stream contains insurance quotation data specific to your Odoo configuration.
| Field | Type | Description |
| :- | :- | :- |
| `id` | Integer | Unique identifier for the insurance quotation record |
| `name` | String | Quotation/proposal number |
| `status` | String | Quotation status |
| `create_date` | DateTime | Timestamp |
| `write_date` | DateTime | Timestamp |
| `display_name` | String | Display name |
| `access_token` | String | External access token |
| `active` | Boolean | Whether quotation is active |
| `validity_date` | DateTime | End of validity period |
| `standard_duration` | String | Standard duration |
| `quotation_fields_readonly` | Boolean | Whether quotation fields are read-only |
| `partner_id` | Integer | Proponent identifier |
| `proponent_name` | String | Proponent name |
| `proponent_type` | String | Proponent type |
| `proponent_vat` | String | CNPJ/CPF |
| `proponent_legal_name` | String | Legal name/company name |
| `proponent_email` | String | Email address |
| `proponent_phone` | String | Phone number |
| `proponent_age` | Integer | Age |
| `proponent_birth_date` | DateTime | Birth date |
| `proponent_gender` | String | Gender |
| `proponent_marital_status` | String | Marital status |
| `proponent_nationality` | String | Nationality |
| `proponent_rg` | String | RG (identity document) |
| `proponent_driver_license` | String | Driver's license (CNH) |
| `proponent_pep` | Boolean | Whether principal driver is PEP (Politically Exposed Person) |
| `proponent_address` | String | Endereço |
| `proponent_street_name` | String | Rua |
| `proponent_street_number` | String | Número |
| `proponent_street2` | String | Complemento |
| `proponent_district` | String | Bairro |
| `proponent_city` | String | City |
| `proponent_city_id` | Integer | City identifier |
| `proponent_state_id` | Integer | State identifier |
| `proponent_country_id` | Integer | Country identifier |
| `proponent_zip` | String | ZIP code |
| `proponent_zip_number` | String | ZIP code (numbers only) |
| `proponent_company_cnae` | String | Company CNAE |
| `proponent_company_email` | String | Company email |
| `proponent_company_phone` | String | Company phone |
| `proponent_company_foundation_date` | DateTime | Company foundation date |
| `proponent_legal_representative_name` | String | Legal representative name |
| `proponent_legal_representative_vat` | String | Legal representative CPF |
| `product_id` | Integer | Insurance product identifier |
| `item_ids` | Array | Certificate/item identifiers |
| `items_display_name` | String | Risk item display name |
| `items_value` | Number | IS value (insured sum) |
| `items_total_premium_amount` | Number | Total premium amount for items |
| `extra_product_ids` | Array | Additional product identifiers |
| `total_premium_amount` | Number | Total premium amount |
| `display_installments` | Number | Installment amount |
| `first_installment_amount` | Number | First installment amount |
| `installments_due_day` | Integer | Installment due day |
| `display_payment_term` | String | Payment term display |
| `factor` | Number | Factor |
| `payment_method_id` | Integer | Payment method identifier |
| `payment_provider_id` | Integer | Payment provider identifier |
| `payment_term_id` | Integer | Payment term identifier |
| `available_payment_method_ids` | Array | Available payment method identifiers |
| `available_payment_provider_ids` | Array | Available payment provider identifiers |
| `discount_id` | Integer | Discount identifier |
| `discount_status` | String | Discount status |
| `is_enable_discount` | Boolean | Whether discount is enabled |
| `enable_commissioning` | Boolean | Whether commissioning is enabled |
| `json_commission_amounts` | String | Commission amounts (JSON) |
| `json_commission_rates` | String | Commission rates (JSON) |
| `is_renewal` | Boolean | Whether quotation is renewal |
| `origin_policy_id` | Integer | Origin policy identifier |
| `is_calc_done` | Boolean | Whether calculation is done |
| `inspection_counter` | Integer | Inspection counter |
| `user_id` | Integer | Salesperson identifier |
| `sales_team_id` | Integer | Sales team identifier |
| `company_id` | Integer | Company identifier |
| `currency_id` | Integer | Currency identifier |
| `activity_ids` | Array | Activity identifiers |
| `activity_calendar_event_id` | Integer | Next activity calendar event identifier |
| `activity_date_deadline` | DateTime | Next activity deadline |
| `activity_state` | String | Activity state |
| `activity_summary` | String | Next activity summary |
| `activity_type_id` | Integer | Next activity type identifier |
| `activity_user_id` | Integer | Responsible user identifier |
| `activity_exception_decoration` | String | Activity exception decoration |
| `activity_exception_icon` | String | Activity exception icon |
| `activity_type_icon` | String | Activity type icon |
| `my_activity_date_deadline` | DateTime | My activity deadline |
| `message_ids` | Array | Message identifiers |
| `message_follower_ids` | Array | Follower identifiers |
| `message_partner_ids` | Array | Follower partner identifiers |
| `message_attachment_count` | Integer | Attachment count |
| `message_has_error` | Boolean | Whether message has error |
| `message_has_error_counter` | Integer | Number of message errors |
| `message_has_sms_error` | Boolean | Whether message has SMS error |
| `message_is_follower` | Boolean | Whether user is follower |
| `message_needaction` | Boolean | Whether action is needed |
| `message_needaction_counter` | Integer | Number of actions needed |
| `has_message` | Boolean | Whether record has messages |
| `website_message_ids` | Array | Website message identifiers |
| `create_uid` | Integer | Created by user identifier |
| `write_uid` | Integer | Last updated by user identifier |
Stream for retrieving policy records from Odoo's `mut.policy` model. Contains insurance policy data including proponent details, coverage, and payment information.
| Field | Type | Description |
| :- | :- | :- |
| `id`