> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nekt.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Profound as a data source
> Bring data from Profound to your Lakehouse.
Profound is an AI search analytics platform — it measures how your brand shows up inside AI-generated answers from engines like ChatGPT, Perplexity, Google Gemini and Claude. It tracks your visibility and share of voice against competitors, the pages those engines cite, the sentiment expressed about your brand, and the AI crawler and referral traffic hitting your website. This connector brings all of it into your Lakehouse.
Profound's API is granted on request. If the **API Keys** page is not available in your Profound account, contact Profound support to have API access enabled before setting up this source.
## 1. Add your Profound access
1. In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the Profound option from the list of connectors.
2. Click **Next** and you'll be prompted to add your access.
* **API Key:** Your Profound API key. In Profound, open **Settings** from the bottom-left menu, select **API Keys** in the sidebar, enter a key name and an expiration date, and click **Create API Key**. Copy the key immediately — Profound will not show it again.
* **Start Date:** (Optional, advanced) The earliest day to sync the reports from. Leave it empty to start 90 days before the first run.
* **Visibility Scope:** (Optional, advanced) Which brands the visibility report covers. Use `all` (the default) to also rank the competitors tracked in your category, or `owned` to return only your own brands.
* **Organization IDs:** (Optional, advanced) Restrict the sync to specific organizations. Leave it empty to sync every organization the API key can read.
* **Category IDs:** (Optional, advanced) Restrict the Answer Engine reports to specific categories. Leave it empty to sync every category the API key can read.
* **Domains:** (Optional, advanced) Restrict the bots and referrals reports to specific websites. Leave it empty to sync every domain connected to Profound's Agent Analytics.
The API key grants read access to every category, report and connected domain of your Profound organization. Treat it as a secret.
3. Click **Next**.
## 2. Select your Profound streams
1. The next step is letting us know which streams you want to bring. You can select entire groups of streams or only a subset of them.
> Tip: The stream can be found more easily by typing its name.
If you don't see a stream you were expecting to find, please check if your access key has access to it. If that's not the issue, then it's probably because we still haven't implemented it. Feel free to get in touch and request it!
2. Click **Next**.
## 3. Configure your Profound data streams
1. Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a name for each table (which will effectively contain the fetched data) and the type of sync.
* **Layer**: choose between the existing [layers](https://docs.nekt.com/get-started/core-concepts/catalog-layers) on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
* **Table name**: we suggest a name, but feel free to customize it. You have the option to add a **prefix** to all tables at once and make this process faster!
* **Sync Type**: depending on the data you are bringing to the lake, you can choose between INCREMENTAL and FULL\_TABLE. Read more about Sync Types [here](https://docs.nekt.com/get-started/core-concepts/types-of-sync).
2. Click **Next**.
## 4. Configure your Profound data source
1. Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
2. To define your [Trigger](https://docs.nekt.com/get-started/core-concepts/triggers), consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
3. Optionally, you can define some additional settings (if available).
* Configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource [here](https://docs.nekt.com/get-started/core-concepts/resource-control).
* Determine when to execute an **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.
### Check your new source!
1. Click **Next** to finalize the setup. Once completed, you'll receive confirmation that your new source is set up!
2. You can view your new source on the [Sources](https://app.nekt.ai/sources) page. Now, for you to be able to see it on your [Catalog](https://app.nekt.ai/catalog), you have to wait for the pipeline to run. You can now monitor it on the [Sources](https://app.nekt.ai/sources) page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
> If you encounter any issues, reach out to us via Slack, and we'll gladly assist you!
## How the data is replicated
The reports are extracted **one day at a time**, in Eastern Time — the timezone Profound uses to bucket its days. Each row carries the `date` it was measured on, which is the incremental replication key. The most recent day already extracted is always re-fetched on the next run, because Profound keeps settling a day's metrics for a while after the day closes.
The Answer Engine reports (visibility, citations, sentiment, query fan-outs and answers) are extracted **per category**, and every row carries the `category_id` and `category_name` it belongs to. Sentiment is additionally extracted per owned brand, and carries the brand in the `asset` column. The Agent Analytics reports (bots and referrals) are extracted **per connected domain**, carried in the `domain` column.
The reference streams (organizations, models, regions, domains, categories, assets, personas, category topics, category tags and prompts) have no notion of time and are always fully re-extracted.
Report dimensions that your category does not use come back empty. Because those columns are part of each report's key, they are stored as an empty string rather than as null.
Some fields hold structured payloads whose shape varies — `sentiment_claims` on answers, `persona` on personas, and `tags`, `regions`, `platforms` and `personas` on prompts. They are delivered as JSON strings, so you can parse them in a Query when you need the individual keys.
## Rate limits and extraction time
Profound allows **600 requests per hour per API key**. The connector stays below that limit on its own and automatically waits and retries when the API asks it to, so no configuration is needed on your side.
Because each Answer Engine report costs one request per category per day, a first extraction can take a while: backfilling 90 days of one category across the five reports is around 450 requests, roughly an hour of runtime. Two ways to keep syncs short:
* Set **Category IDs** to only the categories you actually analyze.
* Deselect the **Answers** stream if you don't need the raw answer text — it's by far the largest stream, and every other report is derived from it.
# Streams and Fields
Below you'll find all available data streams from Profound and their corresponding fields.
Every organization the API key grants access to.
**Key Fields:**
* `id` - Unique identifier of the organization
* `name` - Display name of the organization
AI models (answer engines) monitored for the organization.
**Key Fields:**
* `id` - Unique identifier of the model
* `name` - Display name of the answer engine, such as ChatGPT, Perplexity or Google Gemini
Geographic regions the organization's prompts are run from.
**Key Fields:**
* `id` - Unique identifier of the region
* `name` - Display name of the region, such as United States
Websites connected to Profound's Agent Analytics.
**Key Fields:**
* `id` - Unique identifier of the domain
* `name` - Hostname of the connected website, used to scope the bots and referrals reports
* `created_at` - Timestamp of when the domain was connected to Profound
* `organization_id` - Identifier of the organization the domain belongs to
* `organization_name` - Name of the organization the domain belongs to
Categories, the unit every Answer Engine Insights report is scoped to.
**Key Fields:**
* `id` - Unique identifier of the category, used as the category\_id of every Answer Engine Insights report
* `name` - Display name of the category
* `internal_name` - Internal name of the category, when one is set
* `organization_id` - Identifier of the organization the category belongs to
* `organization_name` - Name of the organization the category belongs to
Brands tracked inside each category, owned ones and competitors alike.
**Key Fields:**
* `id` - Unique identifier of the asset
* `name` - Brand name of the asset. This is the value the sentiment report is keyed on
* `website` - Primary website of the brand
* `alternate_domains` - Additional domains attributed to the brand (JSON array of strings)
* `is_owned` - Whether the brand belongs to the customer. False marks a competitor tracked for comparison
* `created_at` - Timestamp of when the asset was added to the category
* `logo_url` - URL of the brand logo shown in the Profound interface
* `category_id` - Identifier of the category the asset is tracked in
* `category_name` - Name of the category the asset is tracked in
* `organization_id` - Identifier of the organization owning the category
* `organization_name` - Name of the organization owning the category
Buyer personas prompts are executed as.
**Key Fields:**
* `id` - Unique identifier of the persona
* `name` - Display name of the persona
* `persona` - Full persona profile — demographics, employment and behavior (JSON object)
* `category_id` - Identifier of the category the persona belongs to
* `category_name` - Name of the category the persona belongs to
* `organization_id` - Identifier of the organization owning the category
* `organization_name` - Name of the organization owning the category
Topics configured inside a category.
**Key Fields:**
* `category_id` - Identifier of the category the topic belongs to
* `category_name` - Name of the category the topic belongs to
* `id` - Unique identifier of the topic
* `name` - Display name of the topic
* `status` - Whether the topic is currently active or disabled
Tags used to group the prompts of a category.
**Key Fields:**
* `category_id` - Identifier of the category the tag belongs to
* `category_name` - Name of the category the tag belongs to
* `id` - Unique identifier of the tag
* `name` - Display name of the tag
The prompts Profound runs against the answer engines for a category.
**Key Fields:**
* `category_id` - Identifier of the category the prompt belongs to
* `category_name` - Name of the category the prompt belongs to
* `id` - Unique identifier of the prompt
* `prompt` - Text of the question submitted to the answer engines
* `prompt_type` - Legacy prompt type, superseded by analysis\_types
* `analysis_types` - Analyses this prompt feeds — visibility, sentiment or accuracy (JSON array of strings)
* `language` - Language the prompt is written in
* `status` - Whether the prompt is currently active or disabled
* `topic_id` - Identifier of the topic the prompt is filed under
* `topic_name` - Name of the topic the prompt is filed under
* `tags` - Tags applied to the prompt (JSON array of objects)
* `regions` - Regions the prompt is executed from (JSON array of objects)
* `platforms` - Answer engines the prompt is executed against (JSON array of objects)
* `personas` - Personas the prompt is executed as (JSON array of objects)
* `created_at` - Timestamp of when the prompt was created
* `updated_at` - Timestamp of the last update to the prompt
Daily visibility, share of voice and average position per brand.
**Key Fields:**
* `category_id` - Identifier of the category the report was run for
* `category_name` - Name of the category the report was run for
* `date` - Day the metrics were measured on, in Eastern Time. Used as the incremental replication key
* `asset_name` - Brand the metrics describe
* `asset_owned` - Whether the brand belongs to the customer. False marks a competitor tracked in the same category
* `model_id` - Identifier of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `model_name` - Name of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `topic_id` - Identifier of the topic the measured prompts are filed under
* `topic_name` - Name of the topic the measured prompts are filed under
* `region_id` - Identifier of the geographic region the prompts were executed from
* `region_name` - Name of the geographic region the prompts were executed from
* `rank` - Position of the brand in the category ranking. Only returned by ungrouped queries, so it is normally null here
* `visibility_score` - Share of the monitored answers in which the brand appears, as a fraction between 0 and 1
* `share_of_voice` - Share of all brand mentions in the category that went to this brand, as a fraction between 0 and 1
* `average_position` - Average position the brand is mentioned at inside the answers it appears in. Lower is better
Daily citation counts and share per cited page.
**Key Fields:**
* `category_id` - Identifier of the category the report was run for
* `category_name` - Name of the category the report was run for
* `date` - Day the metrics were measured on, in Eastern Time. Used as the incremental replication key
* `domain` - Domain of the cited page
* `page` - URL of the page cited by the answer engine
* `model_id` - Identifier of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `model_name` - Name of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `topic_id` - Identifier of the topic the measured prompts are filed under
* `topic_name` - Name of the topic the measured prompts are filed under
* `region_id` - Identifier of the geographic region the prompts were executed from
* `region_name` - Name of the geographic region the prompts were executed from
* `count` - Number of monitored answers that cited the page
* `citation_share` - Share of all citations in the category that went to this page, as a fraction between 0 and 1
* `rank` - Position of the page in the category citation ranking
* `first_cited_at` - Timestamp of the first time the page was cited
Daily positive and negative sentiment per owned brand.
**Key Fields:**
* `category_id` - Identifier of the category the report was run for
* `category_name` - Name of the category the report was run for
* `asset` - Brand the sentiment was measured for
* `date` - Day the metrics were measured on, in Eastern Time. Used as the incremental replication key
* `model_id` - Identifier of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `model_name` - Name of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `topic_id` - Identifier of the topic the measured prompts are filed under
* `topic_name` - Name of the topic the measured prompts are filed under
* `region_id` - Identifier of the geographic region the prompts were executed from
* `region_name` - Name of the geographic region the prompts were executed from
* `positive_sentiment` - Share of the brand's mentions carrying positive sentiment, as a fraction between 0 and 1
* `negative_sentiment` - Share of the brand's mentions carrying negative sentiment, as a fraction between 0 and 1
* `occurrence` - How often the brand was mentioned in the monitored answers
* `rank` - Position of the row in the report ranking, when returned
Daily fan-out queries generated while answering the monitored prompts.
**Key Fields:**
* `category_id` - Identifier of the category the report was run for
* `category_name` - Name of the category the report was run for
* `date` - Day the metrics were measured on, in Eastern Time. Used as the incremental replication key
* `query` - Search query the answer engine generated from the prompt
* `model_id` - Identifier of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `model_name` - Name of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `region_id` - Identifier of the geographic region the prompts were executed from
* `region_name` - Name of the geographic region the prompts were executed from
* `prompt_id` - Identifier of the prompt the row was measured on
* `prompt_name` - Name of the prompt the row was measured on
* `total_fanouts` - Number of times the query was generated on the day
* `fanouts_per_execution` - Average number of times the query is generated per prompt execution
* `share` - Share of all fan-out queries in the category represented by this query, as a fraction between 0 and 1
* `query_variations` - Number of distinct wordings the query was generated with
* `rank` - Position of the query in the category ranking, when returned
One row per prompt execution, with the full answer text.
**Key Fields:**
* `category_id` - Identifier of the category the report was run for
* `category_name` - Name of the category the report was run for
* `run_id` - Unique identifier of the prompt execution that produced the answer
* `date` - Day the metrics were measured on, in Eastern Time. Used as the incremental replication key
* `model_id` - Identifier of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `model_name` - Name of the answer engine the row was measured on, such as ChatGPT or Perplexity
* `topic_id` - Identifier of the topic the measured prompts are filed under
* `topic_name` - Name of the topic the measured prompts are filed under
* `prompt_id` - Identifier of the prompt the row was measured on
* `prompt_name` - Name of the prompt the row was measured on
* `region` - Geographic region the prompt was executed from
* `persona` - Buyer persona the prompt was executed as
* `tags` - Tags applied to the executed prompt
* `response` - Full text of the answer returned by the engine
* `mentions` - Brands mentioned in the answer
* `citations` - URLs the answer cited as sources
* `search_queries` - Web searches the engine ran while composing the answer, when it exposes them
* `analysis_types` - Analyses this execution feeds — visibility, sentiment or accuracy
* `sentiment_claims` - Claims about the brands extracted from the answer, with their sentiment (JSON array of objects)
Daily AI crawler visits per page, broken down by bot.
**Key Fields:**
* `domain` - Website the traffic was recorded on
* `date` - Day the traffic was recorded on, in UTC. Used as the incremental replication key
* `path` - Path on the website the bot requested
* `bot_name` - Name of the crawler, such as GPTBot or ClaudeBot
* `bot_provider` - Company operating the crawler, such as OpenAI or Anthropic
* `bot_type` - Purpose of the visit: ai\_assistant for live answers, index for search indexing, or ai\_training for model training
* `count` - Number of unique bot visits recorded
* `citations` - Visits made by AI assistants fetching the page to answer a live question
* `indexing` - Visits made by crawlers building a search index
* `training` - Visits made by crawlers collecting model training data
* `last_visit` - Timestamp of the most recent visit in the bucket
Daily visits sent to the website by AI assistants.
**Key Fields:**
* `domain` - Website the traffic was recorded on
* `date` - Day the traffic was recorded on, in UTC. Used as the incremental replication key
* `path` - Path on the website the visitor landed on
* `referral_source` - Assistant that sent the visitor, such as ChatGPT or Perplexity
* `referral_type` - Category of the referrer the visit was attributed to
* `visits` - Number of visits recorded
* `last_visit` - Timestamp of the most recent visit in the bucket