Skip to main content
JSON Content refers to data stored in JSON (JavaScript Object Notation) format, which is a lightweight data interchange format. It’s commonly used for APIs, configuration files, and data storage, providing a human-readable way to represent structured data.

​
Configuring JSON Content as a Source

In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the JSON Content option from the list of connectors. Click Next and you’ll be prompted to add your access.

​
1. Add connection details

You’ll need to provide the connection details to access your JSON data. The following configurations are available:
  • Website URL: The URL of the website to extract data from. It should be a single-page URL containing a JSON payload. This field is required.
  • Records JSON Path: The JSON path to identify where in the JSON payload the data should be extracted. Default is $. This field is required.
  • Number of samples to generate schema: (Advanced setting) The maximum number of records to sample for schema generation. Defines how many items the connector reads to figure out the table columns. You should set a higher number of samples for highly variable payloads. If the records are consistent across the payload, the default (100) is usually sufficient.
Understanding JSON Path: As an example to understand how JSON path works, consider the following payload obtained from the Website URL:
If you want to extract all listings from it, your JSON path should be $.listings[*]. This way, each listing inside the array would represent a row in the extracted table, such as: For more information about how JSON path works, please check the JsonPath documentation. Once you’re done, click Next.

​
2. Select streams

The JSON Content connector exposes a single dynamic stream based on your configured JSON Path. Choose whether you want to sync it. Select the stream and click Next.

​
3. Configure data streams

Customize how you want your data to appear in your catalog. Select a name for the table (which will contain the fetched data) and the type of sync.
  • Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix and make this process faster!
  • Sync Type: you can choose between INCREMENTAL and FULL_TABLE.
    • Incremental: every time the extraction happens, we’ll get only the new data - which is good if, for example, you want to keep every record ever fetched.
    • Full table: every time the extraction happens, we’ll get the current state of the data - which is good if, for example, you don’t want to have deleted data in your catalog.
Once you are done configuring, click Next.

​
4. Configure data source

Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc. To define your Trigger, consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).

​
5. Check your new source

Click Next to finalize the setup. Once completed, you’ll receive confirmation that your new source is set up! You can view your new source on the Sources page. Now, for you to be able to see it on your Catalog, you have to wait for the pipeline to run. You can monitor it on the Sources page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
If you encounter any issues, reach out to us via Slack, and we’ll gladly assist you!

​
Streams and Fields

The JSON Content connector extracts data dynamically based on the payload returned by your URL and the specific JSON path configured.
Because the JSON Content connector extracts data from an arbitrary URL, the resulting fields and data types depend entirely on the target JSON payload.How it works:
  • Dynamic Schema Discovery: The connector reads a sample of records up to the specified “Number of samples to generate schema” parameter (default 100). It automatically infers the column names and data types (e.g., String, Integer, Boolean) from the JSON keys it encounters.
  • JSON Path Filtering: The fields exposed in the stream are the direct keys of the objects returned by your Records JSON Path query. For example, if your JSON Path targets an array of user objects, the resulting table will have columns for id, name, email, etc., exactly matching the keys inside each user object.
Implementation Notes:
  • Ensure your JSON path extracts an array of objects or an object containing a consistent set of keys to correctly populate the table format.
  • If your JSON payload is highly variable or contains sparse fields, consider increasing the Number of samples to generate schema parameter in the source configuration to ensure all possible fields are discovered.

​
Skills for agents

Download JSON Content skills file

JSON Content connector documentation as plain markdown, for use in AI agent contexts.