
1. Add your ClickHouse access
- In the Destinations tab, click on the “Add destination” button located on the top right of your screen. Then, select the ClickHouse option from the list of connectors.
-
Click Next and you’ll be prompted to add your access:
- Host: The host address of your ClickHouse database, without the protocol (
http://orhttps://). For ClickHouse Cloud, copy it from your service’s Connect screen. - Port: The connection port. The default secure (HTTPS) port is
8443; your port might be different. - Database: The name of a database in your ClickHouse account.
- Schema: (Optional) The database to write to. Defaults to the connection database.
- User: The user name for accessing your ClickHouse database.
- Password: The password used for accessing your ClickHouse database.
- Host: The host address of your ClickHouse database, without the protocol (
-
Decide about the Load method:
- Append: Always inserts new rows (uses a
MergeTreetable engine). - Upsert: Inserts or updates based on the table’s primary keys, keeping the latest row per key (uses a
ReplacingMergeTreeengine ordered by the primary keys). - Overwrite: Clears the table before loading.
- Append: Always inserts new rows (uses a
- Click Next.
You must ensure your database accepts incoming connections from Nekt, and the database user has write permissions in order to write data.
In ClickHouse, deduplication for the Upsert method is handled by the table engine and applied during background merges. Querying with
SELECT ... FINAL (or running OPTIMIZE TABLE ... FINAL) returns the deduplicated result. The table engine is set when the table is created, so changing the load method on an existing table requires recreating it.2. Select your data to send
-
The next step is letting us know which data you want to send to the database. Select the layer and then the desired table.
Tip: The table can be found more easily by typing its name.
- Click Next.
3. Map your data to send
- Give a name to be given to each table in the destination.
- For each table, select the column that will work as a primary key. This is required for the Upsert load method, as it defines the ordering key used for deduplication.
- By clicking on the fields mapping button, you will see a list of columns and their correspondent names in the destination. You can modify it if you want.
- Click Next.
4. Configure your ClickHouse data destination
- Describe your destination for easy identification within your organization. You can inform things like what data it sends, to which team it belongs, etc.
- To define your Trigger, consider how often you want data to be sent to this destination. This decision usually depends on how frequently you need the new table data updated (every day, once a week, only at specific times, etc.).
- Click Done.
Check your new destination!
- Once completed, you’ll receive confirmation that your new destination is set up!
- You can view your new destination on the Destinations page. Now, for you to be able to see it on your ClickHouse database, you have to wait for the pipeline to run. You can monitor it on the Destinations page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your data should be seen on the defined ClickHouse location.
If you encounter any issues, reach out to us via Slack, and we’ll gladly assist you!