Bring data from Microsoft SQL Server to your Lakehouse.
SQL Server is Microsoft’s relational database management system designed for enterprise applications and data warehousing. It provides robust data storage, advanced analytics, and business intelligence capabilities with features for high availability, security, and performance optimization.
In order to connect Nekt to a database, you have to do some pre work to ensure access is granted in a secure way. So, make sure to check this guide before proceeding.
Once you have done the pre work defined in section 0, you can inform your database accesses. In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the Postgres option from the list of connectors.
Click Next and you’ll be prompted to add your database access.
You must ensure your database accepts incoming connections from Nekt, and the database user has SELECT permissions in order to fetch data.
Choose which data streams you want to sync - you can select all streams or pick specific ones that matter most to you. We recommend selecting only the tables that are useful for you from you. You can add new tables at anytime after the source is created.
Tip: The stream can be found more easily by typing its name.
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix and make this process faster!
Sync Type: you can choose between INCREMENTAL and FULL_TABLE.
Incremental: every time the extraction happens, we’ll get only the new data - which is good if, for example, you want to keep every record ever fetched.
Full table: every time the extraction happens, we’ll get the current state of the data - which is good if, for example, you don’t want to have deleted data in your catalog.
Describe your data source for easy identification within your organization, not exceeding 140 characters.To define your Trigger, consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).Optionally, you can determine when to execute a full sync. This will complement the incremental data extractions, ensuring that your data is completely synchronized with your source every once in a while.Once you are ready, click Next to finalize the setup.
You can view your new source on the Sources page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your Catalog, you need at least one successful source run.