In order to connect Nekt to a database, you have to do some pre work to ensure access is granted in a secure way. So,
Stablish a peering connection between Nekt VPC and database VPC
On your AWS Console, access the VPC service and go to Peering Connection. Make sure you are logged in with the account that contains the database you want to connect with.
Select Create peering connection
Setup as requested
Give your connection a name (something as ‘Nekt <> Database’)
Select nekt-vpc ID as requester (on VPC ID field)
Choose ‘Another account’ on the Select another VPC to peer with section. Inform the Account ID.
To get the Account ID, go to the RDS service (you’ll find it searching on the AWS Console).
Click on BD Instances
Select the desired database
Copy the Account ID associated with this database.
Inform the VPC ID (Accepter)
On the desired database details, on the Connectivity and Security section, click on the VPC.
Copy the VPC ID
Click on Create peering connection. You’ll notice the status is ‘Pending acceptance’.
Go to Peering Connection again and your new peering connection should be listed, yet still pending acceptance.
On the Actions menu, click on Accept Request and confirm it.
You should edit your Peering Connection name to ‘Nekt <> Database’, to keep the pattern.
Nekt VPC
Access the created Peering Connection - that now should have the status ‘Active’ and a value on Accepter CIDRs. Copy this value, it will be the Nekt VPC IP.
In the VPC dashboard menu, go to Route Tables
In every route table with ‘nekt’ in its name, follow these steps:
Click on the Routes tab
Click on Edit routes
Click on Add route
On the ‘Destination’ column, paste the Nekt VPC IP (Accepter CIDRs previously copied)
On the ‘Target’ column, choose ‘Peering Connection’ and the Nekt <> Database option in the field that will open.
Keep ‘Status’ and ‘Propagated’ colmns as default
Target: Peering connection stablished between Nekt and database
Save changes
Database VPC
Repeat the process done for Nekt VPC, but now use the Nekt VPC IP as Destination
Search for RDS on your AWS Console and access it.
Select your database and go to Connectivity & Security.
Click on VPC security groups.
Selcting your DB security group, go to the Inbound Rules tab
Click on Edit inbound rules
Add the following inbound rule to the security group:
Type: Oracle-RDS
Source: Custom with the Nekt VPC IP as value
Add a description to better identify it. Something like ‘Nekt’
Save rule
Done! With that, you are ready to follow the next steps and connect Nekt with your database hosted on AWS through the interface of our application.
Ask Nekt to create a fixed IP in your AWS infra.
In your database provider, give access to the IP provided by Nekt.
Done! With that, you are ready to follow the next steps and connect Nekt with your database through the interface of our application.
Once you have done the pre work defined in section 0, you can inform your database accesses. In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the Oracle option from the list of connectors.
Click Next and you’ll be prompted to add your database access.
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.
Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix and make this process faster!
Sync Type: you can choose between INCREMENTAL and FULL_TABLE.
Incremental: every time the extraction happens, we’ll get only the new data - which is good if, for example, you want to keep every record ever fetched.
Full table: every time the extraction happens, we’ll get the current state of the data - which is good if, for example, you don’t want to have deleted data in your catalog.
Describe your data source for easy identification within your organization. You can inform things like what data it brings, to which team it belongs, etc.
To define your Trigger, consider how often you want data to be extracted from this source. This decision usually depends on how frequently you need the new table data updated (every day, once a week, or only at specific times).
Click Done to finalize the setup. Once completed, you’ll receive confirmation that your new source is set up!
You can view your new source on the Sources page. Now, for you to be able to see it on your Catalog, you have to wait for the pipeline to run. You can now monitor it on the Sources page to see its execution and completion. If needed, manually trigger the pipeline by clicking on the refresh icon. Once executed, your new table will appear in the Catalog section.
If you encounter any issues, reach out to us via Slack, and we’ll gladly assist you!
