Skip to main content
GitLab is a DevOps platform for source code, code review, issue tracking and CI/CD. This connector reads your GitLab (GitLab.com or a self-managed GitLab) through the GitLab REST API and brings your groups and projects, their members, issues and merge requests with their comments and approvals, commits, branches, tags, releases, CI/CD pipelines and jobs, labels, milestones and epics.
The connector only reads data. It never creates, changes or deletes anything in GitLab.

Configuring GitLab as a Source

In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the GitLab option from the list of connectors. Click Next and you’ll be prompted to add your access.

1. Add account access

The connector uses a GitLab access token with the read_api scope. You can use:
  • a personal access token: in GitLab, click your avatar > Edit profile > Access tokens > Add new token;
  • a group access token (reads everything in the group and its subgroups): open the group > Settings > Access tokens;
  • a project access token (reads one project): open the project > Settings > Access tokens.
Select the read_api scope and, for group and project tokens, at least the Reporter role. Set an expiration date you will remember: when the token expires, the source stops until you paste a new one. The following configurations are available:
  • Access Token: the token you created. It is stored encrypted.
  • GitLab URL: keep https://gitlab.com for GitLab.com. For a self-managed GitLab, use the address you open in the browser (for example https://gitlab.mycompany.com).
  • Start Date: the earliest date read. Issues, merge requests, pipelines and epics updated since this date, and commits made since this date, are synced on the first run.
  • Groups (advanced, optional): the groups to sync, one per line, as their path (for example my-group or my-group/my-subgroup, as in the group’s URL) or numeric ID. Each group’s subgroups and projects are included.
  • Projects (advanced, optional): extra projects to sync, one per line, as their path (for example my-group/my-project) or numeric ID.
If Groups and Projects are empty, the connector syncs every group the token’s user is a member of and every project where it has at least the Reporter role. It never reads the public projects of other people on GitLab.com.
A self-managed GitLab must be reachable from the internet over HTTPS. If your GitLab only accepts known addresses, allow Nekt’s addresses (ask Nekt support for them).
A token whose user has the Guest role cannot read the code, pipelines or releases of private projects. Use a user (or group token) with at least the Reporter role in the projects you want to sync.
Once you’re done, click Next.

2. Select streams

Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.
Tip: The stream can be found more easily by typing its name.
Select the streams and click Next.

3. Configure data streams

Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.
  • Layer: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
  • Folder: a folder can be created inside the selected layer to group all tables being created from this new data source.
  • Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix to all tables at once and make this process faster!
  • Sync Type: keep the suggested type. Issues, merge requests, comments, approvals, commits, pipelines, jobs and epics are Incremental: each sync reads only what changed since the previous one, project by project. Groups, projects, members, users, labels, milestones, branches, tags and releases are Full table: each sync reads them whole and replaces the table.
Once you are done configuring, click Next.

4. Configure data source

Describe your data source for easy identification within your organization, not exceeding 140 characters. To define your Trigger, consider how often you want data to be extracted from this source. Engineering teams usually sync every few hours or daily. Optionally, you can configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource here. Once you are ready, click Next to finalize the setup.

5. Check your new source

You can view your new source on the Sources page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.
For you to be able to see it on your Catalog, you need at least one successful source run.

Streams and Fields

Available streams

The table below lists every stream, its slug (the exact identifier to pass when creating the source via API) and a short description. To join the tables: projects.namespace_id → groups.id; issues.project_id, merge_requests.project_id, commits.project_id, pipelines.project_id → projects.id; issue_notes.issue_id → issues.id; merge_request_notes.merge_request_id and merge_request_approvals.merge_request_id → merge_requests.id; jobs.pipeline_id → pipelines.id; author_id, user_id and member id → users.id.

What each table needs

Projects where a feature is turned off are simply not read for that feature’s tables.

Fields by stream

Fields marked String (JSON) hold a nested object or list as JSON text; use your warehouse’s JSON functions to read them. Every table also has extra_fields: fields GitLab adds in the future land there (as JSON) until they get their own column.
The groups in the source’s scope, with their subgroups.Primary key: id. Sync: Full table.
The projects in the source’s scope.Primary key: id. Sync: Full table.
Direct members of each group, with their role.Primary key: group_id, id. Sync: Full table.
Direct members of each project, with their role (members inherited from a group are in group_members).Primary key: project_id, id. Sync: Full table.
Every user who is a direct member of a group or project of the source.Primary key: id. Sync: Full table.
Labels of the groups and projects (each label once, with the group or project that owns it).Primary key: id. Sync: Full table.
Milestones of the groups and projects.Primary key: id. Sync: Full table.
Branches of each repository, with their last commit.Primary key: project_id, name. Sync: Full table.
Tags of each repository.Primary key: project_id, name. Sync: Full table.
Releases of each project.Primary key: project_id, tag_name. Sync: Full table.
Issues of every project, open and closed.Primary key: id. Sync: Incremental by updated_at.
Comments and system notes of the issues.Primary key: id. Sync: Incremental by issue_updated_at.
Merge requests of every project, in every state.Primary key: id. Sync: Incremental by updated_at.
Comments, review comments and system notes of the merge requests.Primary key: id. Sync: Incremental by merge_request_updated_at.
Approval state of each merge request: who approved and how many approvals are still missing.Primary key: merge_request_id. Sync: Incremental by merge_request_updated_at.
Commits of every branch of each repository.Primary key: project_id, id. Sync: Incremental by committed_date.
CI/CD pipelines of every project.Primary key: id. Sync: Incremental by updated_at.
Jobs of each pipeline, including retried ones.Primary key: id. Sync: Incremental by pipeline_updated_at.
Epics of every group (GitLab Premium and Ultimate only).Primary key: id. Sync: Incremental by updated_at.

Implementation Notes

Data Quality Considerations

  • Incremental tables are read project by project. Each project keeps its own sync point. A project added to the source later (or a new project in a synced group) is read from the Start Date; the others continue from where they stopped, re-reading the last 5 minutes to catch late changes. If a sync fails, no sync point moves: the next sync reads the same period again, and rows are merged by their primary key, so nothing is duplicated.
  • Comments, approvals and jobs follow their parent. Comments are read for the issues and merge requests updated since the last sync (adding a comment updates the issue or merge request), and jobs for the pipelines updated since the last sync. Approvals are read for the merge requests updated since the last sync and for every open merge request.
  • Cross-reference notes can arrive late. Automatic notes such as “mentioned in !123” or “mentioned in commit abc” do not update their issue or merge request in GitLab, so they only arrive when that issue or merge request changes again, or with a full sync. Comments, replies, review comments and the other system notes arrive normally.
  • Deleted records stay in incremental tables. A deleted comment, issue, merge request, pipeline, job or epic is not removed from its table until a full sync. Full table tables (groups, projects, members, branches, tags and the others) reflect deletions on every sync.
  • Jobs keep the values of their last read until their pipeline changes again; for example, artifacts that expire later still appear (artifacts_expire_at tells when they expire). Trigger jobs that start other pipelines (bridges) are not included.
  • Commits are read by commit date. GitLab can only filter commits by the date they were made, and a commit can be pushed days after it was made. Each sync re-reads the last 14 days of every repository; commits pushed more than 14 days after they were made only arrive with a full sync.
  • Commits include every branch and GitLab’s hidden references, so commits of open merge requests (also from forks) and of branches GitLab still keeps after deletion appear in commits. Branches deleted or rewritten during a sync do not make the connector skip other commits.
  • Full table tables are never half updated. The connector reads the whole table before writing it. If rows are removed while the list is read, it is read again. A table that suddenly comes back empty is not emptied: that table fails in the sync. If the data really was removed, run a full sync of the source to accept it.
  • Merge requests and issues edited during a sync are not missed. The connector reads them in order of their last update and never relies on page numbers, so a change in the middle of a sync only moves a record to the end of the list.
  • Projects that leave the source (removed from the configuration, or a feature turned off) are read again from the Start Date if they come back.
  • Users are the members of the synced groups and projects. Emails are only filled for GitLab administrators or SAML/enterprise users, as GitLab returns them.
  • Epics need GitLab Premium or Ultimate. Groups without them are named once in the sync log and skipped; if no group has epics, the epics table is shown as failed and the other tables keep syncing. Deselect it if your plan has no epics.
  • Permissions. If GitLab refuses one table of a project the token should read (for example, the user’s role is too low there), that table is shown as failed with the project’s name, its data is kept as it was, and the other tables keep syncing. A project deleted during a sync is simply left out.

API Limits & Performance

  • GitLab.com allows 2,000 API requests per minute per user; self-managed GitLab sets its own limits. The connector follows GitLab’s rate-limit headers, slows down before the limit, and when GitLab asks it to wait it waits and says so in the sync log. Other integrations that use the same GitLab user share that limit.
  • Lists are read 100 records per request. Comments, approvals and jobs need one request per issue, merge request or pipeline that changed, so the first sync of a large GitLab can take hours; later syncs read only what changed. Approvals also cost one request per open merge request on every sync; deselect merge_request_approvals if you do not need it.
  • Choose a recent Start Date for a faster first sync if you do not need the full history.

Troubleshooting

Skills for agents

Download GitLab skills file

GitLab connector documentation as plain markdown, for use in AI agent contexts.