The connector only reads data. It never creates, changes or deletes anything in GitLab.
Configuring GitLab as a Source
In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the GitLab option from the list of connectors. Click Next and you’ll be prompted to add your access.1. Add account access
The connector uses a GitLab access token with the read_api scope. You can use:- a personal access token: in GitLab, click your avatar > Edit profile > Access tokens > Add new token;
- a group access token (reads everything in the group and its subgroups): open the group > Settings > Access tokens;
- a project access token (reads one project): open the project > Settings > Access tokens.
- Access Token: the token you created. It is stored encrypted.
- GitLab URL: keep
https://gitlab.comfor GitLab.com. For a self-managed GitLab, use the address you open in the browser (for examplehttps://gitlab.mycompany.com). - Start Date: the earliest date read. Issues, merge requests, pipelines and epics updated since this date, and commits made since this date, are synced on the first run.
- Groups (advanced, optional): the groups to sync, one per line, as their path (for example
my-groupormy-group/my-subgroup, as in the group’s URL) or numeric ID. Each group’s subgroups and projects are included. - Projects (advanced, optional): extra projects to sync, one per line, as their path (for example
my-group/my-project) or numeric ID.
2. Select streams
Choose which data streams you want to sync. For faster extractions, select only the streams that are relevant to your analysis. You can select entire groups of streams or pick specific ones.Tip: The stream can be found more easily by typing its name.Select the streams and click Next.
3. Configure data streams
Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table (which will effectively contain the fetched data) and the type of sync.- Layer: choose between the existing layers on your catalog. This is where you will find your new extracted tables as the extraction runs successfully.
- Folder: a folder can be created inside the selected layer to group all tables being created from this new data source.
- Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix to all tables at once and make this process faster!
- Sync Type: keep the suggested type. Issues, merge requests, comments, approvals, commits, pipelines, jobs and epics are Incremental: each sync reads only what changed since the previous one, project by project. Groups, projects, members, users, labels, milestones, branches, tags and releases are Full table: each sync reads them whole and replaces the table.
4. Configure data source
Describe your data source for easy identification within your organization, not exceeding 140 characters. To define your Trigger, consider how often you want data to be extracted from this source. Engineering teams usually sync every few hours or daily. Optionally, you can configure Delta Log Retention and determine for how long we should store old states of this table as it gets updated. Read more about this resource here. Once you are ready, click Next to finalize the setup.5. Check your new source
You can view your new source on the Sources page. If needed, manually trigger the source extraction by clicking on the arrow button. Once executed, your data will appear in your Catalog.Streams and Fields
Available streams
The table below lists every stream, its slug (the exact identifier to pass when creating the source via API) and a short description.
To join the tables:
projects.namespace_id → groups.id; issues.project_id, merge_requests.project_id, commits.project_id, pipelines.project_id → projects.id; issue_notes.issue_id → issues.id; merge_request_notes.merge_request_id and merge_request_approvals.merge_request_id → merge_requests.id; jobs.pipeline_id → pipelines.id; author_id, user_id and member id → users.id.
What each table needs
Projects where a feature is turned off are simply not read for that feature’s tables.
Fields by stream
Fields marked String (JSON) hold a nested object or list as JSON text; use your warehouse’s JSON functions to read them. Every table also hasextra_fields: fields GitLab adds in the future land there (as JSON) until they get their own column.
Groups
Groups
The groups in the source’s scope, with their subgroups.
Primary key:
id. Sync: Full table.Projects
Projects
The projects in the source’s scope.
Primary key:
id. Sync: Full table.Group members
Group members
Direct members of each group, with their role.
Primary key:
group_id, id. Sync: Full table.Project members
Project members
Direct members of each project, with their role (members inherited from a group are in group_members).
Primary key:
project_id, id. Sync: Full table.Users
Users
Every user who is a direct member of a group or project of the source.
Primary key:
id. Sync: Full table.Labels
Labels
Labels of the groups and projects (each label once, with the group or project that owns it).
Primary key:
id. Sync: Full table.Milestones
Milestones
Milestones of the groups and projects.
Primary key:
id. Sync: Full table.Branches
Branches
Branches of each repository, with their last commit.
Primary key:
project_id, name. Sync: Full table.Releases
Releases
Releases of each project.
Primary key:
project_id, tag_name. Sync: Full table.Issues
Issues
Issues of every project, open and closed.
Primary key:
id. Sync: Incremental by updated_at.Issue notes
Issue notes
Comments and system notes of the issues.
Primary key:
id. Sync: Incremental by issue_updated_at.Merge requests
Merge requests
Merge requests of every project, in every state.
Primary key:
id. Sync: Incremental by updated_at.Merge request notes
Merge request notes
Comments, review comments and system notes of the merge requests.
Primary key:
id. Sync: Incremental by merge_request_updated_at.Merge request approvals
Merge request approvals
Approval state of each merge request: who approved and how many approvals are still missing.
Primary key:
merge_request_id. Sync: Incremental by merge_request_updated_at.Commits
Commits
Commits of every branch of each repository.
Primary key:
project_id, id. Sync: Incremental by committed_date.Pipelines
Pipelines
CI/CD pipelines of every project.
Primary key:
id. Sync: Incremental by updated_at.Jobs
Jobs
Jobs of each pipeline, including retried ones.
Primary key:
id. Sync: Incremental by pipeline_updated_at.Epics
Epics
Epics of every group (GitLab Premium and Ultimate only).
Primary key:
id. Sync: Incremental by updated_at.Implementation Notes
Data Quality Considerations
- Incremental tables are read project by project. Each project keeps its own sync point. A project added to the source later (or a new project in a synced group) is read from the Start Date; the others continue from where they stopped, re-reading the last 5 minutes to catch late changes. If a sync fails, no sync point moves: the next sync reads the same period again, and rows are merged by their primary key, so nothing is duplicated.
- Comments, approvals and jobs follow their parent. Comments are read for the issues and merge requests updated since the last sync (adding a comment updates the issue or merge request), and jobs for the pipelines updated since the last sync. Approvals are read for the merge requests updated since the last sync and for every open merge request.
- Cross-reference notes can arrive late. Automatic notes such as “mentioned in !123” or “mentioned in commit abc” do not update their issue or merge request in GitLab, so they only arrive when that issue or merge request changes again, or with a full sync. Comments, replies, review comments and the other system notes arrive normally.
- Deleted records stay in incremental tables. A deleted comment, issue, merge request, pipeline, job or epic is not removed from its table until a full sync. Full table tables (groups, projects, members, branches, tags and the others) reflect deletions on every sync.
- Jobs keep the values of their last read until their pipeline changes again; for example, artifacts that expire later still appear (
artifacts_expire_attells when they expire). Trigger jobs that start other pipelines (bridges) are not included. - Commits are read by commit date. GitLab can only filter commits by the date they were made, and a commit can be pushed days after it was made. Each sync re-reads the last 14 days of every repository; commits pushed more than 14 days after they were made only arrive with a full sync.
- Commits include every branch and GitLab’s hidden references, so commits of open merge requests (also from forks) and of branches GitLab still keeps after deletion appear in
commits. Branches deleted or rewritten during a sync do not make the connector skip other commits. - Full table tables are never half updated. The connector reads the whole table before writing it. If rows are removed while the list is read, it is read again. A table that suddenly comes back empty is not emptied: that table fails in the sync. If the data really was removed, run a full sync of the source to accept it.
- Merge requests and issues edited during a sync are not missed. The connector reads them in order of their last update and never relies on page numbers, so a change in the middle of a sync only moves a record to the end of the list.
- Projects that leave the source (removed from the configuration, or a feature turned off) are read again from the Start Date if they come back.
- Users are the members of the synced groups and projects. Emails are only filled for GitLab administrators or SAML/enterprise users, as GitLab returns them.
- Epics need GitLab Premium or Ultimate. Groups without them are named once in the sync log and skipped; if no group has epics, the epics table is shown as failed and the other tables keep syncing. Deselect it if your plan has no epics.
- Permissions. If GitLab refuses one table of a project the token should read (for example, the user’s role is too low there), that table is shown as failed with the project’s name, its data is kept as it was, and the other tables keep syncing. A project deleted during a sync is simply left out.
API Limits & Performance
- GitLab.com allows 2,000 API requests per minute per user; self-managed GitLab sets its own limits. The connector follows GitLab’s rate-limit headers, slows down before the limit, and when GitLab asks it to wait it waits and says so in the sync log. Other integrations that use the same GitLab user share that limit.
- Lists are read 100 records per request. Comments, approvals and jobs need one request per issue, merge request or pipeline that changed, so the first sync of a large GitLab can take hours; later syncs read only what changed. Approvals also cost one request per open merge request on every sync; deselect
merge_request_approvalsif you do not need it. - Choose a recent Start Date for a faster first sync if you do not need the full history.
Troubleshooting
Skills for agents
Download GitLab skills file
GitLab connector documentation as plain markdown, for use in AI agent contexts.