> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nekt.com/llms.txt
> Use this file to discover all available pages before exploring further.

# GitHub as a data source

> Bring repository, pull request, and commit data from GitHub to Nekt.

GitHub is a software development platform for source control, collaboration, and code review. The Nekt GitHub connector uses the GitHub REST API to extract repository metadata, pull requests, and commits into your Catalog.

<img width="200" src="https://mintcdn.com/nekt/z4W2Fwx-d8vrmHjC/assets/logo/logo-github.png?fit=max&auto=format&n=z4W2Fwx-d8vrmHjC&q=85&s=e61d5faa9a0319882ef7a72e6e9a5f38" data-path="assets/logo/logo-github.png" />

## Configuring GitHub as a Source

In the [Sources](https://app.nekt.ai/sources) tab, click on the "Add source" button located on the top right of your screen. Then, select the **GitHub** option from the list of connectors.

Click **Next** and you'll be prompted to add your access.

### 1. Add account access

You'll need a GitHub Personal Access Token (classic or fine-grained) with permission to read the repositories you want to sync.

The following configurations are available:

* **Personal Access Token**: Your GitHub Personal Access Token (classic or fine-grained) with repo scope. This field is required and stored securely.

* **Repositories**: Optional list of repositories in `owner/repo` format (one per line). If provided, only these repositories are synced. If left empty, the connector syncs all repositories accessible by the token.

* **Start Date**: Optional starting point used by incremental commit syncs. Only sync records created or updated after this date.

Once you're done, click **Next**.

### 2. Select streams

Choose which data streams you want to sync:

* **repositories**
* **pull\_requests**
* **commits**

For faster extractions, select only the streams you need.

Select the streams and click **Next**.

### 3. Configure data streams

Customize how you want your data to appear in your catalog. Select the desired layer where the data will be placed, a folder to organize it inside the layer, a name for each table, and the type of sync.

* **Layer**: Choose the layer where extracted GitHub tables will be created.
* **Folder**: Optionally group all GitHub tables inside a folder.
* **Table name**: A default name is suggested, but you can customize it. You can also add a prefix to all tables.
* **Sync Type**: Choose between **INCREMENTAL** and **FULL\_TABLE**.
  * **Incremental**: Recommended for `commits`, using `committed_at` as the replication key.
  * **Full table**: Useful for one-off backfills or full refreshes.

Once you are done configuring, click **Next**.

### 4. Configure data source

Describe your data source for easy identification within your organization, not exceeding 140 characters.

To define your [Trigger](https://docs.nekt.com/runs/scheduling-and-triggers), consider how often your repositories change:

* **Hourly / every few hours** for active engineering analytics.
* **Daily** for standard operational reporting.
* **Weekly** for low-change repositories.

Optionally, you can define:

* **Delta Log Retention**: How long Nekt keeps previous table states. See [Resource control](https://docs.nekt.com/get-started/core-concepts/resource-control).
* **Additional [Full Sync](https://docs.nekt.com/get-started/core-concepts/types-of-sync#additional-full-sync)**: Periodic full syncs in addition to incrementals.

When you are ready, click **Next** to finalize the setup.

### 5. Check your new source

You can view your new source on the [Sources](https://app.nekt.ai/sources) page. If needed, manually trigger the extraction by clicking on the arrow button. Once a run completes successfully, your data appears in the Catalog.

<Warning>You need at least one successful source run to see the tables in your [Catalog](https://app.nekt.ai/catalog).</Warning>

# Streams and Fields

Below you'll find the available GitHub streams and their core fields.

<AccordionGroup>
  <Accordion title="Repositories">
    Repository metadata for all repositories accessible by the token (or only the configured list in `repositories`).

    **Key fields:**

    | Field               | Type     | Description                                       |
    | :------------------ | :------- | :------------------------------------------------ |
    | `id`                | Integer  | Repository numeric ID (primary key)               |
    | `full_name`         | String   | Repository name in `owner/repo` format            |
    | `private`           | Boolean  | Indicates whether the repository is private       |
    | `visibility`        | String   | Repository visibility (`public`, `private`, etc.) |
    | `default_branch`    | String   | Default branch                                    |
    | `language`          | String   | Primary detected language                         |
    | `stargazers_count`  | Integer  | Number of stars                                   |
    | `forks_count`       | Integer  | Number of forks                                   |
    | `open_issues_count` | Integer  | Number of open issues                             |
    | `created_at`        | DateTime | Timestamp when the repository was created         |
    | `updated_at`        | DateTime | Timestamp when the repository was last updated    |
    | `pushed_at`         | DateTime | Timestamp when the repository was last pushed to  |

    **Notes:**

    * Primary key: `id`
    * Replication: full-table style (no replication key)
    * Child context: each repository emits `owner` and `repo` context used by `pull_requests` and `commits`
  </Accordion>

  <Accordion title="Pull Requests">
    Pull requests for each repository. The connector fetches all pull request states (`open`, `closed`, and `merged`).

    **Key fields:**

    | Field             | Type     | Description                               |
    | :---------------- | :------- | :---------------------------------------- |
    | `id`              | Integer  | Pull request ID (primary key)             |
    | `number`          | Integer  | Pull request number inside the repository |
    | `title`           | String   | Pull request title                        |
    | `body`            | String   | Pull request description/body             |
    | `state`           | String   | Current state (`open`, `closed`)          |
    | `draft`           | Boolean  | Indicates if it is a draft PR             |
    | `locked`          | Boolean  | Indicates if the PR is locked             |
    | `user`            | Object   | Pull request author                       |
    | `head`            | Object   | Source branch metadata                    |
    | `base`            | Object   | Target branch metadata                    |
    | `merged_at`       | DateTime | Timestamp when merged                     |
    | `closed_at`       | DateTime | Timestamp when closed                     |
    | `created_at`      | DateTime | Timestamp when created                    |
    | `updated_at`      | DateTime | Timestamp when last updated               |
    | `additions`       | Integer  | Lines added                               |
    | `deletions`       | Integer  | Lines deleted                             |
    | `changed_files`   | Integer  | Number of files changed                   |
    | `comments`        | Integer  | Number of comments                        |
    | `review_comments` | Integer  | Number of review comments                 |
    | `commits`         | Integer  | Number of commits in the PR               |
    | `_sdc_repository` | String   | Repository context in `owner/repo` format |

    **Notes:**

    * Primary key: `id`
    * Replication: full-table style (no replication key)
    * Includes repository context fields (`owner`, `repo`, `_sdc_repository`) for easier joins
  </Accordion>

  <Accordion title="Commits">
    Commits for each repository. This stream supports incremental sync using commit timestamp.

    **Key fields:**

    | Field                 | Type     | Description                                                    |
    | :-------------------- | :------- | :------------------------------------------------------------- |
    | `sha`                 | String   | Git SHA hash value of the object (primary key).                |
    | `node_id`             | String   | GraphQL node identifier of the record.                         |
    | `html_url`            | String   | Web URL for this resource in GitHub.                           |
    | `url`                 | String   | API URL for this resource.                                     |
    | `commit.message`      | String   | Commit message.                                                |
    | `commit.author`       | Object   | Author metadata embedded in the commit (name, email, date).    |
    | `commit.committer`    | Object   | Committer metadata embedded in the commit (name, email, date). |
    | `commit.tree`         | Object   | Git tree object referenced by this commit.                     |
    | `commit.verification` | Object   | Verification details (verified, reason, signature, payload).   |
    | `author`              | Object   | GitHub user object for the author (when available).            |
    | `committer`           | Object   | GitHub user object for the committer (when available).         |
    | `parents`             | Array    | Parent commit references.                                      |
    | `stats.additions`     | Integer  | Lines added.                                                   |
    | `stats.deletions`     | Integer  | Lines deleted.                                                 |
    | `stats.total`         | Integer  | Total lines changed.                                           |
    | `committed_at`        | DateTime | Replication key (derived from `commit.committer.date`).        |
    | `_sdc_repository`     | String   | Repository context in `owner/repo` format.                     |

    **Notes:**

    * Primary key: `sha`
    * Replication key: `committed_at`
    * Incremental sync sends `since` to GitHub API based on state bookmark (or `start_date` when state is not available)
  </Accordion>
</AccordionGroup>

# Data Model

The connector follows a repository-centered model:

```mermaid theme={null}
graph TD;
    Repositories["repositories"];
    PullRequests["pull_requests"];
    Commits["commits"];

    Repositories -- "owner + repo context" --> PullRequests;
    Repositories -- "owner + repo context" --> Commits;
```

# Use Cases for Data Analysis

This section includes practical SQL examples you can run in [Explorer](https://app.nekt.ai/explorer).

### 1. Pull Request throughput by repository

Measure how many pull requests are created, closed, and merged by repository.

<Accordion title="SQL query">
  <Tabs>
    <Tab title="AWS">
      ```sql theme={null}
      SELECT
         _sdc_repository AS repository,
         COUNT(*) AS total_prs,
         SUM(CASE WHEN state = 'open' THEN 1 ELSE 0 END) AS open_prs,
         SUM(CASE WHEN closed_at IS NOT NULL THEN 1 ELSE 0 END) AS closed_prs,
         SUM(CASE WHEN merged_at IS NOT NULL THEN 1 ELSE 0 END) AS merged_prs
      FROM
         nekt_raw.github_pull_requests
      GROUP BY
         1
      ORDER BY
         total_prs DESC;
      ```
    </Tab>

    <Tab title="GCP">
      ```sql theme={null}
      SELECT
         _sdc_repository AS repository,
         COUNT(*) AS total_prs,
         SUM(CASE WHEN state = 'open' THEN 1 ELSE 0 END) AS open_prs,
         SUM(CASE WHEN closed_at IS NOT NULL THEN 1 ELSE 0 END) AS closed_prs,
         SUM(CASE WHEN merged_at IS NOT NULL THEN 1 ELSE 0 END) AS merged_prs
      FROM
         `nekt_raw.github_pull_requests`
      GROUP BY
         1
      ORDER BY
         total_prs DESC;
      ```
    </Tab>
  </Tabs>
</Accordion>

### 2. Commit activity in the last 30 days

Track commit volume and active contributors by repository.

<Accordion title="SQL query">
  <Tabs>
    <Tab title="AWS">
      ```sql theme={null}
      SELECT
         _sdc_repository AS repository,
         COUNT(*) AS commits_last_30d,
         COUNT(DISTINCT COALESCE(author.login, commit.author.email)) AS active_authors
      FROM
         nekt_raw.github_commits
      WHERE
         CAST(committed_at AS timestamp) >= current_timestamp - interval '30' day
      GROUP BY
         1
      ORDER BY
         commits_last_30d DESC;
      ```
    </Tab>

    <Tab title="GCP">
      ```sql theme={null}
      SELECT
         _sdc_repository AS repository,
         COUNT(*) AS commits_last_30d,
         COUNT(DISTINCT COALESCE(author.login, JSON_VALUE(TO_JSON_STRING(commit), '$.author.email'))) AS active_authors
      FROM
         `nekt_raw.github_commits`
      WHERE
         TIMESTAMP(committed_at) >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
      GROUP BY
         1
      ORDER BY
         commits_last_30d DESC;
      ```
    </Tab>
  </Tabs>
</Accordion>

## Skills for agents

<Snippet file="agent-skills-intro.mdx" />

<Card title="Download GitHub skills file" icon="wand-magic-sparkles" href="/sources/github.md">
  GitHub connector documentation as plain markdown, for use in AI agent contexts.
</Card>