> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nekt.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Dependencies

> Install Python packages, private packages, and Spark jars in your notebooks.

Every notebook runs with the Nekt SDK and its runtime's preinstalled libraries. Any other Python package your code imports must be added as a dependency. Nekt installs the dependencies before your code runs.

You add dependencies in the **Define dependencies** step when you create a notebook, or in the **Dependencies** section when you edit one. Each line holds one dependency.

## Public packages

Add packages from [PyPI](https://pypi.org) with their name and version:

```text theme={null}
holidays==0.48
openpyxl==3.1.5
```

Pin an exact version (`==`) so every run installs the same code. A package without a pinned version is installed once and reused until the notebook's dependencies change, so runs don't pick up its new releases. When you save, Nekt checks that each package exists on PyPI and rejects the ones it can't find.

## Private packages

To share code across notebooks, put it in a Python package in a private Git repository and install it as a dependency. Private repositories need credentials, which you store as a [Secret](https://docs.nekt.com/workspaces/secrets.md) and reference by key instead of writing them into the dependency.

Reference a secret as `${KEY}`, where `KEY` is the secret's key:

```text theme={null}
git+https://${GITHUB_TOKEN}@github.com/acme/data-utils.git@v1.0.0
```

Credentials must always come from a secret. A dependency with a token or password written into its URL is rejected when you save the notebook, and when it is synced from GitHub.

**1. Create a token for the repository**

Use a token that can only read the repository. On GitHub, create a [fine-grained personal access token](https://github.com/settings/personal-access-tokens/new) with access to that repository only and the **Contents: Read-only** repository permission.

**2. Store the token as a secret**

In [Secrets](https://docs.nekt.com/workspaces/secrets.md), create a secret whose value is the token, for example with the key `GITHUB_TOKEN`. A key used in a reference may contain only letters, digits, and underscores, and must not start with a digit.

**3. Add the dependency**

Add the dependency in the `git+https://${KEY}@host/owner/repository.git@ref` format. Set `ref` to a tag, branch, or commit.

**4. Save the notebook**

Nekt checks that the secret exists, that you can read it, and that the repository and `ref` can be reached with its value.

Once the dependency installs, your notebook imports the package like any other:

```python theme={null}
from data_utils import normalize_column_names
```

### Branches, tags, and commits

What `ref` points at decides when the notebook picks up new code:

| `ref` | Example | What each run installs |
| :- | :- | :- |
| A commit | `@4f2c9e1...` | Always that commit. |
| A tag | `@v1.0.0` | The commit the tag points at. |
| A branch | `@main` | The latest commit on the branch. |
| None | `utils.git` | The latest commit on the repository's default branch. |

For a branch, a tag, or no `ref`, Nekt checks the repository for a new commit when each run starts. A run after a push installs the new commit. On a PySpark notebook, a new commit makes the run rebuild the notebook's image before it starts, which adds a few minutes to that run only. PyPI packages are not checked: they install at the versions listed.

The run log shows the check as one line when it starts and one summary when it ends:

```text theme={null}
Dependencies: checking 1 package installed from a git repository for new commits. PyPI packages are not checked: they install at the versions listed.
Dependencies: checked 1 package; this run installs:
- github.com/acme/data-utils@main: commit 4f2c9e1a7b3d, new since 9b81d0c4e2f7.
```

#### When the check fails

What happens depends on whether the problem can fix itself:

| Problem | What the run does |
| :- | :- |
| The Git host doesn't answer, times out, or returns a server error | Tries again after 30 s, 60 s, and 120 s. If the host still doesn't answer, the run starts with the commit it last installed and the summary warns about it. If no commit was installed before, the run fails. |
| The token is invalid or can't read the repository, the repository or `ref` doesn't exist, or a referenced secret has no value | The run fails before it starts, without retrying. The run log says which package can't be installed and what to fix. |

A failed check stops the run before its code runs, so no data is written.

> [!TIP]
>
> Point production notebooks at a tag (`@v1.0.0`) or a commit. Code reaches them only when you change the dependency, so a push to the repository can't break a notebook that is already running. Use a branch while you develop the package.

### How secrets in dependencies are handled

* **Access:** You can only reference secrets you have permission to read. Saving a notebook that references any other secret fails, whether the secret is missing or you lack permission.
* **Linking:** A secret referenced in a dependency is added to the notebook's secrets, the same way as a secret your code loads with [`load_secret`](https://docs.nekt.com/notebooks/sdk#load-secret).
* **Deletion:** A secret a notebook uses can't be deleted. The error lists the notebooks that use it. Remove the reference from their code and dependencies first.
* **Storage:** The notebook stores only the `${KEY}` reference. The value is read when dependencies are installed, so rotating the secret needs no change to the notebook.
* **Exposure:** The secret's value is used only to install the package. Your notebook code can't read it from the environment, and the value is masked in run logs.

> [!NOTE]
>
> The `git+https://` scheme is the only one supported for Git dependencies. SSH URLs (`git+ssh://`) are rejected.

### Other Git hosts

Any Git host that accepts a token over HTTPS works. Put the token where the host expects it in the URL. For example, on GitLab:

```text theme={null}
git+https://oauth2:${GITLAB_TOKEN}@gitlab.com/acme/data-utils.git@v1.0.0
```

## Spark jars

PySpark notebooks can also add **Spark jars**: Maven Central coordinates in the `group:artifact:version` format, added to Spark's classpath.

```text theme={null}
org.apache.sedona:sedona-spark-shaded-3.5_2.12:1.7.1
```

Transitive dependencies of a jar are not downloaded, so prefer shaded jars. If the jar has a Python API, add its package under **Dependencies** too.

## Troubleshooting

| Error | What to check |
| :- | :- |
| `Following modules are invalid: <package>` | The package name or version doesn't exist on PyPI. |
| `These secrets do not exist or you do not have permission to read them: <KEY>` | The secret's key matches the reference exactly (keys are case-sensitive), and you have access to the secret. |
| `Invalid secret references` | The reference uses the `${KEY}` format, with only letters, digits, and underscores in the key. |
| `Repository not found, or the credentials cannot read it.` | The repository URL is correct, and the token in the secret is valid and can read the repository. |
| `Ref '<ref>' not found in the repository.` | The tag, branch, or commit after `@` exists in the repository. |
| `Git dependencies must not contain a token or password` | Store the token as a secret and reference it as `${KEY}`. |
| `Cannot delete secret because it is used by transformations` | Remove the secret from the listed notebooks' code and dependencies, then delete it. |
| Run log: `the package cannot be installed, so the run stopped before it started` | Follow the reason after the package name, for example renew the token in the secret or fix the `ref`. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.