> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nekt.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Common data transformations

> Practical recipes for the most common data transformations, with examples in Athena SQL, BigQuery, and Python (Nekt SDK).

When working with raw data from APIs, databases, and file uploads, you'll often need to reshape, clean, or enrich it before it's ready for analysis. This section provides step-by-step recipes for the most common transformations you'll encounter.

Each recipe includes the same transformation implemented in **Athena SQL**, **BigQuery**, and **Python notebooks** (using the Nekt SDK with PySpark), so you can pick the approach that fits your stack.

***

### Schema and column operations

<CardGroup cols={3}>
  <Card title="Rename columns" icon="pen" href="./rename-columns">
    Standardize column names across sources — apply snake\_case, remove prefixes, or align naming conventions.
  </Card>

  <Card title="Cast or convert data types" icon="arrow-right-arrow-left" href="./cast-data-types">
    Convert strings to dates, numbers, or booleans and handle format differences across sources.
  </Card>

  <Card title="Handle NULL values" icon="circle-half-stroke" href="./handle-null-values">
    Replace NULLs with defaults using COALESCE and clean up missing data for downstream consumers.
  </Card>
</CardGroup>

### Structured data handling

<CardGroup cols={3}>
  <Card title="Unnest arrays" icon="list-tree" href="./unnest-arrays">
    Explode array columns into individual rows — essential for tags, labels, and line items from APIs.
  </Card>

  <Card title="Parse JSON fields" icon="brackets-curly" href="./parse-json-fields">
    Extract values from JSON string columns into proper, typed columns ready for analysis.
  </Card>

  <Card title="Flatten nested structures" icon="layer-group" href="./flatten-nested-structures">
    Normalize deeply nested objects into flat, query-friendly rows.
  </Card>

  <Card title="Expand custom fields" icon="table-columns" href="./expand-custom-fields">
    Pivot a key-value custom fields table into columns on the main entity — common for CRMs like Pipedrive and HubSpot.
  </Card>

  <Card title="Date parsing and calculations" icon="calendar-days" href="./date-calculations">
    Extract date parts, calculate differences, and format dates consistently across SQL engines.
  </Card>
</CardGroup>

### Data quality

<CardGroup cols={3}>
  <Card title="Deduplicate rows" icon="copy" href="./deduplicate-rows">
    Remove exact or near-duplicate records using ROW\_NUMBER, DISTINCT, and other dedup strategies.
  </Card>

  <Card title="Filter invalid rows" icon="filter" href="./filter-invalid-rows">
    Remove or flag rows that fail validation rules before they reach your trusted layer.
  </Card>
</CardGroup>

### Combining and reshaping

<CardGroup cols={3}>
  <Card title="Join tables" icon="code-merge" href="./join-tables">
    Combine related tables with LEFT, INNER, and FULL joins using practical, real-world examples.
  </Card>

  <Card title="Pivot / unpivot data" icon="rotate" href="./pivot-unpivot">
    Reshape rows into columns (and vice versa) to prepare data for reporting and analysis.
  </Card>
</CardGroup>