Skip to main content
This connector reads the invoicing data of Aurora’s ERP (Progress / Datasul) through the ERP’s own API. It was built for one table today, invoice lines (get_f_faturamento), and replaces the generic REST API source that read only the last 45 days. What it does differently:
  • It can load the whole history. It splits the history into periods (30 days each by default) and asks the ERP for several periods at the same time, so the full history since 2022 fits in one sync.
  • It keeps the table you already have. Column names and types are the same as in the table loaded by the REST API source, so your queries keep working.
  • A Full sync removes invoices deleted in the ERP. See Keeping deletions in sync.
The connector only reads data. It never changes anything in the ERP.

Before you start

  • The ERP server must accept connections from Nekt. The address used today (https://aurorapxr1.aurora.com.br:9443/api/esp/v1) already does, for the REST API source.
  • Have the Authorization value at hand. It is the complete value of the Authorization header that the ERP API accepts (for example Basic ... or Bearer ...): the same value configured today in the REST API source.
  • A full history sync takes a while and loads the ERP server. With the default settings it makes about 60 requests, 4 at a time. Expect about 20 to 25 minutes if the server answers the 4 requests as fast as one, and up to about 1.5 hours if it answers one at a time. Schedule it outside business hours.

Configuring Aurora as a Source

In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the Aurora (Progress ERP) option from the list of connectors. Click Next and you’ll be prompted to add your access.

1. Add account access

The following configurations are available:
  • Base URL: Address of the ERP API, up to the version, starting with https://. It comes filled with https://aurorapxr1.aurora.com.br:9443/api/esp/v1.
  • Authorization: The complete value of the Authorization header the ERP API accepts, exactly as sent today.
  • Start Date: The first issue date read on the first sync and on every Full sync. It comes filled with 2022-01-01.
  • Lookback (days) (advanced): Each regular sync reads again the invoices issued this many days before the previous sync, to pick up changes and cancellations. Default: 45, the same period the REST API source reads today.
  • Window Size (days) (advanced): How many days each request to the ERP covers. Default: 30. If the ERP takes too long to answer one request, the sync splits that period in smaller ones by itself.
  • Max Parallel Requests (advanced): How many periods are requested at the same time (1 to 8). Default: 4. Higher is faster but loads the ERP server more.
  • Request Timeout (seconds) (advanced): How long to wait for the ERP to answer one request before splitting its period and trying again. Default: 600.
Once you’re done, click Next.

2. Select streams

Select the Invoice lines stream (get_f_faturamento) and click Next.

3. Configure data streams

Customize how you want your data to appear in your catalog: the layer, a folder, the table name (you can add a prefix, such as the progress_erp_ used today) and the type of sync.
  • Sync Type: keep INCREMENTAL. Each regular sync then reads only the recent period and updates the table, and a Full sync (manual or scheduled) reloads the whole history. Choosing FULL_TABLE instead makes every sync read the whole history.
Once you are done configuring, click Next.

4. Configure data source

Describe your data source for easy identification within your organization, not exceeding 140 characters. To define your Trigger, consider how often you want data to be extracted from this source. The REST API source runs every 2 hours on business days; the same schedule works here. To keep deletions in sync, also set an Additional Full Sync schedule (see below). Once you are ready, click Next to finalize the setup.

5. Check your new source

You can view your new source on the Sources page. If needed, manually trigger the source extraction by clicking on the arrow button. The first sync reads the whole history from the Start Date. Once executed, your data will appear in your Catalog.
For you to be able to see it on your Catalog, you need at least one successful source run.

Keeping deletions in sync

The ERP API only returns invoices that exist today. It does not tell which invoices were deleted.
  • A regular (incremental) sync never removes anything. It reads the last days (45 by default) and adds or updates those invoice lines. An invoice deleted in the ERP stays in your table until the next Full sync.
  • A Full sync rebuilds the table. It reads the whole history from the Start Date and replaces the table with it, so deleted invoices disappear. It also brings changes to invoices older than the lookback period.
To do this automatically, schedule an Additional Full Sync in the source settings, for example every night (or every weekend, if a nightly reload is too heavy for the ERP server), at a time when the ERP is not busy. Between two Full syncs, deleted invoices can still appear in the table.
If a Full sync fails halfway (for example, the ERP stops answering), the table is not replaced: it keeps the data of the last successful sync, and the sync shows as failed.

Streams and Fields

Available streams

The table below lists every stream and its slug (the exact identifier to pass when creating the source via API).

Things to know

  • Same columns as before. Names are in capital letters and the types match the table loaded by the REST API source. Dates (DT_EMIS_DOCTO, DT_TRANSACAO, DT_ENTREGA_NF) are kept as text, exactly as the ERP sends them.
  • Amounts. Columns ending in _RS are in Brazilian reais and columns ending in _USS in US dollars. Amounts the ERP sends as text are converted to numbers; an amount that cannot be read is loaded empty and the sync log names its column.
  • Key. A row is identified by ESTABELEC_ID, NAT_OPERACAO, CLIENTE_ID, PRODUTO_ID, NR_SEQ_FAT, NRO_DOCTO and SERIE_DOCTO.
  • Slow answers. If the ERP takes longer than the Request Timeout to answer a period, the sync splits that period in two and tries again, down to one day, and the sync log says so. If even one day cannot be read, the sync fails (it never finishes with a period missing), and the next sync tries again.
  • Credential errors. If the ERP refuses the Authorization value, the sync stops before reading anything and the sync log asks you to check the Authorization field.

Fields by stream

Stream get_f_faturamento. Incremental by DT_EMIS_DOCTO; key ESTABELEC_ID, NAT_OPERACAO, CLIENTE_ID, PRODUTO_ID, NR_SEQ_FAT, NRO_DOCTO, SERIE_DOCTO.
  • REGIAOVENDAS_ID (string) - Sales region code
  • VL_DESC_RS (number) - Discount amount, in BRL
  • CLIENTE_ID (integer) - Customer code (part of the key)
  • VL_FRETE_RS (number) - Freight amount, in BRL
  • VL_TOT_ITEM_USS (number) - Total amount of the invoice line, in USD
  • VL_COFINS_USS (number) - COFINS tax amount, in USD
  • PRODUTO_ID (string) - Product (item) code (part of the key)
  • ESTABELEC_ID (string) - Issuing establishment (branch) code (part of the key)
  • REPRESENTANTE_ID (integer) - Sales representative code
  • VL_FCP_ST_USS (number) - FCP poverty-fund amount under ICMS tax substitution (FCP-ST), in USD
  • VL_MERC_BRU_RS (number) - Gross merchandise amount, in BRL
  • VL_MERC_LIQ_RS (number) - Net merchandise amount, in BRL
  • NR_SEQ_FAT (integer) - Sequence number of the line within the invoice (part of the key)
  • VL_FRETE_USS (number) - Freight amount, in USD
  • VL_IPI_RS (number) - IPI tax amount, in BRL
  • VL_TOT_ITEM_RS (number) - Total amount of the invoice line, in BRL
  • DT_TRANSACAO (string) - Transaction date, as sent by the ERP
  • VOLUME_FIS_UNID (number) - Physical volume of the line, in units
  • REPRES_CARTEIRA_ID (integer) - Code of the representative who owns the customer account (portfolio)
  • VL_DESPESAS_USS (number) - Additional expenses amount, in USD
  • VL_PIS_RS (number) - PIS tax amount, in BRL
  • VL_DESPESAS_RS (number) - Additional expenses amount, in BRL
  • GERENTEREGIONAL_ID (integer) - Regional manager code
  • DT_EMIS_DOCTO (string) - Issue date of the invoice, as sent by the ERP. The periods read are based on this date
  • QTD_UNID (number) - Quantity invoiced, in units
  • SUPERVISOR_ID (integer) - Sales supervisor code
  • MES_ID (integer) - Month identifier of the invoice, as sent by the ERP
  • VL_IPI_USS (number) - IPI tax amount, in USD
  • VL_FCP_ST_RS (number) - FCP poverty-fund amount under ICMS tax substitution (FCP-ST), in BRL
  • DT_ENTREGA_NF (string) - Delivery date of the invoice, as sent by the ERP
  • GERENTE_AREA_ID (integer) - Area manager code
  • TIPO_DOCTO (string) - Document type
  • VL_MERC_BRU_USS (number) - Gross merchandise amount, in USD
  • SERIE_DOCTO (string) - Invoice series (part of the key)
  • VL_MERC_LIQ_USS (number) - Net merchandise amount, in USD
  • VL_COFINS_RS (number) - COFINS tax amount, in BRL
  • VL_DESC_USS (number) - Discount amount, in USD
  • PZ_MED (number) - Average payment term of the invoice, in days
  • NRO_NF_COMP (string) - Number of the related (complementary or referenced) invoice
  • VL_ICMS_ST_USS (number) - ICMS tax substitution (ICMS-ST) amount, in USD
  • NR_PEDCLI (string) - Customer purchase order number
  • SER_NF_COMP (string) - Series of the related (complementary or referenced) invoice
  • VL_CUSMAT_USS (number) - Material cost of the line, in USD
  • NRO_DOCTO (string) - Invoice (document) number (part of the key)
  • QTD_CXS (number) - Quantity invoiced, in boxes
  • NAT_OPERACAO (string) - Fiscal operation nature code (part of the key)
  • VL_ICMS_ST_RS (number) - ICMS tax substitution (ICMS-ST) amount, in BRL
  • VL_PIS_USS (number) - PIS tax amount, in USD
  • VL_ICMS_RS (number) - ICMS tax amount, in BRL
  • VL_CUSMAT_RS (number) - Material cost of the line, in BRL
  • VL_ICMS_USS (number) - ICMS tax amount, in USD