FastBCP 1.4: Teradata over ADBC and Physical parallel method

FastBCP 1.4 adds two things: Teradata export through an embedded ADBC driver, and a Physical parallel method that splits a table on the physical row location of its engine.

It also fixes Physloc, which on SQL Server only parallelised with an odd degree, and several cases where Ntile, Timepartition or a source filter could lose or duplicate rows. Read the upgrade checks at the end of the article before deploying 1.4.

Teradata through ADBC: -C adbc_teradata

The new adbc_teradata connection type uses ArpeTeradata, an ADBC driver embedded in FastBCP, for both the metadata step and the data step. It keeps the Teradata SQL dialect, so every parallel method of -C teradata works: Random, Ntile, DataDriven, RangeId, Timepartition, and DBPartition on PPI tables.

By default the driver logs on with TLS over the HTTPS port (443), with no Teradata client software. When the server has no TLS, or with SSL Mode=disable, it uses the legacy port (1025) through Teradata Tools and Utilities (TTU):

• Output is Parquet only, as for the other adbc_* types. Windows x64 and Linux x64 only.
• Connect with --server host[:port], --user, --password and an optional --database, or with a --sourceconnectstring using the Teradata provider keys (Server, Port, Database, User Id, Password, SSL Mode, HTTPS Port, TMODE, TTU Path…). Explicit adbc.arpeteradata.* keys take precedence.
• Logon mechanisms: LDAP, JWT, KRB5 and TDNEGO. JWT and Kerberos work over TLS, without TTU. Trusted authentication (-A / --trusted) is not supported.

Windows

.\FastBCP.exe `
  --connectiontype adbc_teradata `
  --server "td-host" `
  --database "TPCH" `
  --user "FastUser" `
  --password "FastPassword" `
  --sourceschema "TPCH" `
  --sourcetable "ORDERS" `
  --directory "D:\export\orders" `
  --fileoutput "orders.parquet" `
  --parallelmethod Ntile `
  --distributekeycolumn "o_orderkey" `
  --paralleldegree 8

Linux

./FastBCP \
  --connectiontype adbc_teradata \
  --server "td-host" \
  --database "TPCH" \
  --user "FastUser" \
  --password "FastPassword" \
  --sourceschema "TPCH" \
  --sourcetable "ORDERS" \
  --directory "/data/export/orders" \
  --fileoutput "orders.parquet" \
  --parallelmethod Ntile \
  --distributekeycolumn "o_orderkey" \
  --paralleldegree 8

To adapt this command to your environment, use the Teradata to local Parquet pipeline builder.

Legacy port and TTU. On Windows, the driver finds wincli32.dll itself (ARPETERADATA_TTU_PATH, its own directory, then the registered TTU installation). On Linux, FastBCP looks in the new FASTBCP_TTU_PATH environment variable, then in a ttu or TeradataToolsAndUtilitiesBase directory next to the binary. Teradata's TTU download can be unpacked there without installing it. FastBCP warns at startup when that folder lacks tdclientdir or the ICU links, or when the client directory is not named after its version.

One method for physical splits: -m Physical

Each engine exposes the physical location of a row under its own name: ctid in PostgreSQL, ROWID in Oracle, %%physloc%% in SQL Server, the data slice in Netezza. Until 1.3, you had to pick the matching FastBCP method yourself. With --parallelmethod Physical, FastBCP picks it from the connection type:

• --sourceschema and --sourcetable are required. --query can be added to filter the table with a simple WHERE clause. --distributekeycolumn is not used.
• Other sources (odbc, oledb, mysql, clickhouse, hana, teradata, adbc_teradata) stop with an explicit error. With odbc and oledb, FastBCP cannot know which engine is behind the driver, so the error asks for a native connection type.
• The log shows the resolved strategy: Distribute Method : Physical (Ctid). In OpenTelemetry, fastbcp.method holds the resolved method and the new fastbcp.method.requested tag holds Physical.

Windows

.\FastBCP.exe `
  --connectiontype pgsql `
  --server "localhost:5432" `
  --database "tpch" `
  --user "FastUser" `
  --password "FastPassword" `
  --sourceschema "tpch_10" `
  --sourcetable "orders" `
  --directory "D:\export\orders" `
  --fileoutput "orders.parquet" `
  --parallelmethod Physical `
  --paralleldegree 8

Linux

./FastBCP \
  --connectiontype pgsql \
  --server "localhost:5432" \
  --database "tpch" \
  --user "FastUser" \
  --password "FastPassword" \
  --sourceschema "tpch_10" \
  --sourcetable "orders" \
  --directory "/data/export/orders" \
  --fileoutput "orders.parquet" \
  --parallelmethod Physical \
  --paralleldegree 8

The same command works against Oracle, SQL Server or Netezza: only the connection parameters change.

Deprecated method names

Ctid, Rowid, Physloc and NZDataSlice are deprecated and will be removed in FastBCP 2.0. They still work in 1.4, with a warning in the log:

Parallel method Ctid is deprecated and will be removed in 2.0 ==> use --parallelmethod Physical

Physloc on SQL Server: even chunks

Before 1.4, Physloc only spread the work with an odd --paralleldegree. The distribution key was a multiple of 256 for most page ids, so with a degree of 2, 4, 8 or 16 one chunk received every row and the other workers exported nothing. With 6 or 12, only three chunks got rows.

1.4 changes the key:

• Rowstore tables: the key is the two low bytes of the page id, so runs of 256 pages go to each chunk. Chunks stay within 3% of even up to degree 192.
• Clustered columnstore tables: %%physloc%% is a row id there, not a page address, so the old key was the same for every row. FastBCP now detects a clustered columnstore index on the source table and splits on the whole row id. The log says Physloc : <schema>.<table> is a clustered columnstore, splitting on the columnstore row id. If the check fails, FastBCP logs a warning and uses the rowstore key.

If you avoided Physloc because it did not scale, try it again with an even degree, through -m Physical.

Timepartition and drivers

Timepartition:

• Partition discovery now returns the row count of each partition with no extra scan. Partitions are exported largest first through a shared queue, instead of a static round-robin.
• Quoted or case-sensitive date columns such as "OrderDate" or [OrderDate] now work.

ADBC drivers:

• ArpeMSSQL 0.7.6 reads sql_variant and the SQL Server 2025 JSON type. An unreadable column no longer drops the connection.
• ArpeTeradata 0.3.4 is new. ArpePGSQL moves to 0.4.7, ArpeOracle to 0.4.1, ArpeNetezza to 0.3.7.

The other dependency versions are listed in the release notes.

Before you upgrade

These changes can alter the result of an existing job:

1. Physloc and Rowid require --sourceschema and --sourcetable, like Ctid, whether used directly or through Physical. Without them, FastBCP stops at the parameter check. A Physloc job based on --query alone must now also give the schema and table. (Before 1.4, Rowid crashed in that case.)
2. adbc_oracle (ArpeOracle 0.4.1) has stricter security defaults. TCPS verifies the server certificate. Kerberos no longer forwards the TGT unless krb5_forward=true is set, and requires mutual authentication. Token and OAuth2 logons require TCPS.
3. --help exits with code 0. -?, --help, /? and /help now print the usage only, without the banner, and exit with code 0. Before, they also printed four ERROR in command line parameters lines and exited with code 1. Update any script that relied on that exit code.
4. Deprecated method names log a warning. If you parse logs for warnings, replace Ctid, Rowid, Physloc and NZDataSlice with Physical.

Notable fixes

• Ntile dropped rows with sub-millisecond timestamp keys (SQL Server datetime2(4..7), PostgreSQL timestamp, Oracle TIMESTAMP(4..9)). Chunk bounds were truncated to milliseconds. Chunks now share their boundaries, so every row lands in exactly one chunk. If you used Ntile on such a key before 1.4, compare source and target row counts.
• Source filters with OR could export extra rows. The predicate added by a parallel method was not parenthesized, so WHERE a = 1 AND k < x OR k IS NULL exported rows with a NULL key outside the filter. This affected every method that rewrites the WHERE clause.
• Timepartition skipped rows with a NULL date. They now go to a <column>=NULL directory.
• Timepartition with week granularity duplicated rows. The week predicate now combines the date range with exact year/week filters, on every engine.
• Random and Physloc showed wrong per-chunk and total row counts in the log: each chunk now counts its own rows.
• Ntile no longer throws on a bucket whose minimum is NULL, and numeric bounds no longer depend on regional settings.
• Output file size and disk write throughput are now reported with DataDriven, Timepartition and DBPartition.

The full list is in the release notes.

Resources

• FastBCP 1.4 release notes
• Connection parameters: adbc_teradata and FASTBCP_TTU_PATH
• Parallel parameters: Physical method
• Teradata to local Parquet pipeline builder
• Request a FastBCP trial