FastBCP 1.0 ADBC SQL Server Benchmark

FastBCP 1.0 introduces a new adbc_mssql connector built on Apache Arrow's ADBC interface. It transfers data in columnar Arrow format — no row-by-row serialization. We benchmarked it against the classic mssql .NET driver, DuckDB (both adbc_scanner and mssql extensions), and Sling on a 15 million row TPC-H table.

The result: 17.3 million cells per second with 8 threads and Zstd compression — 23x faster than Sling and 12x faster than DuckDB.

Test Environment

Component	Details
OS	Windows 11 (10.0.26200)
CPU	Intel Core i7-14650HX
SQL Server	2022 Developer Edition — localhost,11433
Database	tpch10
Table	dbo.orders — 15,000,000 rows, 9 columns = 135,000,000 cells
FastBCP	v1.0.3.0 (.NET 10.0.9)
DuckDB	v1.5.3 + adbc_scanner extension (query-farm) + mssql extension (Tmonster)
Sling	v1.5.20 (standard driver only — ADBC crashes on Windows)
Output	Local disk — --merge false (no file merging)

Full Results

#	Tool	Connector	Mode	Compression	Time (s)	Size (MB)	rows/s	cells/s	Command
1	FastBCP	mssql	single	None	25.2	1,859	604,869	5,357,143	cmd
2	FastBCP	mssql	single	Snappy	25.3	631	602,775	5,335,968	cmd
3	FastBCP	mssql	single	Gzip	49.1	374	307,612	2,749,490	cmd
4	FastBCP	mssql	single	Zstd	29.6	382	514,314	4,560,811	cmd
5	FastBCP	adbc_mssql	single	None	14.3	1,283	1,079,721	9,440,559	cmd
6	FastBCP	adbc_mssql	single	Snappy	15.5	539	989,910	8,709,677	cmd
7	FastBCP	adbc_mssql	single	Gzip	57.2	367	264,074	2,360,140	cmd
8	FastBCP	adbc_mssql	single	Zstd	19.9	386	766,370	6,783,920	cmd
9	FastBCP	mssql	p4	None	14.8	1,859	1,080,550	9,121,622	cmd
10	FastBCP	mssql	p4	Snappy	17.7	632	891,576	7,627,119	cmd
11	FastBCP	mssql	p4	Gzip	24.8	375	625,782	5,443,548	cmd
12	FastBCP	mssql	p4	Zstd	17.2	384	916,471	7,848,837	cmd
13	FastBCP	mssql	p8	None	13.1	1,859	1,227,167	10,305,344	cmd
14	FastBCP	mssql	p8	Snappy	14.7	632	1,080,376	9,183,673	cmd
15	FastBCP	mssql	p8	Gzip	20.3	376	769,231	6,650,246	cmd
16	FastBCP	mssql	p8	Zstd	17.3	385	908,804	7,803,468	cmd
17	FastBCP	adbc_mssql	p4	None	8.0	1,284	1,875,000	16,875,000	cmd
18	FastBCP	adbc_mssql	p4	Snappy	9.4	537	1,595,745	14,361,702	cmd
19	FastBCP	adbc_mssql	p4	Gzip	19.5	366	769,231	6,923,077	cmd
20	FastBCP	adbc_mssql	p4	Zstd	9.8	384	1,530,612	13,775,510	cmd
21	FastBCP	adbc_mssql	p8	None	6.4	1,283	2,343,750	21,093,750	cmd
22	FastBCP	adbc_mssql	p8	Snappy	7.9	541	1,898,734	17,088,608	cmd
23	FastBCP	adbc_mssql	p8	Gzip	14.8	370	1,013,514	9,121,622	cmd
24	FastBCP	adbc_mssql	p8	Zstd	7.8	388	1,923,077	17,307,692	cmd
25	DuckDB	adbc_scanner	single	None	94.2	1,310	159,236	1,433,121	cmd
26	DuckDB	adbc_scanner	single	Snappy	96.8	565	155,040	1,394,628	cmd
27	DuckDB	adbc_scanner	single	Gzip	154.6	362	97,087	873,350	cmd
28	DuckDB	adbc_scanner	single	Zstd	97.7	373	153,531	1,381,781	cmd
29	Sling	sqlserver	single	None	184.5	575	82,231	731,707	cmd
30	Sling	sqlserver	single	Snappy	192.7	575	78,729	700,571	cmd
31	Sling	sqlserver	single	Gzip	274.3	374	55,075	492,160	cmd
32	Sling	sqlserver	single	Zstd	181.6	384	83,492	743,392	cmd
33	DuckDB	mssql	single	None	31.7	1,310	473,186	4,258,490	cmd
34	DuckDB	mssql	single	Snappy	25.3	565	593,160	5,338,440	cmd
35	DuckDB	mssql	single	Gzip	89.2	362	168,161	1,513,453	cmd
36	DuckDB	mssql	single	Zstd	26.2	373	572,519	5,152,672	cmd

note

Sling ADBC crashes on Windows (purego: struct return values only supported on darwin arm64 & amd64). All Sling tests use the standard driver (without use_adbc).

Charts

Tool Comparison — Zstd Compression

Export time (seconds) with Zstd compression across all tools and parallelism modes

Compression Comparison — FastBCP adbc_mssql p8

Time and file size trade-offs for None, Snappy, Gzip, and Zstd with FastBCP adbc_mssql at parallelism degree 8

Connector Impact: mssql vs adbc_mssql

Performance gain of the Arrow-native ADBC connector over the .NET mssql driver at each parallelism level

Parallelism Scaling — FastBCP adbc_mssql

Throughput scaling from single thread to degree 4 and degree 8 with adbc_mssql connector

File Size by Tool and Compression

Output file sizes (MB) across all tools and compression codecs

Key Takeaways

1. FastBCP adbc_mssql is the clear winner: the ADBC Arrow-native connector is 1.5–2.2x faster than the .NET mssql driver, thanks to columnar Apache Arrow data transfer (zero row-by-row serialization).

2. Best configuration: FastBCP adbc_mssql + paralleldegree 8 + Zstd exports 15M rows (135M cells) in 7.8 seconds at 17.3M cells/s for a 388 MB file. That's 23x faster than Sling and 12x faster than DuckDB.

3. Zstd = best trade-off: 3.3x compression ratio with minimal overhead (+22% time vs None). Gzip compresses 5% more but is 2x slower.

4. Parallelism scales well: going from single thread to degree 8 yields a 2.6x speedup (adbc) and 1.9x (mssql). Gains diminish beyond 4 threads on this table.

5. DuckDB mssql extension: significantly faster than adbc_scanner (~26–32s vs ~95s, 3–4x speedup). Produces identical file sizes and Parquet schema.

6. DuckDB adbc_scanner: functional but slow (~95s single thread, ~1.4M cells/s). No native multi-connection parallelism.

7. Sling: slowest (180–274s, ~0.7M cells/s). The ADBC option (use_adbc: true) crashes on Windows due to a purego bug in the Go ADBC driver manager.

Parquet Validation

All 36 output files were verified readable with DuckDB read_parquet(). Every file contains the expected 15,000,000 rows.

Parallel exports (p4 = 4 chunks, p8 = 8 chunks) are also readable via glob: read_parquet('*_chunk_*.parquet').

SQL Server Source Types

Column	SQL Server Type	Precision	Scale
o_orderdate	date	—	—
o_orderkey	bigint	19	0
o_custkey	bigint	19	0
o_orderpriority	char(15)	—	—
o_shippriority	int	10	0
o_clerk	char(15)	—	—
o_orderstatus	char(1)	—	—
o_totalprice	money	19	4
o_comment	varchar(79)	—	—

Logical Types (read by DuckDB)

Column	FastBCP mssql	FastBCP adbc	DuckDB adbc	DuckDB mssql	Sling std
o_orderdate	DATE	DATE	DATE	DATE	DATE
o_orderkey	BIGINT	BIGINT	BIGINT	BIGINT	BIGINT
o_custkey	BIGINT	BIGINT	BIGINT	BIGINT	BIGINT
o_orderpriority	VARCHAR	VARCHAR	VARCHAR	VARCHAR	VARCHAR
o_shippriority	INTEGER	INTEGER	INTEGER	INTEGER	INTEGER
o_clerk	VARCHAR	VARCHAR	VARCHAR	VARCHAR	VARCHAR
o_orderstatus	VARCHAR	VARCHAR	VARCHAR	VARCHAR	VARCHAR
o_totalprice	DECIMAL(19,4)	DECIMAL(19,4)	DECIMAL(19,4)	DECIMAL(19,4)	DECIMAL(24,6)
o_comment	VARCHAR	VARCHAR	VARCHAR	VARCHAR	VARCHAR

Parquet Physical Types & Metadata

Column	SQL Server	FastBCP mssql	FastBCP adbc	DuckDB adbc	DuckDB mssql	Sling std
o_orderdate	date	INT32 / DATE	INT32 / DateType()	INT32 / DATE	INT32 / DATE	INT32 / DATE
o_orderkey	bigint	INT64 / INT_64	INT64	INT64 / INT_64	INT64 / INT_64	INT64 / INT_64
o_custkey	bigint	INT64 / INT_64	INT64	INT64 / INT_64	INT64 / INT_64	INT64 / INT_64
o_orderpriority	char(15)	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8
o_shippriority	int	INT32 / INT_32	INT32	INT32 / INT_32	INT32 / INT_32	INT32 / INT_32
o_clerk	char(15)	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8
o_orderstatus	char(1)	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8
o_totalprice	money(19,4)	FIXED_LEN / Decimal(19,4)	FIXED_LEN / Decimal(19,4)	FIXED_LEN / Decimal(19,4)	FIXED_LEN / Decimal(19,4)	FIXED_LEN / Decimal(24,6)
o_comment	varchar(79)	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8	BYTE_ARRAY / UTF8

Annotation Differences

Aspect	FastBCP mssql	FastBCP adbc	DuckDB adbc	DuckDB mssql	Sling std
converted_type annotations	Full (INT_64, INT_32, DATE)	Partial (missing for INT columns)	Full	Full	Full
logical_type annotations	Full (IntType, StringType)	Partial (DateType, StringType only)	Full	Full	Full
o_totalprice precision	19,4 (matches source)	19,4 (matches source)	19,4 (matches source)	19,4 (matches source)	24,6 (widened)
Schema root name	root	schema	duckdb_schema	duckdb_schema	duckdb_schema

tip

FastBCP mssql and both DuckDB connectors produce full converted_type and logical_type annotations. FastBCP adbc has partial annotations (INT columns missing). Only Sling widens o_totalprice to DECIMAL(24,6).

Commands Used

FastBCP — mssql connector (single thread)

# None compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_single_none_orders.parquet" `
  --parquetcompression None `
  --nobanner --runid "fastbcp_mssql_single_none"

# Snappy compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_single_snappy_orders.parquet" `
  --parquetcompression Snappy `
  --nobanner --runid "fastbcp_mssql_single_snappy"

# Gzip compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_single_gzip_orders.parquet" `
  --parquetcompression Gzip `
  --nobanner --runid "fastbcp_mssql_single_gzip"

# Zstd compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_single_zstd_orders.parquet" `
  --parquetcompression Zstd `
  --nobanner --runid "fastbcp_mssql_single_zstd"

FastBCP — adbc_mssql connector (single thread)

# None compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_single_none_orders.parquet" `
  --parquetcompression None `
  --nobanner --runid "fastbcp_adbc_single_none"

# Snappy compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_single_snappy_orders.parquet" `
  --parquetcompression Snappy `
  --nobanner --runid "fastbcp_adbc_single_snappy"

# Gzip compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_single_gzip_orders.parquet" `
  --parquetcompression Gzip `
  --nobanner --runid "fastbcp_adbc_single_gzip"

# Zstd compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --query "SELECT * FROM dbo.orders" `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_single_zstd_orders.parquet" `
  --parquetcompression Zstd `
  --nobanner --runid "fastbcp_adbc_single_zstd"

FastBCP — mssql connector (parallel degree 4)

# None compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p4_none_orders.parquet" `
  --parquetcompression None `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p4_none"

# Snappy compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p4_snappy_orders.parquet" `
  --parquetcompression Snappy `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p4_snappy"

# Gzip compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p4_gzip_orders.parquet" `
  --parquetcompression Gzip `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p4_gzip"

# Zstd compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p4_zstd_orders.parquet" `
  --parquetcompression Zstd `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p4_zstd"

FastBCP — mssql connector (parallel degree 8)

# None compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p8_none_orders.parquet" `
  --parquetcompression None `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p8_none"

# Snappy compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p8_snappy_orders.parquet" `
  --parquetcompression Snappy `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p8_snappy"

# Gzip compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p8_gzip_orders.parquet" `
  --parquetcompression Gzip `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p8_gzip"

# Zstd compression
.\FastBCP.exe `
  --connectiontype mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_mssql_p8_zstd_orders.parquet" `
  --parquetcompression Zstd `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_mssql_p8_zstd"

FastBCP — adbc_mssql connector (parallel degree 4)

# None compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p4_none_orders.parquet" `
  --parquetcompression None `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p4_none"

# Snappy compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p4_snappy_orders.parquet" `
  --parquetcompression Snappy `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p4_snappy"

# Gzip compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p4_gzip_orders.parquet" `
  --parquetcompression Gzip `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p4_gzip"

# Zstd compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p4_zstd_orders.parquet" `
  --parquetcompression Zstd `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 4 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p4_zstd"

FastBCP — adbc_mssql connector (parallel degree 8)

# None compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p8_none_orders.parquet" `
  --parquetcompression None `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p8_none"

# Snappy compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p8_snappy_orders.parquet" `
  --parquetcompression Snappy `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p8_snappy"

# Gzip compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p8_gzip_orders.parquet" `
  --parquetcompression Gzip `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p8_gzip"

# Zstd compression
.\FastBCP.exe `
  --connectiontype adbc_mssql `
  --server "localhost,11433" `
  --database tpch10 `
  --trusted `
  --sourceschema dbo `
  --sourcetable orders `
  --directory "D:\FastBCP\Test\ADBC\output" `
  --fileoutput "fastbcp_adbc_p8_zstd_orders.parquet" `
  --parquetcompression Zstd `
  --parallelmethod Ntile `
  --distributekeycolumn o_orderkey `
  --paralleldegree 8 `
  --merge false `
  --nobanner --runid "fastbcp_adbc_p8_zstd"

DuckDB — adbc_scanner extension

# None compression
duckdb -unsigned -c "
INSTALL adbc_scanner FROM community;
LOAD adbc_scanner;
SET VARIABLE conn = (SELECT adbc_connect({'driver': 'mssql', 'uri': 'sqlserver://localhost:11433?database=tpch10&trusted_connection=true'}));
COPY (SELECT * FROM adbc_scan(getvariable('conn')::BIGINT, 'SELECT * FROM dbo.orders'))
TO 'D:/FastBCP/Test/ADBC/output/duckdb_adbc_single_none_orders.parquet'
(FORMAT PARQUET, COMPRESSION UNCOMPRESSED);
SELECT adbc_disconnect(getvariable('conn')::BIGINT);
"

# Snappy compression
duckdb -unsigned -c "
INSTALL adbc_scanner FROM community;
LOAD adbc_scanner;
SET VARIABLE conn = (SELECT adbc_connect({'driver': 'mssql', 'uri': 'sqlserver://localhost:11433?database=tpch10&trusted_connection=true'}));
COPY (SELECT * FROM adbc_scan(getvariable('conn')::BIGINT, 'SELECT * FROM dbo.orders'))
TO 'D:/FastBCP/Test/ADBC/output/duckdb_adbc_single_snappy_orders.parquet'
(FORMAT PARQUET, COMPRESSION SNAPPY);
SELECT adbc_disconnect(getvariable('conn')::BIGINT);
"

# Gzip compression
duckdb -unsigned -c "
INSTALL adbc_scanner FROM community;
LOAD adbc_scanner;
SET VARIABLE conn = (SELECT adbc_connect({'driver': 'mssql', 'uri': 'sqlserver://localhost:11433?database=tpch10&trusted_connection=true'}));
COPY (SELECT * FROM adbc_scan(getvariable('conn')::BIGINT, 'SELECT * FROM dbo.orders'))
TO 'D:/FastBCP/Test/ADBC/output/duckdb_adbc_single_gzip_orders.parquet'
(FORMAT PARQUET, COMPRESSION GZIP);
SELECT adbc_disconnect(getvariable('conn')::BIGINT);
"

# Zstd compression
duckdb -unsigned -c "
INSTALL adbc_scanner FROM community;
LOAD adbc_scanner;
SET VARIABLE conn = (SELECT adbc_connect({'driver': 'mssql', 'uri': 'sqlserver://localhost:11433?database=tpch10&trusted_connection=true'}));
COPY (SELECT * FROM adbc_scan(getvariable('conn')::BIGINT, 'SELECT * FROM dbo.orders'))
TO 'D:/FastBCP/Test/ADBC/output/duckdb_adbc_single_zstd_orders.parquet'
(FORMAT PARQUET, COMPRESSION ZSTD);
SELECT adbc_disconnect(getvariable('conn')::BIGINT);
"

DuckDB — mssql extension

# None compression
duckdb -unsigned -c "
INSTALL mssql FROM community;
LOAD mssql;
ATTACH 'SERVER=localhost,11433;DATABASE=tpch10;Trusted_Connection=Yes' AS ms (TYPE mssql);
COPY (SELECT * FROM ms.dbo.orders)
TO 'D:/FastBCP/Test/ADBC/output/duckdb_mssql_single_none_orders.parquet'
(FORMAT PARQUET, COMPRESSION UNCOMPRESSED);
DETACH ms;
"

# Snappy compression
duckdb -unsigned -c "
INSTALL mssql FROM community;
LOAD mssql;
ATTACH 'SERVER=localhost,11433;DATABASE=tpch10;Trusted_Connection=Yes' AS ms (TYPE mssql);
COPY (SELECT * FROM ms.dbo.orders)
TO 'D:/FastBCP/Test/ADBC/output/duckdb_mssql_single_snappy_orders.parquet'
(FORMAT PARQUET, COMPRESSION SNAPPY);
DETACH ms;
"

# Gzip compression
duckdb -unsigned -c "
INSTALL mssql FROM community;
LOAD mssql;
ATTACH 'SERVER=localhost,11433;DATABASE=tpch10;Trusted_Connection=Yes' AS ms (TYPE mssql);
COPY (SELECT * FROM ms.dbo.orders)
TO 'D:/FastBCP/Test/ADBC/output/duckdb_mssql_single_gzip_orders.parquet'
(FORMAT PARQUET, COMPRESSION GZIP);
DETACH ms;
"

# Zstd compression
duckdb -unsigned -c "
INSTALL mssql FROM community;
LOAD mssql;
ATTACH 'SERVER=localhost,11433;DATABASE=tpch10;Trusted_Connection=Yes' AS ms (TYPE mssql);
COPY (SELECT * FROM ms.dbo.orders)
TO 'D:/FastBCP/Test/ADBC/output/duckdb_mssql_single_zstd_orders.parquet'
(FORMAT PARQUET, COMPRESSION ZSTD);
DETACH ms;
"

Sling — sqlserver standard driver

# Connection setup (once)
sling conns set MSSQL_STD type=sqlserver host=localhost port=11433 database=tpch10 trusted_connection=true encrypt=disable

# None compression
sling run `
  --src-conn MSSQL_STD `
  --src-stream "dbo.orders" `
  --tgt-object "file://D:/FastBCP/Test/ADBC/output/sling_std_single_none_orders.parquet" `
  --tgt-options '{format: parquet, compression: none}'

# Snappy compression
sling run `
  --src-conn MSSQL_STD `
  --src-stream "dbo.orders" `
  --tgt-object "file://D:/FastBCP/Test/ADBC/output/sling_std_single_snappy_orders.parquet" `
  --tgt-options '{format: parquet, compression: snappy}'

# Gzip compression
sling run `
  --src-conn MSSQL_STD `
  --src-stream "dbo.orders" `
  --tgt-object "file://D:/FastBCP/Test/ADBC/output/sling_std_single_gzip_orders.parquet" `
  --tgt-options '{format: parquet, compression: gzip}'

# Zstd compression
sling run `
  --src-conn MSSQL_STD `
  --src-stream "dbo.orders" `
  --tgt-object "file://D:/FastBCP/Test/ADBC/output/sling_std_single_zstd_orders.parquet" `
  --tgt-options '{format: parquet, compression: zstd}'

Resources

• FastBCP: https://fastbcp.arpe.io
• Docs: https://fastbcp-docs.arpe.io/latest/
• Wizard: https://fastbcp-docs.arpe.io/latest/wizard
• FastBCP Free 30-Day Trial: https://arpe.io/get-trial?product=FastBCP
• Apache Arrow ADBC: https://arrow.apache.org/adbc/
• DuckDB adbc_scanner: https://github.com/query-farm/adbc_scanner
• DuckDB mssql extension: https://duckdb.org/community_extensions/extensions/mssql
• Sling: https://slingdata.io