NumPy

Overview

NumPy (short for Numerical Python) is the foundational package for numerical computing in Python. Its central object, the ndarray, is a fixed-type, contiguous, N-dimensional array that supports fast element-wise arithmetic, linear algebra, random number generation, and broadcasting. Because it stores data as compact typed buffers rather than lists of Python objects, NumPy operations run orders of magnitude faster than equivalent pure-Python loops. It has been the substrate that pandas, scikit-learn, PyTorch, and much of the scientific Python ecosystem are built on top of.

The ndarray and vectorization

An ndarray has a fixed dtype (e.g. float64, int32) and shape. Operations are vectorized: instead of looping over elements in Python, you apply a function to the whole array at once, and NumPy executes the loop in compiled C code.

Copy code
import numpy as np

a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])

c = a * b + 1          # vectorized, no Python-level loop
mean = c.mean()

Broadcasting lets NumPy apply operations between arrays of different but compatible shapes (e.g. a matrix and a row vector) without manually replicating data.

NumPy and DuckDB

DuckDB's Python client interoperates with NumPy in both directions. DuckDB can query NumPy arrays directly by name via a replacement scan, and query results can be exported as NumPy arrays with .fetchnumpy().

Copy code
import duckdb
import numpy as np

my_arr = np.array([(1, 9.0), (2, 8.0), (3, 7.0)])
duckdb.sql("SELECT * FROM my_arr").show()

result = duckdb.sql("SELECT unnest([1, 2, 3]) AS x").fetchnumpy()

This makes it easy to push heavy filtering, joining, and aggregation into DuckDB's vectorized SQL engine and only pull the smaller, already-reduced result into NumPy for downstream numerical work like modeling or plotting.