Azure Blob Storage

Azure Blob is a Data Sources/Sinks service for storing and retrieving data.

Configure Azure Blob Storage credentials

You can safely store your Azure Blob Storage credentials in MotherDuck for convenience by creating a SECRET object using the CREATE SECRET command.

note

See Azure docs to find the correct connection string format.

Create a SECRET object

SQL

-- to configure a secret manually:
CREATE SECRET IN MOTHERDUCK (
    TYPE AZURE,
    CONNECTION_STRING '[your_connection_string]'
);

-- to store a secret configured through `az configure`:
CREATE SECRET az_secret IN MOTHERDUCK (
      TYPE AZURE,
      PROVIDER credential_chain,
      ACCOUNT_NAME 'some-account'
);

-- test the azure credentials
SELECT count(*) FROM 'azure://[container]/[file]'
SELECT * FROM 'azure://[container]/*.csv';

-- browse objects in a container
FROM md_list_files('azure://[container]/', limit := 50);

Python

import duckdb

con = duckdb.connect('md:')
con.sql("CREATE SECRET IN MOTHERDUCK (TYPE AZURE, CONNECTION_STRING '[your_connection_string]')");

# testing that our Azure credentials work
con.sql("SELECT count(*) FROM 'azure://[container]/[file]'").show()
con.sql("SELECT * FROM 'azure://[container]/*.csv'").show()

UI

Click on your profile to access the Settings panel and click on Secrets menu.

Then click on Add secret in the secrets section.

You will then be prompted to enter your Amazon S3 credentials.

Delete a SECRET object

SQL

You can use the same method above, using the DROP SECRET command.

DROP SECRET <secret_name>;

UI

Click on your profile and access the Settings menu. Click on the bin icon to delete the secret.

Azure credentials as temporary secrets

MotherDuck supports DuckDB syntax for providing Azure credentials.

CREATE SECRET (
    TYPE AZURE,
    CONNECTION_STRING '[your_connection_string]'
);

or if you use the az configure command to store your credentials in the az CLI.

CREATE SECRET az_secret (
      TYPE AZURE,
      PROVIDER credential_chain,
      ACCOUNT_NAME 'some-account'
);

note

Local/In-memory secrets are not persisted across sessions.

info

Even temporary, in-memory secrets are available to MotherDuck's cloud execution engine when you connect your local DuckDB instance to MotherDuck. When you query Azure Blob Storage, the query runs on MotherDuck's servers, not your local machine, and MotherDuck uses the best-matching secret to authenticate, whether it is stored locally or in MotherDuck. For more details, see CREATE SECRET.

Browse files in Azure Blob Storage

To inspect a container before querying individual files, use MD_LIST_FILES():

FROM md_list_files('azure://[container]/');
FROM md_list_files('az://[container]/path/');