Google Cloud Storage

With MotherDuck, you can access files in a private Google Cloud Storage (GCS) bucket.

Google Cloud Storage Requirements

• Enable Amazon S3 API interoperability
• Assign a default project for interoperable access
• Create an access key and secret

Configure Google Cloud Storage credentials

You can safely store your Google Cloud Storage credentials in MotherDuck for convenience by creating a SECRET object using the CREATE SECRET command.

Create a SECRET object

You can safely store your Google Cloud Storage credentials in MotherDuck for convenience by creating a SECRET object using the CREATE SECRET command.

SQL

CREATE SECRET IN MOTHERDUCK (
    TYPE GCS,
	KEY_ID 'access_key',
    SECRET 'secret_key',
);

-- test GCS credentials
SELECT count(*) FROM 'gcs://<bucket>/<file>';

Python

import duckdb

con = duckdb.connect('md:')
con.sql("CREATE SECRET IN MOTHERDUCK (TYPE GCS, KEY_ID 'access_key', SECRET 'secret_key')");

# test GCS
con.sql("SELECT count(*) FROM 'gcs://<bucket>/<file>'").show()
# 42

UI

Click on your profile to access the Settings panel and click on Secrets menu.

Then click on Add secret in the secrets section.

You will then be prompted to enter your Amazon S3 credentials.

You can update your secret by executing CREATE OR REPLACE SECRET command to overwrite your secret.

Delete a SECRET object

SQL

You can use the same method above, using the DROP SECRET command.

DROP SECRET <secret_name>;

UI

Click on your profile and access the Settings menu. Click on the bin icon to delete your current secrets.

Google Cloud Storage credentials as temporary secrets

MotherDuck supports DuckDB syntax for providing GCS credentials.

CREATE SECRET (
    TYPE GCS,
    KEY_ID 's3_access_key',
    SECRET 's3_secret_key'
);

note

Local/In-memory secrets are not persisted across sessions.