Google Cloud Storage
With MotherDuck, you can access files in a private Google Cloud Storage (GCS) bucket. This leverages the GCS S3 compatible connection.
Google Cloud Storage Connection Process
- Create an HMAC key for the service account: Cloud Storage → Settings → Interoperability → Create a key for a service account
- Save the Access ID and Secret (shown once)
- Create the DuckDB secret using the HMAC credentials as described below
Configure Google Cloud Storage credentials
You can safely store your Google Cloud Storage credentials in MotherDuck for convenience by creating a SECRET object using the CREATE SECRET command.
Create a SECRET object
You can safely store your Google Cloud Storage credentials in MotherDuck for convenience by creating a SECRET object using the CREATE SECRET command.
- SQL
- Python
- UI
CREATE SECRET IN MOTHERDUCK (
TYPE GCS,
KEY_ID 'HMAC_ACCESS_ID',
SECRET 'HMAC_SECRET',
);
-- test GCS credentials
SELECT count(*) FROM 'gcs://<bucket>/<file>';
import duckdb
con = duckdb.connect('md:')
con.sql("CREATE SECRET IN MOTHERDUCK (TYPE GCS, KEY_ID 'access_key', SECRET 'secret_key')");
# test GCS
con.sql("SELECT count(*) FROM 'gcs://<bucket>/<file>'").show()
# 42
Click on your profile to access the Settings panel and click on Secrets menu.
Then click on Add secret in the secrets section.
You will then be prompted to enter your Amazon S3 credentials.
You can update your secret by executing CREATE OR REPLACE SECRET command to overwrite your secret.
Delete a SECRET object
- SQL
- UI
You can use the same method above, using the DROP SECRET command.
DROP SECRET <secret_name>;
Click on your profile and access the Settings menu. Click on the bin icon to delete your current secrets.
Google Cloud Storage credentials as temporary secrets
MotherDuck supports DuckDB syntax for providing GCS credentials.
CREATE SECRET (
TYPE GCS,
KEY_ID 's3_access_key',
SECRET 's3_secret_key'
);
Local/In-memory secrets are not persisted across sessions.